Thread: Merge CSV files with perl csv_xs help

hello,

have bunch of csv files merge one. these files rather complicated work (they contain sorts of special characters, commas , newlines inside fields). need merge them 1 , append filename @ each row (so know file came from). tried using csvfix, due special characters rows messed up. awk won't either.

have perl script works fine appends filename on new lines inside fields. here code:

code:

#!/usr/bin/perl $result_file=shift; $pattern=shift; (@file_list) = glob $pattern;  open( out, ">$result_file") or die "can't open '$result_file':$!"; foreach $in_file (@file_list) {     open(in, "$in_file");     while (<in>) {       chomp;       print out $_ . ',' . $in_file . "\n" ;       }    close (in);    }  close (out);

i think perl can solve problem since csv_xs can handle new lines inside fields, can't find example on how this.

appreciated.

fran

what "special characters"? know encoding of text?

if it's utf-8, try:

code:

my $result_file = shift @argv;    $pattern = shift @argv;  @file_list = glob $pattern;    open( $out_fh, '>:encoding(utf8)', $result_file ) or die "could not open '$result_file': $!\n";    foreach $in_file (@file_list) {      open( $in_fh, '<:encoding(utf8)', $in_file ) or die "could not open '$in_file': $!\n";      while (<$in_fh>) {        chomp;        print $out_fh $_ . ',' . $in_file . "\n" ;        }     close ($in_fh);     }    close ($out_fh);

you should use three-argument open since allows specify encoding in open.

should use lexical file handles won't interfere others.

Forum The Ubuntu Forum Community Ubuntu Specialised Support Development & Programming Programming Talk Merge CSV files with perl csv_xs help

Ubuntu