How to make a conclusion from the text :: CSV utf8?

I have a CSV file, say win.csv, whose text is encoded in windows-1252. First I use iconv to make it in utf8.

$iconv -o test.csv -f windows-1252 -t utf-8 win.csv

Then I read the converted CSV file with the following Perl script (utfcsv.pl).

#!/usr/bin/perl 
use utf8;
use Text::CSV;
use Encode::Detect::Detector;

my $csv = Text::CSV->new({ binary => 1, sep_char => ';',});
open my $fh, "<encoding(utf8)", "test.csv";

while (my $row = $csv->getline($fh)) { 
  my $line = join " ", @$row;
  my $enc = Encode::Detect::Detector::detect($line);
  print "($enc) $line\n";
}

$csv->eof || $csv->error_diag();
close $fh;
$csv->eol("\r\n");
exit;

Then the output is as follows.

(UFT-8) .........
() .....

Namely, the encoding of all strings is defined as UTF-8 (or ASCII). But the actual output does not seem to be UTF-8. In fact, if I save the output in a file

$./utfcsv.pl > output.txt

then encoding output.txt is detected as windows-1252.

Question: How to get the output text in UFT-8?

Notes:

  • Environment: openSUSE 13.2 x86_64, perl 5.20.1
  • I do not use Text :: CSV :: Encoded because the installation fails. (Since test.csv is converted to UTF-8, it is therefore strange to use Text :: CSV :: Encoded.)
  • script . ( , CSV win.csv.)

.

#!/usr/bin/perl 
use Encode::Detect::Detector;
open my $in,  "<","$ARGV[0]" || die "open failed";
while (my $line = <$in>) {
  my $enc = Encode::Detect::Detector::detect($line);
  chomp $enc;
  if ($enc) {
    print "$enc\n";
  }
}
+4
1

(, , <:encoding(utf8) - ), , Perl

Unicode , - (ASCII) 0 0x7F Latin-1 0x80 0xFF, Windows 1252. , U 0xFC Unicode CP1252, CP1252, , 0xC3 0xBC UTF-8

binmode STDOUT , , open ,

use open qw/ :std :encoding(utf-8) /;

STDIN, STDOUT STDERR, . , CSV , :

, use strict use warnings, Perl. autodie, -, Perl , , <27 >

#!/usr/bin/perl

use utf8;
use strict;
use warnings 'all';
use open qw/ :std :encoding(utf-8) /;
use autodie;

use Text::CSV;

my $csv = Text::CSV->new({ binary => 1, sep_char => ';' });

open my $fh, '<', 'test.csv';

while ( my $row = $csv->getline($fh) ) {
    print "@$row\n";
}

close $fh;
+10

All Articles