Input Encoding: UTF-8 Receive

Question

I need to get the output of my own application in PowerShell. The problem is that the output is encoded using UTF-8 (without specification), which PowerShell does not recognize and simply converts these funky UTF characters directly to Unicode.

I found that PowerShell has the $OutputEncoding variable, but it doesn't seem to affect the input.

Good ol 'iconv doesn't help either, as this unnecessary Unicode UTF8-as-if-ASCII => conversion happens before the next pipeline member gets the data.

Answer 1

Now I see a problem with the program below (stdout.cpp - cl stdout.cpp):

 #include <stdio.h> void main() { char bytes[] = { 0x41, 0x53, 0x43, 0x49, 0x49, 0x20, 0x6F, 0x75, 0x74, 0x70, 0x75, 0x74, 0xE1, 0xBE, 0xB9}; for (int i = 0; i < 15; i++) { printf("%c", bytes[i]); } } 

And doing this with | Out-File -enc UTF8 foo.txt | Out-File -enc UTF8 foo.txt gives gibberish:

 PS> fhex foo.txt Address: 0 1 2 3 4 5 6 7 8 9 ABCDEF ASCII -------- ----------------------------------------------- ---------------- 00000000 EF BB BF 41 53 43 49 49 20 6F 75 74 70 75 74 0D ...ASCII output. 00000010 9F E2 95 9B E2 95 A3 0D 0A ......... 

Note that fhex is a PSCX utility.

UPDATE: find out how to do this:

 $enc = [Console]::OutputEncoding [Console]::OutputEncoding = [text.encoding]::utf8 .\stdout.exe | out-file fubar3.txt -enc utf8 fhex .\fubar3.txt Address: 0 1 2 3 4 5 6 7 8 9 ABCDEF ASCII -------- ----------------------------------------------- ---------------- 00000000 EF BB BF 41 53 43 49 49 20 6F 75 74 70 75 74 E1 ...ASCII output. 00000010 BE B9 0D 0A .... [Console]::OutputEncoding = $enc 

Answer 2

Probabry you need to run "chcp 65001" (after changing the powershell.exe font).
This command is available with PSISE.

Answer 3

If your goal is to process data from your own command in powershell, you can try

 ./program-that-outputs-utf8 > temp.txt get-content temp.txt -Encoding utf8 | (do_whatever)