How do I pass a literal double quote from PowerShell to my own command?

Question

I want to print a string literal in awk / gawk using the Powershell command line (the specific program is not important). However, I think I misunderstand the citation rules somewhere along the line - PowerShell obviously removes double quotes inside single quotes for native commands, but not when passing them to commands.

This works in bash:

bash$ awk 'BEGIN {print "hello"}' hello <-- GOOD 

And this works in PowerShell - but, importantly, I have no idea why escaping is required:

 PS> awk 'BEGIN {print \"hello\"}' hello <-- GOOD 

In PowerShell, nothing prints:

 PS> awk 'BEGIN {print "hello"}' <-- NOTHING IS BAD 

If this is really the only way to do this in PowerShell, then I would like to understand a citation rule chain that explains why. According to PowerShell's citation rules at http://technet.microsoft.com/en-us/library/hh847740.aspx , this is not necessary.

BEGIN SOLUTION

The line punch kindly provided by Duncan below is that you should add this function to your powershell profile:

  filter Run-Native($command) { $_ | & $command ($args -replace'(\\*)"','$1$1\"') } 

Or specifically for awk:

  filter awk { $_ | gawk.exe ($args -replace'(\\*)"','$1$1\"') } 

END OF DECISION

The quotes are properly passed to the PowerShell echo:

 PS> echo '"hello"' "hello" <-- GOOD 

But when the external "native" program is called, the quotation marks disappear:

 PS> c:\cygwin\bin\echo.exe '"hello"' hello <-- BAD, POWERSHELL REMOVED THE QUOTES 

An even cleaner example is shown here if you are worried that Cygwin might have something to do with this:

 echo @" >>> // program guaranteed not to interfere with command line parsing >>> public class Program >>> { >>> public static void Main(string[] args) >>> { >>> System.Console.WriteLine(args[0]); >>> } >>> } >>> "@ > Program.cs csc.exe Program.cs .\Program.exe '"hello"' hello <-- BAD, POWERSHELL REMOVED THE QUOTES 

DEPRECATED EXAMPLE to go to cmd, which does its own parsing (see Etan comment below):

 PS> cmd /c 'echo "hello"' "hello" <-- GOOD 

DEPRECATED EXAMPLE to go to bash, which does its own parsing (see Etan comment below):

 PS> bash -c 'echo "hello"' hello <-- BAD, WHERE DID THE QUOTES GO 

Any solutions, more elegant workarounds or explanations?

Answer 1

The problem is that the standard Windows C environment prevents double quotation marks from arguments when parsing the command line. Powershell passes arguments to its own commands by placing double quotes around the arguments, but does not escape the double quotes that are contained in the arguments.

Here's a test program that prints the arguments provided with C stdlib, a raw Windows command line and Windows command line processing (which seems to behave the same with stdlib):

 C:\Temp>type tc #include <stdio.h> #include <windows.h> #include <ShellAPI.h> int main(int argc,char **argv){ int i; for(i=0; i < argc; i++) { printf("Arg[%d]: %s\n", i, argv[i]); } LPWSTR *szArglist; LPWSTR cmdLine = GetCommandLineW(); wprintf(L"Command Line: %s\n", cmdLine); int nArgs; szArglist = CommandLineToArgvW(GetCommandLineW(), &nArgs); if( NULL == szArglist ) { wprintf(L"CommandLineToArgvW failed\n"); return 0; } else for( i=0; i<nArgs; i++) printf("%d: %ws\n", i, szArglist[i]); // Free memory allocated for CommandLineToArgvW arguments. LocalFree(szArglist); return 0; } C:\Temp>cl tc "C:\Program Files (x86)\Windows Kits\8.1\lib\winv6.3\um\x86\shell32.lib" Microsoft (R) C/C++ Optimizing Compiler Version 18.00.21005.1 for x86 Copyright (C) Microsoft Corporation. All rights reserved. tc Microsoft (R) Incremental Linker Version 12.00.21005.1 Copyright (C) Microsoft Corporation. All rights reserved. /out:t.exe t.obj "C:\Program Files (x86)\Windows Kits\8.1\lib\winv6.3\um\x86\shell32.lib" 

Running this in cmd will see that all unshielded quotes are stripped and spaces only separate arguments when there was an even number of non-exclusive quotes:

 C:\Temp>t "a"b" "\"escaped\"" Arg[0]: t Arg[1]: ab "escaped" Command Line: t "a"b" "\"escaped\"" 0: t 1: ab "escaped" C:\Temp>t "a"bc"de" Arg[0]: t Arg[1]: ab Arg[2]: cd e Command Line: t "a"bc"de" 0: t 1: ab 2: cd e 

Powershell behaves differently:

 C:\Temp>powershell Windows PowerShell Copyright (C) 2012 Microsoft Corporation. All rights reserved. C:\Temp> .\t 'a"b' Arg[0]: C:\Temp\t.exe Arg[1]: ab Command Line: "C:\Temp\t.exe" a"b 0: C:\Temp\t.exe 1: ab C:\Temp> $a = "string with `"double quotes`"" C:\Temp> $a string with "double quotes" C:\Temp> .\t $a nospaces Arg[0]: C:\Temp\t.exe Arg[1]: string with double Arg[2]: quotes Arg[3]: nospaces Command Line: "C:\Temp\t.exe" "string with "double quotes"" nospaces 0: C:\Temp\t.exe 1: string with double 2: quotes 3: nospaces 

In Powershell, any argument containing spaces is enclosed in double quotes. Also, the command itself gets quotes, even if there are no spaces. Other arguments are not quoted, even if they include punctuation marks such as double quotes, and, and I think this is a mistake. Powershell does not escape double quotes that appear inside arguments.

If you are interested (I was), Powershell did not even bother to give arguments containing new lines, but argument processing does not consider new lines as whitespace:

 C:\Temp> $a = @" >> a >> b >> "@ >> C:\Temp> .\t $a Arg[0]: C:\Temp\t.exe Arg[1]: a b Command Line: "C:\Temp\t.exe" a b 0: C:\Temp\t.exe 1: a b 

The only option, since Powershell does not escape quotes, for you, it seems you need to do this yourself:

 C:\Temp> .\t 'BEGIN {print "hello"}'.replace('"','\"') Arg[0]: C:\Temp\t.exe Arg[1]: BEGIN {print "hello"} Command Line: "C:\Temp\t.exe" "BEGIN {print \"hello\"}" 0: C:\Temp\t.exe 1: BEGIN {print "hello"} 

To avoid this every time, you can define a simple function:

 C:\Temp> function run-native($command) { & $command $args.replace('\','\\').replace('"','\"') } C:\Temp> run-native .\t 'BEGIN {print "hello"}' 'And "another"' Arg[0]: C:\Temp\t.exe Arg[1]: BEGIN {print "hello"} Arg[2]: And "another" Command Line: "C:\Temp\t.exe" "BEGIN {print \"hello\"}" "And \"another\"" 0: C:\Temp\t.exe 1: BEGIN {print "hello"} 2: And "another" 

NB You must avoid backslashes as well as double quotes, otherwise this will not work ( this does not work, see further editing below ):

 C:\Temp> run-native .\t 'BEGIN {print "hello"}' 'And \"another\"' Arg[0]: C:\Temp\t.exe Arg[1]: BEGIN {print "hello"} Arg[2]: And \"another\" Command Line: "C:\Temp\t.exe" "B EGIN {print \"hello\"}" "And \\\"another\\\"" 0: C:\Temp\t.exe 1: BEGIN {print "hello"} 2: And \"another\" 

Other editing: Backslash and quote processing in a Microsoft universe is even weirder than I understood. In the end, I had to go and read the C stdlib sources to find out how they interpret backslashes and quotes:

 /* Rules: 2N backslashes + " ==> N backslashes and begin/end quote 2N+1 backslashes + " ==> N backslashes + literal " N backslashes ==> N backslashes */ 

So run-native should be:

 function run-native($command) { & $command ($args -replace'(\\*)"','$1$1\"') } 

and all backslashes and quotes will withstand command line processing. Or if you want to run a specific command:

 filter awk() { $_ | awk.exe ($args -replace'(\\*)"','$1$1\"') } 

(updated after the comment by @jhclark: it should be a filter that allows you to create pipelines in stdin).

Answer 2

You have a different behavior because you use 4 different echo and in different ways.

<i>

 PS> echo '"hello"' "hello" 

echo is the PowerShell Write-Output cmdlet.

This works because the cmdlet takes a given argument string (text inside an external set of quotes, ie "hello" ), and prints that string to the success output stream.

<i>

 PS> c:\cygwin\bin\echo '"hello"' hello 

echo - Cygwin echo.exe .

This does not work because double quotes are removed from the argument string (the text in the external set of quotes, ie "hello" ) when PowerShell calls the external command.

You will get the same result if, for example, you call echo.vbs '"hello"' with WScript.Echo WScript.Arguments(0) , which is the content of echo.vbs .

<i>

 PS> cmd /c 'echo "hello"' "hello" 

echo is a CMD built-in echo command.

This works because the command line (the text inside the external set of quotes, ie echo "hello" ) is launched in CMD , and the built-in echo command saves the double quotation argument (running echo "hello" in CMD produces "hello" ).

<i>

 PS> bash -c 'echo "hello"' hello 

echo - bash built-in echo command.

This does not work because the command line (the text inside the external set of quotes, ie echo "hello" ) runs in bash.exe , and the built-in echo command does not save the double quote argument (running echo "hello" in bash creates hello ) .

If you want Cygwin echo print external double quotes, you need to add a double quote to your string:

 PS> c:\cygwin\bin\echo '"\"hello\""' "hello" 

I would suggest that this works well for bash -builtin echo es, but for some reason this is not the case:

 PS> bash -c 'echo "\"hello\""' hello 

Answer 3

Quote rules can get confused when you invoke commands directly from PowerShell. Instead, I regularly recommend that people use the Start-Process cmdlet along with their -ArgumentList parameter.

 Start-Process -Wait -FilePath awk.exe -ArgumentList 'BEING {print "Hello"}' -RedirectStandardOutput ('{0}\awk.log' -f $env:USERPROFILE); 

I don't have awk.exe (does this come from Cygwin?), But this line should work for you.