Removing double quotes in batch Script

How do I go about replacing all double quotes in my batch file options with escaped double quotes? This is my current batch file that extends all command line options inside the line:

@echo off call bash --verbose -c "g++-linux-4.1 %*" 

He then uses this line to invoke Cygwin bash executing the Linux cross-compiler. Unfortunately, I get options like these in my batch file:

 "launch-linux-g++.bat" -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -DNDEBUG -c -o "C:\Users\Me\Documents\Testing\SparseLib\bin\Win32\LinuxRelease\hello.o" "c:\Users\Me\Documents\Testing\SparseLib\SparseLib\hello.cpp" 

If the first quote around the first traversed path ends prematurely, the line is passed to GCC and passed to the rest of the parameters directly in bash (which does not work effectively.)

I assume that if I can concatenate the parameters into one line and then escape from the quotes, it should work fine, but itโ€™s hard for me to determine how to do this. Somebody knows?

+74
scripting parameters escaping quotes batch-file
Feb 18 '09 at 17:19
source share
5 answers

Google eventually came up with an answer. The syntax for replacing a string in a package is:

 set v_myvar=replace me set v_myvar=%v_myvar:ace=icate% 

What produces "replicating me". My script now looks like this:

 @echo off set v_params=%* set v_params=%v_params:"=\"% call bash -c "g++-linux-4.1 %v_params%" 

Which replaces all instances of " with \" , was correctly escaped for bash.

+21
Feb 18 '09 at 17:48
source share

Equivalent character in batch scripts ^ . But for double-quoted strings, double quotes:

 "string with an embedded "" character" 
+85
Feb 18 '09 at 17:31
source share

the carefree own answer simply and effectively solves its specific problem: it replaces all " instances in the entire argument list with \" , since Bash requires double quotes inside a string with two quotes.

To answer the question of how to avoid double quotes inside a double-quoted string using cmd.exe , the Windows command line interpreter (both on the command line, often still mistakenly called the "DOS prompt", and in the batch file): see below to view PowerShell.

tl; dr :

  • You must use "" when passing a string to a (different) batch file , and you can use "" with applications created with Microsoft C / C ++ /. NET compilers (which also accept \" ), which on Windows includes Python and Node.js :

    • Example: foo.bat "We had 3"" of rain."

    • For batch files, only the following applies:

      • "" is the only way to get a command interpreter ( cmd.exe ) to process the entire string with two quotation marks as the only argument.

      • However, unfortunately, not only closed double quotes (as usual) are saved, but double escaped ones, so getting the intended string is a two-step process; for example, assuming a double-quoted string is passed as the first argument, %1 :

      • set "str=%~1" removes enclosed double quotes; set "str=%str:""="%" then converts double double quotes to single.
        Be sure to use closed double quotation marks around the target parts to prevent unwanted interpretation of the values.

  • \" is required - as the only option - by many other programs (for example, Ruby, Perl and even Microsoft - PowerShell (!)), but ITS USE IS NOT SAFE :

    • \" is what many executables and interpreters require - including Microsoft PowerShell when passing strings from outside - or, in the case of Microsoft compilers, alternative support for "" - ultimately, however, it is up to the target program to parse the argument list .
    • Example: foo.exe "We had 3\" of rain."
    • HOWEVER, USE \" MAY RESULT IN PERFECT, ARBITRATED PERFORMANCE OF TEAMS AND / OR INPUTS / OUTPUTS :
      • The following symbols represent this risk: & | < > & | < > & | < > & | < >
      • For example, the following results in an unintended execution of ver ; see below below for an explanation and the following point for a workaround:
        • foo.exe "3\" of snow" "& ver."
    • For PowerShell for Windows only, \"" is a reliable alternative.
  • If you must use \" , there are only 3 safe approaches , which, however, are rather cumbersome : Tip hats in TS for his help.

    • Using (possibly a selective) slow expansion of a variable in your batch file, you can save the literal \" in a variable and refer to this variable inside the string "..." using the syntax !var! - see TS Useful Answer .

      • The above approach, despite being cumbersome, has the advantage that you can apply it methodically and that it works reliably with any input.
    • Only with LITERAL strings not associated with VARIABLES you get a similar methodological approach: categorically ^ -escape all cmd.exe metacharacters: " & | < > and - if you also want to suppress the variable extension - % :
      foo.exe ^"3\^" of snow^" ^"^& ver.^"

    • Otherwise, you should formulate your line based on the recognition of which parts of the cmd.exe line are considered unordered due to the misinterpretation of \" as closing delimiters:

      • in literal parts containing shell metacharacters: ^ -escape; using the example above, this & should be ^ -escape d:
        foo.exe "3\" of snow" "^& ver."

      • piecemeal with links to %...% -style: make sure cmd.exe considers them to be part of the "..." and that the variable values โ€‹โ€‹themselves do not have built-in unbalanced quotes, which is not always possible .

Read on for help.




Background

Note. This is based on my own experiments. Let me know if I am wrong.

POSIX-like shells, such as Bash on Unix-like systems, block the argument list (string) before passing the arguments individually to the target program: among other extensions, they break the argument list into separate words (word splitting) and remove quoting characters from the resulting words ( delete quotes). The target program is passed conceptually by an array of separate arguments with (syntactically required) quotation marks.

In contrast, the Windows command interpreter does not appear to eliminate the argument list and simply passes a single line containing all arguments, including character quotes. - to the target program.
However, some preprocessing is done before one whole line is passed to the target program: ^ escape characters. outside double-quoted strings are deleted (they avoid the next character), and variable references are first referenced (for example, %USERNAME% ).

Thus, unlike Unix, the responsibility for the target program is to parse the string of arguments and break it into separate arguments with the quotation marks removed. Thus, different programs may hypothetically require different screening methods and there is no single screening mechanism that is guaranteed to work with all programs. Stack Overflow.site/questions/4522 / ... contains an excellent background of anarchy, which is a Windows parsing command line.

In practice, \" very common, but NOT SAFE , as mentioned above:

Since cmd.exe itself does not recognize \" as an escaped double quote, it may incorrectly interpret markers later on the command line as invalid and potentially interpret them as commands and / or I / O redirects.
In a nutshell: a surface problem if any of the following characters follows an opening or unbalanced \" : & | < > , for example:

 foo.exe "3\" of snow" "& ver." 

cmd.exe sees the following tokens that result from a misinterpretation of \" as a regular double quote:

  • "3\"
  • of
  • snow" "
  • rest: & ver.

Because cmd.exe thinks & ver. unquoted, it interprets it as & (the sequence operator), followed by the name of the command to execute ( ver. - . ignored; ver reports version information for cmd.exe ).
General effect:

  • Firstly, foo.exe is only called using the first 3 tokens.
  • Then the ver command is executed.

Even in cases where a random team does no harm, your common team will not work as it would be designed, given that not all arguments are passed to it.

Many compilers / interpreters recognize ONLY \" for example, the GNU C / C ++ compiler, Python, Perl, Ruby, even Microsoft, owned by PowerShell when called from cmd.exe - and, except for PowerShell with \"" , there is not a simple solution for them this problem.
Essentially, you need to know in advance which parts of your command line are misinterpreted as incorrect, and selectively ^ -escape all instances of & | < > & | < > & | < > & | < > in these parts.

On the contrary, the use of "" SAFE , but, unfortunately, is only supported by executable and file commands -c ompiler -based (in the case of batch files with the quirks described above).

In contrast, PowerShell recognizes only \" and, in Windows, more reliable \"" when called from outside - for example, from cmd.exe , whether from the command line or batch file, although internally PowerShell uses ' as an escape character in double quotes, and also accepts "" , for example:

  • powershell -c " \"ab c\".length" works (outputs 4 ), as well as more reliable ones
    powershell -c " \""ab c\"".length" ,

  • but powershell -c " ""ab c"".length" breaks.




Related Information

  • ^ can only be used as an escape character in unquoted strings - inside double-quoted strings , ^ not special and is treated as a literal.

    • CAVEAT : the use of the ^ parameter in the parameters passed to the call statement is violated (this applies to the use of call : another batch file or binary is called, and calls the subroutine in the same batch file):
      • ^ instances in double quotes inexplicably double , changing the passed value: for example, if the variable %v% contains the literal value a^b , call :foo "%v%" assigns "a^^b" (!) to %1 (first parameter ) in the subroutine :foo .
      • The incorrect use of ^ with call completely violated by the fact that ^ can no longer be used to exit special characters : for example, call foo.cmd a^&b quietly breaks (instead of passing the literal a&b , it is also foo.cmd , as it would be without call ) - foo.cmd never called (!), at least on Windows 7.
  • Unfortunately, the % literal exception is a special case that requires clear syntax depending on whether the line is specified on the command line or inside the batch file ; see https://stackoverflow.com>

    • In short: use %% inside the batch file. On the command line, % cannot be escaped, but if you put a ^ at the beginning, at the end, or inside the variable name in an unquoted string (for example, echo %^foo% ), you can prevent the variable from expanding (interpolation); % instances on the command line that are not part of the variable reference are treated as literals (for example, 100% ).
  • Generally, to work safely with variable values โ€‹โ€‹that may contain spaces and special characters :

    • Purpose : specify both the name of the variable and the value in one pair of double quotes ; for example, set "v=a & b" assigns the literal value a & b variable %v% (on the contrary, set v="a & b" will double quote the part of the value). Exclude the literal % instances as %% (works only in batch files - see above).
    • Link : double quotation marks to make sure their value is not interpolated; for example, echo "%v%" does not interpolate the value %v% and prints "a & b" (but note that double quotes are also printed). In contrast, echo %v% passes the literal a to echo , interprets & as the statement of a command sequence, and therefore tries to execute a command named b .
      Also note the caveat of using ^ with the call statement above.
    • External programs usually take care to remove closed double quotes around parameters, but as noted in batch files you have to do this yourself (for example, %~1 to remove closed double quotes from the first parameter) and, unfortunately, there is the direct way that I know to get echo to print a variable value exactly without enclosed double quotes .
      • Neil offers a for -based workaround that works as long as the value has no built-in double quotes ; eg:
        set "var=^&')|;,%!" for/f "delims=" %%v in ("%var%") do echo %%~v
  • cmd.exe does not recognize single quotes as line separators - they are treated as literals and usually cannot be used to delimit strings with a built-in space; it also follows that tokens abutting in single quotes and any tokens between them are treated as invalid by cmd.exe and interpreted accordingly.

    • However, given that target programs ultimately perform their own argument analysis, some programs, such as Ruby, recognize single-quoted strings, even on Windows; in contrast, C / C ++, Perl, and Python executables do not recognize them.
      However, even if supported by the target program, it is not practical to use single-quoted strings, given that their contents are not protected from the potentially unwanted interpretation of cmd.exe .



Powerhell

Windows PowerShell is a much more sophisticated shell than cmd.exe , and it has been part of Windows for many years (and PowerShell Core brings MacOS and Linux experience to PowerShell).

PowerShell works sequentially with respect to citation:

  • inside double-quoted strings, use '" or "" to avoid double quotes
  • inside single quote strings use '' to avoid single quotes

This works on the PowerShell command line and when passing parameters to scripts or PowerShell functions from PowerShell.

(As discussed above, passing a hidden double quote to PowerShell from the outside requires \" or, more reliably, \"" - nothing else works).

Unfortunately, when calling external programs, you are faced with the need to both take into account your own PowerShell citation rules and run away for the target program:

This problematic behavior is also discussed and summarized in this release of the GitHub documentation.

Double quotes inside double-quoted strings :

Consider the string "3'" of rain" , which PowerShell internally translates to literal 3" of rain .

If you want to pass this line to an external program, you need to apply the escaping of the target program in addition to PowerShell ; let's say you want to pass a string to a C program that expects embedded double quotes to be escaped as \" :

 foo.exe "3\'" of rain" 

Note that both '" - to make PowerShell happy - and \ - to make the target program happy - must be present.

The same logic applies to calling a batch file, where "" should be used:

 foo.bat "3'"'" of rain" 

Conversely, nesting single quotes in a double-quoted string does not require any escaping.

Single quotes inside single quote strings do not require additional acceleration; consider the '2'' of snow' which represents PowerShell' view 2' of snow .

 foo.exe '2'' of snow' foo.bat '2'' of snow' 

PowerShell converts single-quoted strings to double quotes before passing them to the target program.

However, double quotes inside single-quoted strings that do not require escaping for PowerShell must still be escaped for the target program:

 foo.exe '3\" of rain' foo.bat '3"" of rain' 

PowerShell v3 introduced the magic --% option , called the stop-parser symbol, which alleviates some of the pain by passing something after it is not interpreted by the target program, with the exception of references to cmd.exe -style environment variables (for example , %USERNAME% ) that are expanded; eg:

 foo.exe --% "3\" of rain" -u %USERNAME% 

Please note that it is enough to avoid the built-in " like \" for the target program (and also not for PowerShell like \'" ).

However, this approach:

  • does not allow % characters to be avoided to avoid decompositions of environment variables.
  • excludes direct use of variables and PowerShell expressions; instead, the command line should be built into a string variable in the first step, and then invoked with Invoke-Expression in a second.



Thus, despite many achievements, PowerShell did not slip away when calling external programs. However, he introduced support for single-quoted strings.

I wonder if it is possible at all in the Windows world to ever switch to a Unix model that allows the shell to make all conventions and quotes initially predictable, regardless of the target program, and then call the target program, passing the resulting tokens,

+61
Jul 14 '15 at 17:45
source share

In addition to mklement0, an excellent answer :

Almost all executables accept \" as escaped. " Safe use in cmd, however, is only practical with DELAYEDEXPANSION.
To explicitly send the literal " for some process, assign \" an environment variable, and then use this variable whenever you need to pass a quote. Example:

 SETLOCAL ENABLEDELAYEDEXPANSION set q=\" child "malicious argument!q!&whoami" 

Note. SETLOCAL ENABLEDELAYEDEXPANSION only works in batch files. To get DELAYEDEXPANSION in an interactive session, run cmd/V:ON .

If your batch file does not work with DELAYEDEXPANSION, you can temporarily enable it:

 ::region without DELAYEDEXPANSION SETLOCAL ENABLEDELAYEDEXPANSION ::region with DELAYEDEXPANSION set q=\" echoarg.exe "ab !q! & echo danger" ENDLOCAL ::region without DELAYEDEXPANSION 

If you want to transfer dynamic content from a variable containing quotes that are selected as "" you can replace "" with \" when expanding:

 SETLOCAL ENABLEDELAYEDEXPANSION foo.exe "danger & bar=region with !dynamic_content:""=\"! & danger" ENDLOCAL 

This replacement is not safe when expanding %...% !

In the case of OP bash -c "g++-linux-4.1 !v_params:"=\"!" This is a safe version.




If for some reason even temporarily turning on DELAYEDEXPANSION is not an option, read:

\" cmd , , . ( , , ...)

( ^" ), , , ( \^" ). ^ , , & => ^& ; | => ^| ; > => ^> ; etc.

Example:

 child ^"malicious argument\^"^&whoami^" 

: , . " ",




, :
, , "" cmd.exe ( , - %var:""=\"% ). " , " ^ -escaped. CMD- " . Example:

 foo.exe ^"danger ^& bar=\"region with %dynamic_content% & danger\"^" 

, %dynamic_content% .

+8
24 '17 23:19
source share

, Unreal Engine Automation -

: -cmdline = "-Messaging" -device = -addcmdline = "-SessionId = -SessionOwner = '' -SessionName = 'Build' -dataProviderMode = local -LogCmds = ' LogCommodity OFF '-execcmds =' , runtests + + + T1 + T2; quit '"-run

, -, .

0
03 . '18 16:17
source share



All Articles