Powershell: passing json string to curl

12
votes

I'm trying to pass a JSON string from within a powershell script to the build.phonegap.com api, using curl.
According to phonegap's forum, when running on a Windows machine, the JSON data has to be formatted as:

curl.exe -ku user@email:mypass -X PUT -d "data={\"password\":\"keypass\"}" https://build.phonegap.com/api/v1/key

Indeed, this does run fine when invoked from the command line.
However, when I try to invoke this from within a powershell script, the double quotes seem to be stripped.

So far, I have tried:

  • Putting the JSON in single quoted string:
curl.exe -ku user@email:mypass -X PUT -d '"data={\"password\":\"keypass\"}"'  https://build.phonegap.com/api/v1/key
  • Putting the JSON in single quoted string, without the DOS escape backslashes:
curl.exe -ku user@email:mypass -X PUT -d '"data={"password":"keypass"}"'  https://build.phonegap.com/api/v1/key
  • Putting the JSON in single quoted string, escaping the double quotes and backslashes (DOS style with a backslash):
curl.exe -ku user@email:mypass -X PUT -d '\"data={\\\"password\\\":\\\"keypass\\\"}\"'  https://build.phonegap.com/api/v1/key
  • Putting the JSON in a double quoted string, escaping the double quotes with the powershell backtick character (`):
curl.exe -ku user@email:mypass -X PUT -d "`"data={\`"password\`":\`"build*2014`\`"}`""  https://build.phonegap.com/api/v1/key

Any idea how to achieve this?

Thanks for your time, Koen

3
powershellcurlescapingquoting
the double quotes seem to be stripped. how do you assert that? – njzk2
I created a dummy Console application that just echo's the incoming args. – KoenJ
Revisiting your other solution attempts: The last one should actually work, although the inner enclosing " aren't necessary, and while the workaround with the additional \ -escaping is effective, it (a) shouldn't be necessary and (b) would break if and when the underlying PowerShell problem is fixed, though this may be mitigated by the fix - currently (since PowerShell Core 7.2.0-preview.5) only available as experimental feature PSNativeCommandArgumentPassing possibly requiring opt-in - assuming it becomes an official feature. – mklement0
If you want to use the payload data from a file, see this answer. – meJustAndrew

3 Answers

15
votes

Try using the --% operator to put PowerShell into simple (dumb) argument parsing mode:

curl.exe --% -ku user@email:mypass -X PUT -d "data={\"password\":\"keypass\"}" https://build.phonegap.com/api/v1/key

This is quite often useful for invoking exes with argument syntax that runs afoul of PowerShell's argument syntax. This does require PowerShell V3 or higher.

1
votes

  • PowerShell's escape character is ` (backtick), so in order to embed " characters in a "..." (double-quoted, interpolating) string, use `" (or "") rather than \"

    • In your attempt, PowerShell didn't see \" as an escaped " and therefore saw multiple "..." strings, which ultimately - when PowerShell of necessity applied its on demand re-quoting behind the scenes, passed two separate string arguments that individually didn't need double-quoting, due to not containing spaces, namely: verbatim data={\ and password\:\keypass\}

    • Using PowerShell's quoting rules, you should have used:

      • either:

        • "data={`"password`":`"keypass`"}"

      • or, more simply, given that no string interpolation is needed, via a verbatim, single-quoted string, inside of which " chars. don't require escaping:

        • 'data={"password":"keypass"}'

      • Unfortunately, however, as of PowerShell 7.1 this is NOT enough, though the experimental PSNativeCommandArgumentPassing feature available since PowerShell Core 7.2.0-preview.5 may fix this at least for some external programs, including curl.exe; read on for details.

  • As of PowerShell 7.1, an unexpected extra layer of escaping of embedded " characters is needed, using \-escaping when calling (most) external programs:

    • In Windows PowerShell there are edge cases where this approach doesn't work, in which case use of --% is required (see below). Notably, escaping '"foo bar"' as '\"foo bar\"' doesn't work, due to the enclosing \" being at the very start and end of the string - see this answer for details.

    • Also, some external programs on Windows understand ""-escaping only (e.g. msiexec); for them, use -replace '"', '""' in order to programmatically perform the extra escaping, assuming the value contains at least one space. Do the same for programs that do not support embedded " chars. at all (WSH), so that the embedded " at least do not break argument boundaries (but they will be stripped).

    • For programs that expect \"-escaping, use the following -replace operation to robustly perform the extra escaping programmatically:

      • '...' -replace '([\\]*)"', '$1$1\"'
      • If the input string contains no preexisting verbatim \" sequences, you can simplify to '...' -replace '"', '\"'
# Note: Escaping the embedded " chars. as `" is enough for PowerShell itself,
#       but, unfortunately, not when calling *external programs*.
#       The `-replace` operation performs the necessary additional \-escaping.
$passwd = 'foo'
curl.exe -ku user@email:mypass -X PUT -d (
  "data={`"password`": `"$passwd`"}" -replace '([\\]*)"', '$1$1\"'
) https://build.phonegap.com/api/v1/key

  • This shouldn't be required, but is due to a bug since v1 that hasn't been fixed for fear of breaking backward compatibility - see this answer.

  • A - presumably - opt-in fix is now being considered for v7.2 - see GitHub issue #14747 - and using it, once available, would obviate the need for the manual escaping.

    • Update: PowerShell Core 7.2.0-preview.5 now includes experimental feature PSNativeCommandArgumentPassing with an attempted fix, but, unfortunately, it looks like it will lack important accommodations for high-profile CLIs on Windows (though curl.exe wouldn't be affected) - see this summary from GitHub issue #15143.

  • A backward- and forward-compatible helper function is the ie function from the Native module (Install-Module Native), which obviates the need for the extra escaping, contains important accommodations for high-profile CLIs on Windows, and will continue to work as expected even with the opt-in fix in place:
    ie curl.exe ... -d "data={`"password`": `"$passwd`"}" ... )

  • Using --%, the stop-parsing symbol, as in Keith Hill's answer is a suboptimal workaround that also doesn't require the extra \-escaping, however:

    • --% has inherent limitations - see GitHub docs issue #6149 - and is virtually useless on Unix-like platforms - see GitHub docs issue #4963.
    • The only - awkward and side effect-producing - way to embed PowerShell variable values in the arguments following --% is to (a) define them as environment variables (e.g., $env:passwd = 'foo') and (b) to reference these variables cmd.exe-style, even on Unix (e.g., %passwd%).

  • An alternative workaround - especially if you need to include the values of PowerShell variables or expressions in your calls - is to call via cmd /c with a single argument containing the entire command line; for quoting convenience, the following example uses a here-string (see the bottom section of this answer for an overview of PowerShell's string literals):

# Use @"<newline>...<newline>"@ if you need to embed PowerShell variables / expressions.
cmd /c @'
curl.exe -ku user@email:mypass -X PUT -d "data={\"password\":"\keypass\"}" https://build.phonegap.com/api/v1/key
'@
-1
votes

Set the content type:

curl -H "Content-Type: application/json" -d '{"password":"keypass"}' https://build.phonegap.com/api/v1/key