shell - process quoted arguments correctly

Question

Say I have a bash script as below, its purpose is to join all arguments to a command string then execute this command. As you see below the joined command can not be successfully executed and I think I did not process the quoted string correctly. I wonder why and how to correct it.

The script:

$ cat testarg.sh
#!/bin/bash
echo "args number: $#"

args="$@"

cmd=""

#way 1, loop over $@
for arg in $args
do
    echo "arg: $arg"
    cmd="$cmd $arg"
done

ttt=""

#way 2, use shift built-in
while [[ $# -gt 0  ]]
do
    k="$1"
    shift
    echo "k: $k"
    ttt="$ttt $k"
done

echo "args: $args"
echo "cmd: $cmd"
echo "ttt: $ttt"

$ttt

The output running the script with a curl command (failed):

$ testarg.sh curl -k -X POST -H "Content-Type: application/json" -d '{"username":"admin", "password": "passw0rd"}' "https://localhost:8443/token/generate"
args number: 9
arg: curl
arg: -k
arg: -X
arg: POST
arg: -H
arg: Content-Type:
arg: application/json
arg: -d
arg: {"username":"admin",
arg: "password":
arg: "passw0rd"}
arg: https://localhost:8443/token/generate
k: curl
k: -k
k: -X
k: POST
k: -H
k: Content-Type: application/json
k: -d
k: {"username":"admin", "password": "passw0rd"}
k: https://localhost:8443/token/generate
args: curl -k -X POST -H Content-Type: application/json -d {"username":"admin", "password": "passw0rd"} https://localhost:8443/token/generate
cmd:  curl -k -X POST -H Content-Type: application/json -d {"username":"admin", "password": "passw0rd"} https://localhost:8443/token/generate
ttt:  curl -k -X POST -H Content-Type: application/json -d {"username":"admin", "password": "passw0rd"} https://localhost:8443/token/generate
<html>
<head><title>302 Found</title></head>
<body bgcolor="white">
<center><h1>302 Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
<html>
<head><title>302 Found</title></head>
<body bgcolor="white">
<center><h1>302 Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
curl: (3) [globbing] unmatched close brace/bracket in column 11
{
  "timestamp" : "2018-10-31T02:05:48.333+0000",
  "status" : 415,
  "error" : "Unsupported Media Type",
  "message" : "Content type 'application/octet-stream' not supported",
  "path" : "/token/generate"
}

So it looks like I can not loop over "$@" then rejoin them, it splits the quoted string with multiple elements. Therefore I use the shift buit-in, the quoted string was kept.

Same output when directly run the above $ttt string (failed):

$ curl -k -X POST -H Content-Type: application/json -d {"username":"admin", "password": "passw0rd"} https://localhost:8443/token/generate
<html>
<head><title>302 Found</title></head>
<body bgcolor="white">
<center><h1>302 Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
<html>
<head><title>302 Found</title></head>
<body bgcolor="white">
<center><h1>302 Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
curl: (3) [globbing] unmatched close brace/bracket in column 9
{
  "timestamp" : "2018-10-31T02:29:24.973+0000",
  "status" : 415,
  "error" : "Unsupported Media Type",
  "message" : "Content type 'application/octet-stream' not supported",
  "path" : "/token/generate"
}

The output when directly run the exact same curl command (succeeded):

$ curl -k -X POST -H "Content-Type: application/json" -d '{"username":"admin", "password": "passw0rd"}' "https://localhost:8443/token/generate"
{
  "token" : "eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhZG1pbiIsInJvbGUiOiJBRE1JTklTVFJBVE9SIiwianRpIjoiYTE0MWZmMTAtZjNlMi00ZDVhLWFjOGUtODE1Y2YzNzYyNzA0IiwiaXNzIjoiaHR0cDovL2libS5jb20iLCJpYXQiOjE1NDA5NTE1NDAsImV4cCI6MTU0MTAzNzk0MH0.plrjsVAPaRSU6K2WnlTCy-6V1iuSK01F6SULH4lELqs"
}

The reason I want to join all arguments is in future I want to add a header to the other passing curl command. e.g. add a auth token.

For example, say user wants to run this curl command:

testarg.sh curl -k -X POST -H "Content-Type: application/json" -d '{"key": "value"}' "https://localhost:8443/protected_page"

And with testargs.sh, the auth header will be added to the curl command, the final curl command will be executed is:

curl -H "Authorization: Bearer <token>" -k -X POST -H "Content-Type: application/json" -d '{"key": "value"}' "https://localhost:8443/protected_page"

Thanks in advance.

tripleee tripleee · Accepted Answer · 2018-10-31T02:23:38

Copying "$@" to a regular variable wrecks any quoting you had in there. You want to loop over "$@" directly, or copy it to an array if you really genuinely need to copy it first.

shell - process quoted arguments correctly

2 Answers