Automatically add newline at end of curl response body

279
votes

If the HTTP response body for a curl request doesn't contain a trailing newline, I end up with this really annoying condition where the shell prompt is in the middle of the line, and escaping is messed up enough that when I put the last curl command on the screen, deleting characters from that curl command deletes the wrong characters.

For example:

[root@localhost ~]# curl jsonip.com
{"ip":"10.10.10.10","about":"/about"}[root@localhost ~]#

Is there a trick I can use to automatically add a newline at the end of a curl response, to get the prompt back on the left edge of the screen?

5
curl
None of these answers work for me with whatever version of curl my cygwin install has; only wrapping the entire curl command in an echo statement does, e.g. echo "$(curl localhost:8001/api)", re this answer: unix.stackexchange.com/a/217611/110338 – Alex Hall
for json specifically, curl -s https://jsonip.com | jq – hanshenrik

5 Answers

487
votes

From the man file:

To better allow script programmers to get to know about the progress of curl, the -w/--write-out option was introduced. Using this, you can specify what information from the previous transfer you want to extract.

To display the amount of bytes downloaded together with some text and an ending newline:

curl -w 'We downloaded %{size_download} bytes\n' www.download.com

So try adding the following to your ~/.curlrc file:

-w "\n"
114
votes

Use this:

curl jsonip.com; echo

If you need grouping to feed a pipe :

{ curl jsonip.com; echo; } | tee new_file_with_newline

OUTPUT

{"ip":"x.x.x.x","about":"/about"}

This is that simple ;)

(and not limited to curl command but all commands that not finish with a newline)

15
votes

For more info as well as a clean new line after curl

~/.curlrc

-w "\nstatus=%{http_code} %{redirect_url} size=%{size_download} time=%{time_total} content-type=\"%{content_type}\"\n"

(More options are available here)

redirect_url will be blank if the request doesn't get redirected or you use -L to follow the redirect.

Example output:

~ ➤  curl https://www.google.com
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>302 Moved</TITLE></HEAD><BODY>
<H1>302 Moved</H1>
The document has moved
<A HREF="https://www.google.co.uk/?gfe_rd=cr&amp;ei=FW">here</A>.
</BODY></HTML>

status=302 https://www.google.co.uk/?gfe_rd=cr&ei=FW size=262 time=0.044209 content-type="text/html; charset=UTF-8"
~ ➤

Edit, to make things more readable you can add ANSI colours to the -w line, it's not that easy to write directly, but this script can generate a ~/.curlrc file with colours.

#!/usr/bin/env python3
from pathlib import Path
import click
chunks = [
    ('status=', 'blue'),
    ('%{http_code} ', 'green'),
    ('%{redirect_url} ', 'green'),
    ('size=', 'blue'),
    ('%{size_download} ', 'green'),
    ('time=', 'blue'),
    ('%{time_total} ', 'green'),
    ('content-type=', 'blue'),
    ('\\"%{content_type}\\"', 'green'),
]
content = '-w "\\n'
for chunk, colour in chunks:
    content += click.style(chunk, fg=colour)
content += '\\n"\n'

path = (Path.home() / '.curlrc').resolve()
print('writing:\n{}to: {}'.format(content, path))
path.write_text(content)
3
votes

The general solution for bash is to add a newline symbol into the command prompt:

See related question (How to have a newline before bash prompt? ) and corresponding answer

This solution covers each command, not only curl.

echo $PS1 # To get your current PS1 env variable's value aka '_current_PS1_'
PS1='\n_current_PS1_'

The only side-effect is that you get command prompt after each 2nd line.

0
votes

I managed to get a new line added dynamically to prompt when command output didn't have a new line at end. So it works not only with curl but also any other command.

# https://github.com/dylanaraps/pure-bash-bible#get-the-current-cursor-position
new_line_ps1() {
  local _ y x _
  local LIGHT_YELLOW="\001\033[1;93m\002"
  local     RESET="\001\e[0m\002"

  IFS='[;' read -p $'\e[6n' -d R -rs _ y x _
  if [[ "$x" != 1 ]]; then
    printf "\n${LIGHT_YELLOW}^^ no newline at end of output ^^\n${RESET}"
  fi
}

PS1="\$(new_line_ps1)$PS1"

my answer in UL site: https://unix.stackexchange.com/a/647881/14907