How to debug a bash function that returns a value, and how to add newlines to a variable?

Question

I'm in a bash crash course today.

This is a bash function that returns a value, via echo:

#!/bin/bash
get_hello_name() {
  echo 'Hello $1!'
}
msg=$(get_hello_name "x")
echo $msg

Output:

$ bash ./initial_script5.sh
Hello $1!

I then incorrectly thought that the last echo was returned (I come from Java and Python), and was trying to use echo to debug the rest of the function.

And then I was wondering why the heck I could not print out newlines in my echo statements, despite trying every single suggestion in this question.

This script demonstrates the problem:

#!/bin/bash
a_function() {
  echo "---In function"

  printf %"s\n" hello world
  printf "Hello\nworld"
  echo $'hello\nworld'
  echo -e 'hello\nworld'
}

echo "---Pre function"
printf %"s\n" hello world
printf "Hello\nworld"
echo $'hello\nworld'
echo -e 'hello\nworld'

x=$(a_function "x")
echo $x

echo "---Post function"
printf %"s\n" hello world
printf "Hello\nworld"
echo $'hello\nworld'
echo -e 'hello\nworld'


$ bash ./initial_script5.sh
---Pre function
hello
world
Hello
worldhello
world
hello
world
---In function hello world Hello worldhello world hello world
---Post function
hello
world
Hello
worldhello
world
hello
world

The problem is that all of the echos in the function are concatenated together, after being individually trimmed, and then returned as a whole.

So this leads me to two questions: How do you debug a function that returns a value, and how do you append newlines onto a variable (not that I even want to do the latter, necessarily, but I would like to understand it)?

Wrote about this adventure on my blog: aliteralmind.wordpress.com/2015/01/10/bashcrash — aliteralmind

John1024 John1024 · Accepted Answer · 2015-01-10T02:04:24

Why the newlines seemed to disappear from the variable

The newlines are actually retained in the variable. They do not display because the variable in the echo statement is not enclosed in double-quotes. From the code:

echo $x

When using a variable without double-quotes, word splitting is performed. Under the default $IFS (wiki entry on IFS), this means that all collections of whitespace, including newlines and tabs, are replaced with single space.

To avoid that, simply use double quotes as in:

echo "$x"

With that single change, the output of your script becomes:

$ bash a,sh 
---Pre function
hello
world
Hello
worldhello
world
hello
world
---In function
hello
world
Hello
worldhello
world
hello
world
---Post function
hello
world
Hello
worldhello
world
hello
world

The newlines that were always in the variable x are now displayed.

Aside: the two words that remain strung together

Note that the combination worldhello appears on one line because because that is what the code asked for:

printf "Hello\nworld"
echo $'hello\nworld'

The printf does not print a newline after world. Hence, that world appears on the same line as the hello which follows.

Documentation of the Details

man bash explains that double-quotes inhibit word splitting:

If the substitution appears within double quotes, word splitting and pathname expansion are not performed on the results.

Word-splitting happens after variable expansion, command substitution, and arithmetic expansion:

The shell scans the results of parameter expansion, command substitution, and arithmetic expansion that did not occur within double quotes for word splitting.

Another subtlety is that word splitting is performed only if some substitution took place:

Note that if no expansion occurs, no splitting is performed.

Normally, when word splitting is performed, all strings of spaces, tabs, and newlines are replaced by a single space. This default behavior can be changed by changing the value of the IFS variable:

The shell treats each character of IFS as a delimiter, and splits the results of the other expansions into words on these characters. If IFS is unset, or its value is exactly , the default, then sequences of space, tab, and newline at the beginning and end of the results of the previous expansions are ignored, and any sequence of IFS characters not at the beginning or end serves to delimit words. If IFS has a value other than the default, then sequences of the whitespace characters space and tab are ignored at the beginning and end of the word, as long as the whitespace character is in the value of IFS (an IFS whitespace character). Any character in IFS that is not IFS whitespace, along with any adjacent IFS whitespace characters, delimits a field. A sequence of IFS whitespace characters is also treated as a delimiter. If the value of IFS is null, no word splitting occurs.

How to Debug

Use set -x

Place the line set -x at the beginning of the code that you wish to run. The results of evaluating each line will be displayed as the function is run, each preceded by PS4 (default is +, space) to distinguish it from normal output.

The debug output can be turned off by including the line set +x.

set -x and set +x both also work on the command line.
Use stderr

Send debug output to stderr (file descriptor 2) as follows:
```
echo "My Debug Info" >&2
```
By default, pipelines and command substitutions only operate on stderr. Consequently, information sent to stderr will, by default, appear on the terminal.

More on `echo`

By default, echo ignores escape characters and the sequence \n simply means a \ followed by an n:

$ echo "Hello\nworld 4"
Hello\nworld 4

To have \n interpreted as a newline, use -e:

$ echo -e "Hello\nworld 4"
Hello
world 4