Linux Replace With Variable Containing Double Quotes

2
votes

I have read the following:

I have learned that I can use sed "s/STRING/$var1/g" to replace a string with the contents of a variable. However, I'm having a hard time finding out how to replace with a variable that contains double quotes, brackets and exclamation marks.

Then, hoping to escape the quotes, I tried piping my result though sed 's/\"/\\\"/g' which gave me another error sed: -e expression #1, char 7: unknown command: E'. I was hoping to escape the problematic characters and then do the variable replacement: sed "s/STRING/$var1/g". But I couldn't get that far either.

I figured you guys might know a better way to replace a string with a variable that contains quotes.

File1.txt:

Example test here
<tag>"Hello! [world]" This line sucks!</tag>
End example file

Variable: 

var1=$(cat file1.txt)

Example:

echo "STRING" | sed "s/STRING/$var1/g"

Desired output:

Example test here
<tag>"Hello! [world]" This line sucks!</tag>
End example file
3
regexlinuxbashawksed
sed is the not the right tool for this substitution. - anubhava
@anubhava I see. Thanks for the advice Anubhava. I'm open to anything that works. Will try doing more research too. Editing the question to reflect that. - DomainsFeatured
@batMan yea, that's the idea. To keep the data in the variable the same. I want to substitute for STRING with the contents of $var1. That's where I'm stuck. This is just a minimalist example. - DomainsFeatured

3 Answers

2
votes

using awk

$ echo "STRING" | awk -v var="$var1" '{ gsub(/STRING/,var,$0); print $0}'
Example test here
<tag>"Hello! [world]" This line sucks!</tag>
End example file

-v var="$var1": To use shell variable in awk
gsub(/STRING/,var,$0) : To globally substitute all occurances of "STRING" in whole record $0 with var

Special case : "If your var has & in it " say at the beginning of the line then it will create problems with gsub as & has a special meaning and refers to the matched text instead.

To deal with this situation we've to escape & as follows : 

$ echo "STRING" | awk -v var="$var1" '{ gsub(/&/,"\\\\&",var); gsub(/STRING/,var,$0);print $0}'
&Example test here
<tag>"Hello! [world]" This line sucks!</tag>
End example file
2
votes

Here is one awk command that gets text-to-be-replaces from a file that may consist of all kind of special characters such as & or \ etc:

awk -v pat="STRING" 'ARGV[1] == FILENAME {
   # read replacement text from first file in arguments
   a = (a == "" ? "" : a RS) $0
   next
}
{
   # now run a loop using index function and use substr to get the replacements 
   s = ""
   while( p = index($0, pat) ) {
      s = s substr($0, 1, p-1) a
      $0 = substr($0, p+length(pat))
   }
   $0 = s $0
} 1' File1.txt <(echo "STRING")

To be able to handle all kind of special characters properly this command avoids any regex based functions. We use plain text based functions such as index, substr etc.

1
votes

The problem isn't the quotes. You're missing the "s" command, leading sed to treat /STRING/ as a line address, and the value of $var1 as a command to execute on matching lines. Also, $var1 has unescaped newlines and a / character that'll cause trouble in the substitution. So add the "s", and escape the relevant characters in $var1:

var1escaped="$(echo "$var1" | sed 's@[\/&]@\\&@; $ !s/$/\\/')"
echo "STRING" | sed "s/STRING/$var1escaped/"

...but realistically, @batMan's answer (using awk) is probably a better solution.