preserve the 2 line input file headers in the output file for large sequential multi header file using awk

Question

my input file looks like something below:

  X|Y|Z|W|N|M|D
  fits_0001
  1   19193.12    76.73  283.634   30.614 5.119 0.168 
  2   16974.98    73.58  643.020  157.605 5.204 0.199 
  3   21860.38    77.26  292.871  172.139 5.270 0.179 
  4  348148.69    88.90   21.196  187.410 5.224 0.176 
  5   70756.91    78.72  282.039  204.934 5.247 0.177 
  6   11991.60    75.90   34.742  223.909 5.244 0.176 
  7   14004.02    76.69  568.076  238.097 5.326 0.147 
  8   68987.25    77.56  793.915  433.301 5.358 0.169 
  9  699118.88    93.08  530.607  469.153 5.395 0.163 
 10   26622.89    78.09  907.149  571.626 5.266 0.153 
 11   25831.63    76.43  853.957  621.249 6.185 0.316 
 12  255103.98    81.78  709.661  629.690 5.391 0.147 
 13   17330.91    76.18  542.212  730.684 5.337 0.169 
 14   10898.14    74.85  670.807  891.083 5.487 0.171 
 15   35657.71    75.42  841.495 1005.311 5.437 0.133 
  X|Y|Z|W|N|M|D
  fits_0002
  1   18714.96    75.10  282.147   29.946 4.962 0.146 
  2   16462.14    77.47  641.517  156.964 4.946 0.121 
  3   21411.43    76.48  291.370  171.422 5.000 0.136 
  4  343717.99   107.30   19.696  186.715 4.969 0.137 
  5   69321.64    81.23  280.601  204.276 5.040 0.134 
  6   12570.47    74.85  713.948  226.500 5.055 0.141 
  7   12948.01    78.34  566.432  237.319 4.858 0.148 
  8   50249.01    81.32  521.645  421.338 5.018 0.145 
  9   67875.78    83.19  792.415  432.572 5.064 0.140 
 10  684185.58   118.56  529.111  468.482 5.171 0.137 
 11   25851.06    76.97  905.646  570.932 5.056 0.140 
 12   25075.39    77.70  852.496  620.413 5.872 0.324 
 13  117170.90   261.02  708.726  627.624 3.372 0.000 
 14   72320.03   522.45  708.195  630.788 2.829 0.000 
 15   16964.97    75.89  540.661  729.893 5.031 0.098 
 16   34085.88    76.87  840.085 1004.523 5.146 0.142 
  X|Y|Z|W|N|M|D
  fits_0003
  1   17702.96    78.29  282.216   29.895 4.631 0.078 
  2   16332.55    78.54  641.578  156.917 4.685 0.026 
  3   20936.28    78.56  291.540  171.342 4.750 0.079 
  4  338357.14   116.54   19.810  186.654 4.736 0.080 
  5   68776.70    85.36  280.672  204.210 4.805 0.069 
  6   10873.89    78.94   33.320  223.096 4.618 0.042 
  7   12069.17    76.95  713.959  226.461 4.682 0.077 
  8   13372.56    78.52  566.731  237.322 4.804 0.039 
  9   49356.48    82.66  521.748  421.303 4.752 0.059 
 10   66294.65    85.80  792.527  432.465 4.784 0.049 
 11  675405.53   136.39  529.174  468.417 4.869 0.050 
 12   23399.20    78.42  117.812  534.329 4.691 0.053 
 13   25242.30    80.45  905.773  570.944 4.691 0.037 
 14  243032.53   110.32  708.222  628.919 4.789 0.047 
 15   16425.15    76.48  540.795  729.855 4.744 0.033 
 16    9637.39    76.89  669.321  890.158 4.703 0.064 
 17   33870.97    81.00  840.087 1004.454 4.786 0.023

and so forth...

with 2 header line for about 2352 different images for fits files (0001 up to 2352).

The one line awk code below:

awk '{if($4<=709.999 && $4>=662.172 && $5<=629.895 && $5>=599.559) {print > "output.txt"}}' input.txt

prints the output file:

 12  255103.98    81.78  709.661  629.690 5.391 0.147 
 13  117170.90   261.02  708.726  627.624 3.372 0.000 
 14  243032.53   110.32  708.222  628.919 4.789 0.047 
 15  249091.58   102.93  708.353  628.764 4.786 0.151 
 15  257602.60    76.18  708.283  628.726 5.301 0.154 
 15  250071.97    97.72  708.465  628.599 4.765 0.206 
 14  257048.70    80.93  708.472  628.770 5.332 0.184 
 15  252688.07    83.82  708.799  628.782 5.118 0.136 
 13  176222.48   140.10  708.871  627.372 4.400 0.344 
 14  254926.50    83.71  708.928  628.599 5.080 0.086 
 12  121131.29   232.98  709.546  627.529 3.593 0.000 
 14  255246.55    85.01  709.061  628.313 5.326 0.086 
 13  256982.44    80.30  709.001  628.434 5.264 0.098 
 13  259024.53    81.94  709.224  628.211 5.158 0.171 
 14  258342.99    80.72  709.358  628.010 5.218 0.169 
 14  256036.26    83.19  709.371  628.342 5.241 0.140 
 15  252519.59    89.42  709.258  628.429 5.059 0.061 
 16  248010.38   101.19  709.374  627.964 4.845 0.139 
 15  251972.06    88.05  709.218  628.478 5.159 0.138 
 17  259479.98    73.57  708.934  628.601 5.299 0.102 
 14   99555.76   323.85  709.277  626.733 3.366 0.000 
 15  251158.18    95.20  708.995  628.113 5.202 0.147 
 16  253486.35    91.32  709.142  628.154 5.177 0.173 
 12  259105.62    75.92  708.855  628.482 5.189 0.177

and the rest .....

This code prints partially what I want but I need the input headers to be printed in the file for every sequential fits image (like what I had in input file; this enables me to recognise which output line belongs to which image (in other words I need the 2 line headers also inserted to the specific rows in output as necessary).

So my desired output should be something like this;

 X|Y|Z|W|N|M|D
 fits_0001
 12  255103.98    81.78  709.661  629.690 5.391 0.147 
 X|Y|Z|W|N|M|D
 fits_0002
 13  117170.90   261.02  708.726  627.624 3.372 0.000 
 X|Y|Z|W|N|M|D
 fits_0003
 14  243032.53   110.32  708.222  628.919 4.789 0.047 
 15  249091.58   102.93  708.353  628.764 4.786 0.151
 X|Y|Z|W|N|M|D
 fits_0004
 15  257602.60    76.18  708.283  628.726 5.301 0.154 
 X|Y|Z|W|N|M|D
 fits_0005
 15  250071.97    97.72  708.465  628.599 4.765 0.206 

 and so on...

I really appreciate any help in this matter and many thanks in advance.

RavinderSingh13 RavinderSingh13 · Accepted Answer · 2020-04-28T10:43:44

Could you please try following, conditions for fields are picked up from OP's code itself.

awk '
/\|/ || /fits/{
  print
  next
}
$4<=709.999 && $4>=662.172 && $5<=629.895 && $5>=599.559
'  Input_fie

Explanation: Adding detailed explanation for above.

awk '                                                            ##Starting awk program from here.
/\|/ || /fits/{                                                  ##Checking condition if a line contains | OR fits then do following.
  print                                                          ##Print the current line here.
  next                                                           ##next will skip all further statements from here.
}
$4<=709.999 && $4>=662.172 && $5<=629.895 && $5>=599.559         ##Checking conditions for 4th and 5th fields as per OP post.
'  Input_file                                                    ##Mentioning Input_file name here.

preserve the 2 line input file headers in the output file for large sequential multi header file using awk

1 Answers