Rotate lines in a text file with n steps

Question

I have a text file with thousands of lines and I need to rotate the lines . Here is an example:

myfile.txt:

line1
line2
line3
line4
line5
line6
line7

I would like to rotate the lines with n steps. For example, if n = 2. line1 moves to line3, line2 moves to line4, line3 moves to line 5, ... line6 moves to line1, line7 move to line2. So the output should be:

line6
line7
line1
line2
line3
line4
line5

Maybe I could use Python and read the file, and rotate the list, and save to another file. I wonder if there is already a command line utility for this purpose? it should take the filename and n as arguments and outputs the rotated lines to a new file.

I appreciate your help.

Answer 1

Use head and tail, plus some simple bash arithmetic expansion. You also need either tac or wc:

First, create a minimum reproducible example for the input file. Set n - the number of lines from the end of the file to rotate to the beginning:

n = 2
perl -le 'print "line$_" for 1..7' > in_file
cat in_file

Prints:

line1
line2
line3
line4
line5
line6
line7

Method 1: Rotate with tail and head, plus wc.

This is slightly less complex than method 2, and uses wc -l ... - $n to compute the number of lines for the head to print. I prefer this method because the programmer's intentions are more clear here. It is also faster, see benchmarks below.

( tail -n $n in_file ; head -n $(( $( wc -l <in_file ) - $n )) in_file ) > out_file

Prints:

line6
line7
line1
line2
line3
line4
line5

Method 2: Rotate with tail and head, plus tac.

Here,
tac : write the lines in reverse order into STDOUT,
tail -n +3 : write the above lines in reverse order starting from line 3 from the end of the original file (lines 1-2 are thus not written), tac : use tac a total of twice, to reverse the reverse order of lines, in order to write the lines in the original order.

( tail -n $n in_file ; tac in_file | tail -n +$(( $n+1 )) | tac ) > out_file

Benchmarks:

Method 1 using wc is substantially faster that method 2 using tac twice:

perl -le 'print "line$_" for 1..1e7' > in_file
n=2

for i in `seq 1 10` ; do
    ( time ( tail -n $n in_file ; head -n $(( $( wc -l <in_file ) - $n )) in_file ) > out_file ) 2>&1 | grep real
done

Prints:

real    0m0.539s
real    0m0.538s
real    0m0.545s
real    0m0.566s
real    0m0.540s
real    0m0.532s
real    0m0.561s
real    0m0.534s
real    0m0.530s
real    0m0.520s

for i in `seq 1 10` ; do
    ( time ( tail -n $n in_file ; tac in_file | tail -n +$(( $n+1 )) | tac ) > out_file ) 2>&1 | grep real
done

Prints:

real    0m0.855s
real    0m0.884s
real    0m0.916s
real    0m0.829s
real    0m0.838s
real    0m0.873s
real    0m0.877s
real    0m0.862s
real    0m0.835s
real    0m0.867s

I ran this using MacBook Pro with macOS v.10.14.6, running:

bash --version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin18)

Answer 2

a bit verbose with awk, but..

$ cat tst.awk
BEGIN {
  step=(!step)?2:step
}
{
  a[FNR+step]=$0
  nr=FNR
}
END {
  for(i=nr+1; i in a;i++)
    a[(i%step)+1]=a[i]
  for(i=1; i<=nr ;i++)
          print a[i]
}

$ awk -f tst.awk file
line6
line7
line1
line2
line3
line4
line5

$ awk -v step=4 -f tst.awk file
line4
line5
line6
line7
line1
line2
line3

Answer 3

Here's a method using tee, the job control commands wait and &, and bash process substitution, that unlike the answers so far, works with pipes and redirects, so the filename only needs to be mentioned once, or a stream can be used instead:

N=2
tee >(tail -n $N &) \
    >(wait && head -n -$N) > /dev/null < myfile.txt

Or, with a stream instead of a filename:

N=2
seq -f 'line%g' 7 | 
tee >(tail -n $N &) \
    >(wait && head -n -$N) > /dev/null

Output of either:

line6
line7
line1
line2
line3
line4
line5

Answer 4

You can do this with the shell commands head and tail. If you wanted to "rotate" a file of length N 2 times, you'd print the last N-2 lines of the file and then the first 2 lines.

The example below rotates a file called "rotate" with 5 lines 2 "times".

tail --lines 3 rotate
head --lines 2 rotate

or as a one-liner tail --lines 3 rotate; head --lines 2 rotate