From World’s best introduction to sed

sed workflow

4 spaces of sed

• Input Stream
• Pattern Space
• Hold Buffer
• Output Stream

Basic work flow of sed

1. Read from input stream input until ‘\n’, put content into pattern space (not ‘\n’ is not put into)
2. Operate on pattern space. Hold buffer is temporary buffer for we to use.
3. Output to the output stream, add a ‘\n’

Useful options

• -n option tells sed not to output the pattern space by default. Use p command to force output.
• -i is dangerous, safer to use the file extension to create a copy before modifying the content, e.g. -i.bak.

• -e to combine multiple command. Or use ; to separate command.

  • The following 2 are the same:

    sed -n 's/foo/bar/; p'
    sed -n -e 's/foo/bar/' -e 'p'
    

• -r use extended regular expression, make characters like + magic character

Specifying command range

• Line number (similar with Vim’s command range command)

  • Replace on line 5 to 10:

    sed '5,10s/pat/foo/'
    

  • Print only line 5 to 10:

    sed -n '5,10p'
    

  • Invert range, replace except line 5 to line 10:

    sed '5,10!s/pat/foo/'
    

  • $ means the last line

• Use regular expression (similar with awk)

  • This emulates grep:

    sed -rn '/\d*/p'
    

  • Can use regex to specify range. Example, delete lines from the first line which matches /foo/ to the first line matches /bar/, inclusive

    sed '/foo/,/bar/d'
    

Hold buffer

Suppose you have a problem where you want to print the line before the line that matches a regular expression. How do you do this? Hold buffer can solve this problem.

• h copy pattern space to hold buffer
• g copy hold buffer to pattern space
• x exchange pattern space and hold buffer

Solving the problem:

sed -n '/regex/{x;p;x}; h'

Note command grouping here: {x;p;x} is a command group, and only get executed when the pattern matches the regex, while the h is executed for each line.

This does not work if the first line matches the regex, fix it:

sed -n '/regex/{x;1!p;x}; h'