Regular expressions in sed & grep: powerful tools for searching and modifying files in bash (CC022)

In this screencast tutorial, Pat Schloss shows how you can use regular expressions to search for patterns in files and then modify files based on those patterns. We'll also learn new commands that work well with grep including wc and head. Today's content will also be important as we determine the degree to which inter- and intra-genomic variation limit the interpretation of amplicon sequence variants (ASVs; aka exact sequence variants or ESVs) for different regions within the 16S rRNA gene. This is an important question for microbiome researchers.

Pat demonstrates these concepts by live coding at the command line interface using RStudio, GitHub Flow, and make. He then turns viewers loose to work through several activities to answer related questions and finishes by giving his solutions.

[dancorvalan]

Hi Pat, great job! Thank you for making these videos.

[RasmusKirkegaard]

Hi Pat
Thanks for a great video series. I have learnt a lot of nice tricks from this.

I noticed that for issue 16
sed 's/^\.+/-/'
will replace multiple leading dots with a single hyphen. Is that going to be a problem if we are going to use the alignments for something later on?
Maybe we are better of changing all the dots with hyphens using sed 's/\./-/g' even though it is also going to affect the coordinates in the headers. I did not manage to find (google) a simple/human readable sed only way to replace the dots using the + or * with exactly the same number of hyphens.

Since dots only occur in the headers in between the numbers we could do
sed 's/\./-/g' | sed -E
To put the dots back in but it feels a bit like cheating :D.