EVERYONE Needs to Learn a Little Bit of AWK!

If you find yourself creating spreadsheet to process rows of data or numbers then you MUST learn some AWK!

####
####




#garyexplains

[ripwolfe]

I used awk to do data analysis of a large database that output data as a .csv. The company hired an "analyst" (really, he just ran DB scripts) to put the data into Excel, format it and run a few math functions on it. It took 3 hours to complete their way. The awk way took less than 15 minutes.

[RoryIsNotACabbage]

Man I've watched so many awk videos and this has been the only one that has said "this is why you do it" rather than just do the thing
I don't know how people expect you to learn when they don't explain what you're doing

[captainz]

Awk and grep were the heart of many scripts I've written over the years.

[zeke]

I've tried reading more than one awk intro and I've never made it very far. You've successfully taught me way more than all of them, thank you!

[carndt]

Used AWK in 70's and 80's. Had an accounting system written in AWK. Also, had an AWK to C compiler, for the real hardcore number crunchers.

[nickiebanchou]

"forget spreadsheets and excel"
crowd: ooh?

"use command line!!"
crowd : oh...

[jimsteele]

Back when I was contracting for EDS one of the sys admins handed me a book on AWK and asked me if I could figure out a way to extract billing info from the processing logs. I was hooked. :-) One of the handy features was using strings as array subscripts, and having "sparse" arrays, where only the accessed elements existed. Eventually, I had most of my production automated with AWK scripts.

[meowcula]

awk is one of my favourite tools, not because I'm any good at using it, mind, but just because of its history. It's from 1977! The basic apps of our unix/linux ecosystem have a rich history.

[srirachahero]

It's 2:00AM and I'm watching Gary Explain awk... and it was amazing!

Thank you, Mr. Simms!

[michaelburns]

If you like using Awk to do stuff like this, there are a few other cool Unix tools you should have a look at. Split, join, uniq will allow you to do DB-like things on text files. Another is tr. Of course sed, cut, grep as well. I took a grad course in which we had to create software development tools only using Unix tools. That class was both illuminating and fascinating. Learned a lot that I still use to this day.

[IreshDissanayakaM]

This is the best introduction to awk I have encountered.

[aseerose]

I have used it to generate a useful database of user information from emails after concatenating the emails for processing. It was not hard to learn and ended up being a very useful multi-purpose tool in addition to its primary mission success. Thanks for an excellent video!

[BryanChance]

Awk and sed one of the most useful and powerful text manipulation and formatting tools I ever learned to use.

[AuctorisVideo]

Nice video Gary. In case you (or anyone else watching) didn't know – you can simplify your "rounding up the file sizes" example using printf() and a format specifier.


For example to print the file sizes to one decimal place you could use:


{printf("%s %.1fKb\n", $1, $2/1024)}


Or:


{printf("%s %.0fKb\n", $1, $2/1024)}


To recreate your exmaple & round to the nearest integer…

[prettylitch]

00:00 Intro
01:15 more show data in file
01:29 wc -l show number of lines in data
01:48 awk '{print}' prints file
02:24 awk '{print $0}' $0 prints every line, $1 prints first field ect.
03:10 awk '/gcc/ {print $1}' match gcc print file name
03:41 awk '/^w/ {print $1}' Lines start with a double-u
04:23 awk '/path/ {print $1, $2}' Lines start with path, include multiple lines
04:41 awk '/path/ {print $1, $2/1024}' can divide output

[LoneIgadzra]

For small files there are more user-friendly tools, but awk really shines when you have some huge text file that you need to massage because it is just about as fast as you can get. Really important to know about in those cases, because you could easily be lead down an unnecessarily slow or expensive "big data" path.

[mahmoodabdulla]

This is the best awk tutorial I've seen so far.. please make a video for SED

[inwhichidie]

Just want you to know you saved my ass with this video. Procrastinated on an assignment for my CS class and this really helped me understand some stuff I'd missed and get the assignment done in time. Thanks a ton!

[juzujuzu]

Gary, I love these introductions to Linux/unix commands/software.

[damiancosta]

Gary, awesome job giving me the basic understanding of awk. All my little failed projects have been revived since Your walk thru of the AWK!