Xargs Should Be In Your Command Line Toolbag

Xargs is an important shell utility to know because it allows you to execute to pass standard input as an argument to another command. While many command line programs allow standard input to be used as a parameter (things like grep, sed, awk etc.), many other programs do not allow for this (such as echo and rm). This is where Xargs comes in handy.

REFERENCED:

WANT TO SUPPORT THE CHANNEL?

DONATE CRYPTO:
💰 Bitcoin: 1Mp6ebz5bNcjNFW7XWHVht36SkiLoxPKoX
🐶 Dogecoin: D5fpRD1JRoBFPDXSBocRTp8W9uKzfwLFAu
📕 LBC: bMfA2c3zmcLxPCpyPcrykLvMhZ7A5mQuhJ

SOCIAL PLATFORMS:

DT ON THE WEB:

FREE AND OPEN SOURCE SOFTWARE THAT I USE:

Your support is very much appreciated. Thanks, guys!

[first-thoughtgiver-of-will]

I've been writing bash scripts for years now and never bothered to thoroughly read the documentation. I have always used an inferior work around for this thank you for showing me the formal solution.

[sasakanjuh]

Just a tip, if you are trying to create a bunch of sequentially numerated files with touch, you can use bash ranges {} syntax, like "touch {1..1000}".. It's really convenient, and it works with other types of sequences (a..z), and you can even pass a step in which next item will be incremented :)
Nice video, btw! :)

[psapehin]

Great structured intro into xargs, thanks DT!

Here is an approximate ToC for this video:


00:48 basic xargs usage + echo
02:03 xargs -t flag = show expanded output
02:55 xargs + ls = recursive ls
03:36 example: /etc/passwd
04:00 passwd + cut + sort + xargs
04:41 xargs -I (capital i) flag = alias for arguments
06:19 example: make 1000 txt files: seq + xargs + touch
07:21 example: mv *.txt to *.text with ls + cut + xargs
08:35 xargs -n flag (number of args) = multiline output with
09:39 example: seq + xargs + bash + sleep
11:34 example: xargs + find = remove all *.text
13:50 performance: find --exec vs xargs rm
15:18 summary

This table of content was created using "Smart Bookmarks for YouTube" chrome extension.

[emiliadaria]

Hey DT, your comparison of "-exec" and "xargs" wasn't entirely fair.
Using -exec with the semicolon is more similar to calling xargs with the "-L 1" option. A fair comparison would be to use "find ... -exec rm {} +". The "+" tries to fit as many arguments to one command as it can just like xargs by default.

[phylwx]

You're content is getting way better DT, congrats! I love this vid, really nice info.

[stephenflee]

Hands down the best rundown of xargs I've ever seen, you rock my dude!

[InfiniteQuest]

xargs is a true lifesaver when it's needed. One key example is when you have a tool that needs to run on every file in a directory. Using ls and xargs with -n 1 can really blast out the results compared to trying to bash for loop it or something. The problem is people don't even know about it to consider it as an option.

[johnbaroon]

Man I thought you are not a programmer but you are GOOD DAM LINUX PROGRAMMER ANY WAY Thank you DT for all what you do.

[matthewlandry]

Never knew about xargs! Neither did my spellchecker. Now we both do! Thanks DT

[gonzalooviedo]

From Chile, thanks for your dedication and your team also, you are doing fantastic work every day, like the format of your videos and your english is very good, thanks for not talking fast!. I'm addicted to linux.

[alexb]

very informative video, it literally answered all my questions. meaning all important points were considered and answered, like different use cases and comparisons with other similar tools, which is preferable in which scenario and so on. very good work indeed

[josephfredbill]

After 40ish years writing shell scripts I never used xargs. I can see its useful but usually I turned standard output into arguments using the back-quote and when needed combined with sed, grep and others. I never understood man xargs :-). Great explanation.

[rubeo]

Getting the hang of Tmusk, really love it, now trying to get more acquinted with xargs and awk. The latter is the most complicated one but I love being busy with that! Thanks for your videos, love 'm!!

[SylvesterInk]

Correct me if I'm wrong, but isn't the -exec functionality of find slower because it's executing the command for each individual find result, much like running `xargs -n 1`? I tested the find command using `-exec rm {} \+"`, which appends the results to a single command, and got the same time results as using xarg.

So if you're using find, it's pretty much comparable to use -exec or xargs interchangeably, for basic tasks, though xargs is obviously more versatile for tasks that aren't related exclusively to the results of find.

[sdegh]

These command line utility videos are amazing. Thank you for the content.

[fennecbesixdouze]

This was an incredibly well written and produced video. Really spectacular job running down this command.

[MrPontiac]

XARGS is one of those insanely useful tools. It's something in my toolbox that I use EVERY day when I manage any one of the hundreds of servers I could be dealing with each day.

Just for clarification, XARGS doesn't make FIND any more or less efficient. FIND will execute the RM statement (Or whatever it's internal equivalent is) for each result it returns. XARGS will pass as many parameters to RM (or whatever command) as it can if you don't set a limit on the number of parameters to pass in. That means that the RM command may be run multiple times against a large number of files/parameters. RM is intelligent with file transactions, just the same as relational databases are intelligent with their ACID transactions.

As an example to really put this to the test:

for c in `seq 100`
do
echo "----"
echo "${c} step"
echo "Creating Files"
time seq 1000 | xargs -i touch {}.txt
echo "Running RM with ${c} parameters"
time find . -type f -name '*.txt' | xargs -n ${c} rm
done

This will start a loop. In each loop, it'll create 1000 files, then promptly delete them passing a number of files/parameters to be deleted each loop. So on loop one, a single file/parameter will be passed to RM for it to be deleted. On the second pass, two files will be passed to RM, etc.

Note that `seq 100` is wrapped in the BACK-TICKS (Unshifted-Tilde key on standard US keyboards, at least), it's not a standard single quote.

[guyboisvert]

This is really useful! I made a script that processed several million files, output from the find command using -print0 and xargs with the -0 argument (for using paths / filenames with spaces) and launching another script, 16 of them in parallel with -n1 and -P16 arguments to xargs. The performance was phenomenal: the Dual Xeon E5-2660v2 (2 x 8 cores) usage was up to 95%, averaging about 80%. This is multi-threading bash scripting!

[ian_zane]

This is truly much better than reading the manual pages! loved the accent btw :)

[leroyonlinux]

I love this videos explaining those basic CLI tools. Because if you don't know they exist, its hard to find them. ESpecially for new users.