How to Extract Specific Values from Lines in Log Files Using bash

Learn how to write a bash script that effectively extracts specific numerical values from log files based on keyword searches.
---

Visit these links for original content and any more details, such as alternate solutions, latest updates/developments on topic, comments, revision history etc. For example, the original title of the Question was: Printing the numbers after match (bash)

If anything seems off to you, please feel free to write me at vlogize [AT] gmail [DOT] com.
---
How to Extract Specific Values from Lines in Log Files Using bash

If you are working with log files and need to extract specific numerical values based on certain keywords, you might find yourself facing a common challenge in scripting. A frequent requirement is to locate a string within a line and print only the relevant numerical data that follows it. In this guide, we will explore how you can efficiently achieve this using a simple bash script.

Problem Overview

Suppose you have log files containing lines with various data, such as:

[[See Video to Reveal this Text or Code Snippet]]

In this case, you want to extract just the numerical value (-9.68765465413) that comes after the specified keyword (E( ENERGY) =). However, your current script is outputting the entire line, which is not ideal.

Current Approach

Your existing bash script currently looks like this:

[[See Video to Reveal this Text or Code Snippet]]

This script effectively finds the lines containing the keyword but reveals the entire line, making it cumbersome for your needs.

Desired Output

You want to transform the output to look like this:

[[See Video to Reveal this Text or Code Snippet]]

Solution: Using awk

Instead of using grep, you can leverage the power of awk, which is designed for text processing and can manipulate the output more effectively. Below, I’ll share a revised version of your script utilizing awk to accomplish the extraction.

Revised Script

Here’s how you can modify your script to use awk.

[[See Video to Reveal this Text or Code Snippet]]

Explanation of the Script

Variable Definitions:

output='.' sets the current directory as the target for searching files.

ext='log' specifies that you are looking for .log files.

word="E( ENERGY)" defines the keyword to search for.

Finding Files:

find "${output}" -type f -name "*.${ext}" searches for files that match the .log extension.

Using awk:

-v ptn="${word}" passes the search word into awk.

index($0,ptn) checks if the line contains the keyword.

{print FILENAME,$NF} prints the filename along with the last field (which in this case is the numerical value).

Expected Output

After running the revised script, you should see an output that cleanly isolates the filename and the numerical value:

[[See Video to Reveal this Text or Code Snippet]]

Conclusion

With this updated approach using awk, you can effortlessly extract numerical values following specified keywords in your log files. This method not only simplifies your code but also makes the output much cleaner and easier to work with.

If you have any questions or need further customization of your bash scripts, feel free to reach out in the comments below!