9
AWK: Extract Logs for the Given Date(s) from a Log File
source link: https://blog.jakubholy.net/2011/12/18/awk-extract-logs-for-a-given-dates-from-a-log-file/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
AWK: Extract Logs for the Given Date(s) from a Log File
December 18, 2011
If your log file has entries like these:
Then you can use the following bash script snippet to extract logs only for a particular day or consecutive days, including everything - even lines not starting with the date such as stacktraces - between the first log of the date up to the first log of a subsequent date (default: yesterday):
This
You would run it in one of the following ways:
2011-12-10T22:00:27.996+0000 [http-8080-1] INFO my.package.MyClass Hello, I'm alive!
2011-12-11T17:05:46.811+0000 [http-8080-15] ERROR my.package.MyClass - Error caught in DispatcherServlet
at my.package.MyServiceClass(MyServiceClass.java:36)
...
2011-12-11T17:06:10.120+0000 [http-8080-14] DEBUG my.package.MyClass Whoo, that has been a long day!
Then you can use the following bash script snippet to extract logs only for a particular day or consecutive days, including everything - even lines not starting with the date such as stacktraces - between the first log of the date up to the first log of a subsequent date (default: yesterday):
LOGFILE_ORIG="$0"; LOGFILE="${LOGFILE_ORIG}.subset"
if [ -z "$LOGDAY" ]; then LOGDAY=$(date +%F -d "-1 days"); fi
if [ -z "$AFTERLOGDAY" ]; then AFTERLOGDAY=$(date +%F -d "$LOGDAY +1 days"); fi
echo "Extracting logs in the range (>= $LOGDAY && < $AFTERLOGDAY) into $LOGFILE ..." awk "/^$LOGDAY/,/^$AFTERLOGDAY/ {if(!/^$AFTERLOGDAY/) print}" $LOGFILE_ORIG > $LOGFILE
This
date format works on Linux. Date is very flexible and can provide dates in any format, not only yyyy-mm-dd. You may also want to read more about Awk ranges and other tips.You would run it in one of the following ways:
$ ./analysis.sh /path/to/logfile.log
$ LOGDAY=2011-12-12 AFTERLOGDAY=2011-12-17 ./analysis.sh /path/to/logfile.log
Are you benefitting from my writing? Consider buying me a coffee or supporting my work via GitHub Sponsors. Thank you! You can also book me for a mentoring / pair-programming session via Codementor or (cheaper) email.
Allow me to write to you!
Let's get in touch! I will occasionally send you a short email with a few links to interesting stuff I found and with summaries of my new blog posts. Max 1-2 emails per month. I read and answer to all replies.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK