Automatically send unique errors (with count) from Apache error log as email

Sometime back I wrote about a simple awk script that I wrote which allowed me to find unique errors from Apache error log files.

After I wrote that script I found myself executing that script every day in the morning to figure out if there were any errors in my sites. After a couple of days I wrote another script to automatically parse the error log file and email me if there were any errors. As usual I thought of writing about it here so that it would be useful for someone else as well 🙂

Logrotate

First you need to install the logrotate program which will allow you to automatically rotate log files. In most servers it gets installed automatically. If it is not installed that you can install it using the command sudo apt-get install logrotate.

After you have installed the logrotate program you need to edit its configuration file. My configuration files looks like this.

/path/to/log/*.log  {
        daily
        missingok
        rotate 10000
        compress
        delaycompress
        notifempty
        dateext
        create 660 root www-data
        sharedscripts
        postrotate

        if [ -f "`. /etc/apache2/envvars ; echo ${APACHE_PID_FILE:-/var/run/apache2.pid}`" ]; then
            /etc/init.d/apache2 reload > /dev/null
        fi

        endscript
        prerotate

            awk -F'[\[\]]+' '$4 == "error"{print $7}' /path/to/error.log | sort | uniq -c | sor -nr | /path/to/apache-error-mailer.py

        endscript

The important part is the lines between prerotate and endscript. These lines get executed before the log file is rotated and I am calling my awk script to parse the error log file to get unique errors and then I am passing it to a python script. Kindly refer to my original post where I explained the awk script in detail.

Send email

The next step is to send the output of the awk script though email. For this I am using the following python program. This python program reads the lines from stdin and then send them through email. In my case, I am sending the email through mandrill. If you are going to use some other mechanism then you may have to change the send_email function appropriately.

#!/usr/bin/env python

import sys
import mandrill

MANDRILL_API_KEY = "api-key"
EMAIL_FROM = "your@email.com"
EMAIL_TO = "your@email.com"
EMAIL_SUBJECT = "Error Log"

def send_mail(content):
    """Send email using Mandrill API"""
    mandrill_client = mandrill.Mandrill(MANDRILL_API_KEY)
    message = {
        'auto_html': True,
        'auto_text': None,
        'to': [{'email': EMAIL_TO, 'type': 'to'}],
        'from_email': EMAIL_FROM,
        'subject': EMAIL_SUBJECT,
        'html': content,
        'track_clicks': None,
        'tags': ['error-log']
    }
    result = mandrill_client.messages.send(message=message, async=False)


if __name__ == '__main__':
    lines = []
    for line in sys.stdin:
        lines.append(line.strip())

    content = "<br>".join(filter(None, lines))

    if len(content) > 0:
        send_mail(content)

That’s it. Once you have configured it, you would be getting email everyday with the unique errors from your apache error log.

Chaim Keren-Tzion says:

Dec 31, 2014 at 9:47 PM

You wrote:

The important part is the lines between postrotate and endscript. These lines get executed after the log file is rotated and I am calling my awk script to parse the error log file to get unique errors and then I am passing it to a python script.

However in the logrotate config file you have:

prerotate
 
   awk -F'[\[\]]+' '$4 == "error"{print $7}' /path/to/error.log 
                   | sort | uniq -c | sor -nr | /path/to/apache-error-mailer.py
 
endscript

I think you meant to write that: The important part is the lines between PREROTATE and endscript. These lines get executed BEFORE the log file is rotated…

After the rotate the log file will be empty. Your config file has it right but your explanation needs to be tweaked.

Sudar says:

Jan 1, 2015 at 1:02 PM

Thanks for pointing it out. That was a major blunder from my side and I have corrected it now.

Dec 31, 2014 at 9:50 PM

Sorry about the formatting above; there is no preview, post editing or delete option. :-p

Sudar says:

Jan 1, 2015 at 1:03 PM

Don’t worry about the formatting. I just corrected it.

I have “comment preview and editing” in my (rather long) todo list. Need to find a good plugin that can do this.

Prostatitis Treatment says:

Jul 5, 2017 at 1:57 PM

Thank you for posting it helped a lot .
Finally got right after altering

Night Dreaming (by Sudar)