Centralizing Apache Logs

Centralizing Apache Logs

By default, Apache stores all logs on the local disk. This works well for development environments and small deployments but can become unsustainable once you have more than one server. Not only is it frustrating having to open each log file on each server, but trying to trace requests across multiple servers can quickly become time-consuming.

Log centralization services help prevent this by allowing you to store logs from your Apache servers in a single location, making it possible to view all of your web logs without having to open each log file individually. Many log centralization services can also automatically parse your logs and provide a UI allowing you to scroll, search, and filter through your log data in near real time.

This section discusses different methods of aggregating and centralizing logs from your Apache servers.

Syslog

System Logging Protocol (Syslog) is a logging service commonly found on Linux, Unix, and Mac systems. Syslog handles logs from various sources, including applications, system services, daemons, and hardware. Syslog is reliable, standardized, and can forward your logs to another syslog server.

A common approach to reading Apache logs is to configure syslog with file monitoring. With file monitoring enabled, syslog periodically scans a file on the system for changes, then imports those changes into its own log file. The benefit is you get the complete original log message wrapped in the standard syslog message format without modifying the original file.

For details on configuring Apache log file monitoring with rsyslog, see Configuring File Monitoring in Syslog.

Filtering Logs Before Centralization

In some situations, you may want to filter your logs (such as on error codes) to decrease storage on the remote system before sending them to your centralization service. With rsyslog, we can add a condition to our file monitoring rule to only allow events containing certain HTTP status codes.

This configuration example drops all messages where the status code is not 500 or 502; stop tells rsyslog to discard the message.

if $programname == 'apache2-access' and not ($msg contains '500' or $msg contains '502') then stop

Piping to Logger

Apache doesn’t only support logging to files. For example, you can also send logs directly to a syslog service using a custom logging pipeline. The most common method is to use the /usr/bin/logger command, which forwards logs over a syslog socket to the syslog service. This lets you bypass the file monitoring process, which can have performance advantages on slower storage devices. And, you no longer have to store a separate log file for Apache.

The downside to this approach is it removes the local backup provided by your Apache logs. If there’s a problem sending your logs from logger to syslog, you could lose messages. Additionally, logger supports a maximum message size of 1024 bytes. However, you can increase the size of this by adding the --sizeparameter.

To set up a logging pipe, open your Apache configuration file and replace your logging configuration with the following:

ErrorLog  "| /usr/bin/logger -thttpd -plocal6.err"
CustomLog "| /usr/bin/logger -thttpd -plocal6.notice" extended_ncsa

Restart the Apache service to apply the changes.

sudo service apache2 restart

Now your logs will no longer be written to the access.log and error.log files but will instead go straight to syslog. If you want to continue logging to file as well as syslog, you can use the following configuration instead—this uses the tee command first to pipe the log message to file, then pipes the output from that command to logger:

ErrorLog  "|/usr/bin/tee -a /var/log/www/error.log  | /usr/bin/logger -thttpd -plocal6.err"
CustomLog "|/usr/bin/tee -a /var/log/www/access.log | /usr/bin/logger -thttpd -plocal6.notice" extended_ncsa

Last updated 2022