Buffering Apache logfiles with the BufferedLogs directive

On busy web servers, the process of writing to the access_log can sometimes overwhelm the spindles in a server. In Apache 2.0.41, the developers added the experimental “BufferedLogs” directive to buffer access_log entries in memory, and write them out as a single group. The documentation indicates that setting “BufferedLogs” to “On” enables buffered logging, but I couldn’t find anything that described how to configure the size of the buffer. After a bit of poking around in mod_log_config.c, I noticed that the size of the buffer was controlled by the LOG_BUFSIZE macro:

if (len + buf->outcnt > LOG_BUFSIZE) {
if (len >= LOG_BUFSIZE) {
apr_size_t w;

str = apr_palloc(r->pool, len + 1);
for (i = 0, s = str; i < nelts; ++i) {
memcpy(s, strs[i], strl[i]);
s += strl[i];
w = len;
rv = apr_file_write(buf->handle, str, &w);

To see what LOG_BUFSIZE was set to, I searched for the value in mod_log_config.c. This is what my search turned up:

#ifdef PIPE_BUF
#define LOG_BUFSIZE (512)

Nifty! So if PIPE_BUF is defined, that will be used as the size. Now to see what the value of PIPE_BUF is set to on my CentOS 4.4 server:

$ cd /usr/include && find . | xargs grep PIPE_BUF

./linux/limits.h:#define PIPE_BUF 4096 

So PIPE_BUF is set to a 4k buffer on my CentOS 4.4 server, and this value will be used if it exists. I am curious to see if there are any downsides to using extremely large buffers (hosting providers might be interested in using something larger than 4k). More testing is needea.

This article was posted by Matty on 2006-12-21 12:19:00 -0400 -0400