logrotate Explained: Keeping Log Files From Eating Your Disk

logrotate Explained: Keeping Log Files From Eating Your Disk

A busy nginx server writes access logs at a pace that turns megabytes into gigabytes within weeks, and no log is useful enough to justify a full disk. logrotate is the tool every distribution ships to handle this: it ages log files through numbered archives, compresses the old ones, deletes the oldest, and tells applications to start fresh, all on autopilot.

The rotation cycle

One rotation of access.log with compression and a retention of 4 looks like:

access.log       -> access.log.1
access.log.1     -> access.log.2.gz (compressed this cycle)
access.log.2.gz  -> access.log.3.gz
access.log.3.gz  -> access.log.4.gz
access.log.4.gz  -> deleted
(new empty access.log created)

Each run shifts the whole ladder by one. Retention count, compression, schedule, and what happens at each step are all per-log configuration.

Where the configuration lives

The main file /etc/logrotate.conf sets defaults, and /etc/logrotate.d/ holds one snippet per application, dropped there by packages. Installing nginx also installs /etc/logrotate.d/nginx, which is why log rotation mostly just works without anyone thinking about it:

/var/log/nginx/*.log {
    daily
    missingok
    rotate 14
    compress
    delaycompress
    notifempty
    create 0640 www-data adm
    sharedscripts
    postrotate
        invoke-rc.d nginx rotate >/dev/null 2>&1
    endscript
}

Reading it top to bottom: rotate daily, do not error if the file is missing, keep 14 archives, compress them but not the newest one (delaycompress), skip empty logs, create the replacement file with these permissions, run the postrotate script once for all matched logs rather than per file (sharedscripts), and finally tell nginx to reopen its log files.

The open file handle problem

The subtle part of rotation is that renaming a file does not disturb programs that have it open; they keep writing to the same inode under its new name. Rotate a log without telling the application, and your fresh access.log stays empty while access.log.1 keeps growing.

Two solutions exist. The clean one is the postrotate script, which signals the app to reopen its logs (nginx and most daemons accept a signal or subcommand for exactly this). The blunt one is copytruncate: copy the current contents to the archive and truncate the original in place, so the app never needs to know. copytruncate requires zero application cooperation but can lose the lines written in the instant between copy and truncate, an acceptable trade for low-volume application logs and the standard answer for apps that simply have no reopen mechanism.

Writing a config for your own app

A service writing /var/log/myapp/app.log gets a file in /etc/logrotate.d/myapp:

/var/log/myapp/*.log {
    weekly
    rotate 8
    compress
    delaycompress
    missingok
    notifempty
    copytruncate
}

Size-based rotation is available where cadence is the wrong measure: size 100M rotates whenever the file exceeds the threshold, checked at each run, and maxsize combines with a time schedule as “weekly, or sooner if it gets big.” dateext switches archive names from .1, .2 to date suffixes like -20260818, friendlier for archival.

Test before trusting:

sudo logrotate -d /etc/logrotate.d/myapp   # dry run, prints decisions
sudo logrotate -f /etc/logrotate.d/myapp   # force a real rotation now

How it actually gets triggered

logrotate is not a daemon. Something must run it, and on modern distros that is a systemd timer:

systemctl list-timers logrotate.timer

Once a day the timer fires, logrotate evaluates every config, and rotates whatever is due. The state file /var/lib/logrotate/status records when each log last rotated, which is how weekly knows a week has passed.

The journald boundary

One scoping note that saves confusion: the systemd journal does not use logrotate. Journald caps itself via SystemMaxUse in journald.conf and cleans with journalctl --vacuum-size. logrotate’s domain is plain text files in /var/log: web server logs, application logs, and whatever rsyslog writes. On a modern system the two coexist, each managing its own territory, and a disk-space investigation should check both: journalctl --disk-usage for the journal, du -sh /var/log/* for the rest.

Frequently Asked Questions

What does logrotate do?

On a schedule, logrotate renames current log files to numbered archives, optionally compresses them, deletes archives beyond a retention count, and signals applications to start writing fresh files. It is the standard defense against logs filling a disk.

Does journald need logrotate?

No. The systemd journal manages its own size through SystemMaxUse and vacuum settings. logrotate exists for plain text log files written by applications like nginx, or forwarded by rsyslog into /var/log.

How often does logrotate run?

It is triggered externally, on modern systemd distros by a daily timer. Each run evaluates every config and rotates whatever its rules say is due, whether by age like daily or weekly, or by size.

What is the difference between copytruncate and create?

create renames the file and expects the app to open a new one, usually after a postrotate signal. copytruncate copies the file then empties the original in place, needing no app cooperation but risking loss of lines written during the copy.

Why is my rotated log empty or my app still writing to the old file?

The app still holds the renamed file open. Renaming does not affect open file handles, so the app must be told to reopen its logs in a postrotate script, or you must use copytruncate.

How do I test a logrotate config without waiting a day?

Run logrotate in debug mode with -d to see what would happen, or force a rotation with -f against your config file to execute it immediately.