Our journalctl guide covers system logging. auditd is a different thing with a different purpose.

Application logs record what a program chose to tell you. Audit records what the kernel observed. A compromised process can stop writing to its log; it cannot prevent the kernel recording the syscall it just made.

That property is why audit exists and why compliance frameworks ask for it specifically.

Architecture

The audit subsystem lives in the kernel. Rules are loaded into it, matching events are generated during syscall handling, and auditd in userspace writes them to /var/log/audit/audit.log.

Not the journal. Its own file, with its own rotation and its own permissions.

sudo apt install auditd audispd-plugins
sudo systemctl enable --now auditd
sudo auditctl -s

enabled 1
failure 1
pid 892
rate_limit 0
backlog_limit 8192
lost 0
backlog 0

lost counting up means records are being dropped because the daemon cannot keep up. That is a signal your rules are too broad.

Rules

Two kinds.

Watches

-w /etc/passwd -p wa -k identity
-w /etc/shadow -p wa -k identity
-w /etc/sudoers -p wa -k privilege
-w /etc/sudoers.d/ -p wa -k privilege
-w /etc/ssh/sshd_config -p wa -k sshd_config

-p is the permission mask: r read, w write, x execute, a attribute change. -k is a key you use later to search.

wa on a config file catches both content changes and permission changes, and the latter matters because making a file world-writable is a quieter attack than editing it.

Syscall rules

-a always,exit -F arch=b64 -S execve -F euid=0 -F auid>=1000 -F auid!=-1 -k root_exec

-a always,exit -F arch=b64 -S unlink,unlinkat,rename,renameat -F auid>=1000 -F auid!=-1 -k delete

-a always,exit -F arch=b64 -S mount -F auid>=1000 -F auid!=-1 -k mounts

-a always,exit -F arch=b64 -S init_module,delete_module -k modules

The auid filter is the one worth understanding. auid is the login UID, which stays with a session even after su or sudo changes the effective UID.

So -F euid=0 -F auid>=1000 means “running as root, by someone who logged in as a normal user.” That is the interesting case: a real person escalating, not a system daemon doing its job.

-F auid!=-1 excludes processes with no login UID, which are daemons. Without it you record every system service and drown in noise.

init_module and delete_module catch kernel module loading, which is a direct path from code execution to kernel compromise. Our kernel modules guide covers the mechanism, and our service hardening guide covers blocking it per-service.

Making rules persistent

# /etc/audit/rules.d/hardening.rules
-D
-b 8192
-f 1

-w /etc/passwd -p wa -k identity
-a always,exit -F arch=b64 -S execve -F euid=0 -F auid>=1000 -F auid!=-1 -k root_exec

-e 2

sudo augenrules --load
sudo auditctl -l

Rules added with auditctl apply to the running kernel only. Files under /etc/audit/rules.d are compiled by augenrules at boot, and forgetting the file is why rules vanish after a reboot.

-e 2 makes the configuration immutable until the next reboot. Nobody, including root, can change or unload the rules. That is what makes the audit trail trustworthy against an attacker who gained root, and the cost is that legitimate changes also need a reboot.

Put it last, because nothing after it is processed.

-f 1 sets failure mode to printk. -f 2 panics the kernel if audit cannot record, which some compliance regimes require and which will absolutely take a machine down.

Searching

sudo ausearch -k identity -i
sudo ausearch -k root_exec --start today -i
sudo ausearch -ua 1000 --start recent -i
sudo ausearch -m USER_LOGIN --start today -i

sudo aureport --summary
sudo aureport --auth --summary
sudo aureport -f --summary

-i is the flag to remember. It interprets numeric IDs and hex-encoded values into names and readable strings. Without it you get raw values and a bad time.

sudo ausearch -k identity -i | head -20

type=SYSCALL msg=audit(19/09/26 14:22:07.113:4471) : arch=x86_64 syscall=openat
  success=yes exit=3 ppid=2841 pid=2903 auid=colton uid=root gid=root
  euid=root suid=root fsuid=root comm=vim exe=/usr/bin/vim key=identity
type=PATH msg=audit(19/09/26 14:22:07.113:4471) : name=/etc/passwd
  inode=1049159 mode=file,644

auid=colton uid=root is the whole value of the audit trail: it survived the sudo and still names the person.

Note the records are split by type and linked by the event ID. This is why grep on the raw log is unpleasant and ausearch exists.

Keeping the volume manageable

This is where audit deployments fail. Rules that are too broad produce gigabytes a day, overhead you can measure, and a log nobody reads.

Watch specific paths, not directories full of activity. -w /etc/ is a mistake. -w /etc/shadow is useful.

Filter by auid. Excluding daemons removes most of the volume.

Avoid auditing read access on anything busy. -p r on a frequently-read file generates a record per open.

Exclude known noise:

-a never,exit -F arch=b64 -S all -F exe=/usr/bin/prometheus-node-exporter

Set retention deliberately:

# /etc/audit/auditd.conf
max_log_file = 50
num_logs = 10
max_log_file_action = ROTATE
space_left = 500
space_left_action = EMAIL
admin_space_left_action = SUSPEND

disk_full_action = HALT exists and some compliance regimes want it. Understand that it means the machine stops when the audit partition fills, which is a policy decision rather than a technical one.

Shipping it

An audit log that lives only on the audited machine is deletable by whoever compromises it.

sudo apt install audispd-plugins

# /etc/audit/plugins.d/au-remote.conf
active = yes
direction = out
path = /sbin/audisp-remote
type = always

Shipping to a host the audited machine cannot delete from is what makes the trail survive a compromise. Our Loki guide covers centralised logging generally, with the caveat noted there: for audit specifically you want append-only storage rather than something the source can influence.

Where it fits

Use it for: compliance requirements that name it, forensics after an incident, watching a small set of genuinely sensitive files, and recording privilege escalation.

Do not use it for: general application logging, performance monitoring, or as a substitute for SELinux and AppArmor. Audit records; it does not prevent. It tells you what happened, after it happened.

The complementary pairing is worth stating: MAC systems stop the action, audit records the attempt. Running both means you block what you can and have evidence about what you could not.

Frequently Asked Questions

How is auditd different from normal system logging?

auditd records events in the kernel, before any userspace process can influence them, and writes to its own log rather than the journal. A process cannot suppress an audit record about its own behaviour, which is what makes the audit trail suitable for forensics and compliance where application logs are not.

Does auditd slow the system down?

It can, substantially, if the rules are too broad. Every matching syscall generates a record, so auditing all file access on a busy server produces enormous volume and measurable overhead. Well-scoped rules watching specific paths and syscalls cost very little.

What is the difference between a watch and a syscall rule?

A watch monitors a filesystem path for the access types you specify and is written with -w. A syscall rule matches on the system call itself with optional filters on arguments, written with -a, and is more flexible and more expensive to evaluate.

Why do my audit rules disappear after a reboot?

Rules added with auditctl apply only to the running kernel. Persistent rules go in files under /etc/audit/rules.d, which augenrules compiles into the active ruleset at boot. Adding a rule with auditctl and forgetting to write the file is the usual cause.

Can an attacker with root disable auditd?

Yes, unless you set the immutable flag. Adding -e 2 to the ruleset locks the configuration until reboot, so rules cannot be changed or unloaded by anyone including root. The tradeoff is that legitimate rule changes then also require a reboot.

How do I search the audit log?

Use ausearch to filter by key, user, syscall, or time, and aureport for summaries. Reading the raw log with grep is possible and unpleasant, because records are split across multiple lines linked by an event ID and the values are frequently hex encoded.