Symptoms
- 100% capacity used for /dev/sda3.
- Size of audit.log file is very large and /var/log/audit folder consumes majority of the space.
- Saved logs from log rotate policy reference a date that is not in line with the policy.
- Unable to connect to the vCenter Server as services are not started.
- Running /etc/cron.daily/logrotate manually rotates logs as expected.
- Accessing vSphere Web Client might fail with error: 503 service unavailable
Resolution
To resolve this issue, truncate the audit.log file and verify the cron job is working correctly.
Truncate audit.log
- Log in to the vCenter Server Appliance through SSH.
- Run this command to enable access the Bash shell:
shell.set –enabled true
- Type shell and press Enter.
- Navigate to the /var/log/audit folder with this command:
cd /var/log/audit
- Run this command to verify the issue is with the audit.log file being too large (a few GBs):
ls -lh
For example:
ls -lh
total 3.5G
-rw——- 1 root root 3.5G Feb 3 16:55 audit.log
-rw——- 1 root root 445K Apr 8 2016 audit.log-20160408.bz2
-rw——- 1 root root 447K Apr 9 2016 audit.log-20160409.bz2 - Truncate (clean the content without deleting the file) the audit.log file with this command:
truncate -s 0 audit.log
