↓Skip to main content

Docker Logging Limitation

·389 words·2 mins
Sebastian Scheibe
Author
Sebastian Scheibe
Table of Contents

Introduction
#

Ever had the error No space left on device or Cannot create directory? It might be due to logs from your Docker daemon. Here is how you can limit the logs from Docker containers.

Overview of Docker Logging
#

Docker logs are essential for debugging and monitoring container applications. However, unmanaged logs can quickly consume disk space, leading to system errors and degraded performance.

Problem with Default Logging Settings
#

By default, Docker uses the json-file log driver, which does not have a cap on the file size. This can lead to issues like No space left on device, especially on systems with limited storage.

Solution
#

Configure Log Limit
#

You can choose between multiple configurations, I recommend the local file.

Local file
#

Default file limit to 100mb per container. Each container can have up to 5 files with each having maximum 20 MB per file.

sudo nano /etc/docker/daemon.json

{
  "log-driver": "local"
}

systemctl restart docker

One Liner

echo '{"log-driver": "local"}' | sudo tee /etc/docker/daemon.json > /dev/null && sudo systemctl restart docker

See more here on how to configure the local log-driver.

JSON file
#

This is the default log driver. It has no cap on the file limit, leading to issues like No space left on device. The following configuration specifies a maximum size of 50mb per container.

sudo nano /etc/docker/daemon.json

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "50m"
  }
}

systemctl restart docker

One Liner

echo '{"log-driver": "json-file", "log-opts": {"max-size": "50m"}}' | sudo tee /etc/docker/daemon.json > /dev/null && sudo systemctl restart docker

See more here on how to configure the JSON log-driver.

Recreate Containers
#

Important: To apply the new log configuration for each running container, please recreate each container.

Troubleshooting Tips
#

To identify if logs are causing issues:

Check disk usage
#

Monitor disk usage with:

df -h

Example:

df -h

Filesystem      Size  Used Avail Use% Mounted on
tmpfs           1.6G  5.2M  1.6G   1% /run
/dev/vda1       155G   24G  131G  16% /
tmpfs           7.9G     0  7.9G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
/dev/vda15      105M  6.1M   99M   6% /boot/efi
tmpfs           1.6G  4.0K  1.6G   1% /run/user/0

Check log sizes of container
#

Check log sizes with:

docker inspect --format='{{.LogPath}}' <container_id> | xargs du -h

Example:

docker inspect --format='{{.LogPath}}' d9576ca2924e | xargs du -h

38M     /var/lib/docker/containers/d9576ca2924e2111d53a415f62cdbdc57e7d79571c8dd4bd9c2b8dbaf13f46df/d9576ca2924e2111d53a415f62cdbdc57e7d79571c8dd4bd9c2b8dbaf13f46df-json.log

References
#