Some time ago I wrote an article on how to test deployment of Docker containers with Ansible using Molecule. In addition I merrily went ahead and used my own advice and initially I weren’t able to discover any flaws. After some time low disk-space warnings began appearing. Using NCDU, I tried to determine what was filling up my hard-disk but could not quite pinpoint the problem. Finally the computer became unusable and I had no choice but to find the true cause of the problem.

Partial spoiler: The problem was not caused by Ansible, Molecule nor Docker.

Reproduce

To reproduce the problem I took the example from my previous article and made the following modifications:

Replace the contents of the converge.yml file located in the directory dockerdeploy/roles/deploy/molecule/default/ with:

---
- name: Converge
  hosts: all
  tasks:
    - name: "Deploy Docker containers"
      ansible.builtin.include_role:
        name: "deploy"
      vars:
        deployments:
          - {
            docker_image: "elasticsearch:8.6.1",
            container_name: "elasticsearch",
          }

Replace the contents of the verify.yml file located in dockerdeploy/roles/deploy/molecule/default/ with:

---
- name: Verify
  hosts: all
  gather_facts: false
  tasks:
    - name: Verify
      ansible.builtin.debug:
        msg: "Verify does nothing"

The verify_container_logs.yml file will no longer be used and may be deleted or left as desired.

Cause

After quite some investigation I discovered that each time I ran my Molecule test, a Docker volume was created. Any Docker images used in the test were pulled and stored on such a Docker volume. The following illustrates what happens when the test is run using “molecule converge”:

Before Molecule converge:

$ docker system df
TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          2         1         1.049GB   722.9MB (68%)
Containers      1         1         0B        0B
Local Volumes   0         0         0B        0B
Build Cache     0         0         0B        0B

After Molecule converge:

$ docker system df
TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          2         2         1.049GB   0B (0%)
Containers      2         2         298.3MB   0B (0%)
Local Volumes   1         1         1.412GB   0B (0%)
Build Cache     0         0         0B        0B

An attempt were made to clean up after the test using “molecule destroy”.
After Molecule destroy:

$ docker system df
TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          2         1         1.049GB   722.9MB (68%)
Containers      1         1         0B        0B
Local Volumes   1         0         1.412GB   1.412GB (100%)
Build Cache     0         0         0B        0B

While the disk-space used by local volumes did become reclaimable, it was not released.

Solution

In order to reclaim the disk-space used by the Docker volume created when running the Molecule test, the command “docker volume prune” can be used.

In the directory dockerdeploy/roles/deploy/molecule/default/, create a file named “cleanup.yml” with the following contents:

---
- name: Cleanup after Molecule test
  hosts: 127.0.0.1
  connection: local
  tasks:
    - name: Clean up Docker volumes
      ansible.builtin.shell:
        cmd: "docker volume prune --force"

Note that the flag “–force” has been added to the “docker volume prune” command. This was done in order to avoid being prompted during the cleanup.

If we now run the test using “molecule test” we will be able to see that there will be no reclaimable diskspace used by Docker volumes after Molecule has completed executing the test.

Final Words

I have not tested whether the problem also occurs with other Molecule drivers such as the Podman driver. If you have, please let me know in the comments!

Happy coding!