Skip to content

Minikube delete can quietly leave most data on the host on linux when the driver is docker #15222

Open
@allanbarklie

Description

@allanbarklie

What Happened?

Observations

Without carefully matched file permissions minikube delete quietly leaves most data on the host on linux when the driver is docker.

When running as a non root user on linux with the driver set to docker minikube start is successful and most data is stored under the folder
/var/lib/docker/volumes/

When minikube delete is called however there appears to be a permissions issue that can result in the silent failure to delete most data.
When this happens all this data is left intact (full data of containerized host - so all images/volumes etc).

I believe the issue occurs when there is a mismatch between the user that originally called start (and thus created the /var/lib/docker/volumes/minikube folder) and the user that calls delete

Running with sudo does delete the remaining data but this has the side effect that purge then looks in the wrong home folder (root) rather than the current user for config data to clean

Note that in the successful case the user is additionally reminded that (the relatively small) "Kicbase images have not been deleted".

Expectations

Expect either for data to be deleted or for information to be displayed informing the user that most data could not be deleted.
To me the biggest issue is this silent failure, this can lead to a system's hard drive filling up and the system becoming unresponsive.

Expect not to have to call delete twice (once with and once without sudo) to delete and purge all data.

This was seen on a CENTOS 7 system.

Some anonymised logs - first we list the data, then try minikube delete. Notice that some 30GB of data remains under /var/lib/docker/volumes/ and yet no problems are reported. Then I re-run using sudo. This time 30GB is removed. Note that a different home folder is then purged.

[myuseraccount@myserver myfolder]$ sudo du -h / | sort -h -r | head -n 20
53G /
37G /var
34G /var/lib/docker
34G /var/lib
29G /var/lib/docker/volumes/minikube/_data/lib
29G /var/lib/docker/volumes/minikube/_data
29G /var/lib/docker/volumes/minikube
29G /var/lib/docker/volumes
28G /var/lib/docker/volumes/minikube/_data/lib/docker/overlay2
28G /var/lib/docker/volumes/minikube/_data/lib/docker
10G /data/jenkins
10G /data
9.9G /data/jenkins/workspace
4.7G /var/lib/docker/overlay2
2.8G /usr
2.0G /home
...

[myuseraccount@myserver myfolder]$ ./minikube delete --all --purge
🔥 Successfully deleted all profiles
💀 Successfully purged minikube directory located at - [/home/myuseraccount@MYDOMAIN/.minikube]

[myuseraccount@myserver myfolder]$ sudo du -h / | sort -h -r | head -n 20
53G /
37G /var
34G /var/lib/docker
34G /var/lib
29G /var/lib/docker/volumes/minikube/_data/lib
29G /var/lib/docker/volumes/minikube/_data
29G /var/lib/docker/volumes/minikube
29G /var/lib/docker/volumes
28G /var/lib/docker/volumes/minikube/_data/lib/docker/overlay2
28G /var/lib/docker/volumes/minikube/_data/lib/docker
10G /data/jenkins
10G /data
9.9G /data/jenkins/workspace
4.7G /var/lib/docker/overlay2
2.8G /usr
2.0G /home
...

[myuseraccount@myserver myfolder]$ sudo ./minikube delete --all --purge
🔥 Successfully deleted all profiles
💀 Successfully purged minikube directory located at - [/root/.minikube]
📌 Kicbase images have not been deleted. To delete images run:
▪ docker rmi gcr.io/k8s-minikube/kicbase:v0.0.34

[myuseraccount@myserver myfolder]$ sudo du -h / | sort -h -r | head -n 20
23G /
10G /data/jenkins
10G /data
9.9G /data/jenkins/workspace
6.8G /var
3.9G /var/lib
3.7G /var/lib/docker
3.6G /var/lib/docker/overlay2
2.8G /usr
2.0G /home
...

Attach the log file

nologfile.txt

Operating System

No response

Driver

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    co/docker-driverIssues related to kubernetes in containerhelp wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.kind/bugCategorizes issue or PR as related to a bug.priority/backlogHigher priority than priority/awaiting-more-evidence.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions