Containerized software is becoming ever more popular among service providers and administrators, with a traditional focus on docker. With the release of podman, operators can now transition to a more open and advanced tool for this job.
A bit of history
Containerization had been part of the linux kernel since the early 2000s in the form of LXC containers, using cgroups to limit cpu/memory resources and namespaces to isolate processes from one another.
Docker was the first widely adopted tool to enhance this ecosystem by combining them with overlay filesystems, networking capabilities and adding components like volumes and container images, all accessible through a simple CLI tool. Especially the ability to quickly build, share and run images across multiple hosts quickly gained traction among service providers that previously had to rely on more resource-heavy virtualization.
With the emergence of more container runtimes and advanced container environments like kubernetes, the OCI (Open Container Initiative) was born, which defines standards for container runtimes and container images. This standardization allowed multiple alternative container runtimes to flourish, like CRI-O, containerd and Podman.
Docker vs Podman
For users, podman seems very similar to docker, but it differs internally: Where docker brings it's own runtime and builder, podman is only the tool for interacting with containers, relying on systemd as it's container runtime backbone and buildah to build container images. The ecosystem aims to be backward-compatible with docker (even offering optional packages which provide an alias for the docker and docker-compose commands), but sticks to the OCI container standards instead of docker's proprietary design. In addition to the functionality found in docker, podman offers several advantages, such as automatic container image updates, rootless containers and support for pods.
Installation
Podman is available from official software repositories for most distributions, needing no third-party repositories or manual downloads.
Debian-based distributions (debian, ubuntu, mint etc):
sudo apt install podman podman-compose
RHEL-based distributions (RHEL, Fedora, RockyLinux, AlmaLinux):
sudo dnf install podman podman-compose
The installation includes everything a container environment needs to function, including support for docker-compose.yml files out of the box.
Basic operation
Podman uses as much of the docker syntax as possible: podman ps will list containers, podman run can launch a new one and podman rm remove it again. Practically all docker commands and flags are supported in podman, for example:
docker run --rm -it ubuntu:latest /bin/bash
translates to the exact same in podman:
podman run --rm -it ubuntu:latest /bin/bash
While the commands will look the same, there are some nuanced differences between the two container management tools. For example, running docker rm on a running container will throw an error, while podman rm will happily stop and delete active containers.
Configure podman as a drop-in replacement for docker
Since most scripts in use today will expect the container management command to be docker, podman offers a drop-in replacement package to provide the docker command but treat it as an alias for podman instead.
Install it on debian-like systems with
sudo apt install podman-docker
or for RHEL family:
sudo dnf install podman-docker
After installation, the docker command should now be available and behave mostly as expected, but use the podman command under the hood.
Using this drop-in replacement will display a warning message about the system not really being docker when running commands:
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
As the message says, creating the file /etc/containers/nodocker will silence that warning permanently:
sudo touch /etc/containers/nodocker
The podman daemon expects images to have fully qualified names (including a host), which you may not be familiar with when coming from docker. For example trying to pull httpd as you would in docker:
podman pull httpd
Results in an error message:
Error: short-name "httpd" did not resolve to an alias and no unqualified-search registries are defined in "/etc/containers/registries.conf"
This error can be fixed by either prefixing the image with docker.io/ or setting it as a global fallback registry in /etc/containers/registries.conf:
echo 'unqualified-search-registries = ["docker.io"]' | sudo tee -a /etc/containers/registries.conf
Now shorthand names for container images should correctly use the docker.io registry.
Finally, remember to prefix all podman commands with sudo to have the same effect as docker did, because running it as a different user would create rootless containers, which may not be what you expect.
Rootless containers
Podman supports containers that don't require superuser privileges, instead running under the user who started them. This is different from docker, and often confuses newcomers. Assume you start a container with sudo privileges:
sudo podman run -d docker.io/nginx
And then check running containers without sudo:
podman ps
You may be surprised to not see any running containers. The reason is simple: using sudo started the container for the root user, while checking container status as a different user would only list their containers, not all containers on the system.
Rootless containers provide many advantages over docker's traditional approach, mainly better control over filesystem mounts and port forwarding. When running a command that would require elevated privileges as a non-root user:
podman run -p 80:80 -d nginx
Podman will reject the command, citing permission errors:
Error: rootlessport cannot expose privileged port 80, you can add 'net.ipv4.ip_unprivileged_port_start=80' to /etc/sysctl.conf (currently 1024), or choose a larger port number (>= 1024): listen tcp 0.0.0.0:80: bind: permission denied
This enables containerized environments to expose minimal access the underlying host system when running container workloads, resulting in a much more robust and secure system.
Start rootless containers at boot
Podman intentionally doesn't provide a daemon process like docker, which has some consequences with container restart policies. While containers started with sudo run and behave as their docker counterparts (correctly restarting after boot if they restart policy demands it), rootless containers have no daemon that could start them at boot, so they don't. Given this example:
podman run --restart=always --name=mywebserver -d nginx
This container would not restart when rebooting the system. To fix it, you need two things: First, enable lingering for the systemd user session, to allow the user's daemons to keep running even after the user session ends:
sudo loginctl enable-linger $UID
This needs to be done only once per user. The next step is to generate and enable a systemd user service for the container, for example:
mkdir -p $HOME/.config/systemd/user/
cd $HOME/.config/systemd/user
podman generate systemd --name mywebserver --files
systemctl --user enable --now container-mywebserver.service
This needs to be done for every rootless container that should restart after boot!. The first two lines simply ensure the user service config directory exists and is the current working directory. The next one generates a systemd .service file from the container named mywebserver, and the last command enables and immediately starts the new user service.
When generating user service files for containers that already have a restart policy, you may encounter another warning:
WARN[0000] Container 2596f2e50ac8a30118ecc55c77c28ccaeff520f8cea981197f2b1fc18f0fdaae has restart policy "always" which can lead to issues on shutdown: consider recreating the container without a restart policy and use systemd's restart mechanism instead
The mentioned problem can be easily resolved by not providing the --restart flag to podman when creating the container, and instead setting a restart policy for the systemd user service file:
podman run --name myweb -d nginx # no --restart flag here
podman generate systemd --name myweb --files --restart-policy=always # set restart policy here instead
Now systemd solely handles container restarts, preventing conflicts of timeouts when shutting down the system.
Fix systemd dbus issues after installing podman
When installing headless systems for containerized applications, you may log in over SSH, then install podman and run container commands. For root containers this works just fine, but rootless containers run into a little issue here, returning this warning for all podman commands:
WARN[0000] The cgroupv2 manager is set to systemd but there is no systemd user session available
WARN[0000] For using systemd, you may need to login using an user session
WARN[0000] Alternatively, you can enable lingering with: `loginctl enable-linger 1000` (possibly as root)
WARN[0000] Falling back to --cgroup-manager=cgroupfs
WARN[0000] Failed to add pause process to systemd sandbox cgroup: exec: "dbus-launch": executable file not found in $PATH
The problem comes from the order of steps: we started our login session first, then installed podman, which installed a dbus user session package as a dependency. This would normally start the dbus service for a user when they start a session (aka log in), but we were already past that step, so dbus isn't running for us. We can start it manually with
systemctl --user start dbus
Alternatively, a reboot would also fix the issue.
You can potentially trigger this issue by logging into a user account without starting a full session, for example by running
sudo su - my_user
This will also skip the dbus user service, so you either have to start it manually again or log out and log in as my_user directly, without using su.
Automatic container updates
Keeping container images up to date is a common task in container environments, and podman brings a built-in solution for it out of the box: the io.containers.autoupdate label. This label can have one of two values: registry will update to the latest container image available from it's remote registry, while local will only update to the most recent image on the local machine (i.e. the most recent pulled image version).
You can start a container with the auto-update label:
podman run -d --label "io.containers.autoupdate=registry" docker.io/nginx:latest
The container's image, docker.io/nginx:latest in this case, will automatically be updated when a newer version is available. You can force it to check and update now using
podman auto-update
the command will update all containers with the auto-update label set, if a newer image version is available for them. Podman also supports running automatic updates in the background periodically using a systemd service, which you can enable:
sudo systemctl enable podman-auto-update.service
Now the service will run the podman auto-update command every day at midnight. You can change the interval by editing the podman-auto-update.timer systemd unit.
Pods
Another major difference between podmand and docker is podman's native support for pods. A pod is kind of like a namespace for multiple containers, with isolated internal networking and it's own port forwarding rules for all containers inside. Pods were originally introduced by kubernetes to group tightly-integrated containers together as a single logical unit, for example a wordpress instance and it's MySQL database.
To use pods, you first create a pod and it's desired networking config:
podman pod create --name wordpress-pod -p 8080:80
We now have a pod named wordpress-pod that forwards internal port 80 to port 8080 on the host. Containers inside a pod share a single networking stack so you don't have to specify which container's port inside the pod is forwarded, because only one can listen on port 80 anyway.
Next, start a MySQL container inside the pod:
podman run -d \
--name mysql-container \
--pod wordpress-pod \
-e MYSQL_DATABASE=wordpress \
-e MYSQL_USER=wpuser \
-e MYSQL_PASSWORD=wppassword \
-e MYSQL_ROOT_PASSWORD=rootpassword \
docker.io/mysql:5.7
Then add a wordpress container:
podman run -d \
--name wordpress-container \
--pod wordpress-pod \
-e WORDPRESS_DB_HOST=127.0.0.1 \
-e WORDPRESS_DB_USER=wpuser \
-e WORDPRESS_DB_PASSWORD=wppassword \
-e WORDPRESS_DB_NAME=wordpress \
docker.io/wordpress:latest
Now both containers are running inside the pod, and we can use the pod to treat them as a single logical unit (a pseudo-container, if you will). Note how the wordpress container is using 127.0.0.1 (aka localhost) as the mysql database host. The containers behave is if they were running inside a single container, simplifying internal networking significantly.
Try running
podman pod logs wordpress-pod
This prints the logs of both the wordpress and mysql containers inside the pod. You can think if it as a more tightly integrated form of docker-compose.
Adding the --pod flag will add pod ids and names to the list of running containers:
podman ps --pod
The output will be very wide, so we're not displaying it here, but you see that the list still shows both containers individually, but belonging to the same pod. You can still interact with each individual container as normal, for example using exec to get an interactive shell into one of them.
Since pods are a concept directly used by kubernetes, they are a good choice for testing pod environments when debugging larger kubernetes deployments or developing a small portion of a larger infrastructure intended for a kubernetes cluster.
Podman can generate a complete kubernetes manifest from a locally running pod, enabling developers to easily build a configuration locally, then deploy it to a cluster all at once. For our sample pod, this would look file:
podman generate kube wordpress-pod > pod.yaml
Now we have a local file pod.yaml with the entire pod definition read to use for kubernetes. It could be adjusted for the new environment, or deployed directly with something like kubectl apply -f pod.yaml.