on
Docker-compose to podman
I migrated my services from docker-compose files to podman1.
Why? Essentially because it was easier getting rootless mode working with podman.
Actually, I initially tried to get docker to work in rootless mode2 and in user namespace isolation mode3, failing in both tries. More precisely:
For docker rootless mode, installation procedure is not really smooth with archlinux arm (I had to edit the installation script). Even after a successfull installation, I ended up with segfault, from this point I lost confidence in getting it to work correcly.
For user namespace isolation mode, the issue lied in mounting volumes which failed even with the rights correcly setup4.
I am not an expert in container technology nor in linux namespace. Maybe I overlooked something important, thus the issues I faced.
From this point I decided to let podman a chance.
Road to podman
Podman is an alternative to docker, it offers similar features to docker but it lacks docker-compose niceities… This is disappointing since alot of my self-hosted service uses docker-compose features extensively.
However, podman has a concept of pod5 as in kubernetes. The basic idea behind a pod is that it holds several container in a tightly coupled environment, the containers in a pod shares the same resources and are co-located.
For exemple, two containers inside the same pod can reference themselves through localhost
since it shares the same network namespace6 7.
Although not the same feature set than docker-compose. The concept of pod is sufficient for encapsulating software stacks in the context of a single machine self hosted environment.
Additionnaly, Podman has been thought to integrate well with systemd
and offers tools to generate systemd’s units. This can be used to replace the basic orchestrating abilities of docker-compose.
Docker feature | Equivalent in podman’s world |
---|---|
docker build | buildah bud |
docker run | podman run |
docker-compose | podman’s pods and systemd |
Exemple: migrating docker-compose of miniflux8 service
We’ll see the migration more in details taking miniflux as an example.
Create miniflux user and setting it up
Since we are going full ret… rootless, we will be assigning dedicated user for the miniflux service.
# We will be assigning uid 100000 to miniflux user
useradd -m -u 100000 miniflux
groupmod -g 100000 miniflux
passwd miniflux
loginctl enable-linger miniflux # Mandatory for miniflux being able to launch systemd services without having to login manually
# see podman rootless mode documentation to understand subuids
echo "miniflux:100000:65536" \
| sudo tee -a /etc/subuid \
| sudo tee -a /etc/subgid
runuser -l miniflux -c "mkdir -p ~/.config/systemd/user" # Create folder holding definition of systemd unit
sudo machinectl login # login as miniflux
We are using sudo machinectl login
to log as miniflux since a simple su - miniflux
will not set environment correctly and you won’t be able to use user instance of systemd.
A quick look at existing docker-compose service definition
version: '2'
services:
miniflux:
image: miniflux/miniflux:2.0.25
mem_limit: 128m
restart: always
env_file:
- ./miniflux.secrets.env
ports:
- 8680:8080
depends_on:
- db
db:
image: postgres:13.1-alpine
mem_limit: 96m
restart: always
volumes:
- /mnt/disks/miniflux-db:/var/lib/postgresql/data
env_file:
- ./postgres.secrets.env
Note the exposed port: 8680
and the bind mount for db
data.
Assign correct permission to the bind mount source
Since we will run rootless container. root
inside the container doesn’t correspond to the root
of the host. This has an impact on volumes and bind mounts.
username:uid in container | corresponding uid in host |
---|---|
root:0 | 100000 |
postgres:70 | 100070 |
Actually, in our case, we want to grant correct permissions to /mnt/disks/miniflux-db
allowing db
container to access it.
The db
container runs under postgres
user, which corresponds to uid 100070
in host. Fortunately, we can chown
directly to an uid:
sudo chown -R 100070 /mnt/disks/miniflux-db
Create equivalent services in podman
We will be creating a pod holding miniflux application and the database.
First, the pod:
pod create --name miniflux -p 8680:8080
podman run -d --pod=miniflux --memory 96m --name db -v /mnt/disks/miniflux-db:/var/lib/postgresql/data --env-file ./postgres.env postgres:13.1-alpine
podman run -d --pod=miniflux --memory 128m --name app --env-file ./miniflux.env miniflux/miniflux:2.0.25
Note that the port exposition is defined at pod’s level, port 8680
on the host will correspond to 8080
within the pod.
Environment files being:
DATABASE_URL=postgres://miniflux:password@127.0.0.1/miniflux?sslmode=disable
BASE_URL=https://miniflux.nimamoh.net/
RUN_MIGRATIONS=1
CREATE_ADMIN=1
ADMIN_USERNAME=username
ADMIN_PASSWORD=password
POSTGRES_USER=miniflux
POSTGRES_PASSWORD=password
Note that the app
container address the db
container with 127.0.0.1
since both container are on the same pod.
If everything went OK, you’ll have one pod with two running container and host port 8680
pointing to miniflux
service.
Service lifecycle with systemd
We’ll now use systemd to control miniflux service. Podman is able to generate the systemd’s unit files via command:
$ podman generate systemd --new --name --files --restart-policy no miniflux
/home/miniflux/pod-miniflux.service
/home/miniflux/container-app.service
/home/miniflux/container-db.service
It will give you pod-miniflux.service
, container-db.service
and container-app.service
. Respectively responsible of the pod
, db
and app
lifecycles.
Our service definition lacks the notion of app
being dependent of db
container, in docker-compose we would have used depends_on
9.
We can achieve similar result using After
option of the unit file in container-app.service
. Adding container-db.service
to it will tell systemd that container-app
should be started after container-db
.
# container-app.service
# autogenerated by Podman 3.2.2
# Wed Jun 30 19:11:42 CEST 2021
[Unit]
Description=Podman container-app.service
...
BindsTo=pod-miniflux.service
After=pod-miniflux.service container-db.service
[Service]
Environment=PODMAN_SYSTEMD_UNIT=%n
...
You can edit systemd service files further according to your need. Personnaly I would recommand to commentting the ExecStopPost
step, in order to let the container live in case of failure (to be able to debug it).
Once finished, install, enable and start the services:
$ mv *.service ~/.config/systemd/user
$ systemctl --user daemon-reload && systemctl --user enable --now pod-miniflux.service
Conclusion
Now, we have a regular systemd service running with user privileges corresponding to our miniflux service. It better fits the principle of least privilege principle. Additionnaly, podman have no daemon, which is a plus in term of simplicity.
On the other hand, systemd’s unit files are less expressive than docker-compose files and require some systemd knowledge.
Other techniques exists, like using kubernetes yaml10 (which podman is able to read) or using podman-compose project.
Troubleshooting, gotchas, tips and tricks
Address host from a rootless container
The container can target the host using its IP address, we can expose host ip through an env variable like this
$ podman run -ti --rm --env HOST_IP=(ip -br -4 addr | grep -v 'lo' | awk '{ print $3 }' | grep -P -o "[0-9\.]+(?=/)") --name container ubuntu bash
Local volume with options cannot be used in rootless mode
It can be interesting to customize volume creation, for example create a volume which points to a btrfs subvolume:
podman volume create --driver local --opt type=btrfs --opt device=/dev/sda --opt o=subvol=/data data
However, these kind of volume cannot be mounted in rootless mode, you will get this kind of error trying to do so:
$ podman run -ti --rm --name container -v data:/data ubuntu bash
Error: error mounting volume data for container xxx: cannot mount volumes without root privileges: operation requires root privileges
OCI permission denied
On podman’s run command, I had this error.
Error: container_linux.go:380: starting container process caused: error adding seccomp filter rule for syscall bdflush: permission denied: OCI permission denied
Solution: replace runc by crun
As stated here and here, some version of runc
appears to be too old. Replacing the runtime by crun
fixed the issue.
pacman -S crun
echo 'runtime = "crun"' | sudo tee -a /etc/containers/containers.conf
Knowing so, if you think you found an error or inexact content, you are more than welcome to notify it through comment below ⏬.
Also, if you found the content useful and it helped you, consider leaving a comment too or, better, give me fuel buying me a coffee with the link on the top of the website. 🙏