Compose with Podman Service (to build busybox!)

…where we check the interoperability between the Compose tool and the Podman Service systemd’s socket unit file to build busybox.

Edited on 2024-04-21

Compose concentrates in orchestrating multiple containers in a single host. To do this with k8s, you would need kind, minikube, k3s (does not use virtualization) or similar. It was made to be compatible with other OCI runtimes, such as Podman, which was one of the first to enable rootless containers, and can be setup with compose using the Podman Service’s systemd unit file for unix sockets.

The orchestration tool Compose supports Podman Service through the $DOCKER_HOST environment variable. This makes it possible to run containers with podman but with the benefit of rootless.

Although in a more complex setting you can find some problems about the context. Consider the following directory structure:

; tree -C -L 3
.
├── artifacts
│   ├── bzImage
│   └── initramfs.cpio.gz
├── compose.yml
├── initramfs.sh
├── LICENSE
├── Makefile
├── README.md
├── run.sh
└── utils
    └── busybox
        ├── busybox-1.36.1
        ├── busybox-1.36.1.tar.bz2
        └── Dockerfile

with the Dockerfile:

FROM debian:12.1 AS debian-builder
ARG BUSYBOX_VERSION=1.36.1

RUN <<"EOF"
apt-get update
apt-get install build-essential wget musl-tools tree make -y
EOF

WORKDIR /app/
COPY ["initramfs.sh", "Makefile", "."]
COPY ./scripts/ccr.sh ./scripts/

RUN mkdir -p ./utils/busybox/ && \
    make busybox && \
    cp -r ./utils/busybox/busybox-${BUSYBOX_VERSION}/_install ./_install && \
    rm -rf ./utils/busybox/busybox-*

ENTRYPOINT ["/bin/sh"]

when you try to build directly with an OCI contianer runtime like Podman ; podman build -f=./utils/busybox/Dockerfile.bubo_debug -t=debian_deps:01, you get the following error:

--> 52a0e5afd585
STEP 4/8: WORKDIR /app/
--> bc21992a907d
STEP 5/8: COPY ["initramfs.sh", "Makefile", "."]
Error: building at STEP "COPY initramfs.sh Makefile .": checking on sources under "/mnt/ssd/dataStore/coda-relocate/qemu/eulab-poc/utils/busybox": copier: stat: "/initramfs.sh": no such file or directory
;

this is because of the build context that considers the context of the directory where the Dockerfile is placed, that is, ./utils/busybox/*. You could use shellscript workarounds, but it would become a little bit complicated and not exactly aligned with the way you run containers, where you define infrastructure as code into YAML files or Containerfiles such as Dockerfile. One quick way out is using docker-compose and passing tags for the build context directory AND for the dockerfile, as you can see below.

version: "3.8"
services:
  busybox:
    build:
      context: "."
      dockerfile: "./utils/busybox/Dockerfile"
      tags:
        - "bubo_builder:01"

The above code will build an OCI container using Buildkit, a “build engine” like Buildah, used on Podman and Jib, used in Java (actually any language related to the JVM, gradle or maven like Clojure, Kotlin, Groovy and Scala.) settings. But this would mean that tools like podman couldn’t be used since docker-compose depends on docker. That isn’t entirely true because the tool is actually called Compose. When you invoke “docker compose” it is the Compose binary that is running, and moreso, the tool is made to be compatible with any OCI Container Runtime. That means that Podman can be used if you don’t like the docker-daemon or want to use the skopeo-buildah-podman-umoci stack.

Although depends on systemd, this script, invoking docker-compose setup the Podman Service systemd’s socket unit file. There is no need for --file= flag because of the build context that is being specified into the compose.yml. Without this, the COPY build context would be difficult because there would be the need of a script workaround since the build context for the container runtimes is the same as where the Dockerfile is located, passed by the --file= argument flag.

Btw, the same is true for the context of the .dockerignore. So, the OCI build context affects at least: dockerfile COPY build context and .dockerignore

Post-script

Take into account that switch to Podman Service means the context of the blobs is handled by buildah. Which means the blobs will be saved in /var/tmp/buildah*/ for each container image. If you previously setup soft links or bind mounts entry on fstab to handle the storage to another storage device, you would also need to do the same for these directories or else you could run out of space in the disk where the OS is installed, be it for servers or in local configs. But this is for another post.
When inspecting an image with podman-inspect, docker-inspect or skopeo, you got a key-value json. There, graphRoot is the storage location where the OCI container images are stored. The graphroot path is /var/lib/containers/storage for ROOTFUL containers (processes that run with root privileges). For ROOTLESS containers (userspace processes), by default, the path is $XDG_DATA_HOME/containers/storage, and if $XDG_DATA_HOME is not set, $HOME/.local/share/containers/storage/ is used.
Calling docker-compose, using Podman Service or not, will build the container image with buildkit instead of other container runtimes. Buildkit is a fork from the containerd-1.0 repo,using containerd snapshot (Checkpoint and Restore (C/R) with CRIU (Checkpoint Restore In Userspace), a tool also used by LXC, a different specification than OCI from Canonical), execution and image libraries [^2].

Edited on 2024-04-21

Post-script

References: