hckrnws
I have a RPi 2 that’s more than enough for my home server needs and I’ve tried to migrate from docker to podman, it simply failed miserably with weird errors, on my laptop everything worked fine, so maybe that’s an issue with Arch ARM or the 32-bit version of it, but really, I wish they could abstract away quadlet, service files generation and so on.
If you install the `podman-docker` package, you get compatibility with docker commands. If you have errors during runtime, it's probably something in your container/config that's assuming a docker socket/runtime. Some of which can be remedied by pointing `DOCKER_HOST` to your podman socket, etc.
The problem wasn’t running the containers with plain podman, they worked. Auto restart is something podman delegates to quaqlet/systemd, I tried to migrate my docker-compose files to quadlet’s config, the containers were up, supposedly listening to the bound ports but connecting simply didn’t work. I tried rootless, slirp4netns and pasta, nothing worked, then I gave up, some other day I might try again.
Running a very similar setup here, also have issues with networking. Pasta worked, but has bugs and replied UDP packets on the wrong ports. The pasta version from debian packages is too old. Also, I tried making the networking work but now I just slap --network=host on everything and call it a day--works perfectly.
What is the killer feature that will make me want to switch from Docker Compose to Podman Quadlets?
Podman quadlet supports "Socket activation of containers" https://github.com/containers/podman/blob/main/docs/tutorial... This allows you to run a network server with `Network=none` (--network=none). If the server would be compromised, the intruder would not have the privileges to use the compromised server as a spam bot. There are other advantages, such as support for preserved source IP address and better performance when running a container with rootless Podman + Pasta in a custom network.
What's old is new again. That's effectively how inetd worked circa 1986. The inetd daemon had some serious security vulnerabilities so the world move away from using "socket activated daemons" to having always listening services (performance reasons as well).
I never understood the use case for socket activation - is someone really running a web server that mixed workloads, long periods with no network traffic you'd rather prioritize something else, and a web server that's so resource intensive when not handling events it makes sense to stop it? Maybe desktop computers?
The security aspect is something new to me and I'm not sure if that applies to inetd/systemd socket services or if it's specifically a container thing.
Does anyone have more info on use cases for this?
> the security aspect
It’s not a systemd-specific thing, but systemd makes it relatively easy to drop privileges (like network in this case), whilst also allowing socket-activated services to be configured easily. You can probably achieve the same thing with inetd + network namespaces (I think this is what systemd uses under the hood)
You can use the podman option `--network=none` together with the systemd directive `RestrictAddressFamilies=`
I wrote a demo: https://www.redhat.com/en/blog/podman-systemd-limit-access
Podman will then not have the privilege to pull the container image, but a web server container can still serve the internet with socket activation.
What's the use case for that? Multitenant server web hosting where customers provide containers and you want to lock them down I guess? Mostly SaaS/PaaS?
I did it out of pure interest, just to explore ways of locking down a web server.
Oh, fair enough! It is very cool, FWIW.
A public facing web server, I doubt it. But for a private one it can make a lot of sense - you are probably only using N services at a time, where N is the number of users.
As for what can be so resource intensive that it's worth to wait for startup time instead of running everything at the same time - a bunch of specialized LLMs is the obvious example. Or maybe you're a hobbyist cramming a hundred services into a tiny computer.
On-demand applications. But the it should shut itself down after a while too.
It’s more useful for applications that keep open a connection for a while instead of stateless request/response architecture
inetd supported "socket activation" using the "wait" directive, where inetd would listen on the socket and then hand off the listening socket when there was activity as fd 0 where the server would need to call accept, and could continue to call accept for new connections, or exit when all clients were handled, and inetd would respawn when there was new pending connection on the listening socket.
That's neat. Does it require 1 connection = 1 process to work? I don't see how you can have a long running server with this feature.
No, the init process hands over the listener FD allowing the server to accept() connections.
You can also do 1 connection = 1 process, though, but it's absolutely not required nor particular common these days.
Comment was deleted :(
I prefer quadlet for 2 reasons:
1. Podman is simpler than Docker. There is no long-running daemon. Rootless is default.
2. Quadlets can be managed as systemd services, giving me the same tools to manage and view logs for system daemons and containers.
Quadlets have been especially nice for bundling up an AI app I wrote as a cloud-init file, making it easy to deploy the hardware, software and models as one artifact.
Podman seems to have lower memory overhead than Docker. I assume that is a consequence of your point 1
quadlets == systemd which requires root to run. this is NOT the same thing as "systemd cant run non root containers". OBVIOUSLY it can, just as docker can run non root containers.
Making systemd a necessary dependency to run > 1 container kinda negates many of the the nice advantages that podman has of not requiring root.
podman compose doesnt require root and would serve as a substitute but it's a very neglected piece of software.
You can do non-root systemd units, including Quadlets. See <https://docs.podman.io/en/latest/markdown/podman-systemd.uni...> under "Podman rootless unit search path."
I recently started making the switch from docker (and docker compose) to using podman and quadlet, but holy crap is the documentation for podman quadlets a big f-you wall-of-text mandoc that would make Torvalds proud. I've read thru that and am still not quite sure of how to get from point A to point B.
To replace a single docker compose file, sounds like one needs to manually create a number of .container, .volume, .network, .kube files correctly so systemd can spin up a container pod? Is that what I'm reading? Is there nothing that can generate that from a docker-compose.yml?
I agree. That documentation really needs some love. But if you see the discussions on github issues about quadlet features a common theme is maintainers dismissing requests because "that shouldn't be done in production" or "that won't scale". It seems they can't wrap their head around people wanting to do simple things or someone doing things by themselves at home and not for work at a big company or corporation, and that reflects on that documentation.
Working for one myself, which does have a support contract wit Red Hat, I kinda get where they're coming from--if they make it easy to shoot yourself in the foot, dumb people shoot themselves in the foot in production and they have to fix the mess later. But for that they could have a sanctioned build for clients and a community build for everybody else, just like they have Fedora and RHEL.
I've used Podlet <https://github.com/containers/podlet> somewhat successfully for this.
you can run docker containers without them requiring root too.
systemd itself is a root service. it shouldnt be a necessary dependency to run > 1 containers without root. somehow it is.
systemd user units can be run by non-root users.
not the point as i mentioned above.
systemd itself requires root.
How were you planning to run podman compose without an init process running as root?
what else would you like to bundle in the init process? docker compose as well? maybe kubernetes too. a webserver? a word processor perhaps? maybe an email client?
>what else would you like to bundle in the init process? docker compose as well?
You can complain that you don't liken systemd's design or that it does too much (very overplayed complaint, but ok).
But that's an orthogonal point to the initial complaint you've made that it's somehow bad that this requires systemd to run.
That complain is moot, since you'd be running systemd anyway, with or without those containers. And it's double moot, because you can run > 1 containers (with podman) without root too.
(It's also wrong that systemd was added container compose capabilities - podman is what translates things to systemd "speak")
>You can complain that you don't liken systemd's design or that it does too much (very overplayed complaint, but ok).
It's not at all orthogonal. Making the "default" way to run > 1 containers together require the init process is fucking stupid.
Similar to how requiring ROOT to run containers was a stupid design decision made by docker.
This decision to make quadlets the "default podman orchestrator" and to double down on it relegates podman from being "a better docker" to just "docker just with different design mistakes".
>That complain is moot, since you'd be running systemd anyway
false. systemd is not the only pid 1 in existence (much as it likes to pretend it is). when you run a container inside a container there also isn't a systemd.
>And it's double moot, because you can run > 1 containers (with podman) without root too.
except there is NO good orchestration system for doing that. podman compose is a steaming pile of shit. quadlets requires systemd which requires root. docker compose requires a root service. you can run podman compose inside a container but not quadlets.
might as well just use docker at this rate.
>That complain is moot
Your comment comprehensively missed my point 3 times. It's triple moot. It would have been better left unmade.
Podman compose isn't bundled in the init process.
Yep, but have you used it? It's crap. It implements about half of what docker compose does.
I'd stop complaining about quadlets being a hunk of crap if podman compose were decently maintained.
I'm contesting your claim that podman compose is bundled with systemd.
I never made this claim.
So what was this about? What's the thing that the "else" refers to?
> what else would you like to bundle in the init process? docker compose as well? maybe kubernetes too. a webserver? a word processor perhaps? maybe an email client?
systemd is the init process, the Linux kernel non-optionally runs the init process as root
yeah thats what i said.
and that particular init process did way more than any init process ever should even before somebody had the bright idea to add "docker compose substitite" to its ever growing list of responsibilities.
you could put a word processor and games in their too if you really wanted. is that a good idea? ill leave that for the reader's judgment.
systemd just provides the feature to use a custom external application to configure a service based on a declarative spec, which podman uses to create actual systemd services from a declarative container spec.
From the podman docs:
> Podman supports building and starting containers (and creating volumes) via systemd by using a systemd generator.
Putting aside all the other issues one may have with systemd, this feels like a decent feature for a service manager to have (custom generation of service specifications).
> bright idea to add “Docker compose substitute”
Why is this so revolutionary? Docker-compose is just a service manager for containers. Systemd is a service manager. Systemd allowing podman to give it “container” features seems pretty reasonable.
> before somebody had the bright idea to add "docker compose substitite" to its ever growing list of responsibilities.
systemd itself isn’t acting as a docker-compose substitute. Podman simply translates unit files containing docker-esque configuration (image name, volumes, etc.) into plain systemd unit files that contain (among other things) an ExecStart line that starts the container with the proper arguments.
Installing packages (like podman or moby/docker) using dnf and apt requires root as well, so I'm not sure what your point is.
making systemd - a root service - a necessary dependency in order to orchestrate > 1 nonroot containers is both unnecessary and bad architecture.
It was a shitty decision that renders it just "a less popular docker" and not "a better docker".
Podman doesn't have a dependency on systemd. e.g. it is packaged in Void Linux.
Podman has a better architecture than Docker in that it can easily run on a non-privileged user.
Quadlet (aka podman-systemd.unit) is a podman-systemd integration which can make it easy to launch and orchestrate podman containers via systemd. You can get all if the systemd dependency handling, require other units to run after a container finishes, and all sorts of other useful things. Systemd "user" units (systemctl --user) also works here with the containers running as a non-privileged user in a non-root systemd context.
Just to be clear, Quadlet is just an integration and you can still run podman without it. You can still run podman on non-systemd systems as well.
And you can use podman to run multiple containers together (as a Pod). With or without systemd.
>Podman doesn't have a dependency on systemd
Just to be clear we're talking about QUADLETS, red hat's recommended way to orchestrate containers.
>Just to be clear, Quadlet is just an integration and you can still run podman without it.
Just to be clear, nobody was unclear about that.
It is, just to be clear, red hat's recommended way to orchestrate podman containers despite having this nasty dependency analogous to the one docker has on a root service.
Hope that helps.
>Just to be clear, nobody was unclear about that
Oh, you were quite unclear. Also wrong in saying you need systemd with podman to orchestrate multiple containers without root.
>It is, just to be clear, red hat's recommended way to orchestrate podman containers despite having this nasty dependency analogous to the one docker has on a root service.
It's not "red hat's recommended way to orchestrate podman containers" in general. It's "red hat's recommended way to orchestrate containers on top of systemd", that its whole point.
Nothing nasty about it either, you'd already be running systemd on your redhat system (and many non red-hat ones).
>Also wrong in saying you need systemd with podman to orchestrate multiple containers without root
I explicitly said thay it wasnt needed and that there werent other ways just that it was the recommended way.
>It's not "red hat's recommended way to orchestrate podman containers
It is.
wut? Containers need an operating system.
systemd runs on a linux host, the rootless container runs on a linux host, controlled by `systemctl --user ...`.
But you don't understand, it also needs an operating system therefore it is vulnerable, because things on it run as root! /s
systemctl --user ...That still depends on systemd, the most privileged deamon in many distros
But it is fun to see the marketing 360
Cool, so no PID Eins. I have no shares in systemd, so fine. What do you propose?
The biggest one is probably that podman runs as a user and doesn't need suid normally. So you can run services and have more assurances that container breaches won't give someone root on your system.
I really like the user namespace handling `--user-ns=keep-id`. It makes it easy for me to create a new Linux user and then have that user run some container and have bind mounts, etc just work correctly. It is the least fuss way I have found of running little services that need access to the host filesystem.
https://docs.podman.io/en/latest/markdown/podman-run.1.html#...
For local development I found no advantages, if anything I found it a little less convenient.
For servers where you don’t need the complexities of Kubernetes etc, using Quadlets is nice as you can manage containers as regular systemd services and no Docker daemon running as root.
For a docker-compose replacement you should probably look at the 'podman kube' support. That supports a subset of the kubernetes API that roughly matches the docker-compose features.
Then for deployment to a Kubernetes cluster you can reuse your podman kube yaml.
For deployment to a single machine where the full Kubernetes is overkill, you can use the podman quadlet support. Quadlets support a "[Kube]" section where you can point to the yaml file, so you don't have to write all your port and volume etc mappings again in a slightly different syntax.
And why are they called quadlets?
Quadlets came up recently:
Quadlet: Running Podman containers under systemd (mo8it.com) | 298 points | 68 comments | https://news.ycombinator.com/item?id=43456934
Where do quadlets fit in the container management landscape?
Running individual services that would have been RPMs? Docker compose? K8s?
> Running individual services that would have been RPMs?
Essentially this. Rather than adding anything on top of systemd (which is almost certainly running anyway) quadlets are a way of turning containers into systemd units, and directing systemd to run them
I think it is the lightest weight, but still convenient, way of running a container as a daemon, with all the pros and cons that implies
I run container sets (like you'd probably run with docker-compose) as systemd services using this.
How do you manage and deploy it? Git a systemd directory?
It's my homeassistant setup. I run mosquitto, zigbee2mqtt, and homeassistant in it (everything they connect to is on the network, so no need to do USB access mumbo jumbo). I don't "manage" it; I have a yaml file (generated from running containers with "podman kube generate") in a directory along with volumes those containers need, and a file in /etc/containers/systemd pointing to that yaml file. That's it.
And once all peripherals are seen and run correctly, I don't "manage" it as long as it works.
It's just a ~/.config/containers/systemd/ folder for rootless containers, I stuff the whole thing in a git repo
Ansible has support for quadlet creation in the containers.podman collection
quadlets is an answer to the question "but what if we also threw a docker compose substitute into the systemd kitchen sink"?
I started looking into podman quadlets on the weekend. All because docker & nft (nftables firewall) don't play nice together.
I have my finely crafted nftables ruleset that I want to keep. Docker is going to abuse the crap out of my ruleset/make me jump through extra hoops to have it work.
Having the containers operate under systemd looks fine (ignoring the fact that it is systemd, I guess I have finally accepted systemd is here to stay).
I was interested in using traefik & docker labelling to have ingress sorted out and I believe this will all play nicely enough to replace docker-compose on a single server.
If you are an individual dev and coming from the docker swarm / compose world, podman will drive you mad.
If you are a sysadmin / platform engineer / etc type familiar with kubernetes / k3s, podman has some pros and cons about it that make working in that ecosystem (systems, platforms) much nicer to manage.
What we are missing right now is a bridge between the two that podman-compose doesn't exactly fulfill for a lot of people.
This made me remember Fleet[0], which I used in the past on CoreOS. It was a very good abstraction for my simple cluster. It felt as robust as a managed cloud service. It's a shame it didn't catch on (and also what happened to CoreOS).
> (and also what happened to CoreOS)
Arguably worse, IMHO, is that the name lives on but does something wholly different, polluting search results
Anyway, I wanted to plug Flatcar which forked CoreOS back when the nonsense started and have been pretty pleasant to work with. They are still misguided about user-data management, and have since been acquired by Microsoft(!), but the code lives on https://github.com/flatcar/Flatcar#readme and https://www.flatcar.org/docs/latest#upgrading-from-coreos-co...
Small note - https://github.com/flatcar/Flatcar has no code in any branches:
> Flatcar project repository for issue tracking, project documentation, etc.
Per the building page https://www.flatcar.org/docs/latest/reference/developer-guid...
the actual code seems to be at https://github.com/flatcar/scripts.git
Yes, apologies, that was poor wording on my part. The "lives on" was meant in the "continuation" sense but I can see how "lives on https:" implied it was in that specific repo. Sorry about that
Hm. I'm actually deploying some CoreOS systems this weekend as a test.
Some help in case you got lost in the terminolgy as I did :
Containers: Encapsulate application and its environment
Pods: Group one or more containers in Kubernetes
Kubernetes: Orchestrates containers across clusters.
Quadlets: Declarative container management with Podman and systemd
Podlet: Tool to generate Quadlet file from existing podman configuration
Podlet is more than that.
It will generate Podman Quadlet files from a Podman command, docker / compose file, or existing object. It can also create kubernetes pod yaml.
I have been trying to use podman with docker containers to avoid the docker license fee, yet to no avail. Wish there was a tutorial. The docker desktop is a nice interface to interact with containers and podman did not have that last time I checked.
edit: Looks like it changed quite a lot. this looks so much nicer
edit2: I still need a tutorial. had so many errors come up. :(
I don't want to dissuade you from Podman Desktop, because Podman is awesome in its own way, but I do hear you about the "uncanny valley" part.
I wanted to offer that you don't have to jump all the way to podman (and Podman Desktop) just to escape the Docker Desktop licensing problem. There are a lot of alternatives which a whole spectrum of "I am CLI savvy or not" interaction experiences
- Rancher Desktop is A++ and carries a friendly license https://rancherdesktop.io/ -> https://github.com/rancher-sandbox/rancher-desktop/blob/v1.1...
- colima (and the lima it wraps) -> https://github.com/abiosoft/colima#readme
- there's one from AWS called Finch <https://github.com/runfinch/finch> that some of my colleagues like better than colima; YMMV <https://aws.amazon.com/blogs/opensource/introducing-finch-an...>
Comment was deleted :(
If you’re on a Mac, I highly recommend OrbStack. Perfect Docker compatibility so far, actually better Rosetta support than Docker itself.
What OS are you running it on?
Is this essentially a compose file in systemd format? Or is there some other complexity that I’m missing
Yes and no.
This is more a bridge between system level container orchestration and kubernetes.
Can I use podman desktop to manage my quadlets on a server over ssh? That would be awesome.
I would say yes, but never tried:
1. enable remote connection in podman desktop (https://podman-desktop.io/docs/podman/podman-remote)
2. try the quadlet extensions (https://podman-desktop.io/blog/podman-quadlet)
I am curious to have feedback if you give it a try (https://github.com/podman-desktop/extension-podman-quadlet/i...)
Thanks for sharing this. For some reason I haven't heard of Podman Quadlets until this post. I can think of a few places where Quadlets would work better than Kubernetes or Compose. I'll give them a tinker.
Yeah I recently put together a home NAS and was dismayed that I had to use Docker Compose. (There's podman-compose but it's... incomplete).
Immediately after I finished the project I discovered the Quadlet thing!
Seems like the userbase is there, the technology is there, and now the two just need to be put together with some awareness.
Comment was deleted :(
Crafted by Rajat
Source Code