How to configure podman in ubuntu server 22 to provide shared image storage among root-less users?

I had asked this question in podman discussion on Github (see the post here) but no luck getting an answer. So I copy the content and turn to ask Ubuntu for help. I hope that in a more general site with more users the comments and answers may solve my problem or lead me to somewhere nearer to the target.

I want to configure podman to provide a shared image storage between root-less users within the host but failed. I don't care if root user works.

user0 is sudo user, user1 and user2 are root-less users. All shells are ssh-ed with corresponding username directly (i.e. not by su).

• user1 output

user1@test:~$ podman imagesREPOSITORY  TAG         IMAGE ID    CREATED     SIZEuser1@test:~$ podman pull busyboxResolving "busybox" using unqualified-search registries (/etc/containers/registries.conf)Trying to pull gitea.cn:666/busybox:latest...Getting image source signaturesCopying blob a10d77880eaf doneCopying config 87ff76f62d doneWriting manifest to image destinationStoring signatures87ff76f62d367950186bde563642e39208c0e2b4afc833b4b3b01b8fef60ae9euser1@test:~$ podman images --log-level=debugINFO[0000] podman filtering at log level debugDEBU[0000] Called images.PersistentPreRunE(podman images --log-level=debug)DEBU[0000] overlay storage already configured with a mount-programDEBU[0000] Merged system config "/usr/share/containers/containers.conf"DEBU[0000] overlay storage already configured with a mount-programDEBU[0000] Using conmon: "/usr/bin/conmon"DEBU[0000] Initializing boltdb state at /var/lib/sharedimages/libpod/bolt_state.dbDEBU[0000] Overriding run root "/run/user/1001" with "/run/user/1001/containers" from databaseDEBU[0000] systemd-logind: Unknown object '/'.DEBU[0000] Using graph driver overlayDEBU[0000] Using graph root /var/lib/sharedimagesDEBU[0000] Using run root /run/user/1001/containersDEBU[0000] Using static dir /var/lib/sharedimages/libpodDEBU[0000] Using tmp dir /run/user/1001/libpod/tmpDEBU[0000] Using volume path /var/lib/sharedimages/volumesDEBU[0000] overlay storage already configured with a mount-programDEBU[0000] Set libpod namespace to ""DEBU[0000] [graphdriver] trying provided driver "overlay"DEBU[0000] cached value indicated that overlay is supportedDEBU[0000] cached value indicated that metacopy is not being usedDEBU[0000] cached value indicated that native-diff is usableDEBU[0000] backingFs=extfs, projectQuotaSupported=false, useNativeDiff=true, usingMetacopy=falseDEBU[0000] Initializing event backend journaldDEBU[0000] configured OCI runtime runc initialization failed: no valid executable found for OCI runtime runc: invalid argumentDEBU[0000] configured OCI runtime kata initialization failed: no valid executable found for OCI runtime kata: invalid argumentDEBU[0000] configured OCI runtime runsc initialization failed: no valid executable found for OCI runtime runsc: invalid argumentDEBU[0000] Using OCI runtime "/usr/bin/crun"INFO[0000] Found CNI network podman (type=bridge) at /home/user1/.config/cni/net.d/87-podman.conflistDEBU[0000] Default CNI network name podman is unchangeableINFO[0000] Setting parallel job count to 37DEBU[0000] parsed reference into "[overlay@/var/lib/sharedimages+/run/user/1001/containers]@87ff76f62d367950186bde563642e39208c0e2b4afc833b4b3b01b8fef60ae9e"DEBU[0000] exporting opaque data as blob "sha256:87ff76f62d367950186bde563642e39208c0e2b4afc833b4b3b01b8fef60ae9e"DEBU[0000] exporting opaque data as blob "sha256:87ff76f62d367950186bde563642e39208c0e2b4afc833b4b3b01b8fef60ae9e"REPOSITORY                                TAG         IMAGE ID      CREATED        SIZEgitea.cn:666/busybox  latest      87ff76f62d36  16 months ago  4.5 MBDEBU[0000] Called images.PersistentPostRunE(podman images --log-level=debug)user1@test:~$

• user2 output

user2@test:~$ podman images --log-level=debugINFO[0000] podman filtering at log level debugDEBU[0000] Called images.PersistentPreRunE(podman images --log-level=debug)DEBU[0000] Merged system config "/usr/share/containers/containers.conf"DEBU[0000] Using conmon: "/usr/bin/conmon"DEBU[0000] Initializing boltdb state at /var/lib/sharedimages/libpod/bolt_state.dbDEBU[0000] Overriding run root "/run/user/1002" with "/run/user/1001/containers" from databaseDEBU[0000] Overriding tmp dir "/run/user/1002/libpod/tmp" with "/run/user/1001/libpod/tmp" from databaseDEBU[0000] systemd-logind: Unknown object '/'.WARN[0000] XDG_RUNTIME_DIR is pointing to a path which is not writable. Most likely podman will fail.DEBU[0000] Using graph driver overlayDEBU[0000] Using graph root /var/lib/sharedimagesDEBU[0000] Using run root /run/user/1001/containersDEBU[0000] Using static dir /var/lib/sharedimages/libpodDEBU[0000] Using tmp dir /run/user/1001/libpod/tmpDEBU[0000] Using volume path /var/lib/sharedimages/volumesDEBU[0000] Set libpod namespace to ""DEBU[0000] Not configuring container storeDEBU[0000] Initializing event backend journaldError: error creating tmpdir: mkdir /run/user/1001/libpod: permission denieduser2@test:~$

The configurations:

• user1

user1@test:~$ more /home/user1/.config/containers/storage.conf[storage]driver = "overlay"runroot = "/run/user/1001"graphroot = "/var/lib/sharedimages"user1@test:~$ iduid=1001(user1) gid=1001(user1) groups=1001(user1)user1@test:~$

• user2

user2@test:~$ more /home/user2/.config/containers/storage.conf[storage]driver = "overlay"runroot = "/run/user/1002"graphroot = "/var/lib/sharedimages"[storage.options]mount_program = "/bin/fuse-overlayfs"user2@test:~$ iduid=1002(user2) gid=1002(user2) groups=1002(user2)

• user0: no configuration

user0@test:/home/user1$ cd /etc/containers/user0@test:/etc/containers$ lslibpod.conf  policy.json  registries.conf  registries.conf.duser0@test:/etc/containers$user0@test:/etc/containers$ iduid=1000(user0) gid=1000(user0) groups=1000(user0),4(adm),24(cdrom),27(sudo),30(dip),46(plugdev),110(lxd)user0@test:/etc/containers$

• The podman version

user1@test:~$ podman versionVersion:      3.4.4API Version:  3.4.4Go Version:   go1.18.1Built:        Thu Jan  1 00:00:00 1970OS/Arch:      linux/amd64

• The podman info

user1@test:~$ podman infohost:  arch: amd64  buildahVersion: 1.23.1  cgroupControllers:  - memory  - pids  cgroupManager: systemd  cgroupVersion: v2  conmon:    package: 'conmon: /usr/bin/conmon'    path: /usr/bin/conmon    version: 'conmon version 2.0.25, commit: unknown'  cpus: 12  distribution:    codename: jammy    distribution: ubuntu    version: "22.04"  eventLogger: journald  hostname: test  idMappings:    gidmap:    - container_id: 0      host_id: 1001      size: 1    - container_id: 1      host_id: 165536      size: 65536    uidmap:    - container_id: 0      host_id: 1001      size: 1    - container_id: 1      host_id: 165536      size: 65536  kernel: 5.15.0-122-generic  linkmode: dynamic  logDriver: journald  memFree: 19415232512  memTotal: 20680478720  ociRuntime:    name: crun    package: 'crun: /usr/bin/crun'    path: /usr/bin/crun    version: |-      crun version 0.17      commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a      spec: 1.0.0+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL  os: linux  remoteSocket:    exists: true    path: /run/user/1001/podman/podman.sock  security:    apparmorEnabled: false    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT    rootless: true    seccompEnabled: true    seccompProfilePath: /usr/share/containers/seccomp.json    selinuxEnabled: false  serviceIsRemote: false  slirp4netns:    executable: /usr/bin/slirp4netns    package: 'slirp4netns: /usr/bin/slirp4netns'    version: |-      slirp4netns version 1.0.1      commit: 6a7b16babc95b6a3056b33fb45b74a6f62262dd4      liuser0irp: 4.6.1  swapFree: 8589930496  swapTotal: 8589930496  uptime: 13h 8m 55.68s (Approximately 0.54 days)plugins:  log:  - k8s-file  - none  - journald  network:  - bridge  - macvlan  volume:  - localregistries:  search:  - gitea.cn:666store:  configFile: /home/user1/.config/containers/storage.conf  containerStore:    number: 0    paused: 0    running: 0    stopped: 0  graphDriverName: overlay  graphOptions: {}  graphRoot: /var/lib/sharedimages  graphStatus:    Backing Filesystem: extfs    Native Overlay Diff: "true"    Supports d_type: "true"    Using metacopy: "false"  imageStore:    number: 1  runRoot: /run/user/1001/containers  volumePath: /var/lib/sharedimages/volumesversion:  APIVersion: 3.4.4  Built: 0  BuiltTime: Thu Jan  1 00:00:00 1970  GitCommit: ""  GoVersion: go1.18.1  OsArch: linux/amd64  Version: 3.4.4

• The OS

user1@test:~$ uname -aLinux test 5.15.0-122-generic #132-Ubuntu SMP Thu Aug 29 13:45:52 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Seemed the debug log said the configuration of user2 did not take effect

DEBU[0000] Initializing boltdb state at /var/lib/sharedimages/libpod/bolt_state.dbDEBU[0000] Overriding run root "/run/user/1002" with "/run/user/1001/containers" from databaseDEBU[0000] Overriding tmp dir "/run/user/1002/libpod/tmp" with "/run/user/1001/libpod/tmp" from databas

Then what's the expected configurations for this requirement?

Thank you very much!

Edit on 29th Sep.

I removed podman -> removed /var/lib/sharedimages -> removed /var/lib/containers -> removed /home/user1/.config/containers/storage.conf -> removed /home/user2/.config/container/stoarge.conf -> reboot host -> reinstalled podman -> recreated /var/lib/sharedimages and /var/lib/container

• try user2 first

user2@test:~$ podman images --log-level=debugINFO[0000] podman filtering at log level debugDEBU[0000] Called images.PersistentPreRunE(podman images --log-level=debug)DEBU[0000] overlay storage already configured with a mount-programDEBU[0000] Merged system config "/usr/share/containers/containers.conf"DEBU[0000] overlay storage already configured with a mount-programDEBU[0000] Using conmon: "/usr/bin/conmon"DEBU[0000] Initializing boltdb state at /home/user2/.local/share/containers/storage/libpod/bolt_state.dbDEBU[0000] systemd-logind: Unknown object '/'.DEBU[0000] Using graph driver overlayDEBU[0000] Using graph root /home/user2/.local/share/containers/storageDEBU[0000] Using run root /run/user/1002/containersDEBU[0000] Using static dir /home/user2/.local/share/containers/storage/libpodDEBU[0000] Using tmp dir /run/user/1002/libpod/tmpDEBU[0000] Using volume path /home/user2/.local/share/containers/storage/volumesDEBU[0000] overlay storage already configured with a mount-programDEBU[0000] Set libpod namespace to ""DEBU[0000] [graphdriver] trying provided driver "overlay"DEBU[0000] overlay: mount_program=/usr/bin/fuse-overlayfsDEBU[0000] backingFs=extfs, projectQuotaSupported=false, useNativeDiff=false, usingMetacopy=falseDEBU[0000] Initializing event backend journaldDEBU[0000] configured OCI runtime runc initialization failed: no valid executable found for OCI runtime runc: invalid argumentDEBU[0000] configured OCI runtime kata initialization failed: no valid executable found for OCI runtime kata: invalid argumentDEBU[0000] configured OCI runtime runsc initialization failed: no valid executable found for OCI runtime runsc: invalid argumentDEBU[0000] Using OCI runtime "/usr/bin/crun"INFO[0000] Found CNI network podman (type=bridge) at /home/user2/.config/cni/net.d/87-podman.conflistDEBU[0000] Default CNI network name podman is unchangeableINFO[0000] Setting parallel job count to 37REPOSITORY  TAG         IMAGE ID    CREATED     SIZEDEBU[0000] Called images.PersistentPostRunE(podman images --log-level=debug)user2@test:~$

Then user1

user1@test:~$ podman images --log-level=debugINFO[0000] podman filtering at log level debugDEBU[0000] Called images.PersistentPreRunE(podman images --log-level=debug)DEBU[0000] Merged system config "/usr/share/containers/containers.conf"DEBU[0000] Using conmon: "/usr/bin/conmon"Error: error creating runtime static files directory: mkdir /var/lib/sharedimages/libpod: permission denieduser1@test:~$user1@test:~$user1@test:~$ more .config/ccni/        containers/user1@test:~$ more .config/containers/storage.conf[storage]driver = "overlay"runroot = "/run/user/1001"graphroot = "/var/lib/sharedimages"[storage.options]mount_program = "/bin/fuse-overlayfs"