support ostree native containers (container as OS)

[garrett]

Details: https://coreos.github.io/rpm-ostree/container/

rpm-ostree inherits work in ostree-rs-ext to create “container native ostree” functionality. This elevates OCI/docker containers to be natively supported as a transport mechanism for bootable operating systems.

Support is already available as of Fedora 36. It's still being worked on. They may switch to using it as the default format for Fedora 38.

• F36: https://fedoraproject.org/wiki/Changes/OstreeNativeContainer
• F38: https://fedoraproject.org/wiki/Changes/OstreeNativeContainerStable
• CoreOS docs:
  • https://github.com/coreos/enhancements/blob/main/os/coreos-layering.md
  • https://coreos.github.io/rpm-ostree/container/
• Silverblue isn't officially supported yet, but it's coming soon. There's a thread at:
  • https://discussion.fedoraproject.org/t/feature-for-custom-rpm-ostree-container-native-builds/44480
  • Official should be coming soon. PR landed @ https://pagure.io/releng/pull-request/11120
  • Temporary Silverblue container (especially useful to do a FROM in a Containerfile): https://github.com/cgwalters/sync-fedora-ostree-containers
  • And third party builds are available at:
    • https://github.com/GuiltyDoggy/ostree-container
    • https://github.com/castrojo/ublue-image
• CoreOS has an official container for this and there are even third party ones now too.
  • Official: https://quay.io/repository/fedora/fedora-coreos?tab=tags
  • Examples: https://github.com/coreos/layering-examples
  • Third party: https://github.com/jlebon/pet/

Related, but a little different: https://github.com/containers/bootc (we'll likely keep our integration point at rpm-ostree)

[garrett]

https://github.com/coreos/enhancements/blob/main/os/coreos-layering.md#existing-work

Specifically as of today, this functionality is exposed in:

• rpm-ostree ex-container
• rpm-ostree rebase --experimental $containerref

HOWEVER, according to https://coreos.github.io/rpm-ostree/container/ it's even easier, perhaps?

Rebasing a client system

Use this to switch to booting from a container image:

$ rpm-ostree rebase ostree-unverified-registry:quay.io/fedora/fedora-coreos:stable

And:

After a rebase, all further rpm-ostree operations work as you’d expect. For example, rpm-ostree upgrade will look for a new container version. You can also rpm-ostree apply-live, etc. It also does still work to do “client side” rpm-ostree install etc.
...

[garrett]

Jorge Castro made a video to demonstrate this yesterday:

https://www.youtube.com/watch?v=X8h304Jp9N8

[garrett]

OK, I think all we'd need to support it in cockpit-ostree right now is a toggle (off by default) to support --experimental during rebases. And then remember the experimental state for the repo when rebasing back to it (after rebasing to something else).

And when this isn't deemed "experimental" (probably in F38?) then it's probably not even needed. Although other "experiments" could possibly show up in the future.

If we could somehow detect the experimental repos or just always turn it on, we wouldn't even need the flag and just transparently support it.

---

Building on this, we could try a rebase, catch an error, and have a modal that says something about the container format is not supported by default, but is an experimental format and ask to proceed.

I think this would be even better than having an experimental toggle, IMO. And it should be more future proof and the UI wouldn't be seen in F38 and beyond (unless they add another source type afterward).

[garrett]

Curated lists of examples of custom images for OStree native container images @ https://github.com/ublue-os/awesome-custom-images