o3de-flatpak/docs/DESIGN.md

Design & engineering notes

This document explains why the O3DE Flatpak is built the way it is. The README tells you how to use and operate it; this is the companion that records the reasoning and the hard-won lessons, so the next person to touch this repo (very possibly future-you) doesn't have to rediscover them.

If you read only one thing, read The core tension and The read-only /app saga. Almost every non-obvious decision in this repo flows from those two sections.

---

The core tension

O3DE was never designed to run from a read-only location. It is not a normal desktop app that you install and run — it is an engine + SDK that, at runtime:

• compiles C++ (your game project, gems) — so it ships a build toolchain (clang/cmake/ninja/pkg-config), not just runtime libs;
• bootstraps its own Python into a per-user virtualenv and pip-installs a CLI;
• writes into its own install tree — asset caches, user settings, metrics, Python egg-info, registration data.

A Flatpak gives the app a read-only /app at runtime. So the entire project is, essentially, a series of negotiations between "O3DE wants to write here" and "that location is read-only." Every caveat in the README is one of those negotiations.

The official Debian package assumes a normal writable /opt/O3DE/<ver>/. We take that exact payload and drop it into /app/opt/O3DE/<ver>/, then redirect or pre-bake everything O3DE would otherwise try to write.

---

Packaging architecture

We repackage the .deb; we don't build from source

Building O3DE from source is enormous (hours, tens of GB, a full toolchain). The project instead consumes the official prebuilt Linux release, which ships as a Debian package at a predictable URL:

https://o3debinaries.org/main/Latest/Linux/o3de_<ver>.deb   (+ .sha256)

scripts/get-latest-version.sh scrapes the latest filename, derives the version (o3de_2605_0.deb → 26.05.0, engine dir 26.05) and fetches the checksum. scripts/make-flatpak.sh extracts the .deb (ar + tar) and copies its opt/O3DE/<ver>/ payload into the Flatpak's /app.

The version directory name changes every release, so nothing hard-codes it — the wrapper and build discover paths at runtime/build time with find.

No flatpak-builder, no bubblewrap, no privileged container

This was forced by the CI environment and turned out to be the single most important build decision.

flatpak-builder runs every build command inside a bubblewrap (bwrap) sandbox, which needs user namespaces / a privileged container. In a stock Gitea Actions / act_runner job container that fails with:

bwrap: Creating new namespace failed: Operation not permitted

Marking the runner's container privileged did not fix it (wrong layer), and requiring privileged containers is a bad thing to demand of a self-hosted runner.

The escape: our "build" is just unpacking files. We don't need a sandbox to do that. So scripts/make-flatpak.sh uses the lower-level primitives directly:

flatpak build-init   build-dir <app-id> <runtime> <sdk> <ver>
# ... copy files into build-dir/files ...
flatpak build-finish build-dir --command=… <finish-args…>
flatpak build-export repo build-dir <branch>
flatpak build-update-repo repo …

build-init/finish/export only touch files and the OSTree repo — no bwrap — so they run in a plain unprivileged container. org.o3de.O3DE.yaml (a real flatpak-builder manifest) is kept only as an alternative for anyone who does have a bwrap-capable builder. CI does not use it.

Runtime is org.freedesktop.Sdk, not Platform

Counter-intuitive but essential. O3DE compiles project code at runtime, so its dependency list is a build toolchain (clang, cmake, ninja, pkg-config). Those live in the SDK, not the slimmer Platform runtime. Running against org.freedesktop.Sdk//24.08 is what makes get_python.sh (which needs cmake) and project builds work inside the sandbox. End users therefore pull the larger SDK runtime from Flathub on install — that's expected, not a bug.

Hosting: a static OSTree repo on an orphan pages branch

A Flatpak remote is just a directory of files (summary, config, objects/…). CI commits the generated repo/ to an orphan pages branch and force-pushes a single snapshot (so the branch never accumulates multi-GB history). Gitea serves those files over raw URLs, and o3de.flatpakrepo points clients at them. If you ever front this with a real static host, only Url= in the .flatpakrepo changes.

---

The read-only /app saga

This is the heart of the project. Each item below is a distinct place O3DE tried to write into read-only /app, how it manifested, and how it was resolved. They were found one at a time, each revealed only after fixing the previous one.

#	Symptom	Root cause	Fix
1	qt.qpa.xcb could not connect to display	--socket=fallback-x11 suppresses X11 on Wayland; O3DE's bundled Qt only ships the xcb plugin	Use --socket=x11 unconditionally + QT_QPA_PLATFORM=xcb
2	setup.py develop … [Errno 30] Read-only file system	get_python.sh does pip install -e scripts/o3de; editable mode writes egg-info into read-only /app	Pre-build an sdist so the runtime takes its non-editable branch (below)
3	No module named 'o3de'	Half-built venv from a prior failure; the GUI never re-bootstraps Python	Wrapper runs get_python.sh itself; it's self-healing (below)
4	Missing python venv file … pyvenv.cfg	Project Manager GUI does not bootstrap Python — it only errors if the venv is absent	Wrapper bootstraps the venv before launching (below)
5	Invalid engine json …/venv/…/lib/engine.json → Failed to initialize	Non-editable install moved the o3de package into the venv, so get_this_engine_path() (walks up from __file__) resolves into the venv	Set SNAP/SNAP_BUILD to the engine root (below)
6	Couldn't find a valid asset cache folder / cannot write … UserSettings.xml	Project-less Project Manager puts its engine-level user dir under read-only /app	Non-fatal; documented. The documented --regset override is overwritten by bootstrap (below)
—	… is not a valid icon: bwrap during build-export	build-export validates exported icons in a bwrap sandbox (unavailable unprivileged)	Keep the icon inside the app, drop it from the exported set

Python: the sdist trick (items 2)

O3DE installs its own o3de CLI into the per-user venv. By default python/get_python.sh does:

pip install -e <engine>/scripts/o3de   # editable → writes egg-info into /app → fails

The key discovery: get_python.sh already has a built-in immutable-install escape hatch (it's how the official Snap package copes). It is gated on a pre-built sdist tarball:

if [ -f .../scripts/o3de/dist/o3de-1.0.0.tar.gz ]; then
    pip install .../dist/o3de-1.0.0.tar.gz --no-deps   # non-editable ✅
else
    pip install -e .../scripts/o3de --no-deps           # editable ❌
fi

So the build runs python3 setup.py sdist while /app is still writable, producing scripts/o3de/dist/o3de-1.0.0.tar.gz. At runtime the -f test passes, the CLI installs non-editably into the writable ~/.o3de venv, and nothing touches /app. (setup.py hard-codes version="1.0.0", hence the exact filename — the build fails fast if that ever changes.) The build also patches every cmake/LYPython.cmake to a non-editable install for the separate CMake-configure path used during a project build.

Note — the first patch was on the wrong file. Early on we patched only LYPython.cmake, confirmed the patch landed, and still saw setup.py develop at runtime. The tell was the failure path …/python/../scripts/o3de, which is how get_python.sh builds the path, not LYPython.cmake. Two different installers; the GUI's first-launch path is get_python.sh.

Python: the wrapper bootstraps the venv (items 3, 4)

The Project Manager GUI never runs get_python.sh. It assumes the venv already exists and just errors (Missing python venv file … pyvenv.cfg) if it doesn't — which then cascades into engine-registration failure and exit. Nothing on the desktop launches get_python.sh for you.

So o3de-wrapper.sh does it. It checks python.sh --version (which exits non-zero when the venv for this engine is missing or stale) and, if so, runs get_python.sh before launching the GUI. This is:

• idempotent — a healthy venv means python.sh --version succeeds and we skip;
• self-healing — get_python.sh recreates the venv with --clear, so a half-built ~/.o3de/Python/venv/<id> left by an earlier failure is repaired.

The venv lives at ~/.o3de/Python/venv/<engine-id>, where <engine-id> is a hash of the engine path computed by cmake. The first launch downloads Python and can take several minutes with no window on screen — expected; later launches are instant.

Engine path: SNAP / SNAP_BUILD (item 5)

A subtle consequence of fixing item 2. O3DE finds its own engine root in Python via:

def get_this_engine_path():
    if "SNAP" in os.environ and "SNAP_BUILD" in os.environ:
        return pathlib.Path(os.environ["SNAP"]) / os.environ["SNAP_BUILD"]
    return pathlib.Path(os.path.realpath(__file__)).parents[3].resolve()

With the old editable install, __file__ lived in the engine tree (/app/.../scripts/o3de/o3de/manifest.py) and parents[3] correctly gave the engine root. Our non-editable install moved the package into the venv site-packages, so parents[3] now lands inside the venv → engine.json not found → Failed to get engine info → Failed to initialize.

O3DE's own override saves us: when SNAP and SNAP_BUILD are set it skips the __file__ walk and returns $SNAP/$SNAP_BUILD. The wrapper sets:

SNAP=/app/opt/O3DE          # dirname of the engine root
SNAP_BUILD=26.05            # basename of the engine root  → $SNAP/$SNAP_BUILD = engine root

We verified (grep of the Python package and the C++ binaries/.sos) that these two variables are read only in get_this_engine_path — setting them triggers no other snap-specific behaviour. This was confirmed empirically by launching the binary with the vars set: the Project Manager opened cleanly with no engine.json errors.

The user folder we couldn't relocate (item 6)

With no project open, O3DE places its engine-level user folder (asset cache, UserSettings.xml, metrics) under <engine>/…/user — read-only /app. You'll see warnings at startup; they are non-fatal. The Project Manager runs fine; it just doesn't persist its own UI settings between runs.

The documented override is registry key /O3DE/Runtime/FilePaths/SourceProjectUserPath. We tested redirecting it with --regset to a writable path and it did not work: O3DE recomputes that path from the engine root during settings-registry bootstrap and overwrites the command-line value. (This was verified with line-buffered output capture — an earlier "it works!" reading was a buffering artifact from timeout-killing the process, which never flushed its full-buffered pipe. Lesson: when capturing a GUI process you'll kill, force stdbuf -oL -eL.)

Real project data — the projects you create and their caches — lives under your home directory and is fully writable, so actual work is unaffected. We chose to document this rather than chase a fix that the engine actively undoes.

The icon

The .deb ships only in-editor asset icons, no app icon, so we add org.o3de.O3DE.png. But flatpak build-export validates exported icons inside a bwrap sandbox — which, unprivileged, fails (is not a valid icon: bwrap …). The compromise: keep the icon inside the app (so the running window/taskbar shows it) but rm -rf build-dir/export/share/icons before export so the export step skips validation. The cost is a generic icon in the host's app menu. A privileged runner would let us export it properly.

---

How a build flows (CI)

.gitea/workflows/build-flatpak.yml, triggered by daily cron or workflow_dispatch (with an optional force):

1. Install deps — curl jq xz-utils zstd binutils tar python3 python3-setuptools flatpak. (python3-setuptools is needed to pre-build the o3de sdist.)
2. Checkout — a plain git init/remote/fetch/checkout, not actions/checkout. The bare ubuntu:24.04 image has no Node.js, so JavaScript actions exit 127.
3. Resolve version via get-latest-version.sh.
4. Decide to build — skip if a vX.Y.Z tag already exists, unless force.
5. Install org.freedesktop.Sdk//24.08 from Flathub.
6. Download + checksum the .deb.
7. Stamp the version/date into org.o3de.O3DE.metainfo.xml.
8. Build via scripts/make-flatpak.sh; delete build-dir/o3de.deb to reclaim disk before publishing.
9. Generate o3de.flatpakrepo pointing at the pages raw URL.
10. Publish — force-push the repo/ snapshot to the orphan pages branch.
11. Tag vX.Y.Z (idempotent: skipped if it already exists, e.g. a force rebuild).

Auth uses PUBLISH_TOKEN if set, otherwise the auto-provided GITHUB_TOKEN.

---

Maintaining this across O3DE releases

Most releases will "just work" because nothing hard-codes the version. The build is designed to fail fast at the spots most likely to break:

• Package version ≠ 1.0.0 — the sdist filename o3de-1.0.0.tar.gz is what get_python.sh looks for. If scripts/o3de/setup.py bumps its version, the build errors at the sdist step. Re-check the filename in get_python.sh and update.
• Layout move — if scripts/o3de/setup.py or cmake/LYPython.cmake disappears, the build errors. Re-trace the install path (it has moved between the Snap-style hook and LYPython.cmake before).
• get_python.sh drops the sdist hook — then the editable install returns and item 2 reappears. Re-check the if [ -f …/dist/o3de-1.0.0.tar.gz ] branch.
• get_this_engine_path stops honouring SNAP/SNAP_BUILD — item 5 reappears as Invalid engine json …/venv/…. Re-check scripts/o3de/o3de/manifest.py.
• Binary/library path change — o3de-wrapper.sh finds the binary under *bin/Linux* and sets LD_LIBRARY_PATH; adjust if bin/Linux/profile/Default/ moves.

How to debug runtime issues quickly (without a full rebuild)

The expensive loop is: force-rebuild (~15 min) → publish → flatpak update → re-download Python. You can usually avoid it. Because the installed app's payload is just files in /app, you can inspect and even re-run pieces in the live sandbox:

# Read any engine file:
flatpak run --command=bash org.o3de.O3DE -c 'cat /app/opt/O3DE/26.05/python/get_python.sh'

# Re-run the Python bootstrap and watch it:
flatpak run --command=bash org.o3de.O3DE -c 'sh /app/opt/O3DE/*/python/get_python.sh'

# Launch the binary with experimental env/args BEFORE baking them into the wrapper
# (this is how SNAP/SNAP_BUILD was validated). Use stdbuf so output flushes on kill,
# and write logs to $HOME (the sandbox /tmp is private, but home is shared):
timeout 40 flatpak run --command=bash org.o3de.O3DE -c '
  export SNAP=/app/opt/O3DE SNAP_BUILD=26.05
  BIN=$(find /app/opt/O3DE -type f -name o3de -path "*bin/Linux*" | head -1)
  export LD_LIBRARY_PATH="$(dirname "$BIN"):/app/lib:/app/opt/O3DE/lib"
  stdbuf -oL -eL "$BIN" > "$HOME/o3de-test.log" 2>&1
'

A timeout exit code of 124 means the process stayed alive (a window opened) — which, by itself, is strong evidence that startup/engine-registration succeeded.

The upstream source for cross-referencing is github.com/o3de/o3de at the matching tag (e.g. 2605.0 for engine dir 26.05). The files that mattered most here: python/get_python.sh, cmake/LYPython.cmake, scripts/o3de/o3de/manifest.py, and Code/Framework/AzFramework/AzFramework/Application/Application.cpp.

---

Open items / future hardening

• Signing. The repo is published unsigned; users add it with GPG verification disabled. See Signing in the README to add a key.
• Exported icon. Generic in the host menu until built on a bwrap-capable (privileged) runner.
• Engine-level user folder. Cosmetic read-only warnings remain (item 6). If a future O3DE exposes an override the bootstrap doesn't clobber, wire it into the wrapper.
• Project create/build end-to-end. Launch is solid; the first real project build is the next thing to exercise, since it's the first time the runtime compile path and project asset cache are used in anger.