diff --git a/README.md b/README.md index c9b69326..65e53abc 100644 --- a/README.md +++ b/README.md @@ -57,6 +57,13 @@ it (`ar` + `tar`) and copies the payload into the Flatpak's `/app`. The version directory inside the `.deb` changes every release, so the wrapper discovers the executable at runtime rather than hard-coding a path. +> **Want the full story?** O3DE is an engine + SDK that compiles code and writes +> into its own install tree at runtime, while a Flatpak gives it a **read-only +> `/app`**. Reconciling those two facts is most of the work here. +> **[`docs/DESIGN.md`](docs/DESIGN.md)** documents every non-obvious decision, the +> read-only-`/app` problems and their fixes, and how to debug/maintain this across +> O3DE releases. Read it before changing the build. + --- ## CI requirements (Gitea Actions) diff --git a/docs/DESIGN.md b/docs/DESIGN.md new file mode 100644 index 00000000..f267326b --- /dev/null +++ b/docs/DESIGN.md @@ -0,0 +1,329 @@ +# Design & engineering notes + +This document explains **why** the O3DE Flatpak is built the way it is. The +[README](../README.md) tells you how to use and operate it; this is the companion +that records the reasoning and the hard-won lessons, so the next person to touch +this repo (very possibly future-you) doesn't have to rediscover them. + +If you read only one thing, read [The core tension](#the-core-tension) and +[The read-only `/app` saga](#the-read-only-app-saga). Almost every non-obvious +decision in this repo flows from those two sections. + +--- + +## The core tension + +O3DE was never designed to run from a read-only location. It is not a normal +desktop app that you install and run — it is an **engine + SDK** that, at runtime: + +- **compiles C++** (your game project, gems) — so it ships a *build toolchain* + (clang/cmake/ninja/pkg-config), not just runtime libs; +- **bootstraps its own Python** into a per-user virtualenv and `pip`-installs a CLI; +- **writes into its own install tree** — asset caches, user settings, metrics, + Python `egg-info`, registration data. + +A Flatpak gives the app a **read-only `/app`** at runtime. So the entire project +is, essentially, a series of negotiations between "O3DE wants to write here" and +"that location is read-only." Every caveat in the README is one of those +negotiations. + +The official Debian package assumes a normal writable `/opt/O3DE//`. We take +that exact payload and drop it into `/app/opt/O3DE//`, then redirect or +pre-bake everything O3DE would otherwise try to write. + +--- + +## Packaging architecture + +### We repackage the `.deb`; we don't build from source + +Building O3DE from source is enormous (hours, tens of GB, a full toolchain). The +project instead consumes the **official prebuilt Linux release**, which ships as a +Debian package at a predictable URL: + +``` +https://o3debinaries.org/main/Latest/Linux/o3de_.deb (+ .sha256) +``` + +`scripts/get-latest-version.sh` scrapes the latest filename, derives the version +(`o3de_2605_0.deb` → `26.05.0`, engine dir `26.05`) and fetches the checksum. +`scripts/make-flatpak.sh` extracts the `.deb` (`ar` + `tar`) and copies its +`opt/O3DE//` payload into the Flatpak's `/app`. + +The version directory name changes every release, so **nothing hard-codes it** — +the wrapper and build discover paths at runtime/build time with `find`. + +### No `flatpak-builder`, no bubblewrap, no privileged container + +This was forced by the CI environment and turned out to be the single most +important build decision. + +`flatpak-builder` runs **every build command inside a bubblewrap (`bwrap`) +sandbox**, which needs user namespaces / a privileged container. In a stock Gitea +Actions / `act_runner` job container that fails with: + +``` +bwrap: Creating new namespace failed: Operation not permitted +``` + +Marking the *runner's* container privileged did **not** fix it (wrong layer), and +requiring privileged containers is a bad thing to demand of a self-hosted runner. + +The escape: our "build" is just *unpacking files*. We don't need a sandbox to do +that. So `scripts/make-flatpak.sh` uses the lower-level primitives directly: + +``` +flatpak build-init build-dir +# ... copy files into build-dir/files ... +flatpak build-finish build-dir --command=… +flatpak build-export repo build-dir +flatpak build-update-repo repo … +``` + +`build-init/finish/export` only touch files and the OSTree repo — **no `bwrap`** — +so they run in a plain unprivileged container. `org.o3de.O3DE.yaml` (a real +`flatpak-builder` manifest) is kept **only** as an alternative for anyone who *does* +have a bwrap-capable builder. CI does not use it. + +### Runtime is `org.freedesktop.Sdk`, not `Platform` + +Counter-intuitive but essential. O3DE compiles project code at runtime, so its +dependency list is a **build toolchain** (clang, cmake, ninja, pkg-config). Those +live in the **SDK**, not the slimmer Platform runtime. Running against +`org.freedesktop.Sdk//24.08` is what makes `get_python.sh` (which needs `cmake`) +and project builds work inside the sandbox. End users therefore pull the larger SDK +runtime from Flathub on install — that's expected, not a bug. + +### Hosting: a static OSTree repo on an orphan `pages` branch + +A Flatpak remote is just a directory of files (`summary`, `config`, `objects/…`). +CI commits the generated `repo/` to an orphan **`pages`** branch and **force-pushes +a single snapshot** (so the branch never accumulates multi-GB history). Gitea serves +those files over raw URLs, and `o3de.flatpakrepo` points clients at them. If you +ever front this with a real static host, only `Url=` in the `.flatpakrepo` changes. + +--- + +## The read-only `/app` saga + +This is the heart of the project. Each item below is a distinct place O3DE tried to +write into read-only `/app`, how it manifested, and how it was resolved. They were +found **one at a time**, each revealed only after fixing the previous one. + +| # | Symptom | Root cause | Fix | +|---|---------|-----------|-----| +| 1 | `qt.qpa.xcb could not connect to display` | `--socket=fallback-x11` *suppresses* X11 on Wayland; O3DE's bundled Qt only ships the `xcb` plugin | Use `--socket=x11` unconditionally + `QT_QPA_PLATFORM=xcb` | +| 2 | `setup.py develop … [Errno 30] Read-only file system` | `get_python.sh` does `pip install -e scripts/o3de`; editable mode writes `egg-info` into read-only `/app` | **Pre-build an sdist** so the runtime takes its non-editable branch (below) | +| 3 | `No module named 'o3de'` | Half-built venv from a prior failure; the GUI never re-bootstraps Python | Wrapper runs `get_python.sh` itself; it's self-healing (below) | +| 4 | `Missing python venv file … pyvenv.cfg` | Project Manager **GUI does not bootstrap Python** — it only errors if the venv is absent | Wrapper bootstraps the venv before launching (below) | +| 5 | `Invalid engine json …/venv/…/lib/engine.json` → `Failed to initialize` | Non-editable install moved the `o3de` package into the venv, so `get_this_engine_path()` (walks up from `__file__`) resolves into the venv | Set `SNAP`/`SNAP_BUILD` to the engine root (below) | +| 6 | `Couldn't find a valid asset cache folder` / `cannot write … UserSettings.xml` | Project-less Project Manager puts its *engine-level* user dir under read-only `/app` | **Non-fatal**; documented. The documented `--regset` override is overwritten by bootstrap (below) | +| — | `… is not a valid icon: bwrap` during `build-export` | `build-export` validates exported icons in a bwrap sandbox (unavailable unprivileged) | Keep the icon *inside* the app, drop it from the *exported* set | + +### Python: the sdist trick (items 2) + +O3DE installs its own `o3de` CLI into the per-user venv. By default +`python/get_python.sh` does: + +```sh +pip install -e /scripts/o3de # editable → writes egg-info into /app → fails +``` + +The key discovery: **`get_python.sh` already has a built-in immutable-install +escape hatch** (it's how the official Snap package copes). It is gated on a +pre-built sdist tarball: + +```sh +if [ -f .../scripts/o3de/dist/o3de-1.0.0.tar.gz ]; then + pip install .../dist/o3de-1.0.0.tar.gz --no-deps # non-editable ✅ +else + pip install -e .../scripts/o3de --no-deps # editable ❌ +fi +``` + +So the build runs `python3 setup.py sdist` **while `/app` is still writable**, +producing `scripts/o3de/dist/o3de-1.0.0.tar.gz`. At runtime the `-f` test passes, +the CLI installs non-editably into the writable `~/.o3de` venv, and nothing touches +`/app`. (`setup.py` hard-codes `version="1.0.0"`, hence the exact filename — the +build fails fast if that ever changes.) The build also patches every +`cmake/LYPython.cmake` to a non-editable install for the *separate* CMake-configure +path used during a project build. + +> **Note — the first patch was on the wrong file.** Early on we patched only +> `LYPython.cmake`, confirmed the patch landed, and *still* saw `setup.py develop` +> at runtime. The tell was the failure path `…/python/../scripts/o3de`, which is how +> **`get_python.sh`** builds the path, not `LYPython.cmake`. Two different installers; +> the GUI's first-launch path is `get_python.sh`. + +### Python: the wrapper bootstraps the venv (items 3, 4) + +The Project Manager **GUI never runs `get_python.sh`**. It assumes the venv already +exists and just errors (`Missing python venv file … pyvenv.cfg`) if it doesn't — +which then cascades into engine-registration failure and exit. Nothing on the +desktop launches `get_python.sh` for you. + +So `o3de-wrapper.sh` does it. It checks `python.sh --version` (which exits non-zero +when the venv for this engine is missing or stale) and, if so, runs `get_python.sh` +before launching the GUI. This is: + +- **idempotent** — a healthy venv means `python.sh --version` succeeds and we skip; +- **self-healing** — `get_python.sh` recreates the venv with `--clear`, so a + half-built `~/.o3de/Python/venv/` left by an earlier failure is repaired. + +The venv lives at `~/.o3de/Python/venv/`, where `` is a hash +of the engine path computed by cmake. The **first launch downloads Python and can +take several minutes with no window on screen** — expected; later launches are +instant. + +### Engine path: `SNAP` / `SNAP_BUILD` (item 5) + +A subtle consequence of fixing item 2. O3DE finds its own engine root in Python via: + +```python +def get_this_engine_path(): + if "SNAP" in os.environ and "SNAP_BUILD" in os.environ: + return pathlib.Path(os.environ["SNAP"]) / os.environ["SNAP_BUILD"] + return pathlib.Path(os.path.realpath(__file__)).parents[3].resolve() +``` + +With the old **editable** install, `__file__` lived in the engine tree +(`/app/.../scripts/o3de/o3de/manifest.py`) and `parents[3]` correctly gave the +engine root. Our **non-editable** install moved the package into the venv +`site-packages`, so `parents[3]` now lands **inside the venv** → `engine.json` not +found → `Failed to get engine info` → `Failed to initialize`. + +O3DE's own override saves us: when `SNAP` and `SNAP_BUILD` are set it skips the +`__file__` walk and returns `$SNAP/$SNAP_BUILD`. The wrapper sets: + +```sh +SNAP=/app/opt/O3DE # dirname of the engine root +SNAP_BUILD=26.05 # basename of the engine root → $SNAP/$SNAP_BUILD = engine root +``` + +We verified (grep of the Python package **and** the C++ binaries/`.so`s) that these +two variables are read **only** in `get_this_engine_path` — setting them triggers no +other snap-specific behaviour. This was confirmed empirically by launching the +binary with the vars set: the Project Manager opened cleanly with no `engine.json` +errors. + +### The user folder we *couldn't* relocate (item 6) + +With no project open, O3DE places its **engine-level** user folder (asset cache, +`UserSettings.xml`, metrics) under `/…/user` — read-only `/app`. You'll see +warnings at startup; they are **non-fatal**. The Project Manager runs fine; it just +doesn't persist *its own* UI settings between runs. + +The documented override is registry key +`/O3DE/Runtime/FilePaths/SourceProjectUserPath`. We tested redirecting it with +`--regset` to a writable path and it **did not work**: O3DE **recomputes** that path +from the engine root during settings-registry bootstrap and overwrites the +command-line value. (This was verified with line-buffered output capture — an +earlier "it works!" reading was a buffering artifact from `timeout`-killing the +process, which never flushed its full-buffered pipe. Lesson: when capturing a GUI +process you'll kill, force `stdbuf -oL -eL`.) + +Real project data — the projects you create and their caches — lives under your home +directory and is fully writable, so actual work is unaffected. We chose to document +this rather than chase a fix that the engine actively undoes. + +### The icon + +The `.deb` ships only in-editor asset icons, no app icon, so we add +`org.o3de.O3DE.png`. But `flatpak build-export` validates *exported* icons inside a +bwrap sandbox — which, unprivileged, fails (`is not a valid icon: bwrap …`). The +compromise: keep the icon **inside** the app (so the running window/taskbar shows +it) but `rm -rf build-dir/export/share/icons` before export so the export step skips +validation. The cost is a generic icon in the host's app menu. A privileged runner +would let us export it properly. + +--- + +## How a build flows (CI) + +`.gitea/workflows/build-flatpak.yml`, triggered by daily `cron` or +`workflow_dispatch` (with an optional **force**): + +1. **Install deps** — `curl jq xz-utils zstd binutils tar python3 python3-setuptools + flatpak`. (`python3-setuptools` is needed to pre-build the o3de sdist.) +2. **Checkout** — a plain `git init/remote/fetch/checkout`, *not* `actions/checkout`. + The bare `ubuntu:24.04` image has no Node.js, so JavaScript actions exit 127. +3. **Resolve version** via `get-latest-version.sh`. +4. **Decide to build** — skip if a `vX.Y.Z` tag already exists, unless `force`. +5. **Install** `org.freedesktop.Sdk//24.08` from Flathub. +6. **Download + checksum** the `.deb`. +7. **Stamp** the version/date into `org.o3de.O3DE.metainfo.xml`. +8. **Build** via `scripts/make-flatpak.sh`; delete `build-dir`/`o3de.deb` to reclaim + disk before publishing. +9. **Generate** `o3de.flatpakrepo` pointing at the `pages` raw URL. +10. **Publish** — force-push the `repo/` snapshot to the orphan `pages` branch. +11. **Tag** `vX.Y.Z` (idempotent: skipped if it already exists, e.g. a force rebuild). + +Auth uses `PUBLISH_TOKEN` if set, otherwise the auto-provided `GITHUB_TOKEN`. + +--- + +## Maintaining this across O3DE releases + +Most releases will "just work" because nothing hard-codes the version. The build is +designed to **fail fast** at the spots most likely to break: + +- **Package version ≠ `1.0.0`** — the sdist filename `o3de-1.0.0.tar.gz` is what + `get_python.sh` looks for. If `scripts/o3de/setup.py` bumps its version, the build + errors at the sdist step. Re-check the filename in `get_python.sh` and update. +- **Layout move** — if `scripts/o3de/setup.py` or `cmake/LYPython.cmake` disappears, + the build errors. Re-trace the install path (it has moved between the Snap-style + hook and `LYPython.cmake` before). +- **`get_python.sh` drops the sdist hook** — then the editable install returns and + item 2 reappears. Re-check the `if [ -f …/dist/o3de-1.0.0.tar.gz ]` branch. +- **`get_this_engine_path` stops honouring `SNAP`/`SNAP_BUILD`** — item 5 reappears + as `Invalid engine json …/venv/…`. Re-check `scripts/o3de/o3de/manifest.py`. +- **Binary/library path change** — `o3de-wrapper.sh` finds the binary under + `*bin/Linux*` and sets `LD_LIBRARY_PATH`; adjust if `bin/Linux/profile/Default/` + moves. + +### How to debug runtime issues quickly (without a full rebuild) + +The expensive loop is: force-rebuild (~15 min) → publish → `flatpak update` → +re-download Python. You can usually avoid it. Because the installed app's payload is +just files in `/app`, you can **inspect and even re-run pieces in the live sandbox**: + +```sh +# Read any engine file: +flatpak run --command=bash org.o3de.O3DE -c 'cat /app/opt/O3DE/26.05/python/get_python.sh' + +# Re-run the Python bootstrap and watch it: +flatpak run --command=bash org.o3de.O3DE -c 'sh /app/opt/O3DE/*/python/get_python.sh' + +# Launch the binary with experimental env/args BEFORE baking them into the wrapper +# (this is how SNAP/SNAP_BUILD was validated). Use stdbuf so output flushes on kill, +# and write logs to $HOME (the sandbox /tmp is private, but home is shared): +timeout 40 flatpak run --command=bash org.o3de.O3DE -c ' + export SNAP=/app/opt/O3DE SNAP_BUILD=26.05 + BIN=$(find /app/opt/O3DE -type f -name o3de -path "*bin/Linux*" | head -1) + export LD_LIBRARY_PATH="$(dirname "$BIN"):/app/lib:/app/opt/O3DE/lib" + stdbuf -oL -eL "$BIN" > "$HOME/o3de-test.log" 2>&1 +' +``` + +A `timeout` exit code of **124** means the process stayed alive (a window opened) — +which, by itself, is strong evidence that startup/engine-registration succeeded. + +The upstream source for cross-referencing is `github.com/o3de/o3de` at the matching +tag (e.g. **`2605.0`** for engine dir `26.05`). The files that mattered most here: +`python/get_python.sh`, `cmake/LYPython.cmake`, `scripts/o3de/o3de/manifest.py`, and +`Code/Framework/AzFramework/AzFramework/Application/Application.cpp`. + +--- + +## Open items / future hardening + +- **Signing.** The repo is published unsigned; users add it with GPG verification + disabled. See *Signing* in the README to add a key. +- **Exported icon.** Generic in the host menu until built on a bwrap-capable + (privileged) runner. +- **Engine-level user folder.** Cosmetic read-only warnings remain (item 6). If a + future O3DE exposes an override the bootstrap *doesn't* clobber, wire it into the + wrapper. +- **Project create/build end-to-end.** Launch is solid; the first real project build + is the next thing to exercise, since it's the first time the runtime compile path + and project asset cache are used in anger.