CI: also build on push to main (code changes), keep cron for new releases

- push to main triggers a rebuild of the current version so build changes get
  tested; scoped to main so the job's pages/tag pushes can't loop it, and
  docs-only paths are ignored to avoid wasteful full rebuilds.
- the decide step treats a push (like a manual force) as build=true even when
  the version tag already exists.
- cron still handles new upstream binary releases as before.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

.gitea/workflows/build-flatpak.yml

@@ -1,8 +1,22 @@
 name: Build and Publish O3DE Flatpak
 
 on:
+  # New O3DE binary release: the daily check rebuilds when upstream's version has
+  # no matching vX.Y.Z tag yet.
   schedule:
-    - cron: '0 2 * * *'          # daily at 02:00 - checks for a new O3DE release
+    - cron: '0 2 * * *'          # daily at 02:00
+  # Code change: rebuild whenever the build itself changes. Scoped to `main` on
+  # purpose - the job force-pushes the `pages` branch and creates tags, and those
+  # must NOT re-trigger it (that would loop). Docs-only changes are ignored so a
+  # README/typo edit doesn't kick off a ~15-min, multi-GB build.
+  push:
+    branches:
+      - main
+    paths-ignore:
+      - 'README.md'
+      - 'docs/**'
+      - 'LICENSE*'
+      - '.gitignore'
   workflow_dispatch:
     inputs:
       force:
@@ -26,11 +40,10 @@ jobs:
       - name: Install build dependencies
         run: |
           apt-get update
-          # cmake + python3: O3DE's get_python.sh (run on the host, not in the
-          # sandbox) uses cmake to fetch its Python runtime.
           DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
             ca-certificates curl git jq xz-utils zstd binutils tar \
-            flatpak cmake python3
+            python3 python3-setuptools \
+            flatpak
 
       # Done as a plain git clone instead of actions/checkout@v4: the bare
       # ubuntu image has no Node.js, so JavaScript actions fail with exit 127.
@@ -62,9 +75,11 @@ jobs:
         id: check
         run: |
           git config --global --add safe.directory "$GITHUB_WORKSPACE"
-          if [ "${{ inputs.force }}" = "true" ]; then
+          # A code push (or a manual force) rebuilds even if this version is already
+          # published - the point is to test the changed build against it.
+          if [ "${{ inputs.force }}" = "true" ] || [ "${{ github.event_name }}" = "push" ]; then
             echo "build=true" >> "$GITHUB_OUTPUT"
-            echo "Force build requested."
+            echo "Rebuild requested (${{ github.event_name }})."
           elif git ls-remote --tags origin "refs/tags/v${{ steps.ver.outputs.version }}" | grep -q .; then
             echo "build=false" >> "$GITHUB_OUTPUT"
             echo "v${{ steps.ver.outputs.version }} already published - nothing to do."
@@ -144,5 +159,10 @@ jobs:
           TOKEN: ${{ secrets.PUBLISH_TOKEN != '' && secrets.PUBLISH_TOKEN || secrets.GITHUB_TOKEN }}
         run: |
           AUTH_URL="$(echo "${{ github.server_url }}" | sed "s#://#://${{ github.actor }}:${TOKEN}@#")/${{ github.repository }}.git"
-          git tag "v${{ steps.ver.outputs.version }}"
-          git push "$AUTH_URL" "v${{ steps.ver.outputs.version }}"
+          TAG="v${{ steps.ver.outputs.version }}"
+          if git ls-remote --tags "$AUTH_URL" "refs/tags/$TAG" | grep -q .; then
+            echo "Tag $TAG already exists (force rebuild) - leaving it in place."
+          else
+            git tag "$TAG"
+            git push "$AUTH_URL" "$TAG"
+          fi

README.md

@@ -57,6 +57,13 @@ it (`ar` + `tar`) and copies the payload into the Flatpak's `/app`. The version
 directory inside the `.deb` changes every release, so the wrapper discovers the
 executable at runtime rather than hard-coding a path.
 
+> **Want the full story?** O3DE is an engine + SDK that compiles code and writes
+> into its own install tree at runtime, while a Flatpak gives it a **read-only
+> `/app`**. Reconciling those two facts is most of the work here.
+> **[`docs/DESIGN.md`](docs/DESIGN.md)** documents every non-obvious decision, the
+> read-only-`/app` problems and their fixes, and how to debug/maintain this across
+> O3DE releases. Read it before changing the build.
+
 ---
 
 ## CI requirements (Gitea Actions)
@@ -78,17 +85,24 @@ The workflow targets a **self-hosted `act_runner`**. Because O3DE is large:
   access and add it as a secret named **`PUBLISH_TOKEN`** — the workflow prefers
   it automatically.
 
-Trigger it manually from the Gitea Actions UI (`workflow_dispatch`, with an
-optional **force** rebuild), or let the daily `cron` run it. It only rebuilds when
-the upstream version has no matching `vX.Y.Z` tag yet, so reruns are cheap no-ops.
+It runs on three triggers:
+
+- **`push` to `main`** — rebuilds whenever the build itself changes (always builds,
+  even for an already-published version, so you can test your change). Docs-only
+  changes (`README.md`, `docs/**`, `LICENSE*`, `.gitignore`) are ignored. Scoped to
+  `main` so the `pages`/tag pushes the job makes don't re-trigger it.
+- **daily `cron`** — picks up a **new O3DE binary release**; only builds when the
+  upstream version has no matching `vX.Y.Z` tag yet, so most days are cheap no-ops.
+- **`workflow_dispatch`** — manual run from the Gitea Actions UI, with an optional
+  **force** rebuild.
 
 ---
 
 ## Building locally
 
 ```sh
-# flatpak + cmake (cmake is used by O3DE's get_python.sh) + curl
-sudo apt install flatpak cmake      # or your distro's equivalent
+# flatpak + curl + python3/setuptools (to pre-build the o3de sdist)
+sudo apt install flatpak python3-setuptools      # or your distro's equivalent
 ./scripts/build.sh
 ```
 
@@ -127,18 +141,50 @@ These were confirmed by inspecting the v26.05 package (`opt/O3DE/26.05/…`):
   engine compiles project code at runtime. Those live in the SDK. Users therefore
   pull the SDK runtime (larger than Platform) on install — Flatpak does this
   automatically from Flathub.
-- **Python is baked in at build time.** O3DE normally downloads its Python runtime
-  into its own install tree on first use, but that tree is read-only inside a
-  Flatpak. The manifest runs `python/get_python.sh` during the build (with network
-  access) so Python is part of the immutable image. **This is the most likely step
-  to need tweaking** — verify it on the first real CI build.
-- **Runtime writes into the install tree may still fail.** Anything O3DE tries to
-  `pip install` or generate *inside* `/opt/O3DE/...` at runtime (e.g. per-gem
-  Python deps when building certain projects) will hit the read-only `/app`. Base
-  project building should work; exotic gems may not. This is the main open risk.
-- **No launcher icon yet.** The `.deb` ships only in-editor asset icons, so the
-  desktop entry uses a generic icon. Drop a real O3DE logo into the repo and
-  install it in the manifest to fix this.
+- **Python is per-user, not baked in.** O3DE needs a Python venv under `~/.o3de`
+  (writable), keyed to the engine path, so it can only be created at runtime — the
+  build can't bundle it. The Project Manager GUI does **not** create it itself, so
+  the wrapper (`o3de-wrapper.sh`) runs `python/get_python.sh` on launch when the
+  venv is missing or stale. The **first launch downloads Python and can take
+  several minutes with no window on screen** — that's expected; subsequent launches
+  are instant. (cmake, which `get_python.sh` needs, comes from the Sdk runtime.)
+- **The o3de CLI install (read-only `/app`).** On first launch O3DE installs its
+  own `o3de` CLI from `scripts/o3de` into the per-user `~/.o3de` venv. By default
+  `python/get_python.sh` does an *editable* install (`pip install -e`), which
+  writes an `egg-info` next to the source under read-only `/app` and fails.
+  `get_python.sh` has a built-in immutable-install hook (the Snap path): if a
+  prebuilt sdist exists at `scripts/o3de/dist/o3de-1.0.0.tar.gz` it installs that
+  (non-editable) instead. The build pre-creates that tarball (`python3 setup.py
+  sdist`) while the tree is still writable, so at runtime the CLI lands in the
+  writable venv and nothing is written to `/app`. (The build also patches every
+  `cmake/LYPython.cmake` to a non-editable install, covering the separate
+  project-build CMake-configure path.) If a future release renames the package or
+  bumps its version away from `1.0.0`, the build fails fast — those are the lines
+  to re-check.
+- **Engine-path resolution needs `SNAP`/`SNAP_BUILD`.** Because the `o3de` CLI is
+  installed non-editably, its package lives in the per-user venv, so O3DE's
+  `manifest.get_this_engine_path()` (which walks up from the package `__file__`)
+  would resolve into the venv and fail to find `engine.json`. O3DE has a built-in
+  override for immutable installs: when `SNAP` and `SNAP_BUILD` are set it returns
+  `$SNAP/$SNAP_BUILD`. The wrapper sets them to the engine root. (Confirmed these
+  vars are read *only* there — no other snap-specific behaviour is triggered.)
+- **The Project Manager's own user folder is read-only (cosmetic).** With no
+  project open, O3DE puts its engine-level *user* folder (asset cache, metrics,
+  `UserSettings.xml`) under `<engine>/.../user`, i.e. read-only `/app`. You'll see
+  `Couldn't find a valid asset cache folder…`, `cannot write to temp file …
+  UserSettings.xml`, and a metrics warning at startup. These are **non-fatal** —
+  the Project Manager runs fine; it just doesn't persist its *own* window/UI
+  settings between runs. The documented override
+  (`/O3DE/Runtime/FilePaths/SourceProjectUserPath`) does **not** help: O3DE
+  recomputes that path from the engine root during bootstrap and overwrites a
+  `--regset`. Real project data (projects you create, their caches) lives under
+  your home directory and is fully writable, so actual work is unaffected.
+- **Launcher icon.** Shipped as `org.o3de.O3DE.png` (the `.deb` itself has only
+  in-editor asset icons). It's kept *inside* the app so the running window/taskbar
+  shows it, but removed from the *exported* set because `flatpak build-export`
+  validates exported icons in a bwrap sandbox that fails in an unprivileged
+  container. Consequence: the host menu launcher icon is generic. A privileged
+  runner would let us export it properly.
 - **GPU / drivers:** the renderer needs working GPU access. The manifest grants
   `--device=dri`/`--device=all`; on some setups you may also want the matching
   GPU driver extension from Flathub.

docs/DESIGN.md

@@ -0,0 +1,329 @@
+# Design & engineering notes
+
+This document explains **why** the O3DE Flatpak is built the way it is. The
+[README](../README.md) tells you how to use and operate it; this is the companion
+that records the reasoning and the hard-won lessons, so the next person to touch
+this repo (very possibly future-you) doesn't have to rediscover them.
+
+If you read only one thing, read [The core tension](#the-core-tension) and
+[The read-only `/app` saga](#the-read-only-app-saga). Almost every non-obvious
+decision in this repo flows from those two sections.
+
+---
+
+## The core tension
+
+O3DE was never designed to run from a read-only location. It is not a normal
+desktop app that you install and run — it is an **engine + SDK** that, at runtime:
+
+- **compiles C++** (your game project, gems) — so it ships a *build toolchain*
+  (clang/cmake/ninja/pkg-config), not just runtime libs;
+- **bootstraps its own Python** into a per-user virtualenv and `pip`-installs a CLI;
+- **writes into its own install tree** — asset caches, user settings, metrics,
+  Python `egg-info`, registration data.
+
+A Flatpak gives the app a **read-only `/app`** at runtime. So the entire project
+is, essentially, a series of negotiations between "O3DE wants to write here" and
+"that location is read-only." Every caveat in the README is one of those
+negotiations.
+
+The official Debian package assumes a normal writable `/opt/O3DE/<ver>/`. We take
+that exact payload and drop it into `/app/opt/O3DE/<ver>/`, then redirect or
+pre-bake everything O3DE would otherwise try to write.
+
+---
+
+## Packaging architecture
+
+### We repackage the `.deb`; we don't build from source
+
+Building O3DE from source is enormous (hours, tens of GB, a full toolchain). The
+project instead consumes the **official prebuilt Linux release**, which ships as a
+Debian package at a predictable URL:
+
+```
+https://o3debinaries.org/main/Latest/Linux/o3de_<ver>.deb   (+ .sha256)
+```
+
+`scripts/get-latest-version.sh` scrapes the latest filename, derives the version
+(`o3de_2605_0.deb` → `26.05.0`, engine dir `26.05`) and fetches the checksum.
+`scripts/make-flatpak.sh` extracts the `.deb` (`ar` + `tar`) and copies its
+`opt/O3DE/<ver>/` payload into the Flatpak's `/app`.
+
+The version directory name changes every release, so **nothing hard-codes it** —
+the wrapper and build discover paths at runtime/build time with `find`.
+
+### No `flatpak-builder`, no bubblewrap, no privileged container
+
+This was forced by the CI environment and turned out to be the single most
+important build decision.
+
+`flatpak-builder` runs **every build command inside a bubblewrap (`bwrap`)
+sandbox**, which needs user namespaces / a privileged container. In a stock Gitea
+Actions / `act_runner` job container that fails with:
+
+```
+bwrap: Creating new namespace failed: Operation not permitted
+```
+
+Marking the *runner's* container privileged did **not** fix it (wrong layer), and
+requiring privileged containers is a bad thing to demand of a self-hosted runner.
+
+The escape: our "build" is just *unpacking files*. We don't need a sandbox to do
+that. So `scripts/make-flatpak.sh` uses the lower-level primitives directly:
+
+```
+flatpak build-init   build-dir <app-id> <runtime> <sdk> <ver>
+# ... copy files into build-dir/files ...
+flatpak build-finish build-dir --command=… <finish-args…>
+flatpak build-export repo build-dir <branch>
+flatpak build-update-repo repo …
+```
+
+`build-init/finish/export` only touch files and the OSTree repo — **no `bwrap`** —
+so they run in a plain unprivileged container. `org.o3de.O3DE.yaml` (a real
+`flatpak-builder` manifest) is kept **only** as an alternative for anyone who *does*
+have a bwrap-capable builder. CI does not use it.
+
+### Runtime is `org.freedesktop.Sdk`, not `Platform`
+
+Counter-intuitive but essential. O3DE compiles project code at runtime, so its
+dependency list is a **build toolchain** (clang, cmake, ninja, pkg-config). Those
+live in the **SDK**, not the slimmer Platform runtime. Running against
+`org.freedesktop.Sdk//24.08` is what makes `get_python.sh` (which needs `cmake`)
+and project builds work inside the sandbox. End users therefore pull the larger SDK
+runtime from Flathub on install — that's expected, not a bug.
+
+### Hosting: a static OSTree repo on an orphan `pages` branch
+
+A Flatpak remote is just a directory of files (`summary`, `config`, `objects/…`).
+CI commits the generated `repo/` to an orphan **`pages`** branch and **force-pushes
+a single snapshot** (so the branch never accumulates multi-GB history). Gitea serves
+those files over raw URLs, and `o3de.flatpakrepo` points clients at them. If you
+ever front this with a real static host, only `Url=` in the `.flatpakrepo` changes.
+
+---
+
+## The read-only `/app` saga
+
+This is the heart of the project. Each item below is a distinct place O3DE tried to
+write into read-only `/app`, how it manifested, and how it was resolved. They were
+found **one at a time**, each revealed only after fixing the previous one.
+
+| # | Symptom | Root cause | Fix |
+|---|---------|-----------|-----|
+| 1 | `qt.qpa.xcb could not connect to display` | `--socket=fallback-x11` *suppresses* X11 on Wayland; O3DE's bundled Qt only ships the `xcb` plugin | Use `--socket=x11` unconditionally + `QT_QPA_PLATFORM=xcb` |
+| 2 | `setup.py develop … [Errno 30] Read-only file system` | `get_python.sh` does `pip install -e scripts/o3de`; editable mode writes `egg-info` into read-only `/app` | **Pre-build an sdist** so the runtime takes its non-editable branch (below) |
+| 3 | `No module named 'o3de'` | Half-built venv from a prior failure; the GUI never re-bootstraps Python | Wrapper runs `get_python.sh` itself; it's self-healing (below) |
+| 4 | `Missing python venv file … pyvenv.cfg` | Project Manager **GUI does not bootstrap Python** — it only errors if the venv is absent | Wrapper bootstraps the venv before launching (below) |
+| 5 | `Invalid engine json …/venv/…/lib/engine.json` → `Failed to initialize` | Non-editable install moved the `o3de` package into the venv, so `get_this_engine_path()` (walks up from `__file__`) resolves into the venv | Set `SNAP`/`SNAP_BUILD` to the engine root (below) |
+| 6 | `Couldn't find a valid asset cache folder` / `cannot write … UserSettings.xml` | Project-less Project Manager puts its *engine-level* user dir under read-only `/app` | **Non-fatal**; documented. The documented `--regset` override is overwritten by bootstrap (below) |
+| — | `… is not a valid icon: bwrap` during `build-export` | `build-export` validates exported icons in a bwrap sandbox (unavailable unprivileged) | Keep the icon *inside* the app, drop it from the *exported* set |
+
+### Python: the sdist trick (items 2)
+
+O3DE installs its own `o3de` CLI into the per-user venv. By default
+`python/get_python.sh` does:
+
+```sh
+pip install -e <engine>/scripts/o3de   # editable → writes egg-info into /app → fails
+```
+
+The key discovery: **`get_python.sh` already has a built-in immutable-install
+escape hatch** (it's how the official Snap package copes). It is gated on a
+pre-built sdist tarball:
+
+```sh
+if [ -f .../scripts/o3de/dist/o3de-1.0.0.tar.gz ]; then
+    pip install .../dist/o3de-1.0.0.tar.gz --no-deps   # non-editable ✅
+else
+    pip install -e .../scripts/o3de --no-deps           # editable ❌
+fi
+```
+
+So the build runs `python3 setup.py sdist` **while `/app` is still writable**,
+producing `scripts/o3de/dist/o3de-1.0.0.tar.gz`. At runtime the `-f` test passes,
+the CLI installs non-editably into the writable `~/.o3de` venv, and nothing touches
+`/app`. (`setup.py` hard-codes `version="1.0.0"`, hence the exact filename — the
+build fails fast if that ever changes.) The build also patches every
+`cmake/LYPython.cmake` to a non-editable install for the *separate* CMake-configure
+path used during a project build.
+
+> **Note — the first patch was on the wrong file.** Early on we patched only
+> `LYPython.cmake`, confirmed the patch landed, and *still* saw `setup.py develop`
+> at runtime. The tell was the failure path `…/python/../scripts/o3de`, which is how
+> **`get_python.sh`** builds the path, not `LYPython.cmake`. Two different installers;
+> the GUI's first-launch path is `get_python.sh`.
+
+### Python: the wrapper bootstraps the venv (items 3, 4)
+
+The Project Manager **GUI never runs `get_python.sh`**. It assumes the venv already
+exists and just errors (`Missing python venv file … pyvenv.cfg`) if it doesn't —
+which then cascades into engine-registration failure and exit. Nothing on the
+desktop launches `get_python.sh` for you.
+
+So `o3de-wrapper.sh` does it. It checks `python.sh --version` (which exits non-zero
+when the venv for this engine is missing or stale) and, if so, runs `get_python.sh`
+before launching the GUI. This is:
+
+- **idempotent** — a healthy venv means `python.sh --version` succeeds and we skip;
+- **self-healing** — `get_python.sh` recreates the venv with `--clear`, so a
+  half-built `~/.o3de/Python/venv/<id>` left by an earlier failure is repaired.
+
+The venv lives at `~/.o3de/Python/venv/<engine-id>`, where `<engine-id>` is a hash
+of the engine path computed by cmake. The **first launch downloads Python and can
+take several minutes with no window on screen** — expected; later launches are
+instant.
+
+### Engine path: `SNAP` / `SNAP_BUILD` (item 5)
+
+A subtle consequence of fixing item 2. O3DE finds its own engine root in Python via:
+
+```python
+def get_this_engine_path():
+    if "SNAP" in os.environ and "SNAP_BUILD" in os.environ:
+        return pathlib.Path(os.environ["SNAP"]) / os.environ["SNAP_BUILD"]
+    return pathlib.Path(os.path.realpath(__file__)).parents[3].resolve()
+```
+
+With the old **editable** install, `__file__` lived in the engine tree
+(`/app/.../scripts/o3de/o3de/manifest.py`) and `parents[3]` correctly gave the
+engine root. Our **non-editable** install moved the package into the venv
+`site-packages`, so `parents[3]` now lands **inside the venv** → `engine.json` not
+found → `Failed to get engine info` → `Failed to initialize`.
+
+O3DE's own override saves us: when `SNAP` and `SNAP_BUILD` are set it skips the
+`__file__` walk and returns `$SNAP/$SNAP_BUILD`. The wrapper sets:
+
+```sh
+SNAP=/app/opt/O3DE          # dirname of the engine root
+SNAP_BUILD=26.05            # basename of the engine root  → $SNAP/$SNAP_BUILD = engine root
+```
+
+We verified (grep of the Python package **and** the C++ binaries/`.so`s) that these
+two variables are read **only** in `get_this_engine_path` — setting them triggers no
+other snap-specific behaviour. This was confirmed empirically by launching the
+binary with the vars set: the Project Manager opened cleanly with no `engine.json`
+errors.
+
+### The user folder we *couldn't* relocate (item 6)
+
+With no project open, O3DE places its **engine-level** user folder (asset cache,
+`UserSettings.xml`, metrics) under `<engine>/…/user` — read-only `/app`. You'll see
+warnings at startup; they are **non-fatal**. The Project Manager runs fine; it just
+doesn't persist *its own* UI settings between runs.
+
+The documented override is registry key
+`/O3DE/Runtime/FilePaths/SourceProjectUserPath`. We tested redirecting it with
+`--regset` to a writable path and it **did not work**: O3DE **recomputes** that path
+from the engine root during settings-registry bootstrap and overwrites the
+command-line value. (This was verified with line-buffered output capture — an
+earlier "it works!" reading was a buffering artifact from `timeout`-killing the
+process, which never flushed its full-buffered pipe. Lesson: when capturing a GUI
+process you'll kill, force `stdbuf -oL -eL`.)
+
+Real project data — the projects you create and their caches — lives under your home
+directory and is fully writable, so actual work is unaffected. We chose to document
+this rather than chase a fix that the engine actively undoes.
+
+### The icon
+
+The `.deb` ships only in-editor asset icons, no app icon, so we add
+`org.o3de.O3DE.png`. But `flatpak build-export` validates *exported* icons inside a
+bwrap sandbox — which, unprivileged, fails (`is not a valid icon: bwrap …`). The
+compromise: keep the icon **inside** the app (so the running window/taskbar shows
+it) but `rm -rf build-dir/export/share/icons` before export so the export step skips
+validation. The cost is a generic icon in the host's app menu. A privileged runner
+would let us export it properly.
+
+---
+
+## How a build flows (CI)
+
+`.gitea/workflows/build-flatpak.yml`, triggered by daily `cron` or
+`workflow_dispatch` (with an optional **force**):
+
+1. **Install deps** — `curl jq xz-utils zstd binutils tar python3 python3-setuptools
+   flatpak`. (`python3-setuptools` is needed to pre-build the o3de sdist.)
+2. **Checkout** — a plain `git init/remote/fetch/checkout`, *not* `actions/checkout`.
+   The bare `ubuntu:24.04` image has no Node.js, so JavaScript actions exit 127.
+3. **Resolve version** via `get-latest-version.sh`.
+4. **Decide to build** — skip if a `vX.Y.Z` tag already exists, unless `force`.
+5. **Install** `org.freedesktop.Sdk//24.08` from Flathub.
+6. **Download + checksum** the `.deb`.
+7. **Stamp** the version/date into `org.o3de.O3DE.metainfo.xml`.
+8. **Build** via `scripts/make-flatpak.sh`; delete `build-dir`/`o3de.deb` to reclaim
+   disk before publishing.
+9. **Generate** `o3de.flatpakrepo` pointing at the `pages` raw URL.
+10. **Publish** — force-push the `repo/` snapshot to the orphan `pages` branch.
+11. **Tag** `vX.Y.Z` (idempotent: skipped if it already exists, e.g. a force rebuild).
+
+Auth uses `PUBLISH_TOKEN` if set, otherwise the auto-provided `GITHUB_TOKEN`.
+
+---
+
+## Maintaining this across O3DE releases
+
+Most releases will "just work" because nothing hard-codes the version. The build is
+designed to **fail fast** at the spots most likely to break:
+
+- **Package version ≠ `1.0.0`** — the sdist filename `o3de-1.0.0.tar.gz` is what
+  `get_python.sh` looks for. If `scripts/o3de/setup.py` bumps its version, the build
+  errors at the sdist step. Re-check the filename in `get_python.sh` and update.
+- **Layout move** — if `scripts/o3de/setup.py` or `cmake/LYPython.cmake` disappears,
+  the build errors. Re-trace the install path (it has moved between the Snap-style
+  hook and `LYPython.cmake` before).
+- **`get_python.sh` drops the sdist hook** — then the editable install returns and
+  item 2 reappears. Re-check the `if [ -f …/dist/o3de-1.0.0.tar.gz ]` branch.
+- **`get_this_engine_path` stops honouring `SNAP`/`SNAP_BUILD`** — item 5 reappears
+  as `Invalid engine json …/venv/…`. Re-check `scripts/o3de/o3de/manifest.py`.
+- **Binary/library path change** — `o3de-wrapper.sh` finds the binary under
+  `*bin/Linux*` and sets `LD_LIBRARY_PATH`; adjust if `bin/Linux/profile/Default/`
+  moves.
+
+### How to debug runtime issues quickly (without a full rebuild)
+
+The expensive loop is: force-rebuild (~15 min) → publish → `flatpak update` →
+re-download Python. You can usually avoid it. Because the installed app's payload is
+just files in `/app`, you can **inspect and even re-run pieces in the live sandbox**:
+
+```sh
+# Read any engine file:
+flatpak run --command=bash org.o3de.O3DE -c 'cat /app/opt/O3DE/26.05/python/get_python.sh'
+
+# Re-run the Python bootstrap and watch it:
+flatpak run --command=bash org.o3de.O3DE -c 'sh /app/opt/O3DE/*/python/get_python.sh'
+
+# Launch the binary with experimental env/args BEFORE baking them into the wrapper
+# (this is how SNAP/SNAP_BUILD was validated). Use stdbuf so output flushes on kill,
+# and write logs to $HOME (the sandbox /tmp is private, but home is shared):
+timeout 40 flatpak run --command=bash org.o3de.O3DE -c '
+  export SNAP=/app/opt/O3DE SNAP_BUILD=26.05
+  BIN=$(find /app/opt/O3DE -type f -name o3de -path "*bin/Linux*" | head -1)
+  export LD_LIBRARY_PATH="$(dirname "$BIN"):/app/lib:/app/opt/O3DE/lib"
+  stdbuf -oL -eL "$BIN" > "$HOME/o3de-test.log" 2>&1
+'
+```
+
+A `timeout` exit code of **124** means the process stayed alive (a window opened) —
+which, by itself, is strong evidence that startup/engine-registration succeeded.
+
+The upstream source for cross-referencing is `github.com/o3de/o3de` at the matching
+tag (e.g. **`2605.0`** for engine dir `26.05`). The files that mattered most here:
+`python/get_python.sh`, `cmake/LYPython.cmake`, `scripts/o3de/o3de/manifest.py`, and
+`Code/Framework/AzFramework/AzFramework/Application/Application.cpp`.
+
+---
+
+## Open items / future hardening
+
+- **Signing.** The repo is published unsigned; users add it with GPG verification
+  disabled. See *Signing* in the README to add a key.
+- **Exported icon.** Generic in the host menu until built on a bwrap-capable
+  (privileged) runner.
+- **Engine-level user folder.** Cosmetic read-only warnings remain (item 6). If a
+  future O3DE exposes an override the bootstrap *doesn't* clobber, wire it into the
+  wrapper.
+- **Project create/build end-to-end.** Launch is solid; the first real project build
+  is the next thing to exercise, since it's the first time the runtime compile path
+  and project asset cache are used in anger.

o3de-wrapper.sh

@@ -27,4 +27,39 @@ fi
 # Make libraries that sit next to the binary discoverable too.
 export LD_LIBRARY_PATH="$(dirname "$O3DE_BIN"):${LD_LIBRARY_PATH}"
 
+# Bootstrap O3DE's per-user Python venv if needed. The Project Manager GUI does
+# NOT set this up itself - it just errors out ("Missing python venv file ...") if
+# the venv is absent. So we ensure it here. python.sh exits non-zero when the
+# venv for this engine is missing/stale; get_python.sh then downloads Python,
+# creates the venv (~/.o3de/Python, writable) and installs the 'o3de' CLI from
+# the prebuilt sdist. It is idempotent and self-healing (recreates a broken venv
+# with --clear). cmake, needed for the engine-id calc, ships in the Sdk runtime.
+PYTHON_SH=$(find "$O3DE_ROOT" -type f -path '*/python/python.sh' 2>/dev/null | head -n 1)
+GET_PYTHON=$(find "$O3DE_ROOT" -type f -path '*/python/get_python.sh' 2>/dev/null | head -n 1)
+if [ -n "$GET_PYTHON" ]; then
+    ENGINE_ROOT=$(dirname "$(dirname "$GET_PYTHON")")
+
+    # O3DE's Python derives the engine path from the o3de package's __file__. That
+    # breaks with our non-editable install (the package lives in the per-user venv,
+    # not the engine tree), so engine.json is looked for under the venv and not
+    # found ("Failed to get engine info"). O3DE has a built-in override for exactly
+    # this immutable-install case: if SNAP and SNAP_BUILD are set, manifest.py's
+    # get_this_engine_path() returns "$SNAP/$SNAP_BUILD". Point them at the engine
+    # root. (Verified nothing else in O3DE - Python or the C++ binaries - reads
+    # these two vars, so there are no snap-specific side effects.)
+    SNAP=$(dirname "$ENGINE_ROOT")
+    SNAP_BUILD=$(basename "$ENGINE_ROOT")
+    export SNAP SNAP_BUILD
+
+    # Bootstrap the per-user Python venv if needed (the GUI doesn't do it itself).
+    if ! "$PYTHON_SH" --version >/dev/null 2>&1; then
+        echo "O3DE: setting up the per-user Python environment." >&2
+        echo "O3DE: first launch downloads Python and can take several minutes..." >&2
+        if ! "$GET_PYTHON"; then
+            echo "error: O3DE Python setup failed (see the log above)." >&2
+            exit 1
+        fi
+    fi
+fi
+
 exec "$O3DE_BIN" "$@"

org.o3de.O3DE.yaml

@@ -16,8 +16,7 @@ command: o3de-wrapper.sh
 finish-args:
   - --share=ipc
   - --share=network            # Project Manager fetches gems/templates; engine downloads assets
-  - --socket=x11
-  - --socket=fallback-x11
+  - --socket=x11               # O3DE's bundled Qt only ships the xcb plugin, so X11/XWayland is required
   - --socket=wayland
   - --socket=pulseaudio
   - --device=dri               # GPU access for the renderer
@@ -29,11 +28,6 @@ finish-args:
 modules:
   - name: o3de
     buildsystem: simple
-    # The Python bootstrap step (below) downloads O3DE's Python runtime into the
-    # install tree, so this module needs network access during the build.
-    build-options:
-      build-args:
-        - --share=network
     build-commands:
       # The .deb is an `ar` archive containing data.tar.{gz,xz,zst}.
       - ar x o3de.deb
@@ -43,25 +37,33 @@ modules:
       # is kept for robustness in case a future release adds desktop glue there.)
       - 'if [ -d data/opt ]; then mkdir -p "${FLATPAK_DEST}/opt"; cp -a data/opt/. "${FLATPAK_DEST}/opt/"; fi'
       - 'if [ -d data/usr ]; then cp -a data/usr/. "${FLATPAK_DEST}/"; fi'
-      # O3DE fetches its own Python runtime on first use, writing into its install
-      # tree. That tree is read-only at runtime in a Flatpak, so bake Python in now
-      # while ${FLATPAK_DEST} is still writable.
+      # On first launch O3DE installs its 'o3de' CLI from scripts/o3de. By default
+      # python/get_python.sh does an *editable* install ('pip install -e'), which
+      # writes an egg-info into read-only /app and fails. get_python.sh has a
+      # built-in immutable-install hook (the Snap path): if a prebuilt sdist exists
+      # at scripts/o3de/dist/o3de-1.0.0.tar.gz it installs that (non-editable)
+      # instead. Build the tarball now, while /app is still writable.
       - |
         set -e
-        ENGINE_DIR=$(find "${FLATPAK_DEST}/opt/O3DE" -mindepth 1 -maxdepth 1 -type d | head -n1)
-        echo "Engine dir: ${ENGINE_DIR}"
-        if [ -x "${ENGINE_DIR}/python/get_python.sh" ]; then
-          ( cd "${ENGINE_DIR}" && HOME="${PWD}" ./python/get_python.sh )
+        O3DE_PKG=$(find "${FLATPAK_DEST}/opt/O3DE" -maxdepth 3 -type d -path '*/scripts/o3de' | head -n1)
+        if [ -n "$O3DE_PKG" ] && [ -f "$O3DE_PKG/setup.py" ]; then
+          ( cd "$O3DE_PKG" && python3 setup.py sdist )
+          test -f "$O3DE_PKG/dist/o3de-1.0.0.tar.gz"
         else
-          echo "::warning:: get_python.sh not found; Python may fail at runtime"
+          echo "scripts/o3de/setup.py not found; O3DE layout may have changed" >&2
+          exit 1
         fi
+        # Belt-and-suspenders: LYPython.cmake also installs o3de (editable) during a
+        # project-build CMake configure. Patch every copy to non-editable.
+        find "${FLATPAK_DEST}/opt/O3DE" -path '*/cmake/LYPython.cmake' | while read -r LYPYTHON; do
+          grep -qF -- '-m pip install -e ' "$LYPYTHON" && \
+            sed -i 's/-m pip install -e /-m pip install /g' "$LYPYTHON" || true
+        done
       # Launcher + AppStream + desktop entry under the Flatpak app-id.
       - install -Dm755 o3de-wrapper.sh "${FLATPAK_DEST}/bin/o3de-wrapper.sh"
       - install -Dm644 org.o3de.O3DE.desktop "${FLATPAK_DEST}/share/applications/org.o3de.O3DE.desktop"
       - install -Dm644 org.o3de.O3DE.metainfo.xml "${FLATPAK_DEST}/share/metainfo/org.o3de.O3DE.metainfo.xml"
-      # NOTE: the .deb ships no clean application icon (only in-editor asset icons),
-      # so none is installed; the desktop entry falls back to a generic icon. Drop a
-      # real logo into the repo and install it here to fix the launcher icon.
+      - install -Dm644 org.o3de.O3DE.png "${FLATPAK_DEST}/share/icons/hicolor/256x256/apps/org.o3de.O3DE.png"
     sources:
       - type: file
         path: o3de.deb
@@ -71,3 +73,5 @@ modules:
         path: org.o3de.O3DE.desktop
       - type: file
         path: org.o3de.O3DE.metainfo.xml
+      - type: file
+        path: org.o3de.O3DE.png

scripts/make-flatpak.sh

@@ -29,21 +29,44 @@ tar -C data -xf data.tar.*
 mkdir -p "$DEST/opt"
 cp -a data/opt/. "$DEST/opt/"
 
-echo ">> baking O3DE's Python runtime into the image"
-# O3DE fetches Python into its install tree on first use, but that tree is
-# read-only at runtime in a Flatpak, so populate it now while it is writable.
-ENGINE_DIR=$(find "$DEST/opt/O3DE" -mindepth 1 -maxdepth 1 -type d | head -n1)
-echo "   engine dir: $ENGINE_DIR"
-if [ -x "$ENGINE_DIR/python/get_python.sh" ]; then
-  ( cd "$ENGINE_DIR" && HOME="$PWD" ./python/get_python.sh )
+echo ">> pre-building the o3de sdist (read-only /app workaround)"
+# On first launch O3DE builds a per-user Python venv in ~/.o3de and installs its
+# 'o3de' CLI from <engine>/scripts/o3de. By default python/get_python.sh does an
+# *editable* install ('pip install -e'), which writes an egg-info next to the
+# source under read-only /app and fails. get_python.sh has a built-in escape hatch
+# for immutable installs (the Snap path): if a prebuilt sdist exists at
+# scripts/o3de/dist/o3de-1.0.0.tar.gz it installs THAT (non-editable) instead. So
+# we build that tarball here, while the tree is still writable. At runtime the
+# tarball is copied into the writable ~/.o3de venv - nothing is written to /app.
+O3DE_PKG=$(find "$DEST/opt/O3DE" -maxdepth 3 -type d -path '*/scripts/o3de' | head -n1)
+if [ -n "$O3DE_PKG" ] && [ -f "$O3DE_PKG/setup.py" ]; then
+  ( cd "$O3DE_PKG" && python3 setup.py sdist )
+  if [ -f "$O3DE_PKG/dist/o3de-1.0.0.tar.gz" ]; then
+    echo "   built: $O3DE_PKG/dist/o3de-1.0.0.tar.gz"
+  else
+    echo "   ERROR: sdist did not produce o3de-1.0.0.tar.gz" >&2
+    exit 1
+  fi
 else
-  echo "   WARNING: get_python.sh not found; Python may fail at runtime" >&2
+  echo "   ERROR: scripts/o3de/setup.py not found; O3DE layout may have changed" >&2
+  exit 1
 fi
 
+# Belt-and-suspenders for the project-build path: LYPython.cmake also installs the
+# o3de package (editable) during a CMake configure when you build a project. Patch
+# every copy to a non-editable install so that write doesn't hit read-only /app.
+find "$DEST/opt/O3DE" -path '*/cmake/LYPython.cmake' | while read -r LYPYTHON; do
+  if grep -qF -- '-m pip install -e ' "$LYPYTHON"; then
+    sed -i 's/-m pip install -e /-m pip install /g' "$LYPYTHON"
+    echo "   patched: $LYPYTHON"
+  fi
+done
+
 echo ">> installing launcher + metadata"
 install -Dm755 o3de-wrapper.sh "$DEST/bin/o3de-wrapper.sh"
 install -Dm644 org.o3de.O3DE.desktop "$DEST/share/applications/$APP_ID.desktop"
 install -Dm644 org.o3de.O3DE.metainfo.xml "$DEST/share/metainfo/$APP_ID.metainfo.xml"
+install -Dm644 org.o3de.O3DE.png "$DEST/share/icons/hicolor/256x256/apps/$APP_ID.png"
 
 echo ">> build-finish (command + sandbox permissions)"
 flatpak build-finish build-dir \
@@ -51,7 +74,6 @@ flatpak build-finish build-dir \
   --share=ipc \
   --share=network \
   --socket=x11 \
-  --socket=fallback-x11 \
   --socket=wayland \
   --socket=pulseaudio \
   --device=dri \
@@ -61,6 +83,12 @@ flatpak build-finish build-dir \
   --env=QT_QPA_PLATFORM=xcb
 
 echo ">> export to OSTree repo"
+# flatpak build-export validates exported app icons in a bwrap sandbox, which
+# fails in an unprivileged container ("is not a valid icon: bwrap ..."). The
+# icon stays inside the app (so the running window/taskbar shows it); we just
+# drop it from the *exported* set so export skips validation. Trade-off: the
+# host menu launcher icon is generic. (A privileged runner would avoid this.)
+rm -rf build-dir/export/share/icons
 flatpak build-export repo build-dir "$BRANCH"
 flatpak build-update-repo repo --title="O3DE (unofficial Flatpak)" --prune --prune-depth=1