17 Commits

Author SHA1 Message Date
pc-heini c1393383ba build: export the app icon via --disable-sandbox instead of dropping it
Build and Publish O3DE Flatpak / build (push) Successful in 13m9s
flatpak build-export validates exported icons in a bwrap sandbox that
fails in an unprivileged CI container. The previous workaround stripped
build-dir/export/share/icons entirely, but flatpak exports the host icon
from that tree at install time, so the host menu and the window/alt-tab
icon were left generic on every machine - even ones where validation
would have passed.

Use 'flatpak build-export --disable-sandbox', which runs the same icon
validation in-process (no bwrap/userns) and keeps the icon in the export,
so it reaches the host.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 05:32:29 +02:00
pc-heini 70d370e8e2 docs: drop <owner> placeholder and fix repo host
Replace the <owner> stub and stale gitea.pc-heini.de host with the
real git.pc-heini.de/pc-heini URL in the README install snippet and
o3de.flatpakrepo. Retitle the caveats section now that the items are
confirmed rather than pending verification.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 05:24:54 +02:00
pc-heini 2144e11ac3 CI: also build on push to main (code changes), keep cron for new releases
Build and Publish O3DE Flatpak / build (push) Successful in 15m28s
- push to main triggers a rebuild of the current version so build changes get
  tested; scoped to main so the job's pages/tag pushes can't loop it, and
  docs-only paths are ignored to avoid wasteful full rebuilds.
- the decide step treats a push (like a manual force) as build=true even when
  the version tag already exists.
- cron still handles new upstream binary releases as before.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 03:55:27 +02:00
pc-heini a2a52f83ea docs: add DESIGN.md engineering notes and link from README
Captures the why behind the build: the read-only /app core tension, the
packaging architecture (no flatpak-builder/bwrap, Sdk-not-Platform, OSTree on
pages), the full read-only-/app saga (X11, Python sdist hook, wrapper venv
bootstrap, SNAP/SNAP_BUILD engine path, the unrelocatable user folder, the
icon), the CI flow, and a maintenance/debugging guide for future O3DE releases.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 03:52:36 +02:00
pc-heini fe8661476e Document the read-only engine user-folder caveat
The non-editable install requires SNAP/SNAP_BUILD for engine-path resolution
(now in the wrapper). The engine-level user folder (cache/metrics/UserSettings)
still resolves under read-only /app for the projectless Project Manager; tested
that the SourceProjectUserPath regset is overwritten by O3DE's bootstrap and
does not help. These writes are non-fatal - documenting rather than chasing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 03:49:31 +02:00
pc-heini 280238bbd7 Point O3DE's engine-path resolution at the read-only engine root
With the non-editable o3de install, the package lives in the per-user venv,
so manifest.get_this_engine_path() (which walks up from the package __file__)
resolved into the venv and engine.json wasn't found -> "Failed to get engine
info" / "Failed to initialize". O3DE has a built-in override for this
immutable-install case: when SNAP and SNAP_BUILD are set, get_this_engine_path
returns "$SNAP/$SNAP_BUILD". The wrapper now sets them to the engine root.

Verified by launching with these vars set: the Project Manager opens with no
engine.json errors.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 03:43:59 +02:00
pc-heini d83526cbf0 Bootstrap the per-user Python venv from the wrapper
The Project Manager GUI does not set up O3DE's Python venv itself; it only
errors ("Missing python venv file .../pyvenv.cfg") when it's absent, which
cascades into "Failed to get engine info" / "Failed to initialize". Nothing
was running get_python.sh on launch.

The wrapper now runs python/get_python.sh when python.sh reports the venv is
missing/stale. get_python.sh downloads Python, creates the venv under the
writable ~/.o3de, and installs the o3de CLI from the prebuilt sdist. It is
idempotent and self-heals a half-built venv (venv creation uses --clear).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 03:04:52 +02:00
pc-heini 3146cab449 Fix o3de CLI install on read-only /app via prebuilt sdist
The real first-launch installer is python/get_python.sh, not LYPython.cmake.
It does `pip install -e scripts/o3de`, whose egg_info write hits read-only
/app and fails ([Errno 30]). get_python.sh has a built-in immutable-install
hook: if scripts/o3de/dist/o3de-1.0.0.tar.gz exists it installs that
(non-editable) instead. Pre-build that tarball at Flatpak build time
(python3 setup.py sdist) so the runtime CLI lands in the writable ~/.o3de
venv. Verified end-to-end: tarball install -> `import o3de` resolves from
site-packages.

Also patch every LYPython.cmake copy (not just the first) for the separate
project-build CMake-configure path, and add python3/python3-setuptools to CI.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 02:41:20 +02:00
pc-heini 8e3fd053d8 CI: make the tag step idempotent on force rebuilds
Force-rebuilding an already-published version made the final tag push fail
("tag already exists"), turning an otherwise successful run red. Skip
tagging when the tag already exists on the remote.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 12:33:27 +02:00
pc-heini 583c81ed51 Fix editable-install patch for 26.05 + unblock icon in unprivileged build
The previous patch matched a development-branch variable that does not
exist in release 2605.0, so the warning fired and the editable install was
not patched. In 2605.0 the -e is hardcoded in the pip command
("pip install -e ${package_folder_path}"); strip the -e there instead.

Also, flatpak build-export validates exported app icons in a bwrap sandbox,
which fails in an unprivileged container. Keep the icon inside the app (for
the running window/taskbar) but remove it from build-dir/export so export
skips validation. Host menu icon stays generic; build needs no privileges.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 12:17:11 +02:00
pc-heini f4d2dddd02 Fix read-only /app failure + add app icon
On first launch O3DE pip-installs its 'o3de' CLI with 'pip install -e'
(editable), which writes an egg-info next to the source under read-only
/app and fails ([Errno 30] Read-only file system), leaving the venv broken
("unable to install O3DE's built-in Python").

- Patch cmake/LYPython.cmake at build time to force a normal (non-editable)
  install, which builds in a temp dir and lands in the writable ~/.o3de venv.
- Drop the build-time get_python "bake": the venv is per-user (keyed to
  $HOME), so it can only be created at runtime; baking under a throwaway
  build HOME did nothing. Speeds up CI; removes cmake/python3 build deps and
  the network build-arg.
- Add a real launcher icon (org.o3de.O3DE.png).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 12:02:41 +02:00
pc-heini 759c49bbeb Fix X11 display: share x11 unconditionally (drop fallback-x11)
O3DE's bundled Qt only ships the xcb platform plugin, so it needs X11/
XWayland. Using --socket=fallback-x11 alongside --socket=wayland meant X11
was not shared on Wayland sessions, leaving DISPLAY empty and Qt unable to
connect. Share --socket=x11 unconditionally instead.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 11:44:10 +02:00
pc-heini 44d64262e2 CI: install cmake/python3 for O3DE's get_python.sh
get_python.sh runs on the host shell and shells out to cmake to fetch the
engine's Python runtime, but the bare ubuntu container had neither cmake
nor python3. Add them (and tar) to the dependency install.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 10:11:14 +02:00
pc-heini 5ae57f3bbf CI: build without flatpak-builder to avoid bwrap/privileged requirement
flatpak-builder sandboxes each build command in bubblewrap, which needs
user namespaces / a privileged job container that Gitea act_runner does
not grant by default (bwrap: Creating new namespace failed).

Replace it with scripts/make-flatpak.sh, which uses flatpak
build-init/build-finish/build-export plus plain-shell extraction and the
get_python.sh bake. None of these use bwrap, so an unprivileged container
works. The flatpak-builder manifest stays as a documented alternative.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 10:04:32 +02:00
pc-heini ff03166642 CI: replace actions/checkout with a shell git clone
The bare ubuntu:24.04 container has no Node.js, so the JS-based
actions/checkout@v4 fails with exit 127. Clone via plain git instead.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 09:46:22 +02:00
pc-heini 81b5856cbd Fix manifest for O3DE runtime reality found via .deb inspection
Inspecting o3de_2605_0.deb (control.tar md5sums) showed:
- everything installs under /opt/O3DE/<ver>/; no /usr glue, no app icon
- ~270 .so files sit beside the o3de launcher (LD_LIBRARY_PATH is required)
- the package depends on a build toolchain (clang/ninja/cmake/pkg-config)
- no Python runtime is bundled; O3DE fetches it into its install tree on
  first use, which is read-only in a Flatpak

Changes:
- run against org.freedesktop.Sdk so the build toolchain is present at runtime
- bake O3DE's Python in at build time via get_python.sh (network build-arg)
- drop the icon auto-grab (it would have installed a random asset SVG)
- document the remaining open risk: runtime writes into the read-only /app

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 09:37:18 +02:00
pc-heini 5421db2960 Initial O3DE Flatpak packaging + Gitea CI
Repackage the official O3DE Linux .deb as a Flatpak and publish it as a
static OSTree repo on the 'pages' branch for install via flatpak remote.

- org.o3de.O3DE.yaml: flatpak-builder manifest (extracts the .deb into /app)
- o3de-wrapper.sh: launcher that locates the versioned o3de binary at runtime
- desktop + AppStream metadata under the app-id
- scripts/: live version resolver and local build helper
- .gitea/workflows/build-flatpak.yml: daily check -> build -> publish -> tag

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 09:29:55 +02:00
19049 changed files with 1088 additions and 10 deletions
+168
View File
@@ -0,0 +1,168 @@
name: Build and Publish O3DE Flatpak
on:
# New O3DE binary release: the daily check rebuilds when upstream's version has
# no matching vX.Y.Z tag yet.
schedule:
- cron: '0 2 * * *' # daily at 02:00
# Code change: rebuild whenever the build itself changes. Scoped to `main` on
# purpose - the job force-pushes the `pages` branch and creates tags, and those
# must NOT re-trigger it (that would loop). Docs-only changes are ignored so a
# README/typo edit doesn't kick off a ~15-min, multi-GB build.
push:
branches:
- main
paths-ignore:
- 'README.md'
- 'docs/**'
- 'LICENSE*'
- '.gitignore'
workflow_dispatch:
inputs:
force:
description: 'Rebuild even if this version was already published'
type: boolean
default: false
permissions:
contents: write
jobs:
build:
# Adjust the label to match your registered act_runner. The runner needs a
# lot of free disk (O3DE is ~15-18 GB installed; the build needs ~2-3x that)
# and the container must be privileged so Flatpak's sandbox (bubblewrap) works.
runs-on: ubuntu-latest
container:
image: ubuntu:24.04
options: --privileged
steps:
- name: Install build dependencies
run: |
apt-get update
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
ca-certificates curl git jq xz-utils zstd binutils tar \
python3 python3-setuptools \
flatpak
# Done as a plain git clone instead of actions/checkout@v4: the bare
# ubuntu image has no Node.js, so JavaScript actions fail with exit 127.
- name: Checkout
env:
TOKEN: ${{ secrets.PUBLISH_TOKEN != '' && secrets.PUBLISH_TOKEN || secrets.GITHUB_TOKEN }}
run: |
AUTH_URL="$(echo "${GITHUB_SERVER_URL}" | sed "s#://#://${GITHUB_ACTOR}:${TOKEN}@#")/${GITHUB_REPOSITORY}.git"
git init -q
git config --global --add safe.directory "$GITHUB_WORKSPACE"
git remote add origin "$AUTH_URL"
git fetch -q --depth 1 origin "${GITHUB_SHA:-$GITHUB_REF_NAME}"
git checkout -q FETCH_HEAD
- name: Resolve latest O3DE version
id: ver
run: |
chmod +x scripts/*.sh
eval "$(scripts/get-latest-version.sh)"
{
echo "version=$version"
echo "deb_url=$deb_url"
echo "deb_file=$deb_file"
echo "sha256=$sha256"
} >> "$GITHUB_OUTPUT"
echo "Latest O3DE: $version ($deb_file)"
- name: Decide whether to build
id: check
run: |
git config --global --add safe.directory "$GITHUB_WORKSPACE"
# A code push (or a manual force) rebuilds even if this version is already
# published - the point is to test the changed build against it.
if [ "${{ inputs.force }}" = "true" ] || [ "${{ github.event_name }}" = "push" ]; then
echo "build=true" >> "$GITHUB_OUTPUT"
echo "Rebuild requested (${{ github.event_name }})."
elif git ls-remote --tags origin "refs/tags/v${{ steps.ver.outputs.version }}" | grep -q .; then
echo "build=false" >> "$GITHUB_OUTPUT"
echo "v${{ steps.ver.outputs.version }} already published - nothing to do."
else
echo "build=true" >> "$GITHUB_OUTPUT"
echo "New version v${{ steps.ver.outputs.version }} - building."
fi
- name: Install Flatpak runtime and SDK
if: steps.check.outputs.build == 'true'
run: |
flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
flatpak install -y flathub org.freedesktop.Platform//24.08 org.freedesktop.Sdk//24.08
- name: Download O3DE .deb
if: steps.check.outputs.build == 'true'
run: |
curl -fL --retry 3 -o o3de.deb "${{ steps.ver.outputs.deb_url }}"
if [ -n "${{ steps.ver.outputs.sha256 }}" ]; then
echo "${{ steps.ver.outputs.sha256 }} o3de.deb" | sha256sum -c -
else
echo "::warning::No published checksum; skipping verification."
fi
- name: Stamp version into AppStream metadata
if: steps.check.outputs.build == 'true'
run: |
sed -i -E \
"s#<release version=\"[^\"]*\" date=\"[^\"]*\">#<release version=\"${{ steps.ver.outputs.version }}\" date=\"$(date +%F)\">#" \
org.o3de.O3DE.metainfo.xml
- name: Build Flatpak into OSTree repo
if: steps.check.outputs.build == 'true'
run: |
# Bwrap-free build (no flatpak-builder) so no privileged container is needed.
chmod +x scripts/make-flatpak.sh
scripts/make-flatpak.sh
# Free disk before publishing (the repo/ snapshot is all we still need).
rm -rf build-dir o3de.deb data
- name: Generate .flatpakrepo
if: steps.check.outputs.build == 'true'
run: |
BASE="${{ github.server_url }}/${{ github.repository }}/raw/branch/pages"
cat > repo/o3de.flatpakrepo <<EOF
[Flatpak Repo]
Title=O3DE (unofficial Flatpak)
Url=$BASE
Homepage=https://o3de.org/
Comment=Unofficial O3DE engine repackaged as a Flatpak
Description=Install the Open 3D Engine on any Linux distribution via Flatpak.
EOF
- name: Publish OSTree repo to the 'pages' branch
if: steps.check.outputs.build == 'true'
env:
# Prefer a personal access token (PUBLISH_TOKEN secret) with repo write
# access; fall back to the auto-provided Actions token.
TOKEN: ${{ secrets.PUBLISH_TOKEN != '' && secrets.PUBLISH_TOKEN || secrets.GITHUB_TOKEN }}
run: |
AUTH_URL="$(echo "${{ github.server_url }}" | sed "s#://#://${{ github.actor }}:${TOKEN}@#")/${{ github.repository }}.git"
rm -rf publish && mkdir publish && cd publish
git init -q -b pages
git config user.name "Gitea Actions"
git config user.email "actions@pc-heini.de"
cp -a ../repo/. .
touch .nojekyll
git add -A
git commit -q -m "O3DE Flatpak v${{ steps.ver.outputs.version }}"
# Force-push a single snapshot so the pages branch never accumulates history.
git push -f "$AUTH_URL" pages
cd ..
- name: Tag the published version
if: steps.check.outputs.build == 'true'
env:
TOKEN: ${{ secrets.PUBLISH_TOKEN != '' && secrets.PUBLISH_TOKEN || secrets.GITHUB_TOKEN }}
run: |
AUTH_URL="$(echo "${{ github.server_url }}" | sed "s#://#://${{ github.actor }}:${TOKEN}@#")/${{ github.repository }}.git"
TAG="v${{ steps.ver.outputs.version }}"
if git ls-remote --tags "$AUTH_URL" "refs/tags/$TAG" | grep -q .; then
echo "Tag $TAG already exists (force rebuild) - leaving it in place."
else
git tag "$TAG"
git push "$AUTH_URL" "$TAG"
fi
+14
View File
@@ -0,0 +1,14 @@
# flatpak-builder build artifacts
build-dir/
.flatpak-builder/
repo/
# downloaded payload
*.deb
*.deb.sha256
*.flatpak
o3de.deb
# misc
.DS_Store
*.log
View File
View File
+203
View File
@@ -0,0 +1,203 @@
# O3DE Flatpak
Repackage the official [Open 3D Engine](https://o3de.org/) Linux release as a
**Flatpak** so it can be installed on any distribution — not only Debian/Ubuntu.
A Gitea Actions workflow checks daily for a new O3DE release, builds the Flatpak,
and publishes it as a static [OSTree](https://ostreedev.github.io/ostree/) Flatpak
repository on the `pages` branch of this repo. You add that as a Flatpak remote and
`install` / `update` like any other app.
> **Status:** community / unofficial. O3DE is a large application (~1518 GB
> installed); building and hosting it is heavy. Treat this as best-effort.
---
## Installing (end users)
```sh
# Flathub provides the runtime O3DE needs
flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
# Add this repo
flatpak remote-add --if-not-exists o3de \
https://git.pc-heini.de/pc-heini/o3de-flatpak/raw/branch/pages/o3de.flatpakrepo
flatpak install o3de org.o3de.O3DE
flatpak run org.o3de.O3DE
```
Later updates:
```sh
flatpak update org.o3de.O3DE
```
> The repo is currently **unsigned** (no GPG). Flatpak will add it with GPG
> verification disabled. See *Signing* below to harden this.
---
## How it works
| File | Purpose |
| --- | --- |
| `scripts/make-flatpak.sh` | **The build.** Unpacks the official `o3de_*.deb` into `/app`, bakes in Python, and exports an OSTree repo using `flatpak build-init`/`build-finish`/`build-export` — no `flatpak-builder`, no bubblewrap, no privileged container. |
| `o3de-wrapper.sh` | Entry point. Finds the versioned `o3de` Project Manager binary inside the sandbox and sets `LD_LIBRARY_PATH`. |
| `org.o3de.O3DE.desktop` | Desktop entry under the Flatpak app-id. |
| `org.o3de.O3DE.metainfo.xml` | AppStream metadata (version stamped at build time). |
| `scripts/get-latest-version.sh` | Resolves the latest `.deb` URL, version, and SHA-256 from o3debinaries.org. |
| `scripts/build.sh` | Download + build + test the Flatpak locally (wraps `make-flatpak.sh`). |
| `org.o3de.O3DE.yaml` | Equivalent `flatpak-builder` manifest — kept as an **alternative** for builders that have a privileged/bwrap-capable environment. Not used by CI. |
| `.gitea/workflows/build-flatpak.yml` | CI: detect new version → build → publish to `pages` → tag `vX.Y.Z`. |
The engine ships as a Debian package at a predictable URL
(`https://o3debinaries.org/main/Latest/Linux/o3de_<ver>.deb`). The build extracts
it (`ar` + `tar`) and copies the payload into the Flatpak's `/app`. The version
directory inside the `.deb` changes every release, so the wrapper discovers the
executable at runtime rather than hard-coding a path.
> **Want the full story?** O3DE is an engine + SDK that compiles code and writes
> into its own install tree at runtime, while a Flatpak gives it a **read-only
> `/app`**. Reconciling those two facts is most of the work here.
> **[`docs/DESIGN.md`](docs/DESIGN.md)** documents every non-obvious decision, the
> read-only-`/app` problems and their fixes, and how to debug/maintain this across
> O3DE releases. Read it before changing the build.
---
## CI requirements (Gitea Actions)
The workflow targets a **self-hosted `act_runner`**. Because O3DE is large:
- **Disk:** budget **60 GB+** free. The build needs roughly 23× the installed
size (extracted payload in `build-dir` + a copy committed into the OSTree
`repo/`). The job deletes `build-dir` before publishing to cut peak usage, but
it can still be tight. If builds fail on space, that's the first thing to check.
- **No privileged container required.** The build avoids `flatpak-builder`/bwrap
and uses `flatpak build-init`/`build-finish`/`build-export`, which only touch
files and the OSTree repo. A plain unprivileged job container works.
- **Runner label:** the job uses `runs-on: ubuntu-latest`. Change it if your
runner is registered with a different label.
- **Token:** publishing force-pushes the `pages` branch and creates a `vX.Y.Z`
tag. The auto-provided `GITHUB_TOKEN` (with `contents: write`) usually suffices.
If your instance restricts it, create a Personal Access Token with repo write
access and add it as a secret named **`PUBLISH_TOKEN`** — the workflow prefers
it automatically.
It runs on three triggers:
- **`push` to `main`** — rebuilds whenever the build itself changes (always builds,
even for an already-published version, so you can test your change). Docs-only
changes (`README.md`, `docs/**`, `LICENSE*`, `.gitignore`) are ignored. Scoped to
`main` so the `pages`/tag pushes the job makes don't re-trigger it.
- **daily `cron`** — picks up a **new O3DE binary release**; only builds when the
upstream version has no matching `vX.Y.Z` tag yet, so most days are cheap no-ops.
- **`workflow_dispatch`** — manual run from the Gitea Actions UI, with an optional
**force** rebuild.
---
## Building locally
```sh
# flatpak + curl + python3/setuptools (to pre-build the o3de sdist)
sudo apt install flatpak python3-setuptools # or your distro's equivalent
./scripts/build.sh
```
Then install from the local `repo/` and run:
```sh
flatpak remote-add --user --no-gpg-verify o3de-local repo
flatpak install --user o3de-local org.o3de.O3DE
flatpak run org.o3de.O3DE
```
---
## Signing (recommended hardening)
The first iteration publishes an unsigned repo for simplicity. To sign:
1. Generate a key: `gpg --quick-gen-key "O3DE Flatpak" default default never`
2. Export the public key and add `GPGKey=<base64>` to the generated `.flatpakrepo`.
3. Pass `--gpg-sign=<KEYID>` to both `flatpak-builder` and `build-update-repo`,
and provide the private key to CI via a secret. Until then, users add the
remote with GPG verification disabled.
---
## Caveats
These were confirmed by inspecting and running the v26.05 package (`opt/O3DE/26.05/…`):
- **Layout**: the package installs everything under
`/opt/O3DE/<ver>/`, with the launcher at `bin/Linux/profile/Default/o3de` and
~270 `.so` files beside it (hence the wrapper's `LD_LIBRARY_PATH`). If a future
release moves things, adjust `o3de-wrapper.sh` and the manifest.
- **Runs against `org.freedesktop.Sdk`, not `Platform`.** O3DE's package
dependencies are a *build toolchain* (clang/ninja/cmake/pkg-config) because the
engine compiles project code at runtime. Those live in the SDK. Users therefore
pull the SDK runtime (larger than Platform) on install — Flatpak does this
automatically from Flathub.
- **Python is per-user, not baked in.** O3DE needs a Python venv under `~/.o3de`
(writable), keyed to the engine path, so it can only be created at runtime — the
build can't bundle it. The Project Manager GUI does **not** create it itself, so
the wrapper (`o3de-wrapper.sh`) runs `python/get_python.sh` on launch when the
venv is missing or stale. The **first launch downloads Python and can take
several minutes with no window on screen** — that's expected; subsequent launches
are instant. (cmake, which `get_python.sh` needs, comes from the Sdk runtime.)
- **The o3de CLI install (read-only `/app`).** On first launch O3DE installs its
own `o3de` CLI from `scripts/o3de` into the per-user `~/.o3de` venv. By default
`python/get_python.sh` does an *editable* install (`pip install -e`), which
writes an `egg-info` next to the source under read-only `/app` and fails.
`get_python.sh` has a built-in immutable-install hook (the Snap path): if a
prebuilt sdist exists at `scripts/o3de/dist/o3de-1.0.0.tar.gz` it installs that
(non-editable) instead. The build pre-creates that tarball (`python3 setup.py
sdist`) while the tree is still writable, so at runtime the CLI lands in the
writable venv and nothing is written to `/app`. (The build also patches every
`cmake/LYPython.cmake` to a non-editable install, covering the separate
project-build CMake-configure path.) If a future release renames the package or
bumps its version away from `1.0.0`, the build fails fast — those are the lines
to re-check.
- **Engine-path resolution needs `SNAP`/`SNAP_BUILD`.** Because the `o3de` CLI is
installed non-editably, its package lives in the per-user venv, so O3DE's
`manifest.get_this_engine_path()` (which walks up from the package `__file__`)
would resolve into the venv and fail to find `engine.json`. O3DE has a built-in
override for immutable installs: when `SNAP` and `SNAP_BUILD` are set it returns
`$SNAP/$SNAP_BUILD`. The wrapper sets them to the engine root. (Confirmed these
vars are read *only* there — no other snap-specific behaviour is triggered.)
- **The Project Manager's own user folder is read-only (cosmetic).** With no
project open, O3DE puts its engine-level *user* folder (asset cache, metrics,
`UserSettings.xml`) under `<engine>/.../user`, i.e. read-only `/app`. You'll see
`Couldn't find a valid asset cache folder…`, `cannot write to temp file …
UserSettings.xml`, and a metrics warning at startup. These are **non-fatal**
the Project Manager runs fine; it just doesn't persist its *own* window/UI
settings between runs. The documented override
(`/O3DE/Runtime/FilePaths/SourceProjectUserPath`) does **not** help: O3DE
recomputes that path from the engine root during bootstrap and overwrites a
`--regset`. Real project data (projects you create, their caches) lives under
your home directory and is fully writable, so actual work is unaffected.
- **Launcher icon.** Shipped as `org.o3de.O3DE.png` (the `.deb` itself has only
in-editor asset icons). `flatpak build-export` validates exported icons in a
bwrap sandbox that fails in an unprivileged container ("is not a valid icon:
bwrap …"), so the build exports with `--disable-sandbox`: the icon is still
validated, but in-process rather than via bwrap/userns. The icon is now
exported to the host, so the menu launcher shows it; the window/alt-tab icon
also picks it up as long as the compositor matches the window to the desktop
entry (`StartupWMClass=O3DE` must equal the window's `WM_CLASS`).
- **GPU / drivers:** the renderer needs working GPU access. The manifest grants
`--device=dri`/`--device=all`; on some setups you may also want the matching
GPU driver extension from Flathub.
- **Sandbox filesystem:** `--filesystem=home` lets the Project Manager create and
open projects under your home directory. Tighten or widen to taste.
- **Hosting via raw branch URLs** works because Flatpak fetches individual files
(`summary`, `config`, `objects/…`). Gitea serves these from the `pages` branch.
If you later put a real static web host in front of it, just change `Url=` in
the `.flatpakrepo`.
---
*Not affiliated with or endorsed by the Open 3D Foundation. O3DE is licensed under
Apache-2.0 / MIT.*
-7
View File
@@ -1,7 +0,0 @@
[core]
repo_version=1
mode=archive-z2
indexed-deltas=true
[flatpak]
title=O3DE (unofficial Flatpak)
+329
View File
@@ -0,0 +1,329 @@
# Design & engineering notes
This document explains **why** the O3DE Flatpak is built the way it is. The
[README](../README.md) tells you how to use and operate it; this is the companion
that records the reasoning and the hard-won lessons, so the next person to touch
this repo (very possibly future-you) doesn't have to rediscover them.
If you read only one thing, read [The core tension](#the-core-tension) and
[The read-only `/app` saga](#the-read-only-app-saga). Almost every non-obvious
decision in this repo flows from those two sections.
---
## The core tension
O3DE was never designed to run from a read-only location. It is not a normal
desktop app that you install and run — it is an **engine + SDK** that, at runtime:
- **compiles C++** (your game project, gems) — so it ships a *build toolchain*
(clang/cmake/ninja/pkg-config), not just runtime libs;
- **bootstraps its own Python** into a per-user virtualenv and `pip`-installs a CLI;
- **writes into its own install tree** — asset caches, user settings, metrics,
Python `egg-info`, registration data.
A Flatpak gives the app a **read-only `/app`** at runtime. So the entire project
is, essentially, a series of negotiations between "O3DE wants to write here" and
"that location is read-only." Every caveat in the README is one of those
negotiations.
The official Debian package assumes a normal writable `/opt/O3DE/<ver>/`. We take
that exact payload and drop it into `/app/opt/O3DE/<ver>/`, then redirect or
pre-bake everything O3DE would otherwise try to write.
---
## Packaging architecture
### We repackage the `.deb`; we don't build from source
Building O3DE from source is enormous (hours, tens of GB, a full toolchain). The
project instead consumes the **official prebuilt Linux release**, which ships as a
Debian package at a predictable URL:
```
https://o3debinaries.org/main/Latest/Linux/o3de_<ver>.deb (+ .sha256)
```
`scripts/get-latest-version.sh` scrapes the latest filename, derives the version
(`o3de_2605_0.deb``26.05.0`, engine dir `26.05`) and fetches the checksum.
`scripts/make-flatpak.sh` extracts the `.deb` (`ar` + `tar`) and copies its
`opt/O3DE/<ver>/` payload into the Flatpak's `/app`.
The version directory name changes every release, so **nothing hard-codes it**
the wrapper and build discover paths at runtime/build time with `find`.
### No `flatpak-builder`, no bubblewrap, no privileged container
This was forced by the CI environment and turned out to be the single most
important build decision.
`flatpak-builder` runs **every build command inside a bubblewrap (`bwrap`)
sandbox**, which needs user namespaces / a privileged container. In a stock Gitea
Actions / `act_runner` job container that fails with:
```
bwrap: Creating new namespace failed: Operation not permitted
```
Marking the *runner's* container privileged did **not** fix it (wrong layer), and
requiring privileged containers is a bad thing to demand of a self-hosted runner.
The escape: our "build" is just *unpacking files*. We don't need a sandbox to do
that. So `scripts/make-flatpak.sh` uses the lower-level primitives directly:
```
flatpak build-init build-dir <app-id> <runtime> <sdk> <ver>
# ... copy files into build-dir/files ...
flatpak build-finish build-dir --command=… <finish-args…>
flatpak build-export repo build-dir <branch>
flatpak build-update-repo repo …
```
`build-init/finish/export` only touch files and the OSTree repo — **no `bwrap`**
so they run in a plain unprivileged container. `org.o3de.O3DE.yaml` (a real
`flatpak-builder` manifest) is kept **only** as an alternative for anyone who *does*
have a bwrap-capable builder. CI does not use it.
### Runtime is `org.freedesktop.Sdk`, not `Platform`
Counter-intuitive but essential. O3DE compiles project code at runtime, so its
dependency list is a **build toolchain** (clang, cmake, ninja, pkg-config). Those
live in the **SDK**, not the slimmer Platform runtime. Running against
`org.freedesktop.Sdk//24.08` is what makes `get_python.sh` (which needs `cmake`)
and project builds work inside the sandbox. End users therefore pull the larger SDK
runtime from Flathub on install — that's expected, not a bug.
### Hosting: a static OSTree repo on an orphan `pages` branch
A Flatpak remote is just a directory of files (`summary`, `config`, `objects/…`).
CI commits the generated `repo/` to an orphan **`pages`** branch and **force-pushes
a single snapshot** (so the branch never accumulates multi-GB history). Gitea serves
those files over raw URLs, and `o3de.flatpakrepo` points clients at them. If you
ever front this with a real static host, only `Url=` in the `.flatpakrepo` changes.
---
## The read-only `/app` saga
This is the heart of the project. Each item below is a distinct place O3DE tried to
write into read-only `/app`, how it manifested, and how it was resolved. They were
found **one at a time**, each revealed only after fixing the previous one.
| # | Symptom | Root cause | Fix |
|---|---------|-----------|-----|
| 1 | `qt.qpa.xcb could not connect to display` | `--socket=fallback-x11` *suppresses* X11 on Wayland; O3DE's bundled Qt only ships the `xcb` plugin | Use `--socket=x11` unconditionally + `QT_QPA_PLATFORM=xcb` |
| 2 | `setup.py develop … [Errno 30] Read-only file system` | `get_python.sh` does `pip install -e scripts/o3de`; editable mode writes `egg-info` into read-only `/app` | **Pre-build an sdist** so the runtime takes its non-editable branch (below) |
| 3 | `No module named 'o3de'` | Half-built venv from a prior failure; the GUI never re-bootstraps Python | Wrapper runs `get_python.sh` itself; it's self-healing (below) |
| 4 | `Missing python venv file … pyvenv.cfg` | Project Manager **GUI does not bootstrap Python** — it only errors if the venv is absent | Wrapper bootstraps the venv before launching (below) |
| 5 | `Invalid engine json …/venv/…/lib/engine.json``Failed to initialize` | Non-editable install moved the `o3de` package into the venv, so `get_this_engine_path()` (walks up from `__file__`) resolves into the venv | Set `SNAP`/`SNAP_BUILD` to the engine root (below) |
| 6 | `Couldn't find a valid asset cache folder` / `cannot write … UserSettings.xml` | Project-less Project Manager puts its *engine-level* user dir under read-only `/app` | **Non-fatal**; documented. The documented `--regset` override is overwritten by bootstrap (below) |
| — | `… is not a valid icon: bwrap` during `build-export` | `build-export` validates exported icons in a bwrap sandbox (unavailable unprivileged) | Keep the icon *inside* the app, drop it from the *exported* set |
### Python: the sdist trick (items 2)
O3DE installs its own `o3de` CLI into the per-user venv. By default
`python/get_python.sh` does:
```sh
pip install -e <engine>/scripts/o3de # editable → writes egg-info into /app → fails
```
The key discovery: **`get_python.sh` already has a built-in immutable-install
escape hatch** (it's how the official Snap package copes). It is gated on a
pre-built sdist tarball:
```sh
if [ -f .../scripts/o3de/dist/o3de-1.0.0.tar.gz ]; then
pip install .../dist/o3de-1.0.0.tar.gz --no-deps # non-editable ✅
else
pip install -e .../scripts/o3de --no-deps # editable ❌
fi
```
So the build runs `python3 setup.py sdist` **while `/app` is still writable**,
producing `scripts/o3de/dist/o3de-1.0.0.tar.gz`. At runtime the `-f` test passes,
the CLI installs non-editably into the writable `~/.o3de` venv, and nothing touches
`/app`. (`setup.py` hard-codes `version="1.0.0"`, hence the exact filename — the
build fails fast if that ever changes.) The build also patches every
`cmake/LYPython.cmake` to a non-editable install for the *separate* CMake-configure
path used during a project build.
> **Note — the first patch was on the wrong file.** Early on we patched only
> `LYPython.cmake`, confirmed the patch landed, and *still* saw `setup.py develop`
> at runtime. The tell was the failure path `…/python/../scripts/o3de`, which is how
> **`get_python.sh`** builds the path, not `LYPython.cmake`. Two different installers;
> the GUI's first-launch path is `get_python.sh`.
### Python: the wrapper bootstraps the venv (items 3, 4)
The Project Manager **GUI never runs `get_python.sh`**. It assumes the venv already
exists and just errors (`Missing python venv file … pyvenv.cfg`) if it doesn't —
which then cascades into engine-registration failure and exit. Nothing on the
desktop launches `get_python.sh` for you.
So `o3de-wrapper.sh` does it. It checks `python.sh --version` (which exits non-zero
when the venv for this engine is missing or stale) and, if so, runs `get_python.sh`
before launching the GUI. This is:
- **idempotent** — a healthy venv means `python.sh --version` succeeds and we skip;
- **self-healing** — `get_python.sh` recreates the venv with `--clear`, so a
half-built `~/.o3de/Python/venv/<id>` left by an earlier failure is repaired.
The venv lives at `~/.o3de/Python/venv/<engine-id>`, where `<engine-id>` is a hash
of the engine path computed by cmake. The **first launch downloads Python and can
take several minutes with no window on screen** — expected; later launches are
instant.
### Engine path: `SNAP` / `SNAP_BUILD` (item 5)
A subtle consequence of fixing item 2. O3DE finds its own engine root in Python via:
```python
def get_this_engine_path():
if "SNAP" in os.environ and "SNAP_BUILD" in os.environ:
return pathlib.Path(os.environ["SNAP"]) / os.environ["SNAP_BUILD"]
return pathlib.Path(os.path.realpath(__file__)).parents[3].resolve()
```
With the old **editable** install, `__file__` lived in the engine tree
(`/app/.../scripts/o3de/o3de/manifest.py`) and `parents[3]` correctly gave the
engine root. Our **non-editable** install moved the package into the venv
`site-packages`, so `parents[3]` now lands **inside the venv**`engine.json` not
found → `Failed to get engine info``Failed to initialize`.
O3DE's own override saves us: when `SNAP` and `SNAP_BUILD` are set it skips the
`__file__` walk and returns `$SNAP/$SNAP_BUILD`. The wrapper sets:
```sh
SNAP=/app/opt/O3DE # dirname of the engine root
SNAP_BUILD=26.05 # basename of the engine root → $SNAP/$SNAP_BUILD = engine root
```
We verified (grep of the Python package **and** the C++ binaries/`.so`s) that these
two variables are read **only** in `get_this_engine_path` — setting them triggers no
other snap-specific behaviour. This was confirmed empirically by launching the
binary with the vars set: the Project Manager opened cleanly with no `engine.json`
errors.
### The user folder we *couldn't* relocate (item 6)
With no project open, O3DE places its **engine-level** user folder (asset cache,
`UserSettings.xml`, metrics) under `<engine>/…/user` — read-only `/app`. You'll see
warnings at startup; they are **non-fatal**. The Project Manager runs fine; it just
doesn't persist *its own* UI settings between runs.
The documented override is registry key
`/O3DE/Runtime/FilePaths/SourceProjectUserPath`. We tested redirecting it with
`--regset` to a writable path and it **did not work**: O3DE **recomputes** that path
from the engine root during settings-registry bootstrap and overwrites the
command-line value. (This was verified with line-buffered output capture — an
earlier "it works!" reading was a buffering artifact from `timeout`-killing the
process, which never flushed its full-buffered pipe. Lesson: when capturing a GUI
process you'll kill, force `stdbuf -oL -eL`.)
Real project data — the projects you create and their caches — lives under your home
directory and is fully writable, so actual work is unaffected. We chose to document
this rather than chase a fix that the engine actively undoes.
### The icon
The `.deb` ships only in-editor asset icons, no app icon, so we add
`org.o3de.O3DE.png`. But `flatpak build-export` validates *exported* icons inside a
bwrap sandbox — which, unprivileged, fails (`is not a valid icon: bwrap …`). The
compromise: keep the icon **inside** the app (so the running window/taskbar shows
it) but `rm -rf build-dir/export/share/icons` before export so the export step skips
validation. The cost is a generic icon in the host's app menu. A privileged runner
would let us export it properly.
---
## How a build flows (CI)
`.gitea/workflows/build-flatpak.yml`, triggered by daily `cron` or
`workflow_dispatch` (with an optional **force**):
1. **Install deps** — `curl jq xz-utils zstd binutils tar python3 python3-setuptools
flatpak`. (`python3-setuptools` is needed to pre-build the o3de sdist.)
2. **Checkout** — a plain `git init/remote/fetch/checkout`, *not* `actions/checkout`.
The bare `ubuntu:24.04` image has no Node.js, so JavaScript actions exit 127.
3. **Resolve version** via `get-latest-version.sh`.
4. **Decide to build** — skip if a `vX.Y.Z` tag already exists, unless `force`.
5. **Install** `org.freedesktop.Sdk//24.08` from Flathub.
6. **Download + checksum** the `.deb`.
7. **Stamp** the version/date into `org.o3de.O3DE.metainfo.xml`.
8. **Build** via `scripts/make-flatpak.sh`; delete `build-dir`/`o3de.deb` to reclaim
disk before publishing.
9. **Generate** `o3de.flatpakrepo` pointing at the `pages` raw URL.
10. **Publish** — force-push the `repo/` snapshot to the orphan `pages` branch.
11. **Tag** `vX.Y.Z` (idempotent: skipped if it already exists, e.g. a force rebuild).
Auth uses `PUBLISH_TOKEN` if set, otherwise the auto-provided `GITHUB_TOKEN`.
---
## Maintaining this across O3DE releases
Most releases will "just work" because nothing hard-codes the version. The build is
designed to **fail fast** at the spots most likely to break:
- **Package version ≠ `1.0.0`** — the sdist filename `o3de-1.0.0.tar.gz` is what
`get_python.sh` looks for. If `scripts/o3de/setup.py` bumps its version, the build
errors at the sdist step. Re-check the filename in `get_python.sh` and update.
- **Layout move** — if `scripts/o3de/setup.py` or `cmake/LYPython.cmake` disappears,
the build errors. Re-trace the install path (it has moved between the Snap-style
hook and `LYPython.cmake` before).
- **`get_python.sh` drops the sdist hook** — then the editable install returns and
item 2 reappears. Re-check the `if [ -f …/dist/o3de-1.0.0.tar.gz ]` branch.
- **`get_this_engine_path` stops honouring `SNAP`/`SNAP_BUILD`** — item 5 reappears
as `Invalid engine json …/venv/…`. Re-check `scripts/o3de/o3de/manifest.py`.
- **Binary/library path change** — `o3de-wrapper.sh` finds the binary under
`*bin/Linux*` and sets `LD_LIBRARY_PATH`; adjust if `bin/Linux/profile/Default/`
moves.
### How to debug runtime issues quickly (without a full rebuild)
The expensive loop is: force-rebuild (~15 min) → publish → `flatpak update` →
re-download Python. You can usually avoid it. Because the installed app's payload is
just files in `/app`, you can **inspect and even re-run pieces in the live sandbox**:
```sh
# Read any engine file:
flatpak run --command=bash org.o3de.O3DE -c 'cat /app/opt/O3DE/26.05/python/get_python.sh'
# Re-run the Python bootstrap and watch it:
flatpak run --command=bash org.o3de.O3DE -c 'sh /app/opt/O3DE/*/python/get_python.sh'
# Launch the binary with experimental env/args BEFORE baking them into the wrapper
# (this is how SNAP/SNAP_BUILD was validated). Use stdbuf so output flushes on kill,
# and write logs to $HOME (the sandbox /tmp is private, but home is shared):
timeout 40 flatpak run --command=bash org.o3de.O3DE -c '
export SNAP=/app/opt/O3DE SNAP_BUILD=26.05
BIN=$(find /app/opt/O3DE -type f -name o3de -path "*bin/Linux*" | head -1)
export LD_LIBRARY_PATH="$(dirname "$BIN"):/app/lib:/app/opt/O3DE/lib"
stdbuf -oL -eL "$BIN" > "$HOME/o3de-test.log" 2>&1
'
```
A `timeout` exit code of **124** means the process stayed alive (a window opened) —
which, by itself, is strong evidence that startup/engine-registration succeeded.
The upstream source for cross-referencing is `github.com/o3de/o3de` at the matching
tag (e.g. **`2605.0`** for engine dir `26.05`). The files that mattered most here:
`python/get_python.sh`, `cmake/LYPython.cmake`, `scripts/o3de/o3de/manifest.py`, and
`Code/Framework/AzFramework/AzFramework/Application/Application.cpp`.
---
## Open items / future hardening
- **Signing.** The repo is published unsigned; users add it with GPG verification
disabled. See *Signing* in the README to add a key.
- **Exported icon.** Generic in the host menu until built on a bwrap-capable
(privileged) runner.
- **Engine-level user folder.** Cosmetic read-only warnings remain (item 6). If a
future O3DE exposes an override the bootstrap *doesn't* clobber, wire it into the
wrapper.
- **Project create/build end-to-end.** Launch is solid; the first real project build
is the next thing to exercise, since it's the first time the runtime compile path
and project asset cache are used in anger.
+65
View File
@@ -0,0 +1,65 @@
#!/bin/sh
# Launcher for the O3DE Project Manager inside the Flatpak sandbox.
#
# The Debian package installs O3DE under /opt/O3DE/<version>/, which becomes
# /app/opt/O3DE/<version>/ inside the Flatpak. The version directory name
# changes with every release, so we discover the executable at runtime instead
# of hard-coding a path.
set -eu
O3DE_ROOT=/app/opt/O3DE
# Bundled shared libraries that ship inside the .deb.
export LD_LIBRARY_PATH="/app/lib:${O3DE_ROOT}/lib:${LD_LIBRARY_PATH:-}"
# Locate the Project Manager executable ("o3de").
O3DE_BIN=$(find "$O3DE_ROOT" -type f -name o3de -path '*bin/Linux*' 2>/dev/null | head -n 1)
if [ -z "$O3DE_BIN" ]; then
O3DE_BIN=$(find "$O3DE_ROOT" -type f -executable -name o3de 2>/dev/null | head -n 1)
fi
if [ -z "$O3DE_BIN" ]; then
echo "error: O3DE executable not found under $O3DE_ROOT" >&2
echo " (the .deb layout may have changed)" >&2
exit 1
fi
# Make libraries that sit next to the binary discoverable too.
export LD_LIBRARY_PATH="$(dirname "$O3DE_BIN"):${LD_LIBRARY_PATH}"
# Bootstrap O3DE's per-user Python venv if needed. The Project Manager GUI does
# NOT set this up itself - it just errors out ("Missing python venv file ...") if
# the venv is absent. So we ensure it here. python.sh exits non-zero when the
# venv for this engine is missing/stale; get_python.sh then downloads Python,
# creates the venv (~/.o3de/Python, writable) and installs the 'o3de' CLI from
# the prebuilt sdist. It is idempotent and self-healing (recreates a broken venv
# with --clear). cmake, needed for the engine-id calc, ships in the Sdk runtime.
PYTHON_SH=$(find "$O3DE_ROOT" -type f -path '*/python/python.sh' 2>/dev/null | head -n 1)
GET_PYTHON=$(find "$O3DE_ROOT" -type f -path '*/python/get_python.sh' 2>/dev/null | head -n 1)
if [ -n "$GET_PYTHON" ]; then
ENGINE_ROOT=$(dirname "$(dirname "$GET_PYTHON")")
# O3DE's Python derives the engine path from the o3de package's __file__. That
# breaks with our non-editable install (the package lives in the per-user venv,
# not the engine tree), so engine.json is looked for under the venv and not
# found ("Failed to get engine info"). O3DE has a built-in override for exactly
# this immutable-install case: if SNAP and SNAP_BUILD are set, manifest.py's
# get_this_engine_path() returns "$SNAP/$SNAP_BUILD". Point them at the engine
# root. (Verified nothing else in O3DE - Python or the C++ binaries - reads
# these two vars, so there are no snap-specific side effects.)
SNAP=$(dirname "$ENGINE_ROOT")
SNAP_BUILD=$(basename "$ENGINE_ROOT")
export SNAP SNAP_BUILD
# Bootstrap the per-user Python venv if needed (the GUI doesn't do it itself).
if ! "$PYTHON_SH" --version >/dev/null 2>&1; then
echo "O3DE: setting up the per-user Python environment." >&2
echo "O3DE: first launch downloads Python and can take several minutes..." >&2
if ! "$GET_PYTHON"; then
echo "error: O3DE Python setup failed (see the log above)." >&2
exit 1
fi
fi
fi
exec "$O3DE_BIN" "$@"
+3
View File
@@ -1,5 +1,8 @@
[Flatpak Repo]
Title=O3DE (unofficial Flatpak)
# The CI job regenerates this file on the 'pages' branch with the correct URL,
# so the canonical copy to use is:
# https://git.pc-heini.de/pc-heini/o3de-flatpak/raw/branch/pages/o3de.flatpakrepo
Url=https://git.pc-heini.de/pc-heini/o3de-flatpak/raw/branch/pages
Homepage=https://o3de.org/
Comment=Unofficial O3DE engine repackaged as a Flatpak

Some files were not shown because too many files have changed in this diff Show More