Skip to main content

​
The Hill90 AI application

It is deployed. The application runs on the Hill90 VPS as a tenant of the homelab, deployed from a GitHub Actions workflow in its own repository. hill90.com serves the UI on a Let’s Encrypt certificate. Identity comes from the homelab’s Keycloak at auth.hill90.com, realm platform; app-auth.hill90.com is not a live surface and returns 404.Verified 2026-08-09 03:53:04 UTC: 23 containers running, 0 unhealthy — the homelab’s 16 and the application’s 7. That is a dated measurement, not a standing guarantee; re-check before relying on it.Signing in was completed by authorization code on 2026-07-31. Keycloak event storage has been enabled since that date, so later login events are recorded; it does not backfill the period before event storage was enabled. This page does not claim a fresh login check.Deployed is not the same as current. The running containers were last replaced on 2026-07-30 at about 03:20 UTC, so anything merged to the application’s main after that is not in the running images. Two retirements are reflected on the host, because they were performed there directly rather than by a deploy.Where a later page in this section disagrees with this one, this one is newer.

​
How it runs

​
How it is deployed

From a workflow_dispatch GitHub Actions workflow in hill90-app, over SSH from the runner via Tailscale. It is never deployed from a workstation. The workflow has a dry-run mode that runs every guard — secrets, tenancy contract, host paths — and stops before changing anything. It deploys as a tenant: hill90_edge and hill90_internal are consumed as external networks that the homelab creates, and network, volume and container names are parameterised so one set of compose files serves both local and production. The tenancy is detachable, and this was tested rather than assumed. Historical yank-out measurement, 2026-07-29: the application was torn down to a single container and redeployed: the homelab returned to exactly its 13-container baseline with all four shared networks intact, the application came back to 10 healthy containers, hill90.com answered 200, and both user accounts survived in the database. The yank-out test passed. These are not the current 16-platform-plus-7-tenant inventory above.

​
Backups, and a restore that has actually been performed

The application’s data is backed up by the homelab’s scripts/backup.sh, not by anything in the application’s own repository:
That command changed on 2026-07-31, and what it does now is worth being exact about. Since the application’s Postgres was retired, its databases live on the homelab’s instance and are already inside the db target’s pg_dumpall. The app-db target no longer takes a dump of its own: it reports that the container is retired, archives the retained data volume for as long as that volume exists, and succeeds. It was deliberately not made to fail — a nightly job that reports failure for data which is in fact backed up is how a real failure gets ignored later. Verified end to end on 2026-07-29: the dump was restored into a throwaway Postgres container and both user accounts came back with their correct roles — realm roles in the application’s own realm, which is how authorization worked before the move to client roles. That was the first restore this estate had performed. And again on 2026-07-31, before the retirement: all five databases on the application’s Postgres were dumped and the dump was restored into a throwaway, with row counts compared per table and matching exactly. A dump nobody has restored is not a backup. Worth knowing what it replaced. Until that date the application’s volume had never been backed up by anything, and the homelab’s own SQL dump had been failing silently for days — the nightly job reported success while producing only a volume tar. See Operations.

​
Continuous integration

The application’s tests run on every pull request as of 2026-07-29: six suites — api (jest), ui (vitest), and pytest for ai, knowledge, mcp and agentbox. Until then only shell tests gated a merge.

​
One Keycloak, one Postgres

The governing principle: the homelab provides identity, data and storage; tenants consume them. Every decision below follows from it. Keycloak — done, 2026-07-30. One Keycloak, one realm, the existing platform realm. Not a new hill90 realm: you do not create a second directory for one organisation, and infra-versus-app is a matter of role and client assignment inside it. The application’s own Keycloak was retired; app-auth.hill90.com returns 404. Authorization is by client roles, not realm roles. The hill90-ui client carries its own admin and user roles, and the application reads only those. Realm platform also has realm roles of its own — including one named admin — and holding that realm role does not grant application access. Verified 2026-07-31: the client roles exist on hill90-ui, and the three accounts in the realm hold them individually. Postgres — done, 2026-07-31. The application’s own Postgres was retired and it now consumes the homelab’s. Its databases are owned by a dedicated non-superuser role, so a tenant cannot reach the platform’s own data. Verified 2026-07-31: that role is present and is not a superuser, and the three tenant databases exist on the homelab’s instance. The complication was real and turned out to be solvable rather than blocking — the homelab’s health check asserts platform-only databases, and that boundary was revisited deliberately rather than loosened. Storage — done, 2026-07-31. Production object storage is the homelab’s MinIO. The tenant consumes it through a scoped tenant-hill90-app credential rather than running a MinIO of its own. Accounts live in the homelab’s realm platform. Three exist there, each granted the hill90-ui client roles individually — verified 2026-07-31. They are created by an operator rather than by an import: the realm ships with no users, so a rebuild that discarded the directory would lock everyone out. Local development is the exception only for data and storage: its Postgres and MinIO are the application’s own, while the default local tenant path authenticates against the homelab’s Keycloak, realm platform. Only the --standalone fork runs its own Keycloak with a dev account baked in. The local data/storage parity question remains separate from the settled identity boundary.
On the api root path. It is worth being precise, because “api returns 404” invites a hunt for a routing fault that is not there. The API router is enabled and proxying: /health returns 200 through Traefik, and the 404 at / carries X-Powered-By: Express and the API’s own correlation-ID header — it is the application’s own 404, not Traefik’s. By contrast a 404 at ai.localtest.me/ is Traefik’s, because no router matches that path.So the API is reachable and serving; what it does not have is a route registered at /. Whether it should is app-arch’s call, and they are treating it as open.

​
What is NOT verified

A sign-in was completed by authorization code on 2026-07-31. Keycloak event storage is enabled for events from that date forward, but it does not reconstruct the period before it was enabled. This page does not claim a new live sign-in check.Deployed is not the same as proven working. Two distinctions cost this project a night: reachable is not working, and authenticating is not signing in. All the application’s stacks report healthy and hill90.com serves a real certificate, but healthy containers and answered routes are not an exercised application. No agent has been run end to end on the VPS — that remains the largest unproven claim on this page.One gap is worth naming rather than leaving implied: realm platform carries no users in its import. The accounts that exist were made by hand, so rebuilding the directory would lock everyone out — recoverable now, but only because verified backups exist.
For the commands that bring the local stack up, see hill90-app’s README.

​
What it is

An AI agent platform: a REST API for managing sandboxed agent containers, a policy-gated LLM inference gateway with bring-your-own-key model management, persistent agent knowledge with hybrid search, an MCP gateway, and a Next.js frontend behind Keycloak. Eight services in total, consuming the platform’s Keycloak, Postgres and MinIO.

​
Capabilities

  • AI Agent Management — create, configure, start and stop sandboxed AI agents via REST API.
  • Bring Your Own Key — register your own LLM provider keys. Keys are encrypted at rest and never exposed.
  • Model Policies & Usage Tracking — control which models agents can access, with rate limits and token budgets. Track usage and costs.
  • Agent Knowledge & Memory — agents maintain persistent knowledge across sessions. Browse and search what your agents have learned.
  • Identity & Access Control — Keycloak OIDC/OAuth2 with JWT-based API authentication.
Automatic HTTPS, observability and automated infrastructure are the homelab’s capabilities rather than the application’s — see the homelab section.

​
Where the code is

jonhill90/hill90-app holds the eight services. The repository is private; the link will not resolve unless you have access.