Postgres and Valkey

rIDM keeps its system of record in Postgres and its short-lived shared state in Valkey. Both are required; a node that cannot reach either fails /readyz and most requests. Neither should be reachable from the internet: rIDM authenticates to them, but cannot protect them from anyone else who can connect.

Versions

The migrations create three extensions: pgcrypto, and pg_trgm with btree_gin (the indexes behind the organization, client and tenant searches). All three are trusted extensions, so the migrator role can create them without being a superuser, but a managed Postgres service has to offer them.

Migrations run while the previous release keeps serving. An index on an existing table is built with CREATE INDEX CONCURRENTLY, which does not block writes; should one of those migrations fail part way (a lost connection, a deadlock with a long transaction), Postgres leaves an INVALID index behind that the retry skips. Drop it (DROP INDEX CONCURRENTLY <name>, the name is in the failed migration) and run ridm-api migrate again.

Two roles and row level security

rIDM is built to connect as two different Postgres roles:

Neither is a superuser and both are NOBYPASSRLS. The compose file creates them with deploy/postgres/init-app-role.sh. On a database you run yourself, do the same as a superuser before the first migration:

CREATE ROLE ridm_migrator LOGIN PASSWORD '...'
    NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS;
CREATE ROLE ridm_app LOGIN PASSWORD '...'
    NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS;

GRANT CONNECT, CREATE, TEMP ON DATABASE ridm TO ridm_migrator;
GRANT ALL ON SCHEMA public TO ridm_migrator;

GRANT CONNECT, TEMP ON DATABASE ridm TO ridm_app;
GRANT USAGE ON SCHEMA public TO ridm_app;
ALTER DEFAULT PRIVILEGES FOR ROLE ridm_migrator IN SCHEMA public
    GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO ridm_app;
ALTER DEFAULT PRIVILEGES FOR ROLE ridm_migrator IN SCHEMA public
    GRANT USAGE, SELECT ON SEQUENCES TO ridm_app;
ALTER DEFAULT PRIVILEGES FOR ROLE ridm_migrator IN SCHEMA public
    GRANT EXECUTE ON FUNCTIONS TO ridm_app;

The default privileges are what give the app role access to tables the migrator creates later, so every migration must run as the migrator. A migration run as some other role (a superuser, say) creates tables the app role cannot see.

What row level security does here

Every tenant-scoped table has row level security enabled and forced (so it applies to the table owner too), with one policy: a row is visible when its tenant_id equals the transaction’s app.tenant_id setting, or when the transaction has set app.bypass_rls to on. The server binds the tenant at the start of each transaction with set_config(..., true), which resets at commit or rollback, so a pooled connection never carries one request’s tenant into the next. A transaction that binds nothing sees no tenant rows at all. Explicitly cross-tenant work (bootstrap, global administration, background jobs) sets the bypass flag for its own transaction.

That makes row level security a second line behind the application’s own tenant checks: a query that forgets its tenant filter returns nothing from other tenants. It is not a boundary against code running as the app role, which can set the bypass flag itself. The role split is about DDL: a superuser ignores row level security outright, and a table owner can switch it off or rewrite the policies, so the running server holds neither power.

MIGRATE_ON_START=true applies migrations only when some are pending. With the server running as the app role, that is harmless once ridm-api migrate has brought the schema up to date; if a migration is pending, startup fails because the app role cannot apply it. Running the server as the migrator instead, so that it can migrate itself, gives up the split. See Container image.

Audit partitions under the two-role setup

The audit log is partitioned by month. Migrations create the current month’s partition and the next two; after that the daily audit_retention job creates upcoming partitions by calling audit_ensure_partitions, as the server’s role. The DML-only app role has no CREATE privilege, so the function is SECURITY DEFINER: it runs with the rights of its owner (the migrator), with a pinned search_path. EXECUTE on it is revoked from PUBLIC and granted, when the migration runs, to every role that holds INSERT on audit_events, which under the setup above is the app role.

An app role created after that migration ran does not get the grant automatically. Give it by hand, as the migrator:

GRANT EXECUTE ON FUNCTION audit_ensure_partitions(integer) TO ridm_app;

If partition creation fails anyway, the job logs the error and still runs its retention purge; rows written meanwhile land in the audit_events_default partition, where the purge reaches them. A month whose range already has rows in the default partition is skipped rather than failing the whole call, because Postgres refuses to create such a partition. Failures show in ridm_job_runs_total{job="audit_retention",outcome="error"} only when the purge itself fails, so watch the server log for audit: creating partitions failed (see Observability).

Migrations

Migrations live in api/migrations, are embedded in the binary, and are recorded in sqlx’s _sqlx_migrations table. They are forward-only. ridm-api migrate applies whatever is pending, under an advisory lock so concurrent runs are safe, and exits; sqlx migrate run --source api/migrations from a checkout does the same. How to run them in each environment is in Container image. A release’s migrations keep the release before it working, so upgrades can roll; see Upgrading, and take a backup before applying a new release’s migrations.

Connection pools

Each node opens one pool to DATABASE_URL and, when DATABASE_READ_URL is set, a second pool of the same size to the replica.

A request waits at most DB_ACQUIRE_TIMEOUT_MS (2 seconds by default) for a Postgres connection and then fails with 503. Fixed behaviour: idle connections close after 10 minutes, every connection is recycled after 30 minutes, and statements slower than 250 ms are logged as warnings. Connections identify themselves as application_name ridm-api (primary) and ridm-api-read (replica), which is how to find them in pg_stat_activity. Valkey connections time out after 5 seconds waiting, creating or recycling.

Sizing: the connections rIDM can open are nodes × DB_POOL_MAX against the primary, plus the same against the replica if there is one, plus a migrate job. Keep that under the server’s max_connections (the compose file sets 200) with room for maintenance. The README’s rule of thumb for DB_POOL_MAX is roughly twice the database server’s CPU count divided by the number of API nodes.

A connection pooler such as PgBouncer is not tested. rIDM’s tenant binding is transaction-local, which suits transaction pooling in principle, but sqlx uses prepared statements, so a pooler in transaction mode must support them.

Read replicas

DATABASE_READ_URL names a streaming replica. When set, listings and statistics run there inside read-only transactions: users, clients, groups, roles, invitations, webhook deliveries, the audit log and the console overview. A write sent there by mistake fails rather than succeeding somewhere unexpected. Everything else, and every read that feeds a decision (sign-in, token issuance, permission checks), stays on the primary. A listing may trail a change by the replica’s lag. Use the same app role on the replica; unset, the primary serves both.

Valkey

Topologies

REDIS_URL selects the topology by its scheme:

Credentials go before an @ and apply to every listed host: redis+cluster://:secret@host1:7000,host2:7001. The cluster and Sentinel forms build plain redis:// connections to each host, so TLS to Valkey is available only in the single-server rediss:// form.

TLS and certificate authentication

rediss:// alone verifies the server against the system roots and authenticates with the password in the URL. Two settings go further, for the single-server form:

• REDIS_TLS_CA_FILE: PEM roots to trust instead of the system’s, for a Valkey behind a private CA (cert-manager, the OpenShift service CA). It applies to the Valkey connection only; outbound HTTPS keeps the system roots.

• REDIS_TLS_CERT_FILE and REDIS_TLS_KEY_FILE: a client certificate and its key, both PEM. Valkey can then authenticate the connection by the certificate instead of a password: with tls-auth-clients yes it accepts only certificates from its tls-ca-cert-file, and with tls-auth-clients-user CN it logs each connection in as the ACL user the certificate’s CN names. That user needs no password at all, which also keeps Valkey’s own password hashing out of the picture (the FIPS build wants this):

  tls-port 6380
  tls-cert-file /etc/valkey/tls/server.crt
  tls-key-file /etc/valkey/tls/server.key
  tls-ca-cert-file /etc/valkey/tls/ca.crt
  tls-auth-clients yes
  tls-auth-clients-user CN
  user default off
  user ridm on nopass ~ridm:* &* +@all -@admin
  

  with REDIS_URL=rediss://valkey.internal:6380 (no credentials) and a certificate whose subject is CN=ridm.

The files are read when a connection is made and read again whenever one of them changes, so a certificate renewed in place is picked up without a restart; a renewal that cannot be read yet keeps the previous certificate until it can. Each region’s Valkey has the same settings with its suffix (REDIS_TLS_CA_FILE_EU). On Kubernetes the chart’s redis.tls mounts the Secrets and sets the variables (Kubernetes).

Cache invalidation between nodes uses pub/sub on one channel. Under Sentinel the subscriber resolves the current master when it connects, and reconnects with backoff after a failover.

What lives in Valkey

Keys are prefixed ridm:, so a shared Valkey is possible, though a dedicated one is simpler to size and secure.

Memory and eviction

Size Valkey so it does not reach maxmemory. With an allkeys-lru or volatile-* policy a full Valkey evicts whatever it likes, including live browser sessions (users are signed out) and denylist entries (revoked access tokens pass rIDM’s checks again until they expire). With noeviction a full Valkey refuses writes and requests fail loudly instead. The compose file’s 256 MB with allkeys-lru suits evaluation only. Enable append-only persistence (appendonly yes, as the compose file does) so a restart keeps sessions.

If Valkey is lost

Losing Valkey’s data (a flush, a restart without persistence, a failover to an empty replica) loses no durable data, but:

• every browser session ends; users sign in again. Refresh tokens issued with offline_access keep working, being in Postgres; refresh tokens without it are bound to the session and stop with it;
• opaque access tokens (at_…) end early, since their claims were only in Valkey; clients refresh or sign in again;
• sign-ins, authorization codes, device authorizations and emailed links in flight fail and must be started again;
• revoked access tokens that have not yet expired pass rIDM’s own checks again (introspection, userinfo, the account API), because the denylist is gone; APIs that validate JWTs locally never consulted it. Revoke again if that matters, or wait out the access-token lifetime;
• rate-limit windows start from zero;
• caches refill from Postgres, and each node clears its in-process cache when its invalidation subscriber reconnects.

While Valkey is unreachable, rather than empty, /readyz returns 503 and requests that need a session, a flow or the cache fail. The rate limiter alone fails open (with a warning), since it protects against abuse and is not an authorization control.

Backups

Back up Postgres; Valkey holds nothing that cannot be lost at the cost above. A Postgres backup contains encrypted secrets that decrypt only with the master key in use (and any listed in MASTER_KEY_PREVIOUS), so keep a copy of the key, stored apart from the backups, for as long as you keep the backups. Backups hold user data and credential hashes: encrypt them and control access. A logical dump must run as a role that bypasses row level security. Backup and restore has the commands, the restore procedure and what to do if the master key is lost.