Webhooks and the audit log

Every state-changing action in rIDM raises a domain event with a stable dotted name such as user.created. Two consumers matter to an administrator: webhooks, which POST selected events to your endpoints, signed, with retries; and the audit log, which appends every event to a per-tenant hash chain that can be listed, exported, verified and shipped to an external system. For the event model itself see Events, audit and webhooks.

Event catalogue

These are the names webhooks subscribe to and the audit log records. The names are part of the public contract and do not change once shipped. The payload fields listed are those of the event’s kind document (next to type).

Three of them never reach a webhook: master_key.rotated and master_key.generation_created are global events (they belong to no tenant and are recorded in the global audit chain), and webhook.delivery_dead is kept out of deliveries so that a failing endpoint cannot breed a new delivery for every dead one. All three are in the audit log.

Webhooks

Creating a webhook

In the console: Webhooks (/console/webhooks/). Through the admin API, under ridm:webhooks:read and ridm:webhooks:write (owners and administrators):

curl -X POST https://id.example.com/admin/tenants/acme/webhooks \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{
        "name": "HR sync",
        "url": "https://hooks.acme.example/ridm",
        "events": ["user.*", "group.member_added", "group.member_removed"],
        "headers": {"X-Api-Key": "…"},
        "max_attempts": 8
      }'

The answer is the webhook plus secret (whsec_…), shown this once. Store it with the receiver.

Webhooks are part of the tenant configuration document; a webhook created by an import gets a fresh secret, reported once.

What a delivery looks like

POST /ridm HTTP/1.1
Host: hooks.acme.example
Content-Type: application/json
User-Agent: rIDM-Webhooks/1
X-RIDM-Event: user.created
X-RIDM-Timestamp: 1789722764
X-RIDM-Delivery: 0192f5d0-6a1e-7c3b-9d2a-4b8e1f0c7a55
X-RIDM-Webhook: 0192f5c4-1b2c-7d3e-8f40-5a6b7c8d9e0f
X-RIDM-Signature: t=1789722764,v1=5f2b9c…
X-Api-Key: …

{
  "delivery_id": "0192f5d0-6a1e-7c3b-9d2a-4b8e1f0c7a55",
  "attempt": 1,
  "event": {
    "id": "0192f5d0-69f0-7a11-b3c4-2d5e6f708192",
    "tenant_id": "0192f0a1-…",
    "occurred_at": "2026-09-18T09:12:44.512Z",
    "actor": {"type": "admin", "id": "0192f0a2-…"},
    "kind": {"type": "user_created", "user_id": "0192f5d0-69e8-…"}
  }
}

event.actor.type is user, client, admin or system (the last without an id). event.impersonator names the administrator when the event happened in a session they opened as the user (impersonation). event.ip and event.user_agent appear when the event was raised by a request that recorded them, such as sign-in attempts. attempt counts from 1 and restarts at 1 after a manual redelivery. Payloads carry ids, not full records: fetch the current state from the admin API when you need it.

Verifying a delivery

The signature is an HMAC-SHA256, keyed with the whole secret string (including the whsec_ prefix) as UTF-8 bytes, over the timestamp, a full stop, and the raw request body exactly as received: "<t>.<body>". A receiver should:

1. recompute the HMAC over the raw body before parsing it, and compare in constant time;
2. reject a timestamp more than a few minutes from its own clock, so a captured request cannot be replayed later (every retry is signed afresh, so an honest delivery is always recent);
3. remember the X-RIDM-Delivery ids it has processed and acknowledge a repeat without acting on it again. Delivery is at least once: a timeout after your endpoint did the work, or an administrator’s redelivery, sends the same delivery again.

import hashlib, hmac, time

TOLERANCE = 300  # seconds

def verify(secret: str, headers, body: bytes, seen: set) -> bool:
    parts = dict(p.split("=", 1) for p in headers["X-RIDM-Signature"].split(","))
    t, sig = parts.get("t", ""), parts.get("v1", "")
    if not t.isdigit() or abs(time.time() - int(t)) > TOLERANCE:
        return False                                   # stale or malformed
    mac = hmac.new(secret.encode(), t.encode() + b"." + body, hashlib.sha256)
    if not hmac.compare_digest(mac.hexdigest(), sig):
        return False                                   # not from rIDM
    delivery = headers["X-RIDM-Delivery"]
    if delivery in seen:
        return True                                    # duplicate: acknowledge, do nothing
    seen.add(delivery)                                 # use a durable store in production
    return True

The same in Rust with the hmac, sha2, hex and subtle crates:

#![allow(unused)]
fn main() {
use hmac::{Hmac, Mac};
use sha2::Sha256;
use subtle::ConstantTimeEq;

fn signature_ok(secret: &str, header: &str, body: &[u8], now: i64) -> bool {
    let (mut t, mut v1) = (None, None);
    for part in header.split(',') {
        match part.split_once('=') {
            Some(("t", v)) => t = v.parse::<i64>().ok(),
            Some(("v1", v)) => v1 = hex::decode(v).ok(),
            _ => {}
        }
    }
    let (Some(t), Some(v1)) = (t, v1) else { return false };
    if (now - t).abs() > 300 {
        return false;
    }
    let mut mac = Hmac::<Sha256>::new_from_slice(secret.as_bytes()).expect("any key length");
    mac.update(t.to_string().as_bytes());
    mac.update(b".");
    mac.update(body);
    mac.finalize().into_bytes().as_slice().ct_eq(&v1).into()
}
}

Rotating the secret (POST …/secret) switches signing at once; there is no period in which both secrets sign. Have the receiver accept either secret while you roll the new one out, then drop the old one.

Retries and dead deliveries

A matching event is queued as a delivery for every enabled webhook that wants it and sent straight away; up to eight deliveries of a tenant are attempted at a time. Each attempt has a 10-second timeout and does not follow redirects.

The backoff after the first failure is 30 seconds, then 2 minutes, 10 minutes, 30 minutes, 2 hours, and 6 hours for every attempt after that. With the default of eight attempts a delivery is given up about 15 hours after the first try. Retries are sent by the webhook_delivery job, which runs every 30 seconds on one node at a time; a delivery stuck in sending for ten minutes (a node died mid-attempt) is picked up again.

A delivery that dies raises webhook.delivery_dead (audited, not delivered) and is logged as a warning. Fix the endpoint, then redeliver from the console’s delivery log or with redeliver-dead. Redelivery resets the attempt count and keeps the delivery id.

Deliveries are not ordered: concurrent attempts and retries mean user.updated can arrive before the user.created it follows. Use event.occurred_at or fetch the current state when order matters.

Delivered and dead deliveries are deleted by the hourly cleanup job once they are older than RETENTION_DAYS (default 30). The counter ridm_webhook_deliveries_total{outcome} (delivered, retry, dead) tracks the outcomes.

Target restrictions

• The URL must be https. Plain http is allowed only for localhost, 127.0.0.1 and [::1], for development.
• A literal private address is refused when the webhook is saved (400): RFC 1918 ranges, link-local (including 169.254.169.254), 100.64.0.0/10, 0.0.0.0/8, IPv6 unique-local (fc00::/7) and link-local (fe80::/10), and IPv6 forms that embed one of those.
• A hostname is checked at delivery time, on every attempt: it is resolved to public addresses only, so a name that resolves (or later rebinds) to an internal address fails the attempt instead of reaching it.
• Redirects are never followed and HTTP(S)_PROXY is ignored, so an endpoint cannot bounce a delivery elsewhere.

These are the rules of the outbound request policy, which applies to every URL tenant administrators choose.

The audit log

What is recorded

Every event in the catalogue is appended to its tenant’s chain as a row with:

Global events, today only master_key.rotated and master_key.generation_created, form a chain of their own.

Querying

In the console: Audit log (/console/audit/), with filters, paging, expandable rows, JSON and CSV export and a verify button; global administrators can switch to the global chain. Every built-in administrator role holds ridm:audit:read.

# Newest first; name ending in "." or "*" is a prefix
curl -G https://id.example.com/admin/tenants/acme/audit \
  -H "Authorization: Bearer $TOKEN" \
  --data-urlencode 'name=login.' \
  --data-urlencode 'from=2026-09-01T00:00:00Z' \
  --data-urlencode 'limit=100'

The answer is {"items": [...], "next_cursor": "..."}; next_cursor is absent on the last page. Tenant listings are served from DATABASE_READ_URL when one is configured.

GET /admin/tenants/{slug}/audit/export?format=json|csv (same filters, no paging) streams the chain oldest first as a download, with the hashes included. GET /admin/audit, /admin/audit/export and /admin/audit/verify do the same for the global chain and require a global administrator.

The hash chain

Each row’s hash is SHA-256(prev_hash || canonical row), where the canonical row is the UTF-8 string

id|chain|seq|occurred_at|name|actor_type|actor_id|subject_id|ip|user_agent|payload

with chain the tenant id (the nil UUID for the global chain), occurred_at in RFC 3339 UTC with exactly six fractional digits and a Z, absent values as empty strings, and payload as compact JSON with its keys sorted. A row with an impersonator_id has |impersonator:<uuid> appended; rows without one hash as they always did. The first row of a chain has no prev_hash. Writers of one chain are serialised with a Postgres advisory lock, so seq has no gaps.

GET /admin/tenants/{slug}/audit/verify walks the retained chain from its oldest row and recomputes every hash:

{
  "checked": 48211, "valid": true, "first_seq": 1, "last_seq": 48211,
  "last_hash": "9f2c…",
  "scheduled": { "verified_seq": 48190, "verified_at": "2026-09-21T03:10:44Z" }
}

A row that was altered, removed from the middle or reordered makes it answer "valid": false with broken_at_seq and a reason (gap in chain after seq …, prev_hash does not link to the previous row, row hash does not match its contents). scheduled is what the daily check (below) last established.

The chain proves that retained history was not edited in place. On its own it can’t prove that the newest rows were not cut off, or that the whole log was not rewritten by someone holding the database. Two things close that gap: keep last_hash somewhere else, and keep a copy outside rIDM (Shipping to an external system).

Checking an export yourself

ridm audit verify --file checks a JSON export with nothing but the file. It never contacts the server, so a server that rewrote its own log can’t vouch for itself. The algorithm is the one above, and it lives in the ridm-core crate, so you can also check exports with your own tooling.

ridm audit export -o acme-2026-09.json         # the whole chain, oldest first
ridm audit verify -f acme-2026-09.json --head 9f2c…
# Intact: 48211 rows (seq 1–48211), ending on 9f2c….

It fails, exiting non-zero, at the first row that does not hash or link, and names its seq. An export filtered by event name or user has gaps by design, and verification says so. Export the whole chain, or a time window, to check it. ridm audit verify without --file asks the server to walk its copy, and --global does the same for the global chain. The file is read as a stream, so an export of millions of rows is fine.

The scheduled check

The daily audit_verify job checks every chain that grew since it was last found intact. It starts from that checkpoint rather than from the oldest row, which keeps it cheap on a large log. It rehashes the checkpoint row itself, so a rewrite that reaches the checkpoint is caught; the verify endpoint and ridm audit verify always walk everything. When a chain does not verify, the job:

• stores where and why on the chain, which the verify endpoint shows as scheduled.broken_at_seq, and checks that chain from the start on every run until it verifies again;
• logs an error and counts it in ridm_audit_chain_breaks_total, and sets ridm_audit_chains_broken to the number of broken chains (alert on anything above 0);
• records audit.chain_broken (seq, reason) in that tenant’s chain, once per break, so the tenant’s webhooks hear of it.

Retention

settings.audit.retention_days (default 365; 0 keeps everything) sets how long a tenant’s rows are kept. The global chain follows the master tenant’s setting. The daily audit_retention job removes each chain’s expired prefix: every row up to the newest one older than the cutoff, in batches of 5,000. Removing a prefix rather than scattered rows keeps what remains verifiable; the oldest retained row’s prev_hash then points at a row that no longer exists, which verification accepts. The same job creates the monthly partitions of the audit table two months ahead, and drops a whole month’s partition, instead of deleting its rows, once the month ended before the longest retention of the tenants in that database and every chain with rows in it belongs to a tenant with a retention (a tenant keeping everything, one being moved between regions, or a deleted tenant’s trail keeps its months). The job remembers when each chain’s oldest remaining row was written and only visits the chains whose oldest row is past their retention.

The setting is in the console under Settings → Keys, discovery & audit, or:

curl -X PATCH https://id.example.com/admin/tenants/acme \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"settings": {"audit": {"retention_days": 730}}}'

Shipping to an external system

Set AUDIT_SINK_URL to copy every recorded row, from every tenant, to a SIEM or log store:

The sink ships from the database, not from memory. A worker on one node at a time (a leader lock) reads each chain’s rows after the last one it delivered, sends them, and records how far it got only once the receiver accepted them. When a delivery fails, nothing moves: the worker backs off, doubling up to a minute, and tries the same rows again. A receiver that is down for an hour, or a rolling restart of rIDM, delays rows but never loses them.

• At least once. A row can arrive twice, for example when a node dies between a delivery and recording it. Deduplicate on the row’s id, not on seq: after a database restore, a chain reuses the seq numbers written after the backup.
• A new destination starts now. The first time rIDM sees an AUDIT_SINK_URL (compared without credentials or query), it starts every chain at its current head rather than replaying history. Backfill older rows with ridm audit export.
• Rows the retention job removed before they were shipped are skipped.

Watch ridm_audit_sink_rows_total, ridm_audit_sink_failures_total and ridm_audit_sink_lag_rows (rows recorded but not yet shipped, over every chain). A lag that keeps growing means the receiver is refusing rows or can’t keep up.

Caveats

• Events travel on an in-process bus, and the audit writer and the webhook dispatcher record them after the action has committed. Each has its own bounded queue, so a burst delays events rather than skipping them (watch ridm_audit_queue_depth and ridm_webhook_dispatch_queue_depth); a node whose queue passes 80% of EVENT_QUEUE_CAPACITY reports itself not ready until it catches up, and only an event published into a full queue is dropped (ridm_event_queue_dropped_total). A node that is stopped gives its queues 15 seconds to drain and loses what was still waiting after that.
• An event is dispatched only on the node where it happened; webhooks and audit rows are not duplicated across nodes.
• Webhook payloads contain identifiers, email addresses (in invitation.created, user.email_changed) and login identifiers (in login.failed). Treat the endpoint as holding personal data.