Write Operations

Core Plus (Core Plus) is the only tier that can change cluster state. Every write goes through an approval gate — Tiphys never silently applies a change.

The safety model

Non-negotiable invariants (hard-coded, not configurable):

  1. Dry-run always runs first. The tool handler renders the change and sends it to the K8s API with dryRun=All before asking for approval. If the dry-run fails (invalid manifest, admission webhook rejection, etc.), the request never reaches the approval queue.
  2. Operator approval is required. The tool's first invocation creates an ApprovalRequest and returns status: pending_approval. The second invocation (with approval_id) proceeds only if the operator clicked Approve.
  3. Protected namespaces are off-limits. kube-system, kube-public, kube-node-lease, tiphys, and the namespace Tiphys runs in reject writes at the gate regardless of RBAC or approval state.
  4. Narrow write ClusterRole. tiphys-writer has create/update/patch on workload resources. No delete. No cluster-admin. No secret-content reads.
  5. Every change is in the audit trail, with the approval it ran under: who asked for it, who approved it, the digest of what was approved, when it was dispatched and how it ended. See The audit trail.

These invariants are tested in backend/tests/unit/test_idp_base.py, test_write_safety.py, and test_permission_errors.py.

The two-source approval flow

Approvals come from two places:

Source: chat

An operator asks Tiphys to do something. The agent calls a write tool. The tool dry-runs, creates an ApprovalRequest, returns a pending status. The UI shows an inline approval card in the chat with the rendered manifest + Approve / Reject buttons.

Operator clicks Approve → the backend flips the approval state to approved. The next agent turn re-invokes the same tool with approval_id, the tool's gate sees the approved state, and the write actually runs.

Source: report

A scheduled health-check rule emits a Recommendation attached to a finding. When reports.recommenderMode=false (Helm value), the Reporter auto-creates ApprovalRequests at cron time — the Approvals view populates overnight without any chat interaction.

The morning oncall opens the UI, sees "3 proposed fixes from last night's scan", reviews each, clicks Approve. The approve endpoint itself dispatches the write (there's no agent turn waiting to re-invoke the tool in this flow). Every dispatch lands in the audit trail identically to chat-sourced approvals.

Advisory-only recommendations (no dispatchable tool payload — "raise memory limit" has no safe generic value) render an Acknowledge button instead of Approve/Reject. The operator clicks to dismiss; Tiphys applies nothing. The intent is honest UX: "we flagged the fix direction; size and apply manually."

The 24-hour TTL

Approvals expire 24 hours after creation by default. This was 5 minutes pre-v0.0.1 — that only made sense for actively-watched chat approvals. Report-sourced approvals obviously need longer.

Teams who expect weekends between cron + approval should bump to 48h:

approvals:
  ttlSeconds: 172800

Expired approvals flip to expired state on the next read; they no longer appear in the Pending list, and the audit trail records each one (approval.expired, dated when it lapsed).

De-dupe

A rule firing every 30 minutes against the same finding doesn't stack duplicate approvals. Before creating a new approval, the Reporter checks for existing pending or approved approvals tagged with the same source_finding_id and skips if any exist.

Rejected / expired approvals don't block re-staging — the operator said no (or ignored it long enough to expire), so when the rule fires again, a fresh approval gets created. Design choice: the rule's job is to keep flagging what it sees; the operator's job is to decide.

Per-tick cap

reports.maxAutoStagedPerTick (default 10) caps auto-staged approvals per scheduler tick so a rule that fires across every deployment in the cluster doesn't produce 50 simultaneous approval cards.

The cap is severity-ordered: critical findings stage first, warnings next, infos last. If the cap kicks in, the low-severity tail is dropped rather than starving the high-severity head.

Dispatch failures

When the operator clicks Approve on a report-sourced approval, the dispatched write can still fail — RBAC denied after staging, the target workload was deleted, the admission webhook rejected the patch. The approval state flipped to approved regardless (the operator's decision succeeded), but nothing changed in the cluster.

The UI surfaces a warning banner with the dispatch error on the next poll cycle. The audit trail records both the approval and the failed dispatch (change.failed, or change.rolled_back when Helm rolled an install back). The operator is expected to re-stage the fix from the finding or approve the write manually via kubectl.

The audit trail

Tiphys records what it does as events, one record each, written once and never changed. Each event says:

  • id (unique, and sorts by time) and at (epoch milliseconds)
  • actor: who (person, with their id, email and name; system for Tiphys on its own, such as a scheduled scan or an approval expiring)
  • action: what, as a dotted verb (below)
  • target: what it acted on (a resource, a Helm release, a namespace, a cloud service, a setting), and cluster
  • trace: the chat and the turn it came from, and a link that opens the chat at that turn
  • outcome: ok, error, refused, denied or not_found, with a code
  • approval: for a change, the approval it ran under (its id, the SHA-256 digest of what was approved, the approver, its scope and, where an approval proof was minted, the proof's id and claims). Never a token.
  • details: a short summary of what was asked (at most 2 KB). Values that look like secrets or tokens are redacted.

What's recorded:

ActionsWhen
approval.requestedA change is staged (or refused before staging: a protected namespace, paused writes, the license, inline Secret data). Carries the digest of the payload.
approval.approved, approval.rejected, approval.acknowledged, approval.expiredA decision, an advisory one acknowledged, or the TTL passing. A decision refused (already decided, writes paused) or denied (the license, the person's role) is recorded too.
change.dispatched, change.succeeded, change.failed, change.rolled_backAn approved change running, and how it ended.
change.verifiedThe deployment monitor's verdict on it.
cluster.read, cloud.read, metrics.read, knowledge.readEach read Tiphys makes, one per tool call, with how much came back. Calls that only make files (manifests, chart scaffolds, bundles) aren't recorded.
model.turnEach chat turn: the provider, the model, how many calls, input, output and cached tokens, and what kinds of data were sent. Never the content.
scan.runEach health report and knowledge scan, with counts.
conversation.deleted, audit.exported, signinA chat deleted, the trail exported, someone signing in.

Retention. Each event is kept audit.retentionDays days (365 by default; -1 keeps everything), or what the license's audit_retention_days says when it has one. The retention is fixed on the event when it's written; an event past it is no longer shown, and is deleted within the hour as new events are written. With persistence.mode: sqlite (the default) the trail is a table in the state database; with configmap it's the newest 800 events in tiphys-audit-events.

Who sees what. With auth.mode: builtin the admin account sees every event. With proxy, members of audit.adminGroups see everyone's events and everyone else sees their own.

Reading it. The Audit tab shows changes and approvals first; choose Reads, Model turns or Scans to see those. Filter by outcome, time and person, page back, open an event to see its approval and follow its link to the chat, and export CSV or JSON (at most 50,000 events; the export is itself recorded). Or call the API:

# Newest changes, a page at a time (pass next_cursor as cursor for the next)
curl -s "$TIPHYS/api/audit?action=change.&limit=100"
# A day as CSV
curl -s -o audit.csv "$TIPHYS/api/audit/export?format=csv&from=2026-10-19&to=2026-10-20"

Before this version the audit log was capped at 500 entries (tiphys-audit). It's copied into the trail on the first start and left as it was, read only.

Golden paths

golden_path_apply renders a pre-blessed workload template + applies it as a single approval bundle. The catalog ships with:

  • standard-web-service — Deployment + Service + HPA + PDB.
  • scheduled-job — CronJob with sensible defaults.
  • internal-api — same as standard-web-service plus NetworkPolicy.
  • worker — Deployment sized for async workload patterns.

Each path auto-detects matching examples in the current cluster so the operator can see "this path matches the 5 services you already run in prod" before committing.