The operating rules for the data estate (classification → policy, zones, products, MDM, retention) and for ML / generative AI / RAG / agents (risk classes, autonomy levels, gateway policy, evaluation, kill switch). Companion views: Data & Analytics and AI & Agentic.
Honesty header: the source document (S19, S40–S42) motivates these disciplines but specifies none of the mechanics. Everything operational below is a proposal (A-02); regulatory overlays stay dormant until OQ-02/OQ-03 are answered (A-11).
The model is federated (the data-mesh operating idea from S40, without its infrastructure fragmentation — ADR-08): domains own their data products and the pager that goes with them; the platform team owns shared infrastructure (lakehouse, catalog, quality tooling, pipelines); the data governance council (gov-data) owns policy — classification scheme, retention schedules, residency rules, quality standards, certification criteria — and arbitrates access disputes. Nothing enters the estate outside this triangle: sources onboard via runbook RB-09 (catalog-first), products publish via RB-11 (certification), and both leave an audit trail.
| Role | Owns | Cannot do |
|---|---|---|
| Domain data-product owner | Product contract, semantics, quality rules, consumer support, freshness SLO | Bypass classification; grant access outside policy; publish uncertified products to enterprise consumers |
| Data platform team | Lakehouse/catalog/quality/pipeline infrastructure, platform SLOs, ingestion patterns | Change product semantics; approve its own policy exceptions |
| Data governance council | Policy, certification, dispute arbitration, overlay activation | Operate pipelines (governance is not a delivery bottleneck by charter — review SLAs apply) |
Four levels — public, internal, confidential, restricted — assigned at registration
(RB-09 gate: unclassified = no pipeline) and carried as catalog tags on datasets, columns, data products, embedded chunks and event schemas.
Tags drive controls automatically (C-DP-01): access policy (zone- and column-scoped grants), masking/tokenization outside
production need-to-know (C-DP-03), encryption expectations (C-DP-02), retention class defaults (C-DP-04), residency placement when the
overlay activates (C-DP-05), and embeddability for AI (only cataloged, classification-tagged sources may enter the embedding pipeline).
A tag change is a policy change: it re-evaluates downstream grants and triggers re-certification of affected products.
| Zone | Contents | Entry gate | Access |
|---|---|---|---|
| raw | As-landed data with capture metadata (source, position, load timestamp) | RB-09 onboarding complete (registered, classified, owned) | Platform + owning-domain engineers only |
| validated | Schema-conformant, quality-checked data | Quality expectations green (freshness, volume, schema, rules) | Domain engineering; no BI/AI consumption |
| curated | Modeled, documented domain tables | Transformation tests pass; lineage complete; documentation present | Cross-domain engineering per grants |
| serving | Consumption-optimized marts / product materializations | Product certification (RB-11) for enterprise consumers | BI, ML, operational consumers via classification policy |
Quality failures page the owning domain, not a central team (data-quality → owner routing), and block promotion rather than annotating it — a red gate is a stop, which is what makes certification mean something (mitigates R-04).
A data product is publishable when its contract is complete: schema with column classifications; semantic definitions (what a row means, grain, keys); freshness SLO and delivery pattern; quality rules with thresholds; owner and support channel; access policy statement; PII handling (consent filtering where personal data — C-PRV-01); versioning and deprecation policy. Certification (RB-11) checks contract completeness, policy conformance, lineage, and a duplication review (reuse before duplication, PR-05). Certified products are the only enterprise-consumable surfaces: BI semantic models bind to products, not raw tables; ML training reads versioned product datasets; RAG embeds product-governed content. The two flagship products (Customer 360, Orders & Fulfillment) set the pattern in wave 1.
Master data (customer, product, partner) suffers a real SoR overlap: CRM owns relationship data while the customer-profile service serves runtime traffic. MDM (data-mdm) resolves it with explicit survivorship rules per attribute (which source wins, on what evidence), match/merge with stewardship queues for ambiguous cases, and golden-record ids distributed back to both systems. Reference data (codes, hierarchies) is versioned and distributed from one governed store — every copy elsewhere is a cache with a refresh contract, never a fork. Disputes land at the data governance council with the survivorship register as the deciding artifact.
Every record class carries a retention schedule and a deletion procedure (C-DP-04); deletion jobs produce evidence, and backups age out on their own schedules so deletion is real, not cosmetic. Legal hold suspends deletion for named scopes with an auditable register — holds override schedules, and hold release re-activates them. Residency (C-DP-05) is a dormant overlay: region allowlists in landing-zone policy plus storage placement rules, activated when OQ-02 answers arrive; until then no residency promise is made. Data-subject-request hooks (access/export/delete) are wired to the profile service, products and document store so a future privacy overlay activates against plumbing that already exists (C-PRV-01) — without claiming any regulation is satisfied today (A-11).
The AI governance council (gov-ai) owns: the use-case intake and risk classification below, autonomy-level grants, tool-grant approvals (with security review), eval thresholds per risk class, the AI logging/retention policy, incident review, and the kill-switch policy. The AI platform team owns the enforcement machinery: gateway, guardrails, registries, evals, observability. The split matters: the council decides what is allowed; the platform makes only that technically possible. Phase-6 exit criteria (runbook P6) hard-gate production autonomy on this council operating — agents never get broad production permissions by default (ADR-14, source S42's ambition bounded).
| Risk class | Definition (examples) | Minimum controls |
|---|---|---|
| RC-1 · Assistive, internal | Employee drafting/search/summarization on internal-classified content | Gateway + guardrails + logging; standard evals |
| RC-2 · Assistive, sensitive data | Assistants touching confidential/restricted content (Customer 360-backed answers) | RC-1 + entitlement-filtered RAG with bypass tests (C-AI-03) + redacted logging + council sign-off |
| RC-3 · Customer-visible generation | Content shown to customers/partners (drafted messages, explanations) | RC-2 + content-safety thresholds + injection red-team pass + human review paths + brand/accuracy evals |
| RC-4 · Action-taking (agentic) | Agents invoking tools that change state (create case, adjust order, spend) | RC-3 + tool registry grants with transaction limits + HITL thresholds + kill-switch drill + full action audit (C-AI-04/05) |
| Autonomy level | Meaning | Graduation evidence (council-decided) |
|---|---|---|
| L0 · Assist | Suggests; human performs the action | — (entry level) |
| L1 · Recommend | Prepares the action; human approves each execution (HITL) | Eval scores stable over window; audit trail complete; incident-free record at L0 |
| L2 · Bounded auto | Executes within tight limits (amount/scope/rate); humans audit samples + approve threshold-crossers | L1 approval history with low override rate; kill-switch drill current; rollback path demonstrated for the action class |
| L3 · Broad auto | — | Not offered in this architecture. A future proposal would require its own ADR, new evidence classes, and steering-level risk acceptance. |
Model access: only through the AI gateway (C-AI-01) — provider allowlists, per-consumer virtual keys, quotas, redacted prompt/response logging per the council's retention policy, fallback routing, and hosted-first sourcing with documented self-host triggers (ADR-13). Prompts: versioned artifacts in the prompt registry with review, eval gating and staged rollout (RB-22) — never string literals in code. Embedding: only cataloged, classified, consent-filtered sources; chunk records carry provenance + classification + entitlement metadata; re-embedding pipelines run on source change so the index never outlives its truth. Retrieval: the RAG service resolves the caller's entitlements before context assembly (deny by default), and every answer carries citations back to governed sources — an uncited answer is a defect. Entitlement-bypass tests are a release gate (mitigates R-05).
Agents act only through registered tools: each grant names the use case, the tool scopes (least privilege — a tool, not a service account), transaction limits, HITL thresholds, the accountable human, and an expiry (RB-12). Execution runs in a sandbox that enforces limits outside the model's reach — a prompt cannot negotiate with a rate limiter. Iterations are bounded; memory is scoped per use case; tool outputs are sanitized before re-entering context (C-AI-02). Every step — plan, tool call, inputs/outputs (redacted), approval, result — lands in the audit trail. The kill switch disables an agent, a tool, a model route or the whole scope immediately, and is drilled quarterly; using it is a normal operational act, not an admission of failure. Grants are recertified quarterly; expired grants auto-disable (mitigates R-06).
Evaluation: golden-set suites per use case — task quality, groundedness/hallucination checks against cited sources, safety categories, injection resistance — run at promotion (RB-22) and on schedule thereafter; thresholds per risk class are council-set. Drift: live eval-proxy metrics (citation rates, override rates, refusal anomalies) trend in AI observability; regression pages the owner and can trip the kill switch. Incidents: AI incidents (leakage, harmful output, runaway actions, provider failure) follow RB-16/RB-23 with an AI-specific annex — capture the full prompt/tool context (redacted), disable the scope, review at the council. Cost: token/inference spend is metered per consumer at the gateway, budgeted, anomaly-alerted and fed to FinOps (C-AI-06, RB-19) — AI spend is treated exactly like any other cloud cost, including the monthly review and the unit-economics line.