Architecture

Full 10-layer architecture with data flows and technology choices

d# Architecture Reference

Technical architecture for the Enterprise Agentic AI Platform — 10 layers from experience to infrastructure, supporting platform-managed, pro-code, and n8n execution modes.

Platform Overview

The following diagram shows all 10 architectural layers and how governance (L8) cross-cuts the request path.

＋−⟲
⛶ Full Screen
Scroll to zoom · Drag to pan · 100%

Governance (L8) enforces default-deny policies at L1 (user entitlements), L5 (tool grants, agent interaction), and L7 (model access), ensuring every request is authenticated, authorized, and audited.

Business Context

The following diagram shows how the Agentic AI Platform interacts with external systems, users, and services. The platform acts as a governance and orchestration layer between human users, AI agents, LLM providers, enterprise tools, and identity systems.

＋−⟲
⛶ Full Screen
Scroll to zoom · Drag to pan · 100%

Key External Interactions

Deployment Diagram (Docker Compose)

The following diagram shows all services as defined in docker-compose.yml grouped by tier, with their ports and inter-service connections.

＋−⟲
⛶ Full Screen
Scroll to zoom · Drag to pan · 100%

Service Inventory

Request Lifecycle

Dual-Protocol Execution

External agents communicate via AG-UI (SSE streaming) for real-time events and A2A (JSON-RPC 2.0) for lifecycle management; the platform intercepts tool calls for governance enforcement.

＋−⟲
⛶ Full Screen
Scroll to zoom · Drag to pan · 100%

The key pattern is "agent declares, platform executes" — the agent requests tool calls but the platform enforces D2 grants and runs the tools through the MCP Gateway.

Chat Message Lifecycle (Platform-Managed)

For platform-managed agents, the API Gateway delegates to a Temporal workflow that orchestrates LLM inference, tool execution, and guardrails with durable checkpointing.

＋−⟲
⛶ Full Screen
Scroll to zoom · Drag to pan · 100%

Temporal provides durability — if the process crashes mid-execution, the workflow resumes from its last checkpoint without losing state.

Tool Execution

The MCP Gateway enforces D2 tool grants with default-deny semantics, rate limiting, circuit breaking, HITL checks, and Sigstore signature verification before any tool executes.

＋−⟲
⛶ Full Screen
Scroll to zoom · Drag to pan · 100%

Every tool call produces an audit event regardless of outcome — denied, circuit-open, and successful calls are all logged.

Access Control

The platform enforces default-deny access across four dimensions within each tenant:

• D1 — User to Agent: Agents have visibility settings (public/restricted/private) and entitlements (allowed groups, roles, users). Enforced server-side at L1.
• D2 — Agent to Tool: Tool access is explicitly granted per agent with per-tool conditions (rate limits, HITL requirements). Enforced at L5 MCP Gateway.
• D3 — Agent to Agent: Interaction policies define who can invoke whom, with delegation depth limits and loop prevention. Enforced at L2/L5.
• D4 — Agent to Model: Model access policies control which models each agent can invoke, with hosting-tier restrictions for data sovereignty. Enforced at L7 LLM Gateway.

All four dimensions are default-deny. Access must be explicitly granted. All decisions are audit-logged. See docs/ACCESS_CONTROL.md for the full model, Cedar policy examples, data schemas, and UI impact.

Memory

Memory is organized into four scopes, each with dedicated storage backends:

• User Scope (private per-user) — Redis (episodic context) + Qdrant (semantic recall). TTL-based expiry for short-term, vector embeddings for long-term.
• Team Scope (group shared) — Qdrant (shared semantic memory) + Neo4j (entity relationships). Enables cross-user knowledge within teams.
• Agent Scope (domain knowledge) — Qdrant (domain embeddings) + Neo4j (domain graph). Procedural memory stores learned multi-step workflows for reuse.
• Global Scope (platform-wide) — Neo4j (enterprise knowledge graph) + Qdrant (organization-wide embeddings). GraphRAG for multi-hop reasoning.

Memory orchestration (Mem0) provides a unified API: extract, deduplicate, conflict-resolve, store. GDPR forget is supported at all scopes.

Credential Resolution

The platform resolves credentials through a 5-tier waterfall. Each tier is checked in order; the first match wins.

1. Vault — Short-lived, auto-rotated secrets managed by HashiCorp Vault. Preferred for production.
2. User BYOK — User-provided API keys submitted via /api/me/credentials. Stored with AES-256-GCM encryption.
3. Agent Override — Per-agent credentials set by the builder for agent-specific integrations.
4. Tenant Shared — Organization-wide credentials shared across all agents in the tenant.
5. ENV Variable — Platform-level defaults from environment variables. Fallback of last resort.

Resolution is implemented in services/api-gateway/src/lib/credential-resolver.ts. A dry-run preview endpoint (GET /api/credentials/resolve-preview) shows which tier a credential would resolve from without exposing the secret value.

OBO Token Exchange

The platform supports RFC 8693 On Behalf Of (OBO) token exchange, allowing agents to call downstream services (SAP, Salesforce, ServiceNow) as the authenticated user. The OBO Context Middleware extracts the user's bearer token, then resolveOBOCredential() uses a cache-first strategy with AES-256-GCM encrypted storage in the tenantCredentials table. Cache keys are derived via SHA-256 with a 60-second expiry buffer to prevent race conditions. Two exchange flows are supported: generic RFC 8693 (Keycloak) and SAP Destination Service. Delegation chain forwarding propagates OBO tokens through agent-to-agent calls via X-OBO-Token-{audience} and X-Agent-Actor headers. Security layers include AES-256-GCM encryption at rest, SPIFFE agent identity verification, scoped delegation (audience + scopes), and configurable delegation depth limits. Admin endpoints provide monitoring (/api/admin/obo/tokens, /api/admin/obo/stats) and bulk revocation (/api/admin/obo/revoke). Every exchange is audit-logged with risk-level classification. See docs/OBO_TOKEN_EXCHANGE.md for the comprehensive guide.

Layer Details

L1 — Experience & Interaction

• API Gateway: Kong/Envoy. REST, gRPC, WebSocket, SSE. Injects tenant_id from JWT.
• Input Guardrails: Prompt injection detection, PII detection/redaction, jailbreak prevention, content classification. Runs BEFORE orchestration.
• Output Guardrails: Hallucination scoring (vs grounding docs), toxicity filtering, PII leak prevention, citation verification. Runs BEFORE response delivery.
• HITL Gateway: Risk scoring: Financial Impact × Data Sensitivity × Irreversibility. Low → auto-execute. Medium → execute + notify. High → block until human approves. SLA timeout auto-rejects.
• Rate Limiter: Per-tenant, per-agent, per-user. Token budget enforcement.

L2 — Orchestration

• Workflow Orchestrator (Temporal): Deterministic, predefined workflows. Durable state, retries, compensation, checkpointing.
• Agent Orchestrator: Dynamic, goal-directed planning. Decomposes goals → sub-tasks → delegates to best runtime (pro-code or n8n).
• Patterns: Prompt chaining, routing, parallelization, evaluator-optimizer, orchestrator-workers.
• Event Bus (Kafka): Inter-agent async communication. Partitioned by tenant_id.

L3 — Agent Execution (DUAL RUNTIME)

• Agent Execution Model: plan() → execute() → reflect() → report(). Both runtimes implement this via AG-UI + A2A dual-protocol.
• Pro-Code Runtime: LangGraph, AutoGen, CrewAI, custom. Kata/gVisor sandbox. Native SPIFFE. Full memory/semantic access.
• n8n Runtime: Visual workflows, 500+ connectors, AI Agent nodes, MCP Server/Client. Queue mode (separate main/webhook/worker).
• Tri-Protocol Integration: External agents use three standard protocols in concert:
  • A2A (lifecycle): Manages the agent lifecycle — registration, Agent Cards, turn-based execution. Governance is a formal A2A extension with URI urn:agentic-platform:governance:v1, carried in RUN_FINISHED events to report HITL status, token usage, and compliance metadata.
  • AG-UI (pipe): Streams real-time execution events (text deltas, tool calls, state snapshots) from agent to frontend over SSE.
  • A2UI (content): Declarative UI surface protocol (a2ui.org, v0.8) for human-in-the-loop approvals. Agents declare approval surfaces (selection lists, parameter forms, text inputs) as structured data; any compliant frontend renders them without custom widget code.
• n8n Bridge Layer (4 sidecars):
  • Memory Bridge (~20MB Go): /memory/read, /memory/write, /memory/search → routes to Redis/Qdrant/Neo4j
  • Guardrails Proxy (~30MB): Reverse proxy between n8n and LLM. Intercepts all LLM calls. Adds input/output guardrails. Zero n8n config change (point n8n's API base URL to localhost:8081).
  • OTel Exporter (~15MB): Transforms n8n execution logs → OTel spans. LLM semantic conventions (model, tokens, cost).
  • Identity Injector: SPIRE agent sidecar. Assigns spiffe://platform/n8n/workflow/{id}. Provides Vault tokens for platform services.

L4 — Memory

• Short-term (Redis): Active conversation context, scratchpad. Sub-ms latency. TTL expiry.
• Long-term Vector (Qdrant): Semantic recall across sessions. Embeddings of past interactions. Per-tenant namespace.
• Graph (Neo4j): Entity relationships. (Sarah) —[manages]→ (Chicago Office). Cypher queries. Temporal properties.
• Procedural: Learned workflows. Successful multi-step processes stored for reuse.
• Memory Orchestration (Mem0): Unified API. Extract → deduplicate → conflict-resolve → store. GDPR forget.

L5 — Tool & Protocol

• MCP Gateway: Centralized proxy for all tool calls. Auth, rate limit, log, cache, per-server circuit breaker (5 failures → OPEN → 30s recovery → HALF_OPEN probe). Enforces agent-level tool grants — agents can only invoke tools they've been explicitly granted access to (see docs/ACCESS_CONTROL.md Dimension 2). Default-deny: no grant = no access.
• MCP Registry: Dynamic tool discovery by capability. Auto-registers n8n workflows with MCP Server Trigger. Each tool has a risk level (low/medium/high/critical) that drives default grant policies.
• A2A Protocol: Agent-to-agent communication. Agent Cards. JSON-RPC 2.0 over HTTPS. Enforces agent interaction policies — agents can only invoke other agents they've been explicitly granted access to (see docs/ACCESS_CONTROL.md Dimension 3). Loop prevention via execution depth counter and cycle detection. Governance is carried as a formal A2A extension (urn:agentic-platform:governance:v1) — HITL status, compliance metadata, and token usage are attached to standard A2A lifecycle events rather than using a custom protocol.
• AG-UI Protocol: Real-time streaming from agent to frontend. SSE-based event stream carrying text deltas, tool call progress, and state snapshots. Used as the "pipe" layer between agent execution and the Portal UI.
• A2UI Protocol: Declarative UI surface protocol (a2ui.org, v0.8) for content-level HITL. Agents emit structured A2UI surfaces (selection lists, parameter editors, free-text inputs) that frontends render natively. Replaces custom HITL widget formats with a standard component vocabulary.
• Tool Signing: Sigstore/Cosign. SLSA Level 3 attestations. Unsigned tools blocked.
• Tool Sandbox: Isolated execution for untrusted tools. Network egress filtering. Timeout enforcement.

L6 — Semantic & Knowledge

• Embedding Pipeline (LlamaIndex): Ingest → Chunk → Embed → Index. Multi-strategy chunking.
• Semantic Query Engine: NL → data queries. Hybrid search (vector + keyword + metadata). Re-ranking. Citation tracking.
• Data Catalog: Metadata registry with ontologies, schemas, quality scores.
• GraphRAG: Knowledge graph + vector search for multi-hop reasoning.

L7 — AI/ML Services

• LLM Gateway — T-Systems AI Foundation Services (AIFS): Unified OpenAI-compatible API across all providers. GDPR-compliant EU hosting (Germany/Switzerland). Base URL: https://llm-server.llmhub.t-systems.net/v2. Available models include: Claude Sonnet 4, GPT-4.1, Gemini 2.5 Pro/Flash, Llama 3.3 70B (self-hosted), DeepSeek R1 (self-hosted), Qwen3-32B (self-hosted), o3/o4-mini. AIFS replaces LiteLLM — it IS the unified gateway. Model routing and fallback chains are implemented at the platform API gateway level. n8n calls flow through Guardrails Proxy → AIFS. Embedding model: jina-embeddings-v2-base-code. Speech: whisper-large-v3.
• Model Access PEP: Policy Enforcement Point sitting in front of AIFS. Enforces agent-level and tenant-level model access policies (see docs/ACCESS_CONTROL.md Dimension 4). Checks: is this agent/tenant allowed to call this model? Is the model EU-hosted or external? Does the agent's daily spend exceed its cap? Default-deny: agents only access explicitly allowed models. Handles fallback routing when primary model is denied.
• Model Registry: Version history, benchmarks, bias audits, capability boundaries. Populated dynamically from AIFS /models endpoint. Each model tagged with: hosting type (eu-self-hosted / external), source type (open-source / proprietary), cost tier (budget / standard / premium / reasoning), deployment region.
• RAG Service: Retrieval + generation + grounding verification + citation injection.
• FinOps: Token budgets per agent/tenant. Track usage.prompt_tokens and usage.completion_tokens from each AIFS response. Circuit breakers for runaway loops. Cost attribution. Spend alerts.
• Model Eval Engine: Golden test sets (100+ Q&A), latency monitoring, drift detection, safety regression.

L8 — Governance & Security

• Cedar PDP: Attribute-based runtime authorization. (agent X, action Y, resource Z, conditions) → allow/deny. Enforces all four access control dimensions (see docs/ACCESS_CONTROL.md): (1) User→Agent visibility via entitlements, (2) Agent→Tool permissions via tool grants, (3) Agent→Agent permissions via interaction policies, (4) Agent→Model access via model policies. All dimensions are default-deny.
• OPA/Gatekeeper: Infrastructure policy (K8s admission, network policies).
• SPIFFE/SPIRE: Every agent gets SVID (15-min TTL). n8n via Identity Injector.
• User Identity (Hybrid Model): Authentication always delegated to external IdP via OIDC/OAuth 2.0. Pluggable per tenant: Keycloak (self-hosted, EU sovereign), Azure AD/Entra ID (Microsoft ecosystem), or any generic OIDC provider. User provisioning: SCIM 2.0 sync (recommended — IdP pushes user/group changes), JIT provisioning (record created on first login from JWT claims), or bundled Keycloak (standalone/PoC). Platform maintains local user records for role assignment (consumer/builder/admin), agent entitlements, activity tracking, and cost attribution. IdP remains source of truth for credentials and group memberships.
• Agent Registry: Single source of truth. Agent Cards. Status management (active/suspended/quarantined). Stores entitlements, tool grants, interaction policies, and model access policies per agent.
• Vault: Zero-standing-privilege secrets. Short-lived, scoped credentials. Auto-rotation.
• Supply Chain: SLSA L3, SBOMs, Sigstore signing. Blocks unsigned artifacts.
• Compliance: EU AI Act, NIST AI RMF, ISO 42001 mapping.

L9 — Observability

• OTel + Prometheus + Grafana: Metrics, traces, logs. Custom LLM semantic conventions.
• Agent Tracing: End-to-end trace per execution. Spans for each step.
• Decision Audit Log: Tamper-proof. What was decided, why, what data, what alternatives. Append-only.
• Behavioral Baselines: ML-based anomaly detection. Unusual tool usage, data access, API spikes → SIEM.
• Cost Attribution: Per agent/workflow/tenant/model/tool. Chargeback/showback.

L10 — Infrastructure

• K8s namespaces: platform-core, agents-procode, agents-n8n, data
• Databases: PostgreSQL (operational), Redis (cache/queue), Qdrant (vector), Neo4j (graph)
• Storage: S3-compatible object storage for docs, models, audit archives
• Messaging: Kafka (events), RabbitMQ (task queues)
• GPU: NVIDIA MIG for multi-tenant GPU sharing

Cross-Runtime Patterns

1. n8n as MCP Tool: Pro-code agent calls n8n workflow via MCP for integrations (Jira, Slack, CRM)
2. Pro-Code as MCP Tool: n8n workflow calls pro-code agent via MCP for complex reasoning
3. Mixed Orchestration: Temporal workflow delegates steps to whichever runtime fits best
4. Prototype to Promote: Build in n8n, validate, promote reasoning to pro-code if needed

Execution Lifecycle & Termination

Executions follow this state machine: starting → running → completed | cancelled | error | timeout.

Cancellation flow:

1. User clicks stop in Portal → POST /api/chat/conversations/:id/cancel
2. API Gateway looks up active Temporal workflow for the conversation
3. Sends cancelWorkflow signal to the Temporal workflow
4. Workflow checks cancelled flag at the next iteration boundary and stops
5. RUN_CANCELLED event published via Redis pub/sub to SSE clients
6. For external agents: the HTTP fetch abort signal propagates to the AG-UI connection

Automatic termination:

• Execution watchdog — background job (30s interval) cancels workflows exceeding maxExecutionTimeMinutes
• Cost circuit breaker — cancels all running workflows for an agent when its daily cost cap is exceeded
• Behavioral baselines — auto-suspends agents on 3σ+ anomaly detection and cancels their running workflows

Configurable limits (per-agent, admin-adjustable at runtime):

• Max execution time, max tokens, max cost per execution, daily cost cap
• Max concurrent executions, max requests per minute

Per-user limits (admin-adjustable):

• Messages per minute, daily cost, daily tokens, concurrent conversations

All cancellations and limit violations are recorded in the decision audit log.

Security Model

• Zero-trust: Every agent = untrusted workload. Authenticate + authorize + audit everything.
• Three-layer auth: RBAC (role baseline) → ABAC (Cedar context) → IBAC (intent classification)
• Four access control dimensions (see docs/ACCESS_CONTROL.md): D1 User→Agent, D2 Agent→Tool, D3 Agent→Agent, D4 Agent→Model. All default-deny. All decisions audit-logged.
• Isolation: Kata/gVisor (pro-code), K8s namespace (n8n), network egress allowlists, token/CPU/memory limits
• Detection: Behavioral baselines → anomaly → alerting → SIEM → auto-suspend (3σ+ anomalies trigger automatic agent suspension)

Multi-Tenancy

• Control Plane (shared): Agent Registry, Cedar PDP, LLM Gateway, Observability, Billing
• Data Plane (isolated): Vector namespace, graph partition, memory store, audit log, execution environment
• Isolation tiers: Shared → Dedicated NS → Dedicated Nodes → Fully Isolated
• tenant_id propagation: JWT → API GW → every layer → every data store → every audit log

Implementation Status