LLM Provider (AIFS)

T-Systems AIFS configuration, models, and SDK usage

LLM Provider: T-Systems AI Foundation Services (AIFS)

All LLM calls in this platform go through T-Systems AI Foundation Services. No direct calls to Anthropic, OpenAI, or Google APIs. AIFS provides a unified, OpenAI-compatible API that routes to all providers, hosted GDPR-compliant in Germany/EU.

Connection Details

Base URL:  https://llm-server.llmhub.t-systems.net/v2
Auth:      API key via Bearer token or OpenAI-compatible header
Protocol:  OpenAI Chat Completions API (fully compatible)
Streaming: Supported (SSE, same as OpenAI streaming)

Environment Variables

# T-Systems AIFS (required)
TSYSTEMS_AIFS_API_KEY=<provided-by-user>
TSYSTEMS_AIFS_BASE_URL=https://llm-server.llmhub.t-systems.net/v2

# Legacy names (for OpenAI SDK compatibility)
OPENAI_API_KEY=${TSYSTEMS_AIFS_API_KEY}
OPENAI_BASE_URL=${TSYSTEMS_AIFS_BASE_URL}

Available Models

Source of truth: the models DB table (seeded from this catalogue, then kept current via POST /api/models/sync, which reads the live AIFS /models endpoint). The sync is contract-scoped: it sets availableOnContract per model from what the platform API key actually grants. Off-contract models stay in the registry (priced, historical) but render disabled in selection UIs. Pricing is stored as EUR per 1M tokens (cost_per_1m_input/output), matching the AIFS price list. The table below is illustrative, not exhaustive.

Chat / Reasoning Models

Embedding Models

Speech Models

SDK Usage (TypeScript / Node.js)

Use the standard OpenAI SDK. Just point the base URL to AIFS.

Install

pnpm add openai

Client Setup

// services/api-gateway/src/lib/llm-client.ts
import OpenAI from 'openai';

export const llmClient = new OpenAI({
  apiKey: process.env.TSYSTEMS_AIFS_API_KEY,
  baseURL: process.env.TSYSTEMS_AIFS_BASE_URL || 'https://llm-server.llmhub.t-systems.net/v2',
});

Chat Completion (non-streaming)

const response = await llmClient.chat.completions.create({
  model: 'claude-sonnet-4',  // or any model from the list above
  messages: [
    { role: 'system', content: agent.config.systemPrompt },
    { role: 'user', content: userMessage },
  ],
  temperature: agent.config.model.temperature,
  max_tokens: agent.config.model.maxTokens,
});

const reply = response.choices[0].message.content;
const usage = response.usage; // { prompt_tokens, completion_tokens, total_tokens }

Chat Completion (streaming — for Portal chat)

const stream = await llmClient.chat.completions.create({
  model: 'claude-sonnet-4',
  messages: [
    { role: 'system', content: agent.config.systemPrompt },
    { role: 'user', content: userMessage },
  ],
  temperature: agent.config.model.temperature,
  max_tokens: agent.config.model.maxTokens,
  stream: true,
});

// Forward to client via SSE
for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content;
  if (content) {
    // Send SSE event to client
    controller.enqueue(`data: ${JSON.stringify({ type: 'token', content })}\n\n`);
  }
}
controller.enqueue(`data: ${JSON.stringify({ type: 'done' })}\n\n`);

Embeddings (for semantic search / RAG)

const embedding = await llmClient.embeddings.create({
  model: 'jina-embeddings-v2-base-code',
  input: 'text to embed',
});

const vector = embedding.data[0].embedding; // number[] with 1024 dimensions

List Available Models (for Model Selector screen S12)

const models = await llmClient.models.list();
const available = models.data.map(m => ({
  id: m.id,
  type: m.meta_data?.model_type,     // 'LLM', 'EMBEDDING', 'NLP'
  source: m.meta_data?.source_type,   // 'OPEN SOURCE' or provider name
  region: m.meta_data?.deployment_region, // 'otc-germany' etc.
  isExternal: m.meta_data?.is_externally_hosted,
}));

Model Routing Strategy

The platform uses model routing — different agents can use different models based on their needs:

Default Recommendations

Agent Config Model Selection

Each agent's config specifies primary + fallback model:

interface AgentModelConfig {
  primary: string;     // e.g. 'claude-sonnet-4'
  fallback?: string;   // e.g. 'gpt-4.1'
  temperature: number;
  maxTokens: number;
}

The backend tries the primary model first. If it fails (rate limit, timeout, error), it falls back to the fallback model. This is handled in the API gateway, not the frontend.

Architecture Implications

What AIFS replaces in the architecture

• L7 LLM Gateway (LiteLLM): NOT needed. AIFS is the unified multi-provider gateway. It handles model routing, authentication, and rate limiting.
• Direct Anthropic/OpenAI/Google SDKs: NOT needed. Everything goes through the OpenAI SDK pointed at AIFS base URL.

What AIFS does NOT replace

• Model Access PEP: A policy enforcement point sits between the platform and AIFS. It enforces per-agent and per-tenant model access policies (Dimension 4 of the access control model — see docs/ACCESS_CONTROL.md). This controls which agents can call which models, enforces data sovereignty (EU-hosted only for certain tenants), and applies cost caps.
• FinOps / cost tracking: Still needed at the platform level. Track usage.prompt_tokens and usage.completion_tokens per request and attribute cost.
• Semantic caching: Still valuable to implement at the platform level (before calling AIFS) to reduce costs.
• Guardrails: Still run at the platform level (L1) before and after AIFS calls.
• Model fallback logic: Implement at the API gateway level using primary/fallback config.

Data Sovereignty — Three-Tier Hosting

AIFS provides three hosting tiers with increasing data sovereignty:

de-hosted is the strictest tier — models run entirely on Telekom Open Telekom Cloud infrastructure in Germany. This is the recommended tier for German enterprise customers with strict data residency requirements.

eu-hosted models are hosted by partners within the EU. Data stays within the EU and is GDPR-compliant, but may cross EU member state borders.

external models are routed through AIFS but inference runs on the provider's own infrastructure (via AIFS partnerships with Microsoft/AWS/Google). Data may leave the EU.

For maximum sovereignty, agents can be configured to use only de-hosted models via the allowedModelTags: ['de-hosted'] policy. The Model Access PEP (Dimension 4) enforces this per-agent and per-tenant, with automatic fallback to permitted models when a denied model is requested.

The AIFS API provides hosting metadata per model (meta_data.deployment_region, meta_data.is_externally_hosted) which the platform uses to automatically classify models during sync.

n8n Integration

n8n's AI Agent nodes natively support the OpenAI API format. To route n8n's LLM calls through AIFS:

Without Guardrails Proxy (simple)

Configure n8n's OpenAI credential with:

• API Key: ${TSYSTEMS_AIFS_API_KEY}
• Base URL: https://llm-server.llmhub.t-systems.net/v2

With Guardrails Proxy (recommended)

The Guardrails Proxy sidecar intercepts n8n's LLM calls:

• n8n credential points to: http://localhost:8081/v2 (Guardrails Proxy)
• Guardrails Proxy applies input/output safety checks
• Then forwards to: https://llm-server.llmhub.t-systems.net/v2 (AIFS)

Testing Without API Key

For development without a real AIFS key, set MOCK_MODE=true. The API gateway will return simulated streaming responses instead of calling AIFS. This allows full frontend development without consuming tokens.