intelligent_route

Selects an upstream cluster from a site/capability descriptor by matching either an inference model name or MCP tool name.

Configuration Notes

This filter is registered by the AI proxy (not Praxis core) because it encodes AI-specific routing semantics: ordered candidate consumption, admission-state filtering, session affinity, and MCP tool-call routing. Praxis core provides the generic filter runtime; this filter adds the intelligent routing candidate model on top.

Modes: - Static: candidates are declared inline in the YAML config. - Overlay: candidates are loaded from an overlay file (routing-overlay.json envelope or legacy routing-config.json) and hot-reloaded via [ArcSwap] when the file changes.

Behavior: - If the request path matches a configured skip_paths prefix (management and discovery endpoints), the filter returns Continue before any model lookup, setting ctx.cluster to management_cluster when one is configured. - If ctx.cluster is already set by an earlier filter, the selection is preserved and no metadata is written. - If no routing source is present, the filter returns Continue without routing. - If the model header or MCP tool name is blank, oversized, or invalid, the filter rejects with 400. - If a matching candidate is found, ctx.cluster is set and bounded route-decision metadata is written. - If no matching candidate is found, the filter rejects with 404.

Selection: session affinity is resolved first. New requests use the overlay’s selection mode within the first viable producer-defined group. Missing group or policy metadata uses deterministic first-admitted ordering. Praxis AI does not recompute source geography, load, or score. admission_state=none is never eligible. existing_only is eligible only through an already-bound session affinity entry.

Metadata: on successful selection, bounded in-process filter metadata is written under the intelligent_route. namespace (kind, name, site, cluster, local_site, stable_id, admission_state, and optionally rank, selection_group, selection_mode, and selection_tier). When session affinity is enabled, session.bound, session.reused, and session.failover keys are also written. When the selected cluster is present in provider_hop_clusters, client-supplied x-ai-routing-candidate, x-ai-routing-request-id, and x-ai-routing-revision values are removed and replaced with the selected stable ID, a generated provider-hop request ID, and the serving overlay revision (envelope mode only). These AI-owned, non-reserved headers are sent only to an mTLS-authenticated provider gateway; the provider must run peer_identity_trust before consuming them. No credential reference or value is forwarded. No request-time database, control-plane, or metrics lookups are performed.

MCP lookup: if mcp.method filter metadata is set to tools/call and mcp.name is present, mcp_tool candidates are matched. Other MCP methods (initialize, notifications/*, etc.) skip routing.

Hot reload: when reload.enabled is true (the default in overlay mode), the filter watches the overlay file’s parent directory for filesystem events. On change, the file is re-read, SHA-256 hashed, parsed, validated, and atomically swapped in when its semantic revision changes via ArcSwap. In-flight requests continue using their previously loaded snapshot. Unreadable or invalid files retain the previous snapshot. Kubernetes ConfigMap projected volumes use atomic symlink replacement (..data), which the watcher detects as a Create/Modify event on the parent directory. The overlay ConfigMap must not use subPath volume mounts — subPath bypasses the ..data symlink mechanism and the watcher will not detect updates.

Envelope contract: The configuration producer publishes routing-overlay.json as a versioned, content-addressed envelope alongside the legacy routing-config.json payload. The AI-owned v1 shape is:

source_generation is a positive monotonic generation of the source resource. Unknown additive envelope, provenance, and candidate fields are accepted for forward compatibility; credential objects reject unknown fields to prevent secret material from entering the overlay.

Any reserved envelope field selects strict envelope parsing; malformed envelopes never fall back to legacy. When expected_overlay_scope is configured, legacy payloads are rejected so scope validation cannot be bypassed. Praxis AI recomputes the RFC 8785 canonical SHA-256 digest over network, local_site, and the ordered candidate list before accepting a snapshot.

The revision lifecycle is observable as: - rendered: The producer constructed the envelope. - distributed: The producer applied it to the destination ConfigMap. - accepted: Praxis AI parsed, scope-checked, and digest-verified it. - serving: a request selected a route from that exact snapshot.

Invalid cold-start envelopes fail filter construction. Invalid reloads retain the same-process last-known-good snapshot. Envelope-mode provider hops carry the serving revision from the same immutable snapshot used for candidate selection.

Scope: overlay hot reload swaps the candidate list and local_site only. It cannot add or remove load_balancer clusters, change cluster endpoints or TLS configuration, or inject credential values. Those changes require a full pipeline reload or pod restart. Every cluster name that may appear in any overlay version must already be configured in the downstream load_balancer filter. An overlay that references an unknown cluster will cause request-time failures, not a reload rejection.

Supports two modes:

Static mode — candidates and local_site are specified inline:

Overlay mode — candidates are loaded from an overlay envelope:

Configuration