An AI gateway banks can actually run.
On-premise LLM platform for financial institutions, architected in C#/.NET. Multi-provider model routing, RAG pipelines with vector search, and multi-agent orchestration — with zero external data egress. Employer and client names withheld; specifics available under NDA.
01 // Problem
Banks want LLM capability — document understanding, retrieval-augmented answers, agent workflows — but regulation and risk mean customer data cannot leave the building. Public-cloud AI APIs are a non-starter; every byte must stay on infrastructure the institution controls.
The gateway's job: give internal teams one API surface for AI capability, run entirely on-premise, with the same reliability bar as any other core-banking dependency.
02 // Decisions
- Provider abstraction first. Models change monthly; the routing layer normalizes providers behind one contract so swapping engines is config, not code.
- RAG as a pipeline, not a library call. Chunking, embedding, vector search, and prompt assembly are explicit stages — each one tunable and auditable.
- Multi-agent orchestration for workflows where one model call isn't enough — deliberate, bounded, logged.
- Zero egress as an architectural property, not a policy document: the deployment has no route to external AI endpoints.
- C#/.NET because it's what the institution's teams can operate, extend, and audit.
03 // Architecture (sanitized)
04 // Properties
Throughput and deployment specifics are the client's to share, not mine. The personal-lab equivalents of every moving part here — RAG bot, embeddings store, agent deliberation — run on my own hardware and are documented on the fintech-ai page.