Traditional application architectures separate routing, security, and observability from business logic - so that developers can move faster without the tax of reinventing the wheel. LLM applications should be no different. ArchGW applies these patterns to prompts, providing a structured approach to building LLM applications.
How It Works
ArchGW runs as a separate process alongside application servers. It intercepts prompts before they reach the backend and applies transformations based on predefined rules and models:
Preprocessing: Normalizes and analyzes prompt structure.
Security Filtering: Rejects jailbreak attempts and unsafe inputs.
Intent Mapping: Determines if a request maps to an API function.
Function Invocation: Extracts arguments and calls backend APIs.
LLM Routing: Chooses the right LLM provider based on latency/cost constraints.
Tracing & Metrics: Adds W3C Trace Context headers, tracks errors, token usage, and request latency.
Why a Dedicated Proxy?
Traditional application architectures separate routing, security, and observability from business logic—LLM applications should be no different. ArchGW applies these patterns to prompt processing, providing a structured approach to LLM integration.
1 comment
[ 5.0 ms ] story [ 25.9 ms ] threadTraditional application architectures separate routing, security, and observability from business logic - so that developers can move faster without the tax of reinventing the wheel. LLM applications should be no different. ArchGW applies these patterns to prompts, providing a structured approach to building LLM applications.
How It Works
ArchGW runs as a separate process alongside application servers. It intercepts prompts before they reach the backend and applies transformations based on predefined rules and models:
Why a Dedicated Proxy?Traditional application architectures separate routing, security, and observability from business logic—LLM applications should be no different. ArchGW applies these patterns to prompt processing, providing a structured approach to LLM integration.