What is SecondStack
SecondStack is a self-hosted, multi-model LLM platform for deploying, managing, and scaling conversational AI across an organization. It bundles a chat app, an LLM gateway, guardrails, cost controls, retrieval (RAG), and an agent runtime into a single platform that runs entirely on infrastructure you control.
The umbrella promise is simple: own your AI infrastructure. Conversations, attached documents, and usage data stay in your own environment, and you decide which models, providers, and policies your teams can use.
Who it is for
Section titled “Who it is for”SecondStack serves two closely related needs, often in the same organization:
- A self-hosted AI platform — a chat, code, and agent experience for teams that want a multi-provider AI workspace without sending their data to a third-party SaaS.
- An enterprise LLM gateway — a central control point for platform, security, and IT teams that need visibility, access control, budgets, and guardrails over all LLM traffic.
Why self-host
Section titled “Why self-host”- Data sovereignty — chat history, documents, and usage logs stay in your own PostgreSQL database and infrastructure. You control retention, backups, and access.
- Multi-model flexibility — route to Claude, GPT, Gemini, Databricks, OpenRouter, and self-hosted open-weight models from one place, instead of locking into a single vendor.
- Consumption pricing — pay for actual LLM API usage rather than per-seat subscriptions. In larger organizations, where most users are low-volume, this is typically far more cost-effective than flat per-seat fees.
- Customizable behavior — shape how the AI behaves with system prompts, guardrails, and agent Skills, tuned to your policies.
What’s inside
Section titled “What’s inside”SecondStack is a productized, self-hostable platform composed of several services that work together:
| Component | What it does |
|---|---|
| Chat App | Multi-model chat UI with image generation, voice input, attachments, RAG, and agents. |
| Cowork Agent | Agentic mode that runs Claude Code in per-thread Docker sandboxes with built-in Skills. |
| API Routing | A LiteLLM gateway that centralizes traffic with virtual API keys, guardrails, and budgets. |
| Cost Management | Per-user, per-team, and per-API-key budgets with threshold alerts and usage analytics. |
| Knowledge & RAG | Named, shareable document sets auto-indexed for retrieval inside chats. |
| ControlTower | Admin dashboard for models, providers, prompts, guardrails, and access control. |
| User Dashboard | Self-service API keys, personal budgets, and per-request usage logs. |
Under the hood, the platform also runs supporting infrastructure — PostgreSQL for data, OpenResty as the API gateway/ingress, content-extraction services for documents, Meilisearch for chat search, a vector database for embeddings, and a built-in Authentik identity core that federates to your upstream identity provider, such as Entra ID (Azure AD), Okta, or Keycloak, for single sign-on.
Next steps
Section titled “Next steps”- Learn how the platform is deployed in Self-Hosting.
- Browse the features to see what each part does.
- Check the FAQ for deployment, security, and model questions.