An MCP gateway for the whole company
There was a stretch last year where every week someone at work shipped a new MCP server. One team wrapped the deploy system. Another wrapped the data catalog. Someone wired up incident tooling so an agent could read a runbook and open a ticket. Each one was genuinely useful. And each one made the overall picture a little worse, in a way nobody owned.
This post is about the fix: a single proxy that every AI tool in the company connects through before it talks to any MCP server. But first, for anyone who hasn't gone down this rabbit hole, let's cover what an MCP server even is and why everyone was building them in the first place.
MCP in one minute
The Model Context Protocol is an open standard, released by Anthropic in late 2024 and since adopted across the industry, for connecting AI tools to the systems where your work actually lives. An MCP server is a small program that exposes a set of tools ("search the catalog", "create a ticket", "send a message") in a format any MCP-aware AI client understands.
The magic word is standard. Before MCP, connecting three AI tools to five internal systems meant fifteen custom integrations, each with its own glue code and its own way of breaking. With MCP, the team that owns a system builds one server, and every AI tool your company uses (a chat assistant, a coding agent in VS Code, an automation runner) can call it the same way. People describe it as USB-C for AI tools, and the analogy holds up: one connector, everything plugs in.
The good and the bad
The good part is real. Write a server once and every AI client gets the capability. The protocol handles discovery, so a client can ask a server "what can you do?" and get a machine-readable answer. It's the reason agents stopped being chatbots and started being able to do things.
The bad part arrives at scale, and it arrives fast. Point-to-point connections mean every client wires directly to every server, each with its own credentials pasted into local config files. Nobody can answer the only question security actually cares about: who can call what, and what happened when they did? Employees can just as easily point their AI tool at an external MCP server nobody has reviewed as an internal one. And there's a quieter failure too: discoverability. A team would build a great server and then have to market it, posting in channels, demoing it in all-hands, hand-distributing config snippets. Half the value of the ecosystem was invisible to the people it was built for.
Every client wired directly to every server. Tokens and configs copied everywhere. External servers one config edit away.
One front door. Clients trust the gateway; servers trust one caller.
Why enterprises need a layer in the middle
None of the problems above are protocol problems. MCP is doing exactly what it promised. They are governance problems, and governance is not something you can ask every team to re-implement in every server. A company needs a consistent answer to what is allowed in and out of its AI tools, and the only place to put a consistent answer is a layer everything passes through.
So we built one: a proxy MCP server that acts as a gateway. To every AI client (the chat assistants people use, the coding agents in editors, developer tools, automations) it looks like one ordinary MCP server. Behind it sit all the real ones. Every request crosses that middle layer, which is where all the boring important things live: authentication, access policy, rate limits so one runaway agent loop can't take down an upstream a whole team depends on, logging, and an audit trail good enough to replay any tool call months later for compliance. Not because any one of those is exciting, but because it's the one place they can be done once instead of a hundred times.
The best internal AI platform is not the one that feels magical. It is the one where, six months later, you can answer exactly who called what, and prove it.
Publishing a server became one step
The part that made this adopted rather than mandated: the gateway solved the discoverability problem better than self-promotion ever did. The deal we offer server authors is simple. Build your server, deploy it, and register it with the gateway. Registration kicks off a vetting process, the same idea as any onboarding review: does the server expose sensitive data it shouldn't, are its tools scoped sanely, does an owner stand behind it. Pass that, and you're in.
And "in" means something. The moment a server is approved and connected to the proxy, its tools show up in every AI tool in the company, for exactly the people allowed to see them. No launch post, no config snippets in chat threads, no marketing your own infrastructure. One registration, distribution everywhere. The manifest is the entire contract:
{
"name": "data-catalog",
"owner": "data-platform",
"upstream": "https://catalog.internal/mcp",
"visibility": "team:data-platform, team:analytics",
"tools": {
"search_tables": { "access": "read" },
"request_export": { "access": "write", "approval": "required" }
},
"limits": { "per_user_rpm": 60 }
}The gateway reads it, stands up a lightweight proxy for that server, applies the visibility rules, and the right people can simply use it. Keeping a proxy per server (rather than merging everything into one giant toolset) also means a slow or misbehaving upstream is contained, and the team that owns the server owns its policy.
Batteries included
Some servers earned a special status: defaults. The ones almost everyone touches daily, like the email server, the internal chat server, and the file-sharing server, are pre-wired into the gateway for everybody. Install any of our supported AI tools, sign in, and those capabilities are just there, out of the box. Nobody hunts for a config. From the user's side there is exactly one setup step, ever:
// one config, set once, for any MCP-aware AI tool
{
"mcpServers": {
"work-gateway": {
"url": "https://mcp-gateway.internal",
"auth": "sso"
}
}
}Because clients only ever point at the gateway, this configuration never changes. New servers, rotated credentials, and revoked access all happen behind the front door. The user's config stays still; their available tools move.
The hard problem now: is that a person or an agent?
Here's what changed after launch, and what keeps the work interesting. At first the callers were humans in AI tools. Then people did the obvious next thing: they built agents and automations that call MCP servers on their behalf, on schedules, in pipelines, overnight.
That breaks the oldest assumption in enterprise auth, that a credential maps to a person who is present. When an agent calls a tool, who is accountable? The person who launched it? The person whose credentials it carries? What happens when an agent outlives the session that spawned it, or when one agent calls another? Authenticating the user or the agent, and being precise about which one you're talking to, has become the most important design problem on the gateway. We're working through it with our AI security experts now: identity for agents, short-lived scoped tokens instead of long-lived secrets, and an audit trail that records not just who, but on whose behalf. If there's a part 2 to this post, it's that.
What I'd tell someone building this
- Treat the gateway as a product, not a proxy you forget about. Its users are engineers, and adoption follows ergonomics.
- Make publishing a server self-serve and boring. If it needs a meeting, it will not scale to hundreds.
- Vet at the front door. A review at registration time beats a hundred reviews at call time.
- Put identity, limits, and audit in the middle once. Never ask each team to re-implement them.
- Ship defaults. The out-of-the-box experience is what convinces non-engineers the platform is real.
- Log enough to replay any tool call. Future-you, and security, will need it.
The model was never the hard part. The hard part was giving a lot of people a safe way to publish their own tools without each one turning into a security review, and giving everyone else those tools without a scavenger hunt. A governed gateway is how that becomes routine.
Since writing this, the same thinking has followed us to the next layer of the stack: agent skills, the markdown-based packages that teach agents how to work. Same pattern, new package format, one governed front door. That story is in Agent skills: teaching your AI how you work.