Outcome
Gateway overhead under 100 ms per turn, and a demo stack that can front any customer's agent by swapping a single secret.
The problem
Copilot Studio’s native real-time voice channel limits voice choice. Customers wanted natural local-accent voices, and the ability to put their existing agents on the phone without rebuilding them.
What I did
- Built a Function App that answers inbound Teams Phone Extensibility calls through ACS, runs speech recognition, relays text to the agent over Direct Line, and speaks responses with Azure Speech or ElevenLabs.
- Added routing by dialled number — two numbers on one ACS resource, each with a different voice provider — by carrying the choice across events on the call’s correlation ID.
- Implemented escalation to a named Teams user: the agent emits a handoff, the gateway resolves the person through Graph and adds them to the live call.
- Diagnosed latency with Application Insights: the gateway itself was under 100 ms; the real variance was the agent’s knowledge routing. Rewriting 14 near-identical knowledge-source descriptions and cutting filler dead air from 4.0 s to 1.5 s made calls feel noticeably tighter.
- Tested voice availability per region with real synthesis calls instead of trusting published tables.
Result
The gateway runs in production on the supported Teams Phone Extensibility path, with a separate demo stack that can’t affect production.