Building the layer between a live phone call and a language model.
3+ years shipping at the intersection of generative AI, voice interfaces, and cloud infrastructure — production systems, not prototypes, handling real callers under real latency budgets.
Three engagements, three different shapes of problem.
I build systems that have to work the first time — because a caller doesn't retry a bad response, they hang up.
I built a production voice AI pipeline handling live phone calls at sub-2-second latency, serving 5,000+ monthly callers — then migrated the entire system from AWS to Azure in one week when the client's licensing changed three weeks before handoff.
Alongside that, I delivered an AI-powered restaurant SaaS platform as the sole engineer, piloted across 15–20 US restaurants, and independently built a phone-accessible voice assistant backed by a live CRM.
I'm now looking to go deeper — building AI-native products and the infrastructure layer underneath them, for teams who ship real systems, not demos.
From CS undergrad to production AI pipelines.
Tools I ship with.
Real-Time Voice AI Pipeline for a Genesys Contact Center
The full signal chain, caller to model and back.
Owned the complete cloud pipeline within a cross-functional team — BA, tester, Genesys dev, project manager, client manager, solutions architect — as the sole AI/cloud/voice engineer. Built from scratch: audio in from the caller, transcribed in real time, reasoned over by an LLM, and spoken back.
End-to-end round trip held at 1.0–2.1 seconds, live on real phone calls.
MVP running ~5,000 calls/month, architected toward ~2 million calls/year at full rollout.
Running on ECS Fargate with load balancing, auto-scaling, and validated multi-call context preservation.
Migrated the entire pipeline from AWS to Azure in one week, solo, mid-engagement.
- The initial design was Lambda-based, routing through Genesys's web flow and workflow designer — latency from that approach wasn't meeting client expectations.
- Rather than waiting for direction, I worked with the solutions architect to prototype a streaming alternative using Genesys's audio connector.
- Built a browser-based PCM16 stream as a working demo for the team and stakeholders before presenting the approach to the client.
- The finished pipeline supported two audio paths simultaneously — PCMU over Genesys's AudioHook for live calls, PCM16 over a browser WebSocket for development — sharing the same call-handling logic rather than maintaining two systems.
- A live phone call means a hard latency budget, no chance to retry or revise a response once speech synthesis has started, and a caller who simply hangs up if the experience feels broken.
- LLM responses were scoped to approved intents and topics via system prompts — the model couldn't go off-script or hallucinate actions outside the defined flow.
- PII was excluded from model context and stripped from logs, keeping caller data out of inference and audit trails.
- Not every turn needed the LLM — authentication, appointment booking, satisfaction surveys, and call transfers ran through deterministic logic that skipped the model entirely.
- A watchdog layer — 800ms end-of-speech detection paired with a 5-second silence timeout and short filler responses — kept the conversation from stalling during slower lookups.
- Nova Pro and Nova Lite were benchmarked head-to-head across 80 real turns, measuring both average and p95 latency.
- Lite came out 27% faster on average and 66% faster at the tail (1.7s vs. 5.0s p95) — tail latency is what a caller actually notices, so that's what decided production.
- The system was validated across 50–200 real test sessions before handoff.
- The BA and tester evaluated the bot through continuous real conversations — what shipped was driven by conversational quality and use-case fit, not latency metrics alone.
- Each version was tracked through git branching and Docker image versioning.
- Azure Log Analytics fed back into every iteration, surfacing latency regressions and conversation failures as they happened.
- BA and tester conversations generated eval data naturally — every real interaction was a potential test case.
- Conversation logs surfaced failure patterns: misunderstood intents, wrong slot fills, unexpected caller phrasing.
- Eval sets were built from production conversations to regression-test prompt changes, so fixes never broke previously working scenarios.
- The upstream API providing caller data took up to 12 seconds to respond — enough to stall the opening of every call.
- Fix: pre-fetch the client data on call connect and mask the lookup behind the intro and authentication flow, so the caller never experienced the delay.
- The same filler-based pattern was applied across other slow operations throughout the pipeline.
- Three weeks before handoff, a late client-side licensing change forced the pipeline off AWS and onto Azure.
- STT, LLM, and TTS layers were re-implemented against Azure Container Apps, Cosmos DB, Azure Speech SDK, and Azure OpenAI — shipped in one week, solo, using AI-assisted development for rapid API migration and debugging unfamiliar services.
- This meant working inside the client's live production Azure account — provisioning and securing every service in coordination with their infrastructure team.
- Azure Container Apps' native WebSocket endpoint rejected the Genesys AudioHook connection despite valid DNS/SSL — fixed with an API Management gateway fronting the container, secured with API-key auth and Key Vault-managed secrets.
- End-to-end latency held at the same 1.0–2.1s range post-migration, with no measurable regression.
- Architected an ASP.NET MVC to .NET + React migration pipeline.
- Built a multi-provider LLM gateway (LiteLLM proxy across AWS Bedrock, Azure OpenAI, and hosted models), used internally.
- Built a RAG evaluation playground for benchmarking embedding strategies and LLM configurations.
AI-Powered Restaurant Management Platform
Full-stack SaaS product for restaurant operations, built and shipped alone from an empty repository to a live pilot with 2,000+ active users across 18 restaurants. Worked directly with the founder as his implementation partner, introducing AI-driven development practices across the product lifecycle.
Recommendation engine driven by structured LLM-enriched flags, not raw text similarity.
OCR + LLM onboarding pipeline digitized 50+ real menus across 10+ layout styles.
Multi-tenant FastAPI backend: JWT auth, subscription-tier access, three role types.
Platform remains live post-handoff; the founder continues to operate it independently.
- Recommendation engine generating personalized dish suggestions inside the customer-facing iOS and Android apps, driven by structured DB flags enriched by LLM during onboarding — not raw text similarity.
- Claude-based analytics layer surfacing dish-performance insights back to restaurant management.
- Image-generation pipeline producing visuals for menu items without photography; owners can also upload existing photos.
- Restaurant onboarding required digitizing existing paper and PDF menus with no consistent format across customers.
- A computer-vision pipeline (OpenCV + Tesseract OCR) parsed 50+ real menus across 10+ distinct layout styles without manual re-entry per restaurant.
- An LLM enrichment step then generates structured flags per dish — spicy, allergen, dietary, cuisine type, flavor profile — which drive the recommendation engine.
- When descriptions are too sparse for reliable inference, owners fill the gaps through the management portal.
- Multi-restaurant chain support: separate dashboards per restaurant, gated by subscription tier.
- Flexible menu structure — one branch can hold multiple menus (seasonal, special occasion), and multiple branches can share a single menu.
- Owner management portal for adding, editing, removing, and hiding dishes, including uploading or generating dish images.
- Multi-tenant FastAPI service with JWT-based auth, subscription-tier access control, and role separation across customer, waiter, and management.
- Owner-controlled access grants per user; ordering, cart, and payment gateway integrations built end to end.
- Deployed on DigitalOcean behind Nginx with SSL.
- Dev, staging, and production environments with CI/CD per branch.
- Staging went to peers and a close circle of real users whose feedback shaped what shipped to production.
- Separate admin users tracked every server-side change by author; log monitoring during client demos and live usage surfaced what was working and what wasn't.
Conversational IVR Assistant Over a Real Phone Line
A phone-accessible voice assistant that authenticates a caller, looks up their account, and completes account actions — password reset, contact update, support ticket status — entirely by voice, backed by a live CRM.
LangGraph structured the conversation as an explicit, stateful flow rather than one open-ended prompt.
CRM operations went through an MCP server — tool access described declaratively, not hand-wired.
Verification gates every account-specific action; sensitive actions require a spoken confirmation.
Vapi handled telephony: barge-in, silence timeouts, and re-prompting on unclear input.
Key highlights from freelance engagements.
All clients are available to provide referrals and recommendations on request.
Hamsterit
- OCR + ML pipeline extracting highlighted content from image-based PDFs, replacing a manual database-update process.
- Semantic search (OpenAI embeddings) deployed to the client's live production website for product search.
DoubtBuddy
- RAG-based JEE question-answering chatbot — diagram detection (OpenCV + PyTorch) through semantic retrieval to LLM-reasoned answers, reaching 95%+ accuracy.
- Deployed on AWS with load balancing and caching.
EpowerX
- B2B messaging automation (Google Sheets + Apps Script + WhatsApp Business API) distributing templated messages at scale.
- 33% reduction in image-generation processing time.
Looking for someone who builds the whole thing?
I've gone from empty repo to production as the sole engineer — voice pipelines, SaaS platforms, LLM infrastructure. If your team needs someone who ships end-to-end and owns what they build, let's talk.