End-to-end AI engineering: research, fine-tuning, deployment, monitoring, evaluation.
Quantized a 4B-parameter medical LLM (MedGemma) to W4A16 and deployed it to production on NVIDIA Jetson AGX Orin, cutting memory footprint enough to run reliably on constrained unified-memory hardware most teams avoid for LLM serving. Resolved multiple blocking upstream bugs along the way (vLLM validation errors, tokenizer round-trip issues, transformers multimodal refactor conflicts) that stall this deployment path for other teams.
What this proves: I can take a model from research to a working, resource-constrained production deployment — not just a notebook demo.
Stack: llm-compressor, vLLM, Jetson AGX Orin, Docker, NGC containers
Outcome: Published open-source to HuggingFace (confamnode/medgemma-1.5-4b-it-W4A16-G128) as a working reference deployment.
Designed and built an AI inference gateway from the ground up — async streaming APIs, multi-provider routing, and multi-node GPU infrastructure — so data-residency-sensitive organizations can run AI without depending on external API providers.
What this proves: I can architect and operate infrastructure-grade AI systems, not just call an API.
Stack: FastAPI (async, SSE streaming), LiteLLM, Qdrant, vLLM, Cloudflare Tunnels, Jetson AGX Orin + Orin Nano nodes
Outcome: Live inference gateway with tiered routing (confam-health, confam-local), built around NDPA-compliant, data-sovereign architecture.
Built a test-first Python SDK wrapping multi-provider LLM routing, keeping the dependency footprint minimal by integrating directly with LiteLLM rather than a heavier framework layer.
What this proves: I write maintainable, well-tested engineering — not just prototype scripts.
Stack: Python, dataclasses, TDD, LiteLLM (raw integration, minimal dependency footprint) Outcome: Production SDK powering client integrations for ConfamNode.
Architected a multi-tenant RAG system — shared vector store with per-user data isolation, cloud-hosted embeddings, and edge-routed chat inference — to keep infrastructure costs down while keeping each customer's data properly separated.