LLM Infrastructure
Routly // Carbon-Aware LLM Router
Founding Engineer · Jan 2025 – Present
Carbon-aware LLM routing system optimizing across carbon intensity, latency, cost, and model quality using live grid signals. Production streaming with fallback chains and zero data loss.
- + ~40% carbon intensity reduction per session
- + Multi-region routing with vLLM backends
- + 8.5% fallback rate, zero data loss
Engineering Challenge
Balancing real-time carbon grid signals with latency SLAs across distributed regions