How we built the most efficient inference engine for Cloudflare’s network
Existing LLM serving engines aren’t efficient enough when deployed across a globally distributed network, so we built our own, in Rust. Infire is an LLM inference engine that employs a range of techniques to maximize resource utilization, allowing us to serve AI models more efficiently and with better performance for Cloudflare workloads.