Satellite inference on continents without degradation
Satellite inference on terabytes of data: how the OlmoEarth platform works and the architectural solutions that eliminate I/O and scaling bottlenecks
Serverless clean architecture without vendor lock-in
How serverless clean architecture reduces vendor lock-in and isolates business logic in a multi-cloud environment using Spring Cloud Function and Terraform CDK
Client-side load balancing at 1M RPS
How client-side load balancing reduces latency at 1M RPS: An analysis of architecture, consistent hashing, and the trade-offs of high fan-out systems
LLM serving platform vLLM and Triton in production
LLM serving platform based on vLLM and Triton: architecture, trade-offs, and bottlenecks in production when scaling inference
Orbital AI Data Centers Hit Network Limits
Orbital AI data centers promise cheap energy and cooling, but network bandwidth, latency, and topology may limit large-scale LLM training in space
LLM-MAS protection against dual drift attacks
Securing LLM-MAS via continual learning and OOD detection: An analysis of OpenEvoShield and its architecture against dynamic attacks
MoE routing acceleration without dynamic networking
MoE routing in direct topologies is limited by unpredictable traffic. An analysis of MoX shows how static routing reduces congestion and approaches switch performance. In classic ML clusters, the network is designed for regular collective operations. This works as long as the traffic is predictable. MoE routing breaks this assumption. Each token selects top-K experts, … Read more
Byzantine Agreement round complexity lower bound
A tight lower bound is established for the round complexity of Byzantine Agreement. An analysis of the impact of an adaptive adversary on latency and consensus architecture.
LLM Serving Workload Analysis FineServe Without Illusions
LLM serving workload turns out to be significantly more complex than is typically modeled. FineServe demonstrates how real workloads break simplified assumptions and impact architecture. Modern LLM platforms operate as always-on services with strict requirements for latency and throughput. The main issue is the unstable and heterogeneous LLM serving workload. Most studies have relied on … Read more
In-band SDN coordination without state explosion
How in-band SDN control plane scales without state growth: an analysis of Periplus and its model of multi-controller coordination