Hyperflux and Microsecond-Scale Core Elasticity
Microsecond-scale cross-VM core elasticity explained: how Hyperflux shifts cores across VMs to cut tail latency without losing ultralight VM properties
Highload
Highload on ThecoreGrid focuses on designing and operating systems that handle massive scale, traffic, and data under strict reliability requirements.
We explore architectures and patterns for horizontal scaling, load distribution, fault tolerance, and performance optimization in distributed environments. Topics include sharding, replication, caching strategies, queueing systems, backpressure handling, and latency reduction under peak load. We analyze real-world trade-offs between consistency, availability, and cost, along with failure scenarios and recovery strategies. Content is grounded in BigTech practices, including incident post-mortems and lessons from operating systems at global scale. You’ll find deep dives into infrastructure behavior, traffic management, autoscaling, and resilience engineering. Instead of simplified guides, the Highload tag delivers practical engineering insights for backend engineers, architects, platform teams, and SREs responsible for building and maintaining systems that must perform reliably under extreme demand.
QoS-aware autoscaling for distributed AI inference
QoS-aware autoscaling for AI inference: distributed scheduling with user devices, lower dedicated capacity, and better tail latency under growth
FHIR and Kafka for Wearable Analytics
FHIR and Kafka for wearable analytics: analysis of cloud-native architecture, FHIR normalization, low-latency ingestion, and clinical workloads
PTX Tensor Core GEMM: When Hand-Written Code Pays Off
PTX Tensor Core GEMM on NVIDIA L4: why hand-written kernels help for INT8 and INT4, and why FP16 still favors WMMA
GPU LZ77 Decode: Where Serialization is Hidden
GPU LZ77 decoding on the H100: where serialization is hidden, why parsing matters more than copying, and the trade-offs involved in data addressability
Kueue for Netflix: seamless batch migration
Kueue migration at Netflix: how to replace CMB with a Kubernetes-native batch platform, maintain API parity, and improve resource utilization
RAP in Spotify: an index for point queries
RAP in Spotify demonstrates how an external index on top of Parquet accelerates point queries in the data lake without copying data to serving databases
Service Topology: how Netflix keeps the dependency map live
Netflix Service Topology: how the real-time service map maintains integrity under load, using backpressure, SSE, and three processing stages
LLM serving with latency budget instead of queues
Latency budget in LLM serving changes the priorities of scheduling. CASCADE demonstrates how to link scheduling and KV-cache for increased goodput. The problem arises when all requests are formally equal in SLO, but in reality, they are not. In one cluster, chat, code generation, and reasoning coexist simultaneously. Their costs differ by orders of magnitude: … Read more
DTMC Planning for URLLC Considering HARQ RTT
DTMC analysis of URLLC with proactive HARQ, accounting for HARQ RTT: How to properly schedule resources and avoid reliability and latency degradation