Orchestration
CloudPlay — GPU game-streaming orchestration
Four Spring Boot microservices that place live game sessions onto a pool of GPU hosts, and reclaim them the moment a session dies.
Role
Sole engineer
When
2026
Outcome
Best-fit bin-packing across an autoscaled EKS GPU pool
Stack
- Problem
- GPU hosts are expensive and game sessions are long-lived and stateful. Naive scheduling either fragments the pool or leaks capacity when clients disappear without signing off.
- Move
- Split the system into four services — API gateway, session orchestrator, WebRTC SDP/ICE signaling, and a distributed node registry — and made every reservation an atomic Redis/Lua operation guarded by a heartbeat lease.
- Result
- A best-fit bin-packing allocator keeps the pool dense, crashed sessions self-release when their lease lapses, and a demand-based autoscaler drives Kubernetes Deployments and HPAs over a Terraform-provisioned EKS GPU pool.
$ s1 needs 3 → host 1 (fullest host that fits)
- running session
- just placed
- free capacity
CONTEXT
Streaming a game is not a request/response problem. A session occupies a specific GPU for as long as the player keeps playing, negotiates a peer connection that can fail independently of the control plane, and has no reliable way to tell you it has ended — the client just stops.
That combination makes the interesting work allocation and reclamation rather than throughput. Pack sessions too loosely and you pay for idle GPUs; pack them without a liveness mechanism and capacity quietly disappears every time a client crashes.
WHAT I DID
- Built four Spring Boot microservices: an API gateway, a session orchestrator, a WebRTC SDP/ICE signaling service, and a distributed node registry tracking per-host GPU capacity.
- Implemented a best-fit bin-packing allocator that places each session on the most-loaded host that still fits it, keeping the pool dense instead of fragmented.
- Made reservations atomic with Redis/Lua scripts, so two concurrent requests can never double-book the same GPU.
- Attached a heartbeat lease to every reservation — when a session stops renewing, the capacity self-releases without operator intervention.
- Drove Kubernetes Deployments and HPAs from a demand-based autoscaler over an EKS GPU pool provisioned with Terraform.
RESULT
The control plane holds a correct view of GPU capacity under concurrent placement and abrupt client loss — the two failure modes that make naive schedulers leak capacity.