Capacity, Drain & Health

Overview

Every Altruist server is a server node (IServerNode): it knows how much work it runs, whether it takes more, and how to stop without cutting anyone off. Three things come with it, with no code to write:

• Capacity — the work the server runs (rooms, worlds, …) adds up to a load, compared with a budget from config.yml. A full server keeps running what it has and starts nothing new.
• Drain — when the server is asked to stop (SIGTERM, a Kubernetes pod termination, an operator), it stops taking new work, lets the running work finish, and only then exits.
• Health and metrics — liveness, readiness and capacity endpoints for orchestrators and load balancers, and standard .NET metrics for autoscalers and dashboards.

Nothing here touches the network or the engine's hot path. Until you configure it, a server is unlimited and drains instantly when nothing is bound to it — existing apps behave exactly as before.

altruist:
  server:
    capacity:
      max-load: 200        # the budget; 0 (default) = unlimited
    drain:
      timeout: 360         # seconds running work may take to finish (default 30)
      force-grace: 5       # seconds stopped work gets after a timeout (default 5)
      on-shutdown: true    # drain when the host stops (default true)
      token: ""            # optional: allow POST /altruist/server/drain with this token

Capacity

Anything that runs on the server and costs something is an ICapacityContributor. It reports its units (how many rooms, worlds, …) and their load (in whatever unit max-load is counted in):

public interface ICapacityContributor
{
    string Kind { get; }           // "rooms", "worlds", ...
    CapacitySample Sample();       // thread-safe and cheap: called from HTTP and metrics threads
}

Built-in contributors:

Register your own as a service ([Service(typeof(ICapacityContributor))]) or at run time with node.Register(contributor) (dispose the result to remove it).

The node sums them into a ServerCapacity report:

var c = node.Capacity();
c.State;        // Starting | Ready | Full | Draining | Drained
c.Load;         // summed load
c.MaxLoad;      // the budget (0 = unlimited)
c.Free;         // MaxLoad - Load, null when unlimited
c.Utilization;  // Load / MaxLoad
c.Kinds;        // units and load per kind

node.CanAccept(cost) is the admission check: true when the server is Ready and the cost still fits. Matchmaking and lobbies use it before opening a room; a room of a mode that does not fit waits in the queue (see Matchmaking).

Drain

Work that has to wind down is an IDrainParticipant:

public interface IDrainParticipant
{
    string Kind { get; }
    void BeginDrain();      // take no new work, let the running work finish (once, any thread)
    bool IsDrained { get; } // nothing runs any more
    void ForceStop();       // the drain timed out: stop what still runs (once at most)
}

node.DrainAsync(timeout):

1. marks the server Draining (CanAccept is false from now on) and calls BeginDrain on every participant — also on ones that register while the drain runs;
2. waits until every participant is drained, up to drain:timeout;
3. on a timeout, calls ForceStop on the ones still running and waits drain:force-grace more;
4. marks the server Drained and returns whether everything finished in time.

Every caller shares the same drain. Cancelling a caller only stops its wait. A participant that throws cannot hang the drain.

A room host bound with UseServer(node) is a participant: queues and lobbies close (or move to another server in a fleet), running rooms play to their end, rejoins still work, connections without a room are told the server is going away (IRoomGame.ServerDraining), and on a timeout the remaining rooms are closed (IRoomGame.OnForcedClose).

Drain on shutdown

ServerDrainOnShutdown drains the node in the host's StoppingAsync, before any hosted service stops — so the engine keeps stepping the rooms while they finish. The host's shutdown timeout is raised to cover drain:timeout + drain:force-grace. Keep your orchestrator's grace period above it:

spec:
  terminationGracePeriodSeconds: 380   # > drain:timeout (360) + force-grace (5)

Endpoints

{
  "nodeId": "game-7f9c-1-3b2e...",
  "state": "Ready",
  "load": 37,
  "maxLoad": 200,
  "free": 163,
  "utilization": 0.185,
  "accepting": true,
  "unlimited": false,
  "kinds": [ { "kind": "rooms", "units": 31, "load": 37 } ]
}

livenessProbe:
  httpGet: { path: /altruist/server/live, port: 8080 }
readinessProbe:
  httpGet: { path: /altruist/server/ready, port: 8080 }
  periodSeconds: 2

Metrics

Standard System.Diagnostics.Metrics instruments; export them with OpenTelemetry, Prometheus or watch them with dotnet-counters:

A Kubernetes HPA can scale on altruist.server.load / altruist.server.max_load (keep a margin of free rooms), and altruist.engine.overruns tells you a server is overloaded before players feel it.

dotnet-counters monitor -n MyGame.Server --counters Altruist.Server,Altruist.Engine

Configuration Reference

altruist:
  server:
    capacity:
      max-load: 0          # budget; 0 = unlimited
      world-load: 0        # load per persistent world
    drain:
      timeout: 30          # seconds the running work may take
      force-grace: 5       # seconds after a forced stop
      on-shutdown: true    # drain when the host stops
      token: ""            # token for POST /altruist/server/drain from outside loopback

Next: run many servers as one — Fleet & Scaling.