Keyed and unkeyed traffic need different selectors.
Keyed traffic often wants affinity. OpenMesh uses rendezvous hashing so the same key tends to select the same peer while minimizing remapping when membership changes. That works well for cache affinity, user/session locality and other stable-key workloads.
Unkeyed traffic defaults to power-of-two choices using available peer health/load state. Sampling two candidates and choosing the better one avoids the blind spots of pure round-robin while staying inexpensive.
Selection happens after traffic policy narrows the eligible subset, so a peer algorithm cannot accidentally escape a canary, metadata or locality rule.
One deadline covers queueing, transport and retries.
A timeout applied only to the socket does not bound total caller latency. A request may spend time waiting for an admission slot and then consume several attempts. OpenMesh treats the deadline as a budget for the complete call path.
If the budget expires in the queue, no network request should be started. If the budget is nearly exhausted after an attempt, a retry should not begin just because its individual socket timeout would fit.
Retry policy is constrained by method semantics.
Retries can turn a transient transport failure into a duplicated side effect. OpenMesh therefore does not implicitly retry unsafe methods such as POST or PATCH. A caller can opt in when it has an idempotency key or an application guarantee that makes replay safe.
Retries stay inside the traffic subset selected for the request. Circuit state is per peer, so a failing instance can be bypassed without treating the whole service as unhealthy.
Overload should create a bounded queue, not infinite work.
A managed service pool can set an in-flight limit and a bounded FIFO queue. Requests beyond those limits fail predictably rather than accumulating unbounded promises and memory. Optional adaptive concurrency can adjust admission from observed success and latency signals.
| Primitive | Purpose |
|---|---|
maxInflight | hard cap on concurrent admitted work |
maxQueue | bound waiting work and memory |
| circuit state | isolate repeatedly failing peers |
| adaptive concurrency | change admission based on observed runtime signals |
| deadline | bound total end-to-end waiting and attempts |
A stream cannot be safely replayed after it is committed.
Buffered requests and long-lived streams have different failure semantics. For SSE, LLM streams and other streaming bodies, OpenMesh commits the response once successful non-5xx headers are handed to the caller. If the body later fails, the error is surfaced to the consumer and is not replayed against another peer.
This explicit no-replay rule prevents duplicated or interleaved stream output. Read the distributed peer documentation for the lower-level options and telemetry fields.