User:Tim Starling (WMF)/LoadBalancer connection metric

Take the adjusted connection count $C_{i}$ which is the number of open connections plus the number of connection failures over the last 10 seconds.

Load weights which satisfy $\sum_{i = 1}^{n} w_{i} = 1$

The discrepancy between the desired connection count and the actual connection count is

$Δ C_{i} = C_{i} - w_{i} T$

where $T = \sum_{j = 1}^{n} C_{j}$

A positive number indicates that the server has too many connections. We could normalize by the total connection count:

$w_{i}^{'} = w_{i} - q \frac{Δ C_{i}}{T}$

But that would be noisy when the number of connections is small. We could simply scale it:

$w_{i}^{'} = w_{i} - r Δ C_{i}$

That's defensible and easy to understand. Maybe it would get noisy with high traffic. We could combine the two:

$w_{i}^{'} = w_{i} - \frac{q Δ C_{i}}{E + T}$

or if you like, equivalently

$w_{i}^{'} = w_{i} - \frac{s Δ C_{i}}{1 + \frac{T}{E}}$

where q, r, s and E are tunable constants.

Obviously you then have to clamp it so that $w_{i}^{'} \geq 0$ , and then scale it so that the weights add up to 1 again.

With $S = \frac{1}{s}$ the two tunable constants S and E can be thought of as connection counts. S is the connection count discrepancy which causes the load to be increased or decreased by 100% under low-traffic conditions. E is the total connection count at which the adjustment will be scaled down by half.