Topic on Talk:Core Platform Team/Initiatives/API Gateway

User story 13, Anonymous rate limit, refers to the use of a "global pool". This has been clarified (elsewhere) to mean that requests from all anonymous agents will contribute to a single rate, with the limit to be applied against that rate. I believe this should be reconsidered.

Assuming some limit L and concurrency C (and assuming for the sake of argument that all agents make requests at the same rate), the effective rate becomes L/C. L is constant, but C is not, so the effective per-agent limit is unknowable (without visibility of C). How would we communicate this to users? How do we set expectations?

I believe the more conventional approach would be to apply limits on a per host/IP basis. These limits would be communicated directly to users; Interpretation would be straightforward.

If the concern is that for any given limit, a sufficiently large concurrency will push the system over capacity, then limiting concurrency is probably the appropriate response (and a sufficiently large concurrency could take us down anyway, regardless of rate).

Finally, since (to the best of my knowledge) this requirement isn't about implementing certain semantics for the sake of the product (the API), but about safe guarding the infrastructure that hosts it, we should really wait for SRE to weigh in. I believe they already have ideas (plans?) with regards to this; It may not even be necessary for us to rate-limit anonymous users (if for example this is occurring upstream of the gateway).

What I'm hearing is that instead of a global pool of API calls for anonymous users, we'd have a rate limit for anonymous users based on their IP address?

As long as we still have all the requirements for rate limits around API keys, and that the per-IP limit wouldn't apply to OAuth-authorised requests, I'm fine with that.

One thing I want to make sure I understand w/r/t capacity planning.

I think if we have a fixed capacity X, and a fixed anonymous pool Y, and a per-client pool Z, and a known number of registered clients C, then as long as X > Y + (C * Z), we are under capacity.

If we have a per-IP limit B instead of a fixed anonymous pool, and A possible IP addresses, we can only be sure we're under capacity if X > (A * B) + (C * Z). A is really big, here. We could probably use D as "number of IP addresses we actually expect to connect", but that's still big and fuzzy.

I'm OK living with that ambiguity, but it seems to me that the fixed anonymous pool is easier to do capacity planning for, rather than the per-IP limits.

Another question is whether we would have limits for IPv6 addresses, too.

Yes; I cannot see any reason to treat IPv4 & IPv6 any differently.

Also, I think having concurrency limits is an OK idea. There are some for public APIs that I've seen; usually something like N connections per [IP, client ID] pair where N is small, like 1 or 2 or 3.

So if an end user is using two different apps, it's OK to use two connections. And if two different users are using the same app on two different machines, that's OK, too.