This squashed commit makes a fairly significant structural change to significantly reduce Flodgatt's CPU usage.
Flodgatt connects to Redis in a single (green) thread, and then creates a new thread to handle each WebSocket/SSE connection. Previously, each thread was responsible for polling the Redis thread to determine whether it had a message relevant to the connected client. I initially selected this structure both because it was simple and because it minimized memory overhead – no messages are sent to a particular thread unless they are relevant to the client connected to the thread. However, I recently ran some load tests that show this approach to have unacceptable CPU costs when 300+ clients are simultaneously connected.
Accordingly, Flodgatt now uses a different structure: the main Redis thread now announces each incoming message via a watch channel connected to every client thread, and each client thread filters out irrelevant messages. In theory, this could lead to slightly higher memory use, but tests I have run so far have not found a measurable increase. On the other hand, Flodgatt's CPU use is now an order of magnitude lower in tests I've run.
This approach does run a (very slight) risk of dropping messages under extremely heavy load: because a watch channel only stores the most recent message transmitted, if Flodgatt adds a second message before the thread can read the first message, the first message will be overwritten and never transmitted. This seems unlikely to happen in practice, and we can avoid the issue entirely by changing to a broadcast channel when we upgrade to the most recent Tokio version (see #75).
* Tweak release profile & micro optimizations
* Replace std HashMap with hashbrown::HashMap
The hashbrown::HashMap is faster than the std::collections::HashMap,
though it does not protect as well against malicious hash collisions
(e.g., in a DDoS). Since we don't expose the hashing externally,
we should switch to the faster implementation.
* Remove use of last_polled_time [WIP]
This commit stops removing subscriptions based on their last polled
time to test the impact of this change on CPU use. This is a WIP
because it does not yet remove subscriptions in any other way, which
(if deployed in production) would cause a memory leak – memory use
would grow with each new subscription and would never be reduced as
clients end their subscriptions.
* Fix bug with RedisConnection polling freqeuency
* Improve performance of EventStream
This commit changes the EventStream so no longer polls client
WebSocket connections to see if it should clean up the connection.
Instead, it cleans up the connection whenever it attempts to send a
ping or a message through the connection and receives an error
indicating that the client has disconnected. As a result, client
connections aren't cleaned up quite as quickly, but overall sys CPU
time should be dramatically improved.
* Remove empty entries from MsgQueues hashmap
Before this change, entries in the MsgQueue hashmap would remain once
added, even if their value fell to 0. This could lead to a very
slight memory leak/increase, because the hashmap would grow each time
a new user connected and would not decrease again. This is now fixed.
* Bump version and remove unused benchmark
* Initial work to support structured errors
* WIP error handling and RedisConn refactor
* WIP for error handling refactor
* Finish substantive work for Redis error handling
* Apply clippy lints
Daniel Sockwell