* Read user and domain blocks from Postgres
This commit reads the blocks from pg and stores them in the User
struct; it does not yet actually filter the responses. It also does
not update the tests.
* Update tests
* Filter out toots involving blocked/muted users
* Add support for domain blocks
* Update test and bump version
* Add type-safe wrapper types to deployement_cfg
* Before deleting redundnat macros
* Store error messages as data
* Significant progress on type safety
* Add type safety to RedisConfig
* Refactor configuration
* Fix bug with incorrect Host env variable
* Improve logging of REDIS_NAMESPACE
* Update test for Postgres configuration
* Conform Redis config to Postgres changes
* Refactor organazation of SSE
This commit refactors how SSE requests are handled to bring them into
line with how WS requests are handled and increase consistency.
* Add websocket tests
* Bump version to 0.2.0
Bump version and update name from ragequit to flodgatt.
* Add test for non-existant endpoints
* Update documentation for recent changes``
* Upgrade postgres dependency to support ssl
* Clean up configuration code
* Add support for SSL with postgres [WIP]
* Add unit tests with mock Postgres
Previously, the access token needed to be passed via the query string;
with this commit, the token can be passed *either* through the query
string or the Sec-WebSocket-Protocol header.
This was done to correspond to the changes made to the streaming.js
version in [Improve streaming server security](https://github.com/tootsuite/mastodon/pull/10818).
However, I am not sure that it *does* increase security; as explained
at <https://support.ably.io/support/solutions/articles/3000075120-is-it-secure-to-send-the-access-token-as-part-of-the-websocket-url-query-params->,
there is generally no security advantage to passing sensitive information
via websocket headers instead of the query string—the entire connection
is encrypted and is not stored in the browser history, so the typical
reasons to keep sensitive info out of the query string don't apply.
I would welcome any corrections on this/reasons this change improves
security.
This commit revises the code structure to share a single connection
to Redis (with multiple subscriptions on that connection) rather than
mutiple connections (each with one subscription). It also simplifies the code based on that change.
This commit implements a shared stream of data from Redis, which
allows all SSE connections that send the same data to the client
to share a single connection to Redis. (Previously, each client
got their own connection, which would significantly increase the
number of open Redis connections—especially since nearly all clients
will subscribe to `/public`.)
This commit tracks the number of active Pub/Sub streams and adds code to
keep the total number of streams below 400. (When additional users
attempt to connect past that point, the server will wait for an slot
to open up). This prevents "too many open file" panics and makes the
server better behaved in general. However, we may need to revisit it
based on what capacity we want the server to have.
This commit also includes some general refactoring.
This commit tracks the existence of the SSE stream and closes the
connection to the redis pub/sub channel when the stream is closed. This
prevents the number of redis connections from growing over time.
Note, however, that the current code still subscribes to one redis
channel per SSE connection rather than reusing existing subscriptions.
This will need to be fixed in a later PR.
This commit rolls up several days of work that took place in a side repo
which was originally intended to be a test/proof-of-concept repo but
that grew into a full implementation.
This implementation switches to Warp as the framework for the primary
web server (in large part because Warp offers built-in support for
SSE and several other advantages). This implementation relies on
a custom Redis interface built on Tokio (because the current Redis
Rust crate does not support asnyc PubSub). Using a custom interface
should also be faster, since it does not contain logic for anything
other than pubsub—which is all we need.
Finally, I note that the SSE support is currently as feature-complete
as it is possible to be without having yet added the Postgress interface
this means that all of the endpoints are present and are accessible
on localhost. However, none of the endpoints have authentication yet,
and the endpoints that should provide user-specific data (e.g.,
`/api/v1/streaming/user`) are currently hard-coded to provide data for
user 1 (the admin). Other than those limitations, however, the SSE
implementation is feature complete.
Daniel Sockwell