From ecc436ab3928ac86dae66a4e48f3d1fa31a9d433 Mon Sep 17 00:00:00 2001 From: Claire Date: Sun, 9 Jul 2023 18:17:37 +0200 Subject: [PATCH] Make it clearer that you should not use read-replicas with sidekiq processes --- content/en/admin/scaling.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/content/en/admin/scaling.md b/content/en/admin/scaling.md index 6ed9f7f3..97c62470 100644 --- a/content/en/admin/scaling.md +++ b/content/en/admin/scaling.md @@ -284,7 +284,11 @@ As far as configuring the Redis database goes, basically you can get rid of back To reduce the load on your Postgresql server, you may wish to setup hot streaming replication (read replica). [See this guide for an example](https://cloud.google.com/community/tutorials/setting-up-postgres-hot-standby). You can make use of the replica in Mastodon in these ways: - The streaming API server does not issue writes at all, so you can connect it straight to the replica. But it’s not querying the database very often anyway so the impact of this is little. -- Use the Makara driver in the web and sidekiq processes, so that writes go to the primary database, while reads go to the replica. Let’s talk about that. +- Use the Makara driver in the web processes, so that writes go to the primary database, while reads go to the replica. Let’s talk about that. + +{{< hint style="warning" >}} +Read replicas are currently not supported for the Sidekiq processes, and using them will lead to failing jobs and data loss. +{{< /hint >}} You will have to edit the `config/database.yml` file and replace the `production` section as follows: @@ -307,5 +311,5 @@ production: Make sure the URLs point to wherever your PostgreSQL servers are. You can add multiple replicas. You could have a locally installed pgBouncer with configuration to connect to two different servers based on database name, e.g. “mastodon” going to the primary, “mastodon_replica” going to the replica, so in the file above both URLs would point to the local pgBouncer with the same user, password, host and port, but different database name. There are many possibilities how this could be setup! For more information on Makara, [see their documentation](https://github.com/taskrabbit/makara#databaseyml). {{< hint style="warning" >}} -Sidekiq cannot reliably use read-replicas because even the tiniest replication lag leads to failing jobs due to queued up records not being found. +Make sure the sidekiq processes run with the stock `config/database.yml` to avoid failing jobs and data loss! {{< /hint >}}