zelo72
/
documentation
mirror of https://github.com/mastodon/documentation
1
0
Fork 0
Code Issues 6 Releases Wiki Activity

Make it clearer that you should not use read-replicas with sidekiq processes (#1255) 

* Make it clearer that you should not use read-replicas with sidekiq processes

* Update scaling.md
This commit is contained in:
Claire 2023-07-09 18:20:23 +02:00 committed by GitHub
parent a61403023a
commit ead55cda6c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 7 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
content/en/admin/scaling.md
View File

@ -284,9 +284,13 @@ As far as configuring the Redis database goes, basically you can get rid of back
To reduce the load on your Postgresql server, you may wish to setup hot streaming replication (read replica). [See this guide for an example](https://cloud.google.com/community/tutorials/setting-up-postgres-hot-standby). You can make use of the replica in Mastodon in these ways:
- The streaming API server does not issue writes at all, so you can connect it straight to the replica. But it’s not querying the database very often anyway so the impact of this is little.
- Use the Makara driver in the web and sidekiq processes, so that writes go to the primary database, while reads go to the replica. Let’s talk about that.
- Use the Makara driver in the web processes, so that writes go to the primary database, while reads go to the replica. Let’s talk about that.
You will have to edit the `config/database.yml` file and replace the `production` section as follows:
{{< hint style="warning" >}}
Read replicas are currently not supported for the Sidekiq processes, and using them will lead to failing jobs and data loss.
{{< /hint >}}
You will have to use a separate `config/database.yml` file for the web processes and edit it to replace the `production` section as follows:
```yaml
production:
@ -307,5 +311,5 @@ production:
Make sure the URLs point to wherever your PostgreSQL servers are. You can add multiple replicas. You could have a locally installed pgBouncer with configuration to connect to two different servers based on database name, e.g. “mastodon” going to the primary, “mastodon_replica” going to the replica, so in the file above both URLs would point to the local pgBouncer with the same user, password, host and port, but different database name. There are many possibilities how this could be setup! For more information on Makara, [see their documentation](https://github.com/taskrabbit/makara#databaseyml).
{{< hint style="warning" >}}
Sidekiq cannot reliably use read-replicas because even the tiniest replication lag leads to failing jobs due to queued up records not being found.
Make sure the sidekiq processes run with the stock `config/database.yml` to avoid failing jobs and data loss!
{{< /hint >}}