From cd3d3f9fa09b15c520b68f77dccf672d07f7a8ea Mon Sep 17 00:00:00 2001 From: Nolan Lawson Date: Thu, 27 Apr 2017 07:29:44 -0700 Subject: [PATCH] Add PgBouncer guide --- Running-Mastodon/PgBouncer-guide.md | 166 ++++++++++++++++++++++++++++ 1 file changed, 166 insertions(+) create mode 100644 Running-Mastodon/PgBouncer-guide.md diff --git a/Running-Mastodon/PgBouncer-guide.md b/Running-Mastodon/PgBouncer-guide.md new file mode 100644 index 00000000..b85dbbfa --- /dev/null +++ b/Running-Mastodon/PgBouncer-guide.md @@ -0,0 +1,166 @@ +PgBouncer Guide +==== + +The following guide explains how to use [PgBouncer](https://pgbouncer.github.io/) as an efficient database pooler on top of Postgres. For a bit of background, you might read ["Scaling Mastodon"](https://medium.com/@Gargron/scaling-mastodon-1becde463090) which briefly describes this approach. + +Why you might need PgBouncer +---- + +If you start running out of available Postgres connections (the default is 100) then you may find PgBouncer to be a good solution. This document describes some common gotchas as well as good configuration defaults for Mastodon. + +Note that you can check "PgHero" in the administration view to see how many Postgres connections are currently being used. + +Installing PgBouncer +----- + +On Ubuntu: + + sudo apt install pgbouncer + +Restarting: + + sudo service pgbouncer restart + +(Note that this guide assumes you aren't using Docker.) + +Configuring PgBouncer +----- + +### Setting a password + +First off, if your `mastodon` user in Postgres is set up wthout a password, you will need to set a password. There seems to be no way to use PgBouncer with an empty password. + +Here's how you might reset the password: + + psql -p 5432 -U mastodon mastodon_production -w + +Then: + + ALTER USER "mastodon" WITH PASSWORD "password"; + +Then `\q` to quit. + +### Configuring PgBouncer + +PgBouncer has two config files: `pgbouncer.ini` and `userlist.txt` both in `/etc/pgbouncer/`. The first contains the configuration, whereas the second just contains a list of usernames and passwords. + +#### Configuring userlist.txt + +Add the `mastodon` user to the `userlist.txt`: + + "mastodon" "md599dff7ae01ae2dc33ae052264bf22bf4" + +Here we're using the md5 scheme, where the md5 password is just the md5sum of `username + password` with the string `md5` prepended. For instance, to derive the hash for user `mastodon` with password `password`, you can do: + +```bash +# ubuntu, debian, etc. +echo -n "mastodonpassword" | md5sum +# macOS, openBSD, etc. +md5 -s "mastodonpassword" +``` + +Then just add `md5` to the beginning of that. + +You'll also want to create a `pgbouncer` admin user to log in to the admin database. So here's a sample `userlist.txt`: + +``` +"mastodon" "md599dff7ae01ae2dc33ae052264bf22bf4" +"pgbouncer" "md509c950063a6cf1b217ee831d0f4c6771" +``` + +In both cases the password is just `password`. + +#### Configuring pgbouncer.ini + +Add a line under `[databases]` listing the Postgres databases you want to connect to. Here we'll just have Postgres use the same username/password and database name to connect to the underlying Postgres database: + +```ini +[databases] + +mastodon_production = host=127.0.0.1 port=5432 dbname=mastodon_production user=mastodon password=password +``` + +The `listen_addr` and `listen_port` tells PgBouncer which address/port to accept connections. The defaults are fine: + +```ini +listen_addr = 127.0.0.1 +listen_port = 6432 +``` + +Put `md5` as the `auth_type` (assuming you're using the md5 format in `userlist.txt`): + +```ini +auth_type = md5 +``` + +Make sure the `pgbouncer` user is an admin: + +```ini +admin_users = pgbouncer +``` + +**This next part is very important!** The default pooling mode is session-based, but for Mastodon we want transaction-based. In other words, a Postgres connection is created when a transaction is created and dropped when the transaction is done. So you'll want to change the `pool_mode` from `session` to `transaction`: + +```ini +pool_mode = transaction +``` + +Next up, `max_client_conn` defines how many connections PgBouncer itself will accept, and `default_pool_size` puts a limit on how many Postgres connections will be opened under the hood. (In PgHero the number of connections reported will correspond to `default_pool_size` because it has no knowledge of PgBouncer.) + +The defaults are fine to start, and you can always increase them later: + +```ini +max_client_conn = 100 +default_pool_size = 20 +``` + +### Debugging that it all works + +You should be able to connect to PgBouncer just like you would with Postgres: + + psql -p 6432 -U mastodon mastodon_production + +And then use your password to log in. + +### Configuring Mastodon to talk to PgBouncer + +In your `.env.production` file, first off make sure that this is set: + +```bash +PREPARED_STATEMENTS=false +``` + +Since we're using transaction-based pooling, we can't use prepared statements. + +Next up, configure Mastodon to use port 6432 (PgBouncer) instead of 5432 (Postgres) and you should be good to go: + +```bash +DB_HOST=localhost +DB_USER=mastodon +DB_NAME=mastodon_production +DB_PASS=password +DB_PORT=6432 +``` + +### Administering PgBouncer + +The easiest way to reboot is: + + sudo service pgbouncer restart + +But if you've set up a PgBouncer admin user, you can also connect as the admin: + + psql -p 6432 -U pgbouncer pgbouncer + +And then do: + + RELOAD; + +Then use `\q` to quit. + +### Resources + +- ["Scaling Mastodon"](https://medium.com/@Gargron/scaling-mastodon-1becde463090) +- [PgBouncer documentation](https://pgbouncer.github.io/) +- [Connection Pooling in PostgreSQL using pgbouncer](https://www.slideshare.net/sameerkasi200x/5th-pugs-meetupjuly2014pgbouncer) +- [PgBouncer on the Postgres wiki](https://wiki.postgresql.org/wiki/PgBouncer) \ No newline at end of file