Update content for 4.0 (part 1) (#991)

* add rules

* join date on profiles

* deprecate follow scope

* deprecate identity proofs

* familiar followers

* use definition lists instead of tables for defining activitypub properties

* reformat notifications page into markdown

* fix broken links to publicKey header

* Application website is now nullable

* update environment variables added and removed

* fix typo

* fix heading level

* min_id and max_id can be used at the same time (3.3)

* fix typo

* new tootctl options

* reformat tootctl page to use definition lists for params

* add rules and configuration to Instance

* fix typo

* refactor instance api page

* 3.3.0 duration on mutes

* 3.3.0 mute_expires_at

* improve section headings

* 3.4.0 resend email confirmation api

* 3.4.0 policy on push subscriptions

* 3.4.0 add details to account registration error

* refactor accounts api page and start adding relrefs to entity pages

* 3.4.0 accounts/lookup api

* add see also to accounts methods

* add more see-also links

* 3.5.0 appeal mod decisions

* 3.5.0 reformat reports and add category/rule_ids params

* document report entity and missing responses

* fix typos

* fix relrefs and url schema, add aliases to old urls

* add archetypes for new methods/entities

* update archetypes with see-also stubs

* clearer presentation of rate limits

* announcements api methods

* refactor apps methods

* refactor bookmarks methods + some anchors

* refactor conversations methods

* custom_emojis methods refactor

* anchors

* refactor directory methods

* refactor domain_blocks methods

* add see also to emails methods

* fix page relref shortcodes to specific methods + refactor endorsements methods

* min_id max_id

* refactor favourites methods

* refactor featured_tags methods

* refactor filters methods, make path params consistent, i18n required shortcode

* follow_requests methods

* lists methods

* markers methods

* forgot to add entity links

* media methods, also fix formatting of some json errors

* mutes methods, add more see-also links

* oembed methods

* preferences methods

* proofs methods

* push methods

* suggestions methods

* 3.5.0 add new trend types, fix formatting

* refactor streaming methods

* refactor oauth methods

* note that streaming api casts payload to string

* refactor search methods

* refactor polls methods

* remove unnecessary link

* reformat scheduled_statuses methods

* reformat timelines methods

* reformat statuses methods

* 3.5.0 editing statuses

* consistent use of array brackets in form data parameters

* update dev setup guide, add vagrant and clean up text

* add admin/accounts methods

* 3.6 role entity

* admin/accounts methods v2

* minor fix

* stub admin/reports methods

* document admin reports

* add 403 example to methods archetype

* cleanup entities for admin reports and add new attrs

* 3.6.0 domain allows methods + normalize admin entity namespace

* fix search-and-replace error

* add aliases for admin entities

* 3.6.0 canonical email blocks entity

* 3.5.0 admin/retention api

* 3.5.0 add admin::ip doc

* 3.5.0 admin/reports

* 3.6.0 admin/domain_allows

* 3.5.0 admin/dimensions

* 3.6.0 permissions and roles

* minor formatting fix

* add anchor link to headings

* checkpoint

* add update commands to dev env setup guide

* change mentions of v3.6 to v4.0

* tootctl now uses custom roles

* fix formatting

* v2 instance api

* update frontmatter, add better titles to pages

* minor wording change

* consistency

* add more aliases

* add placeholders and WIP notices

* explain link pagination and stub out todos

* switch baseURL to https

* 422 on reports with rules but category!=violation

* document bug fixes

* fix typo

* remove duplicate API method definition

* s/tootsuite/mastodon for github links

* remove unnecessary escaping

* s/tootsuite/mastodon in Entity archetype

* add missing nullable shortcode

* clarify oauth scope when requesting a user token

* api/v2/media now synchronous for images

* DISALLOW_UNAUTHENTICATED_API_ACCESS

* add undocumented env variables

* add instance domain blocks and extended description api

* add SMTP_ENABLE_STARTTLS

* add description to SMTP_ENABLE_STARTTLS

* take suggestions from open PRs

* normalize links and flavour language

* Fully document streaming API based on source code

* Add mention of MIME types

* bump to ruby 3.0.4

* clarify how to check on async media processing

* validation of replies_policy

* remove TODOs on admin account action

* EmailDomainBlocks

* IpBlocks

* Admin::DomainBlock

* remove TODOs

* following hashtags

* followed_tags

* remove reference to unused parameter

* add new oauth scopes for admin blocks and allows

* fix command signature for i18n-tasks normalize

* reformat code structure page

* document fixes for following tags (assume 4.0.3)

* Add warning about pre-4.0 hardcoded roles

* add note about case sensitivity

* remove use of 'simply' from docs

* remove reference to silencing

* add reference to IDN normalization for verified links

* add lang parameter
pull/1036/head^2
trwnh 7 months ago committed by GitHub
parent 69e9017730
commit 3628b6d434
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

1
.gitignore vendored

@ -1,3 +1,4 @@
public/
static/.sass-cache
resources/
.hugo_build.lock

@ -0,0 +1,27 @@
---
title: "{{ replace .Name "-" " " | title }}"
description:
menu:
docs:
parent: entities
---
## Example
```json
```
## Attributes
### `id` {#id}
**Description:** The ID of the SOMETHING in the database.\
**Type:** String (cast from an integer, but not guaranteed to be a number)\
**Version history:**\
x.x.x - added
## See also
{{< page-relref ref="methods/SOMETHING#anchor" caption="POST /api/v1/something/" >}}
{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/app/serializers/rest/SOMETHING_serializer.rb" caption="app/serializers/rest/SOMETHING_serializer.rb" >}}

@ -0,0 +1,123 @@
---
title: "{{ replace .Name "-" " " | title }}"
description:
menu:
docs:
parent: methods
---
## What the method does {#anchor}
```http
GET https://mastodon.example/api/v1/example HTTP/1.1
```
**Returns:** [SOMETHING]({{< relref "entities/SOMETHING" >}})\
**OAuth:** User token + `oauth:scope`\
**Permissions:** Manage ???\
**Version history:**\
x.x.x - added
#### Request
##### Path parameters
:id
: {{<required>}} String. The ID of the SOMETHING in the database.
##### Headers
Authorization
: {{<required>}} Provide this header with `Bearer <user token>` to gain authorized access to this API method.
##### Query parameters
**Internal parameter.** Use HTTP `Link` header for pagination.
max_id
: String. Return results older than ID.
since_id
: String. Return results newer than ID.
min_id
: String. Return results immediately newer than ID.
limit
: Integer. Maximum number of results to return. Defaults to 20. Max 40.
##### Form data parameters
#### Response
##### 200: OK
```json
```
Because SOMETHING IDs are generally not exposed via any API responses, you will have to parse the HTTP `Link` header to load older or newer results. See [Paginating through API responses]({{<relref "api/guidelines#pagination">}}) for more information.
```http
Link:
```
##### 401: Unauthorized
Invalid or missing Authorization header.
```json
{
"error": "The access token is invalid"
}
```
##### 403: Forbidden
Authorized user is missing a permission, or invalid or missing Authorization header
```json
{
"error": "This action is not allowed"
}
```
##### 404: Not found
SOMETHING is not owned by you or does not exist
```json
{
"error": "Record not found"
}
```
##### 410: Gone
##### 422: Unprocessable entity
##### 429: Rate limited
```json
{
"error": "Too many requests"
}
```
##### 503: Service unavailable
```json
{
"error": "There was a temporary problem serving your request, please try again"
}
```
---
## See also
{{< page-relref ref="methods/SOMETHING#anchor" caption="POST /api/v1/something/" >}}
{{< page-ref page="client/authorized" >}}
{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/app/controllers/api/v1/SOMETHING_controller.rb" caption="app/controllers/api/v1/SOMETHING_controller.rb" >}}
{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/app/controllers/api/v1/SOMETHING" caption="app/controllers/api/v1/SOMETHING/" >}}

@ -445,6 +445,26 @@ main {
padding: 0;
}
dl {
color: $lightest;
font-size: 16px;
line-height: 28px;
font-weight: normal;
margin-bottom: 26px;
hyphens: auto;
dt {
font-weight: 700;
background: lighten($darkest, 8%);
padding: 2px 16px;
max-width: max-content;
margin-bottom: 4px;
}
dd {
margin-left: 16px;
margin-bottom: 16px;
}
}
.mastodon-embed,
iframe {
width: 100%;
@ -810,3 +830,14 @@ main {
padding: 20px;
}
}
.heading {
display: flex;
align-items: center;
&__anchor-link {
display: inline-flex;
align-content: center;
margin-left: 0.25em;
}
}

@ -1,4 +1,4 @@
baseURL = "http://docs.joinmastodon.org/"
baseURL = "https://docs.joinmastodon.org/"
languageCode = "en-us"
defaultContentLanguage = "en"
title = "Mastodon documentation"
@ -7,12 +7,15 @@ pygmentsStyle = "dracula"
metaDataFormat = "yaml"
paginate = 100
enableGitInfo = true
disablePathToLower = true
[markup]
[markup.tableOfContents]
endLevel = 2
endLevel = 3
ordered = false
startLevel = 1
[markup.goldmark.renderer]
unsafe = true
[languages.en.menu]
[[languages.en.menu.docs]]

@ -30,7 +30,7 @@ A Mastodon website can operate alone. Just like a traditional website, people si
Mastodon uses a standardized, open protocol to implement federation. It is called **ActivityPub**. Any software that likewise implements federation via ActivityPub can seamlessly communicate with Mastodon, just like Mastodon websites communicate with one another.
The **fediverse** \(“federated universe”\) is the name for all websites that can communicate with each other over ActivityPub and the World Wide Web. That includes all Mastodon servers, but also other implementations:
The **fediverse** (“federated universe”) is the name for all websites that can communicate with each other over ActivityPub and the World Wide Web. That includes all Mastodon servers, but also other implementations:
* Pleroma, a modular microblogging engine,
* Pixelfed, federated image sharing platform, which lets you share and consume media posts,
@ -45,13 +45,13 @@ The fediverse does not have its own brand, so you might more often hear “follo
### Choice of service provider and policy {#choice}
Because Mastodon is simply software that can be used to power any website, potential users of Mastodon have the option of choosing a service provider from already-existing Mastodon websites, or to create their own Mastodon website if they wish. The Mastodon project maintains a list of recommended service providers at [joinmastodon.org](https://joinmastodon.org), sortable by category and/or language. Some websites may have moderation policies that go beyond this, such as requiring the use of certain tags on potentially sensitive content, and some websites may have more relaxed moderation policies, but websites listed in the picker all agree to adopt the [Mastodon Server Covenant](https://joinmastodon.org/covenant), meaning that they pledge to actively moderate against hate speech, to take daily backups, to have at least one emergency admin, and to provide at least 3 months advance notice in case of shutdown.
Because Mastodon is just software that can be used to power any website, potential users of Mastodon have the option of choosing a service provider from already-existing Mastodon websites, or to create their own Mastodon website if they wish. The Mastodon project maintains a list of recommended service providers at [joinmastodon.org](https://joinmastodon.org), sortable by category and/or language. Some websites may have moderation policies that go beyond this, such as requiring the use of certain tags on potentially sensitive content, and some websites may have more relaxed moderation policies, but websites listed in the picker all agree to adopt the [Mastodon Server Covenant](https://joinmastodon.org/covenant), meaning that they pledge to actively moderate against hate speech, to take daily backups, to have at least one emergency admin, and to provide at least 3 months advance notice in case of shutdown.
> Maintaining communities that feel safe for all of its members is not easy. Mastodon provides a lot of foundational framework and tools for doing it, and shifts the power to effect change from one commercial entity to the communities themselves.
>
> -- Eugen Rochko, Jul 6 2018, ["Cage the Mastodon"](https://blog.joinmastodon.org/2018/07/cage-the-mastodon/)
> A centralized social media platform has a hierarchical structure where rules and their enforcement, as well as the development and direction of the platform, are decided by the CEO \[...\] A decentralized network deliberately relinquishes control of the platform owner, by essentially not having one.
> A centralized social media platform has a hierarchical structure where rules and their enforcement, as well as the development and direction of the platform, are decided by the CEO [...] A decentralized network deliberately relinquishes control of the platform owner, by essentially not having one.
>
> -- Eugen Rochko, Dec 30 2018, ["Why does decentralization matter?"](https://blog.joinmastodon.org/2018/12/why-does-decentralization-matter/)
@ -91,19 +91,19 @@ Unlike proprietary services, **anyone has the complete freedom to run, examine,
Learn how to use Mastodon:
{{< page-ref page="user/signup.md" >}}
{{< page-ref page="user/signup" >}}
Learn how to install Mastodon:
{{< page-ref page="admin/prerequisites.md" >}}
{{< page-ref page="admin/prerequisites" >}}
Learn how to write an app for Mastodon:
{{< page-ref page="client/intro.md" >}}
{{< page-ref page="client/intro" >}}
Learn about the Mastodon backend and how to contribute:
{{< page-ref page="dev/overview.md" >}}
{{< page-ref page="dev/overview" >}}

@ -13,7 +13,7 @@ For convenience, it can read them from a flat file called `.env.production` in t
## Basic {#basic}
### Federation {#federation}
### Federation and display {#federation}
#### `LOCAL_DOMAIN`
@ -37,13 +37,17 @@ location /.well-known/webfinger {
If you have multiple domains pointed at your Mastodon server, this setting will allow Mastodon to recognize itself when users are addressed using those other domains. Separate the domains by commas, e.g. `foo.com,bar.com`
#### `AUTHORIZED_FETCH` {#authorized_fetch}
#### `ALLOWED_PRIVATE_ADDRESSES`
Comma-separated specific addresses/subnets allowed in outgoing HTTP queries.
#### `AUTHORIZED_FETCH`
Also called "secure mode". When set to `true`, the following changes occur:
- Mastodon will stop generating linked-data signatures for public posts, which prevents them from being re-distributed efficiently but without precise control. Since a linked-data object with a signature is entirely self-contained, it can be passed around without making extra requests to the server where it originates.
- Mastodon will require HTTP signature authentication on ActivityPub representations of public posts and profiles, which are normally available without any authentication. Profiles will only return barebones technical information when no authentication is supplied.
- Mastodon will require any REST/streaming API access to have a user context (i.e. having gone through an OAuth authorization screen with an active user), when normally some API endpoints are available without any authentication.
- Prior to v4.0.0: Mastodon will require any REST/streaming API access to have a user context (i.e. having gone through an OAuth authorization screen with an active user), when normally some API endpoints are available without any authentication.
As a result, through the authentication mechanism and avoiding re-distribution mechanisms that do not have your server in the loop, it becomes possible to enforce who can and cannot retrieve even public content from your server, e.g. servers whose domains you have blocked.
@ -55,7 +59,7 @@ Unfortunately, secure mode is not without its drawbacks, which is why it is not
Secure mode does not hide HTML representations of public posts and profiles. HTML is a more lossy format compared to first-class ActivityPub representations or the REST API but it is still a potential vector for scraping content.
{{</ hint >}}
#### `LIMITED_FEDERATION_MODE` {#limited_federation_mode}
#### `LIMITED_FEDERATION_MODE`
When set to `true`, Mastodon will restrict federation to servers you have manually approved only, as well as disable all public pages and some REST APIs. Limited federation mode is based on secure mode (`AUTHORIZED_FETCH`).
@ -73,6 +77,92 @@ This mode is intended for private use only, such as in academic instituations or
This setting was known as `WHITELIST_MODE` prior to 3.1.5.
{{</ hint >}}
#### `DISALLOW_UNAUTHENTICATED_API_ACCESS`
As of Mastodon v4.0.0, the web app is now used to render all requests, even for logged-out viewers. In order to make these views work, the web app makes public API requests in order to fetch accounts and statuses. If you would like to disallow this, then set this variable to `true`. Note that disallowing unauthenticated API access will cause profile and post permalinks to return an error to logged-out users, essentially making it so that the only ways to view content is to either log in locally or fetch it via ActivityPub.
#### `SINGLE_USER_MODE`
If set to `true`, the frontpage of your Mastodon server will always redirect to the first profile in the database and registrations will be disabled.
#### `DEFAULT_LOCALE`
By default, Mastodon will automatically detect the visitor's language from browser headers and display the Mastodon interface in that language (if it's supported). If you are running a language-specific or regional server, that behaviour may mislead visitors who do not speak your language into signing up on your server. For this reason, you may want to set this variable to a specific language.
Example value: `de`
Supported languages:
- `ar`
- `ast`
- `bg`
- `bn`
- `br`
- `ca`
- `co`
- `cs`
- `cy`
- `da`
- `de`
- `el`
- `en`
- `eo`
- `es`
- `es-AR`
- `et`
- `eu`
- `fa`
- `fi`
- `fr`
- `ga`
- `gl`
- `he`
- `hi`
- `hr`
- `hu`
- `hy`
- `id`
- `io`
- `is`
- `it`
- `ja`
- `ka`
- `kab`
- `kk`
- `kn`
- `ko`
- `lt`
- `lv`
- `mk`
- `ml`
- `mr`
- `ms`
- `nl`
- `nn`
- `no`
- `oc`
- `pl`
- `pt-BR`
- `pt-PT`
- `ro`
- `ru`
- `sk`
- `sl`
- `sq`
- `sr`
- `sr-Latn`
- `sv`
- `ta`
- `te`
- `th`
- `tr`
- `uk`
- `ur`
- `vi`
- `zh-CN`
- `zh-HK`
- `zh-TW`
### Secrets {#secrets}
#### `SECRET_KEY_BASE`
@ -147,7 +237,7 @@ This variable cannot be defined in dotenv (`.env`) files as it's used before the
### Scaling options {#scaling}
{{< page-ref page="admin/scaling.md" >}}
{{< page-ref page="admin/scaling" >}}
#### `WEB_CONCURRENCY`
@ -157,6 +247,10 @@ Specific to Puma, this variable determines how many different processes Puma for
Specific to Puma, this variable determines how many threads each Puma process maintains. Defaults to `5`.
#### `PERSISTENT_TIMEOUT`
Specific to Puma, this variable determines how long Puma should wait before closing a connection. Defaults to `20`.
#### `PREPARED_STATEMENTS`
By default, Mastodon uses the prepared statements feature of PostgreSQL, which offers some performance advantages. This feature is not available if you are using a connection pool where connections are shared between transactions and must thus be set to `false`. When you are scaling up, the advantages of having a transaction-based connection pool outweigh those provided by prepared statements.
@ -171,7 +265,7 @@ Example value: `wss://streaming.example.com`
Specific to the streaming API, this variable determines how many different processes the streaming API forks into. Defaults to the number of CPU cores minus one.
## Database connections {#connections}
## Backend {#backend}
### PostgreSQL {#postgresql}
@ -249,9 +343,11 @@ If provided, takes precedence over `CACHE_REDIS_HOST` and `CACHE_REDIS_PORT`. De
Defaults to value of `REDIS_NAMESPACE`.
#### `SIDEKIQ_REDIS_URL`
### ElasticSearch {#elasticsearch}
{{< page-ref page="admin/optional/elasticsearch.md" >}}
{{< page-ref page="admin/optional/elasticsearch" >}}
#### `ES_ENABLED`
@ -265,6 +361,14 @@ Host of the ElasticSearch server. Defaults to `localhost`
Port of the ElasticSearch server. Defaults to `9200`
#### `ES_USER`
Used for optionally authenticating with ElasticSearch
#### `ES_PASS`
Used for optionally authenticating with ElasticSearch
#### `ES_PREFIX`
Useful if the ElasticSearch server is shared between multiple projects or different Mastodon servers. Defaults to value of `REDIS_NAMESPACE`.
@ -281,137 +385,32 @@ Example value: `localhost:8125`
If set, all StatsD keys will be prefixed with this. Defaults to `Mastodon.production` when `RAILS_ENV` is `production`, `Mastodon.development` when it's `development`, etc.
## Limits {#limits}
### SMTP email delivery {#smtp}
#### `SINGLE_USER_MODE`
#### `SMTP_SERVER`
#### `SMTP_PORT`
#### `SMTP_LOGIN`
#### `SMTP_PASSWORD`
#### `SMTP_FROM_ADDRESS`
#### `SMTP_DOMAIN`
#### `SMTP_DELIVERY_METHOD`
#### `SMTP_AUTH_METHOD`
#### `SMTP_CA_FILE`
#### `SMTP_OPENSSL_VERIFY_MODE`
#### `SMTP_ENABLE_STARTTLS_AUTO`
#### `SMTP_ENABLE_STARTTLS`
If set to `true`, the frontpage of your Mastodon server will always redirect to the first profile in the database and registrations will be disabled.
Set to `auto` (default), `always`, or `never`.
#### `EMAIL_DOMAIN_ALLOWLIST`
**Version history:**\
4.0.0 - added
If set, registrations will not be possible with any e-mails **except** those from the specified domains. Pipe-separated values, e.g.: `foo.com|bar.com`
#### `EMAIL_DOMAIN_DENYLIST`
If set, registrations will not be possible with any e-mails from the specified domains. Pipe-separated values, e.g.: `foo.com|bar.com`
{{< hint style="warning" >}}
This option is deprecated. You can dynamically block e-mail domains from the admin interface or from the `tootctl` command-line interface.
{{</ hint >}}
#### `DEFAULT_LOCALE`
By default, Mastodon will automatically detect the visitor's language from browser headers and display the Mastodon interface in that language (if it's supported). If you are running a language-specific or regional server, that behaviour may mislead visitors who do not speak your language into signing up on your server. For this reason, you may want to set this variable to a specific language.
#### `SMTP_TLS`
#### `SMTP_SSL`
Example value: `de`
## File storage {#files}
Supported languages:
- `ar`
- `ast`
- `bg`
- `bn`
- `br`
- `ca`
- `co`
- `cs`
- `cy`
- `da`
- `de`
- `el`
- `en`
- `eo`
- `es`
- `es-AR`
- `et`
- `eu`
- `fa`
- `fi`
- `fr`
- `ga`
- `gl`
- `he`
- `hi`
- `hr`
- `hu`
- `hy`
- `id`
- `io`
- `is`
- `it`
- `ja`
- `ka`
- `kab`
- `kk`
- `kn`
- `ko`
- `lt`
- `lv`
- `mk`
- `ml`
- `mr`
- `ms`
- `nl`
- `nn`
- `no`
- `oc`
- `pl`
- `pt-BR`
- `pt-PT`
- `ro`
- `ru`
- `sk`
- `sl`
- `sq`
- `sr`
- `sr-Latn`
- `sv`
- `ta`
- `te`
- `th`
- `tr`
- `uk`
- `ur`
- `vi`
- `zh-CN`
- `zh-HK`
- `zh-TW`
#### `MAX_SESSION_ACTIVATIONS`
How many browser sessions are allowed per-user. Defaults to `10`. If a new browser session is created, then the oldest session is deleted, e.g. user in that browser is logged out.
#### `USER_ACTIVE_DAYS`
Mastodon stores home feeds in RAM (specifically, in the Redis database). This makes them very fast to access and update, but it also means that you don't want to keep them there if they're not used, and you don't want to spend resources on inserting new items into home feeds that will not be accessed. For this reason, Mastodon periodically clears out home feeds of users who haven't been online in a while, and if they re-appear, it regenerates those home feeds from database data. By default, users are considered active if they have been online in the past `7` days.
Regeneration of home feeds is computationally expensive, if your Sidekiq is constantly doing it because your users come online every 3 days but your `USER_ACTIVE_DAYS` is set to 2, then consider adjusting it up.
{{< hint style="info" >}}
This setting has no relation to which users are considered active for the purposes of statistics, such as the Monthly Active Users number.
{{</ hint >}}
#### `ALLOWED_PRIVATE_ADDRESSES`
Comma-separated specific addresses/subnets allowed in outgoing HTTP queries.
## E-mail {#email}
* `SMTP_SERVER`
* `SMTP_PORT`
* `SMTP_LOGIN`
* `SMTP_PASSWORD`
* `SMTP_FROM_ADDRESS`
* `SMTP_DOMAIN`
* `SMTP_DELIVERY_METHOD`
* `SMTP_AUTH_METHOD`
* `SMTP_CA_FILE`
* `SMTP_OPENSSL_VERIFY_MODE`
* `SMTP_ENABLE_STARTTLS_AUTO`
* `SMTP_TLS`
* `SMTP_SSL`
## File storage {#cdn}
### CDN {cdn}
#### `CDN_HOST`
@ -434,7 +433,7 @@ It is *extremely recommended* to use your own host instead, for a few reasons:
Example value: `files.example.com`
{{< page-ref page="admin/optional/object-storage-proxy.md" >}}
{{< page-ref page="admin/optional/object-storage-proxy" >}}
{{< hint style="info" >}}
You must serve the files with CORS headers, otherwise some functions of Mastodon's web UI will not work. For example, `Access-Control-Allow-Origin: *`
@ -442,119 +441,218 @@ You must serve the files with CORS headers, otherwise some functions of Mastodon
### Local file storage {#paperclip}
* `PAPERCLIP_ROOT_PATH`
* `PAPERCLIP_ROOT_URL`
#### `PAPERCLIP_ROOT_PATH`
#### `PAPERCLIP_ROOT_URL`
### Amazon S3 and compatible {#s3}
* `S3_ENABLED`
* `S3_BUCKET`
* `AWS_ACCESS_KEY_ID`
* `AWS_SECRET_ACCESS_KEY`
* `S3_REGION`
* `S3_PROTOCOL`
* `S3_HOSTNAME`
* `S3_ENDPOINT`
* `S3_SIGNATURE_VERSION`
* `S3_OVERRIDE_PATH_STYLE`
* `S3_OPEN_TIMEOUT`
* `S3_READ_TIMEOUT`
#### `S3_ENABLED`
#### `S3_BUCKET`
#### `AWS_ACCESS_KEY_ID`
#### `AWS_SECRET_ACCESS_KEY`
#### `S3_REGION`
#### `S3_PROTOCOL`
#### `S3_HOSTNAME`
#### `S3_ENDPOINT`
#### `S3_SIGNATURE_VERSION`
#### `S3_OVERRIDE_PATH_STYLE`
#### `S3_OPEN_TIMEOUT`
#### `S3_READ_TIMEOUT`
#### `S3_FORCE_SINGLE_REQUEST`
### Swift {#swift}
* `SWIFT_ENABLED`
* `SWIFT_USERNAME`
* `SWIFT_TENANT`
* `SWIFT_PASSWORD`
* `SWIFT_PROJECT_ID`
* `SWIFT_AUTH_URL`
* `SWIFT_CONTAINER`
* `SWIFT_OBJECT_URL`
* `SWIFT_REGION`
* `SWIFT_DOMAIN_NAME`
* `SWIFT_CACHE_TTL`
#### `SWIFT_ENABLED`
#### `SWIFT_USERNAME`
#### `SWIFT_TENANT`
#### `SWIFT_PASSWORD`
#### `SWIFT_PROJECT_ID`
#### `SWIFT_AUTH_URL`
#### `SWIFT_CONTAINER`
#### `SWIFT_OBJECT_URL`
#### `SWIFT_REGION`
#### `SWIFT_DOMAIN_NAME`
#### `SWIFT_CACHE_TTL`
## External authentication {#external-authentication}
* `OAUTH_REDIRECT_AT_SIGN_IN`
### OmniAuth
#### `OMNIAUTH_ONLY`
### LDAP {#ldap}
* `LDAP_ENABLED`
* `LDAP_HOST`
* `LDAP_PORT`
* `LDAP_METHOD`
* `LDAP_BASE`
* `LDAP_BIND_DN`
* `LDAP_PASSWORD`
* `LDAP_UID`
* `LDAP_SEARCH_FILTER`
* `LDAP_MAIL`
* `LDAP_UID_CONVERSTION_ENABLED`
#### `LDAP_ENABLED`
#### `LDAP_HOST`
#### `LDAP_PORT`
#### `LDAP_METHOD`
#### `LDAP_BASE`
#### `LDAP_BIND_DN`
#### `LDAP_PASSWORD`
#### `LDAP_UID`
#### `LDAP_SEARCH_FILTER`
#### `LDAP_MAIL`
#### `LDAP_UID_CONVERSTION_ENABLED`
### PAM {#pam}
* `PAM_ENABLED`
* `PAM_EMAIL_DOMAIN`
* `PAM_DEFAULT_SERVICE`
* `PAM_CONTROLLED_SERVICE`
#### `PAM_ENABLED`
#### `PAM_EMAIL_DOMAIN`
#### `PAM_DEFAULT_SERVICE`
#### `PAM_CONTROLLED_SERVICE`
### CAS {#cas}
* `CAS_ENABLED`
* `CAS_URL`
* `CAS_HOST`
* `CAS_PORT`
* `CAS_SSL`
* `CAS_VALIDATE_URL`
* `CAS_CALLBACK_URL`
* `CAS_LOGOUT_URL`
* `CAS_LOGIN_URL`
* `CAS_UID_FIELD`
* `CAS_CA_PATH`
* `CAS_DISABLE_SSL_VERIFICATION`
* `CAS_UID_KEY`
* `CAS_NAME_KEY`
* `CAS_EMAIL_KEY`
* `CAS_NICKNAME_KEY`
* `CAS_FIRST_NAME_KEY`
* `CAS_LAST_NAME_KEY`
* `CAS_LOCATION_KEY`
* `CAS_IMAGE_KEY`
* `CAS_PHONE_KEY`
#### `CAS_ENABLED`
#### `CAS_DISPLAY_NAME`
#### `CAS_URL`
#### `CAS_HOST`
#### `CAS_PORT`
#### `CAS_SSL`
#### `CAS_VALIDATE_URL`
#### `CAS_CALLBACK_URL`
#### `CAS_LOGOUT_URL`
#### `CAS_LOGIN_URL`
#### `CAS_UID_FIELD`
#### `CAS_CA_PATH`
#### `CAS_DISABLE_SSL_VERIFICATION`
#### `CAS_UID_KEY`
#### `CAS_NAME_KEY`
#### `CAS_EMAIL_KEY`
#### `CAS_NICKNAME_KEY`
#### `CAS_FIRST_NAME_KEY`
#### `CAS_LAST_NAME_KEY`
#### `CAS_LOCATION_KEY`
#### `CAS_IMAGE_KEY`
#### `CAS_PHONE_KEY`
#### `CAS_SECURITY_ASSUME_EMAIL_IS_VERIFIED`
### SAML {#saml}
* `SAML_ENABLED`
* `SAML_ACS_URL`
* `SAML_ISSUER`
* `SAML_IDP_SSO_TARGET_URL`
* `SAML_IDP_CERT`
* `SAML_IDP_CERT_FINGERPRINT`
* `SAML_NAME_IDENTIFIER_FORMAT`
* `SAML_CERT`
* `SAML_PRIVATE_KEY`
* `SAML_SECURITY_WANT_ASSERTION_SIGNED`
* `SAML_SECURITY_WANT_ASSERTION_ENCRYPTED`
* `SAML_SECURITY_ASSUME_EMAIL_IS_VERIFIED`
* `SAML_ATTRIBUTES_STATEMENTS_UID`
* `SAML_ATTRIBUTES_STATEMENTS_EMAIL`
* `SAML_ATTRIBUTES_STATEMENTS_FULL_NAME`
* `SAML_ATTRIBUTES_STATEMENTS_FIRST_NAME`
* `SAML_ATTRIBUTES_STATEMENTS_LAST_NAME`
* `SAML_UID_ATTRIBUTE`
* `SAML_ATTRIBUTES_STATEMENTS_VERIFIED`
* `SAML_ATTRIBUTES_STATEMENTS_VERIFIED_EMAIL`
#### `SAML_ENABLED`
#### `SAML_ACS_URL`
#### `SAML_ISSUER`
#### `SAML_IDP_SSO_TARGET_URL`
#### `SAML_IDP_CERT`
#### `SAML_IDP_CERT_FINGERPRINT`
#### `SAML_NAME_IDENTIFIER_FORMAT`
#### `SAML_CERT`
#### `SAML_PRIVATE_KEY`
#### `SAML_SECURITY_WANT_ASSERTION_SIGNED`
#### `SAML_SECURITY_WANT_ASSERTION_ENCRYPTED`
#### `SAML_SECURITY_ASSUME_EMAIL_IS_VERIFIED`
#### `SAML_ATTRIBUTES_STATEMENTS_UID`
#### `SAML_ATTRIBUTES_STATEMENTS_EMAIL`
#### `SAML_ATTRIBUTES_STATEMENTS_FULL_NAME`
#### `SAML_ATTRIBUTES_STATEMENTS_FIRST_NAME`
#### `SAML_ATTRIBUTES_STATEMENTS_LAST_NAME`
#### `SAML_UID_ATTRIBUTE`
#### `SAML_ATTRIBUTES_STATEMENTS_VERIFIED`
#### `SAML_ATTRIBUTES_STATEMENTS_VERIFIED_EMAIL`
## Hidden services {#hidden-services}
{{< page-ref page="admin/optional/tor.md" >}}
### TOR {#tor}
{{< page-ref page="admin/optional/tor" >}}
#### `http_proxy`
#### `http_hidden_proxy`
#### `ALLOW_ACCESS_TO_HIDDEN_SERVICE`
## Limits {#limits}
### Email domains
* `http_proxy`
* `ALLOW_ACCESS_TO_HIDDEN_SERVICE`
#### `EMAIL_DOMAIN_ALLOWLIST`
If set, registrations will not be possible with any e-mails **except** those from the specified domains. Pipe-separated values, e.g.: `foo.com|bar.com`
#### `EMAIL_DOMAIN_DENYLIST`
If set, registrations will not be possible with any e-mails from the specified domains. Pipe-separated values, e.g.: `foo.com|bar.com`
{{< hint style="warning" >}}
This option is deprecated. You can dynamically block e-mail domains from the admin interface or from the `tootctl` command-line interface.
{{</ hint >}}
### Sessions
#### `MAX_SESSION_ACTIVATIONS`
How many browser sessions are allowed per-user. Defaults to `10`. If a new browser session is created, then the oldest session is deleted, e.g. user in that browser is logged out.
### Home feeds
#### `USER_ACTIVE_DAYS`
Mastodon stores home feeds in RAM (specifically, in the Redis database). This makes them very fast to access and update, but it also means that you don't want to keep them there if they're not used, and you don't want to spend resources on inserting new items into home feeds that will not be accessed. For this reason, Mastodon periodically clears out home feeds of users who haven't been online in a while, and if they re-appear, it regenerates those home feeds from database data. By default, users are considered active if they have been online in the past `7` days.
Regeneration of home feeds is computationally expensive, if your Sidekiq is constantly doing it because your users come online every 3 days but your `USER_ACTIVE_DAYS` is set to 2, then consider adjusting it up.
{{< hint style="info" >}}
This setting has no relation to which users are considered active for the purposes of statistics, such as the Monthly Active Users number.
{{</ hint >}}
## Other {#other}
### DB migrations {#migrations}
#### `SKIP_POST_DEPLOYMENT_MIGRATIONS`
This variable only has any effect when running `rake db:migrate` and it is extremely specific to the Mastodon upgrade process. There are two types of database migrations, those that run before new code is deployed and running, and those that run after. By default, both types of migrations are executed. If you shut down all Mastodon processes before running migrations, then there is no difference. The variable makes sense for zero-downtime upgrades. You will see in the upgrade instructions of a specific Mastodon version if you need to use it or not.
### Uncategorized or unsorted
#### `BUNDLE_GEMFILE`
#### `DEEPL_API_KEY`
#### `DEEPL_PLAN`
#### `LIBRE_TRANSLATE_ENDPOINT`
#### `LIBRE_TRANSLATE_API_KEY`
#### `CACHE_BUSTER_ENABLED`
#### `CACHE_BUSTER_SECRET_HEADER`
#### `CACHE_BUSTER_SECRET`
#### `GITHUB_REPOSITORY`
Defaults to `mastodon/mastodon`
#### `SOURCE_BASE_URL`
Defaults to `https://github.com/$GITHUB_REPOSITORY`
#### `FFMPEG_BINARY`
#### `LOCAL_HTTPS`
#### `PATH`
#### `MAX_FOLLOWS_THRESHOLD`
Defaults to `7500`
#### `MAX_FOLLOWS_RATIO`
Defaults to `1.1`
#### `IP_RETENTION_PERIOD`
Defaults to `31536000` (1 year)
#### `SESSION_RETENTION_PERIOD`
Defaults to `31536000` (1 year)
#### `BACKTRACE`
Set to `1` to allow backtracing to Rails framework code.
#### `DISABLE_SIMPLECOV`
#### `EMAIL_DOMAIN_LISTS_APPLY_AFTER_CONFIRMATION`
#### `DISABLE_FOLLOWERS_SYNCHRONIZATION`
#### `MAX_REQUEST_POOL_SIZE`
Defaults to `512`.
#### `GITHUB_API_TOKEN`
Used in a rake task for generating AUTHORS.md from Github commit history.

@ -10,7 +10,7 @@ menu:
## Pre-requisites {#pre-requisites}
* A machine running **Ubuntu 20.04** or **Debian 11** that you have root access to
* A **domain name** \(or a subdomain\) for the Mastodon server, e.g. `example.com`
* A **domain name** (or a subdomain) for the Mastodon server, e.g. `example.com`
* An e-mail delivery service or other **SMTP server**
You will be running the commands as root. If you arent already root, switch to root:
@ -84,8 +84,8 @@ git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build
Once this is done, we can install the correct Ruby version:
```bash
RUBY_CONFIGURE_OPTS=--with-jemalloc rbenv install 3.0.3
rbenv global 3.0.3
RUBY_CONFIGURE_OPTS=--with-jemalloc rbenv install 3.0.4
rbenv global 3.0.4
```
Well also need to install bundler:
@ -104,7 +104,7 @@ exit
### Setting up PostgreSQL {#setting-up-postgresql}
#### Performance configuration \(optional\) {#performance-configuration-optional}
#### Performance configuration (optional) {#performance-configuration-optional}
For optimal performance, you may use [pgTune](https://pgtune.leopard.in.ua/#/) to generate an appropriate configuration and edit values in `/etc/postgresql/14/main/postgresql.conf` before restarting PostgreSQL with `systemctl restart postgresql`
@ -140,7 +140,7 @@ su - mastodon
Use git to download the latest stable release of Mastodon:
```bash
git clone https://github.com/tootsuite/mastodon.git live && cd live
git clone https://github.com/mastodon/mastodon.git live && cd live
git checkout $(git tag -l | grep -v 'rc[0-9]*$' | sort -V | tail -n 1)
```
@ -173,7 +173,7 @@ This will:
* Run asset precompilation
* Create the database schema
The configuration file is saved as `.env.production`. You can review and edit it to your liking. Refer to the [documentation on configuration.]({{< relref "config.md" >}})
The configuration file is saved as `.env.production`. You can review and edit it to your liking. Refer to the [documentation on configuration.]({{< relref "config" >}})
Youre done with the mastodon user for now, so switch back to root:

@ -15,10 +15,10 @@ This guide was written with Ubuntu Server in mind; your mileage may vary for oth
## Basic steps {#basic-steps}
1. Set up a new Mastodon server using the [Production Guide]({{< relref "install.md" >}}) \(however, dont run `mastodon:setup`\).
2. Stop Mastodon on the old server \(e.g. `systemctl stop 'mastodon-*.service'`\).
1. Set up a new Mastodon server using the [Production Guide]({{< relref "install" >}}) (however, dont run `mastodon:setup`).
2. Stop Mastodon on the old server (e.g. `systemctl stop 'mastodon-*.service'`).
3. Dump and load the Postgres database using the instructions below.
4. Copy the `system/` files using the instructions below. \(Note: if youre using S3, you can skip this step.\)
4. Copy the `system/` files using the instructions below. (Note: if youre using S3, you can skip this step.)
5. Copy the `.env.production` file.
6. Run `RAILS_ENV=production bundle exec rails assets:precompile` to compile Mastodon
7. Run `RAILS_ENV=production ./bin/tootctl feeds build` to rebuild the home timelines for each user.
@ -33,19 +33,19 @@ This guide was written with Ubuntu Server in mind; your mileage may vary for oth
At a high level, youll need to copy over the following:
* The `~/live/public/system` directory, which contains user-uploaded images and videos \(if using S3, you dont need this\)
* The Postgres database \(using [pg\_dump](https://www.postgresql.org/docs/9.1/static/backup-dump.html)\)
* The `~/live/public/system` directory, which contains user-uploaded images and videos (if using S3, you dont need this)
* The Postgres database (using [pg\_dump](https://www.postgresql.org/docs/9.1/static/backup-dump.html))
* The `~/live/.env.production` file, which contains server config and secrets
Less crucially, youll probably also want to copy the following for convenience:
* The nginx config \(under `/etc/nginx/sites-available/default`\)
* The systemd config files \(`/etc/systemd/system/mastodon-*.service`\), which may contain your server tweaks and customizations
* The pgbouncer configuration under `/etc/pgbouncer` \(if youre using it\)
* The nginx config (under `/etc/nginx/sites-available/default`)
* The systemd config files (`/etc/systemd/system/mastodon-*.service`), which may contain your server tweaks and customizations
* The pgbouncer configuration under `/etc/pgbouncer` (if youre using it)
### Dump and load Postgres {#dump-and-load-postgres}
Instead of running `mastodon:setup`, were going to create an empty Postgres database using the `template0` database \(which is useful when restoring a Postgres dump, [as described in the pg\_dump documentation](https://www.postgresql.org/docs/9.1/static/backup-dump.html#BACKUP-DUMP-RESTORE)\).
Instead of running `mastodon:setup`, were going to create an empty Postgres database using the `template0` database (which is useful when restoring a Postgres dump, [as described in the pg\_dump documentation](https://www.postgresql.org/docs/9.1/static/backup-dump.html#BACKUP-DUMP-RESTORE)).
Run this as the `mastodon` user on your old system:
@ -66,7 +66,7 @@ pg_restore -Fc -U mastodon -n public --no-owner --role=mastodon \
-d mastodon_production backup.dump
```
\(Note that if the username is not `mastodon` on the new server, you should change the `-U` AND `--role` values above. Its okay if the username is different between the two servers.\)
(Note that if the username is not `mastodon` on the new server, you should change the `-U` AND `--role` values above. Its okay if the username is different between the two servers.)
### Copy files {#copy-files}
@ -86,7 +86,7 @@ Optionally, you may copy over the nginx, systemd, and pgbouncer config files, or
You can edit the `~/live/public/500.html` page on the old machine if you want to show a nice error message to let existing users know that a migration is in progress.
Youll probably also want to set the DNS TTL to something small \(30-60 minutes\) about a day in advance, so that DNS can propagate quickly once you point it to the new IP address.
Youll probably also want to set the DNS TTL to something small (30-60 minutes) about a day in advance, so that DNS can propagate quickly once you point it to the new IP address.
### After migrating {#after-migrating}

@ -7,10 +7,12 @@ menu:
parent: admin
---
## Individual moderation {#individual-moderation}
## Moderating individual users {#individual-moderation}
Moderation in Mastodon is always applied locally, i.e. as seen from the particular server. An admin or moderator on one server cannot affect a user on another server, they can only affect the local copy on their own server.
As of v3.5.0, all default user moderation decisions will notify the affected user by email. The user can access an appeal page, where they can submit one appeal within 20 days of the decision. Moderators can approve or reject the appeal.
### Sensitive {#sensitive-user}
When an account is marked as sensitive, all media that user posts will be automatically [marked as sensitive](https://docs.joinmastodon.org/user/posting/#cw).
@ -27,9 +29,9 @@ When the user's account is un-frozen, normal functionality resumes.
### Limit {#limit-user}
A limited account is hidden to all other users on that instance, except for its followers. All of the content is still there, and it can still be found via search, mentions, and following, but the content is invisible publicly.
Previously known as "silencing". A limited account is hidden to all other users on that instance, except for its followers. All of the content is still there, and it can still be found via search, mentions, and following, but the content is invisible publicly.
At this moment, limit does not affect federation. A locally limited account is _not_ limited automatically on other servers. Account limitations are reversible.
At this moment, limit does not affect federation. A locally limited account is *not* limited automatically on other servers. Account limitations are reversible.
### Suspend {#suspend-user}
@ -37,9 +39,9 @@ A Mastodon suspension means the account is effectively deleted. The account no l
If the account is reinstated within the 30 day period, all data is once again accessible publicly without any adverse affects. If the 30 day period lapses, **all** that user's data is purged from the instance. Admins also have the option to immediately delete the user's account data at any point during the 30 days.
Once the data has been deleted, whether than be after the 30 day period, or if an admin has force deleted it, the account can still be un-suspended. However, the account will have no data (toots, profile information, avatar or header image) associated with it.
Once the data has been deleted, whether than be after the 30 day period, or if an admin has force deleted it, the account can still be un-suspended. However, the account will have no data (statuses, profile information, avatar or header image) associated with it.
## Server-wide moderation {#server-wide-moderation}
## Moderating entire websites {#server-wide-moderation}
Because individually moderating a large volume of users from a misbehaving server can be exhausting, it is possible to pre-emptively moderate against all users from that particular server using a so-called **domain block**, which comes with several different levels of severity.
@ -47,13 +49,13 @@ Because individually moderating a large volume of users from a misbehaving serve
With this option active, no files from the server will be processed locally. That includes avatars, headers, emojis and media attachments.
### Silence {#silence-server}
### Limit {#limit-server}
Applies a silence to all past and future accounts from the server.
Equivalent to [limiting](#limit-user) all past and future accounts from the server. Previously known as "silencing".
### Suspend {#suspend-server}
Applies a suspension to all past and future accounts from the server. No content from the server will be stored locally except for usernames.
Equivalent to [suspending](#suspend-user) all past and future accounts from the server. No content from the server will be stored locally except for usernames.
## Spam-fighting measures {#spam-fighting-measures}

@ -7,7 +7,7 @@ menu:
parent: admin-optional
---
Mastodon supports full-text search when ElasticSearch is available. Mastodons full-text search allows logged in users to find results from their own toots, their favourites, and their mentions. It deliberately does not allow searching for arbitrary strings in the entire database.
Mastodon supports full-text search when ElasticSearch is available. Mastodons full-text search allows logged in users to find results from their own statuses, their mentions, their favourites, and their bookmarks. It deliberately does not allow searching for arbitrary strings in the entire database.
## Installing ElasticSearch {#install}

@ -8,5 +8,7 @@ menu:
{{< hint style="danger" >}}
This page is under construction.
For the moment, please refer to [this pull request](https://github.com/mastodon/mastodon/pull/16221)
{{< /hint >}}

@ -8,7 +8,7 @@ menu:
If you are setting up a fresh machine, it is recommended that you secure it first. Assuming that you are running **Ubuntu 20.04**:
## Do not allow password-based SSH login \(keys only\)
## Do not allow password-based SSH login (keys only)
First make sure you are actually logging in to the server using keys and not via a password, otherwise this will lock you out. Many hosting providers support uploading a public key and automatically set up key-based root login on new machines for you.

@ -11,11 +11,11 @@ menu:
Mastodon has three types of processes:
* Web \(Puma\)
* Web (Puma)
* Streaming API
* Background processing \(Sidekiq\)
* Background processing (Sidekiq)
### Web \(Puma\) {#web}
### Web (Puma) {#web}
The web process serves short-lived HTTP requests for most of the application. The following environment variables control it:
@ -37,7 +37,7 @@ The streaming API handles long-lived HTTP and WebSockets connections, through wh
One process can handle a reasonably high number of connections. The streaming API can be hosted on a different subdomain if you want to e.g. avoid the overhead of nginx proxying the connections.
### Background processing \(Sidekiq\) {#sidekiq}
### Background processing (Sidekiq) {#sidekiq}
Many tasks in Mastodon are delegated to background processing to ensure the HTTP requests are fast, and to prevent HTTP request aborts from affecting the execution of those tasks. Sidekiq is a single process, with a configurable number of threads.
@ -84,7 +84,7 @@ As a solution, it is possible to start different Sidekiq processes for the queue
### Why you might need PgBouncer {#pgbouncer-why}
If you start running out of available Postgres connections \(the default is 100\) then you may find PgBouncer to be a good solution. This document describes some common gotchas as well as good configuration defaults for Mastodon.
If you start running out of available Postgres connections (the default is 100) then you may find PgBouncer to be a good solution. This document describes some common gotchas as well as good configuration defaults for Mastodon.
Note that you can check “PgHero” in the administration view to see how many Postgres connections are currently being used. Typically Mastodon uses as many connections as there are threads both in Puma, Sidekiq and the streaming API combined.
@ -108,7 +108,7 @@ Heres how you might reset the password:
psql -p 5432 -U mastodon mastodon_production -w
```
Then \(obviously, use a different password than the word “password”\):
Then (obviously, use a different password than the word “password”):
```sql
ALTER USER mastodon WITH PASSWORD 'password';
@ -164,13 +164,13 @@ listen_addr = 127.0.0.1
listen_port = 6432
```
Put `md5` as the `auth_type` \(assuming youre using the md5 format in `userlist.txt`\):
Put `md5` as the `auth_type` (assuming youre using the md5 format in `userlist.txt`):
Make sure the `pgbouncer` user is an admin:
**This next part is very important!** The default pooling mode is session-based, but for Mastodon we want transaction-based. In other words, a Postgres connection is created when a transaction is created and dropped when the transaction is done. So youll want to change the `pool_mode` from `session` to `transaction`:
Next up, `max_client_conn` defines how many connections PgBouncer itself will accept, and `default_pool_size` puts a limit on how many Postgres connections will be opened under the hood. \(In PgHero the number of connections reported will correspond to `default_pool_size` because it has no knowledge of PgBouncer.\)
Next up, `max_client_conn` defines how many connections PgBouncer itself will accept, and `default_pool_size` puts a limit on how many Postgres connections will be opened under the hood. (In PgHero the number of connections reported will correspond to `default_pool_size` because it has no knowledge of PgBouncer.)
The defaults are fine to start, and you can always increase them later:
@ -211,7 +211,7 @@ PREPARED_STATEMENTS=false
Since were using transaction-based pooling, we cant use prepared statements.
Next up, configure Mastodon to use port 6432 \(PgBouncer\) instead of 5432 \(Postgres\) and you should be good to go:
Next up, configure Mastodon to use port 6432 (PgBouncer) instead of 5432 (Postgres) and you should be good to go:
```bash
DB_HOST=localhost
@ -222,7 +222,7 @@ DB_PORT=6432
```
{{< hint style="warning" >}}
You cannot use pgBouncer to perform `db:migrate` tasks. But this is easy to work around. If your postgres and pgbouncer are on the same host, it can be as simple as defining `DB_PORT=5432` together with `RAILS_ENV=production` when calling the task, for example: `RAILS_ENV=production DB_PORT=5432 bundle exec rails db:migrate` \(you can specify `DB_HOST` too if its different, etc\)
You cannot use pgBouncer to perform `db:migrate` tasks. But this is easy to work around. If your postgres and pgbouncer are on the same host, it can be as simple as defining `DB_PORT=5432` together with `RAILS_ENV=production` when calling the task, for example: `RAILS_ENV=production DB_PORT=5432 bundle exec rails db:migrate` (you can specify `DB_HOST` too if its different, etc)
{{< /hint >}}
#### Administering PgBouncer {#pgbouncer-admin}
@ -249,13 +249,13 @@ Then use `\q` to quit.
## Separate Redis for cache {#redis}
Redis is used widely throughout the application, but some uses are more important than others. Home feeds, list feeds, and Sidekiq queues as well as the streaming API are backed by Redis and thats important data you wouldnt want to lose \(even though the loss can be survived, unlike the loss of the PostgreSQL database - never lose that!\). However, Redis is also used for volatile cache. If you are at a stage of scaling up where you are worried if your Redis can handle everything, you can use a different Redis database for the cache. In the environment, you can specify `CACHE_REDIS_URL` or individual parts like `CACHE_REDIS_HOST`, `CACHE_REDIS_PORT` etc. Unspecified parts fallback to the same values as without the cache prefix.
Redis is used widely throughout the application, but some uses are more important than others. Home feeds, list feeds, and Sidekiq queues as well as the streaming API are backed by Redis and thats important data you wouldnt want to lose (even though the loss can be survived, unlike the loss of the PostgreSQL database - never lose that!). However, Redis is also used for volatile cache. If you are at a stage of scaling up where you are worried if your Redis can handle everything, you can use a different Redis database for the cache. In the environment, you can specify `CACHE_REDIS_URL` or individual parts like `CACHE_REDIS_HOST`, `CACHE_REDIS_PORT` etc. Unspecified parts fallback to the same values as without the cache prefix.
As far as configuring the Redis database goes, basically you can get rid of background saving to disk, since it doesnt matter if the data gets lost on restart and you can save some disk I/O on that. You can also add a maximum memory limit and a key eviction policy, for that, see this guide: [Using Redis as an LRU cache](https://redis.io/topics/lru-cache)
## Read-replicas {#read-replicas}
To reduce the load on your Postgresql server, you may wish to setup hot streaming replication \(read replica\). [See this guide for an example](https://cloud.google.com/community/tutorials/setting-up-postgres-hot-standby). You can make use of the replica in Mastodon in these ways:
To reduce the load on your Postgresql server, you may wish to setup hot streaming replication (read replica). [See this guide for an example](https://cloud.google.com/community/tutorials/setting-up-postgres-hot-standby). You can make use of the replica in Mastodon in these ways:
* The streaming API server does not issue writes at all, so you can connect it straight to the replica. But its not querying the database very often anyway so the impact of this is little.
* Use the Makara driver in the web and sidekiq processes, so that writes go to the master database, while reads go to the replica. Lets talk about that.

@ -17,6 +17,10 @@ After signing up in the browser, you will need to use the command line to give y
RAILS_ENV=production bin/tootctl accounts modify alice --role Owner
```
{{<hint style="warning">}}
Prior to Mastodon 4.0, roles were hardcoded to be one of `user`, `moderator`, or `admin`. Since Mastodon 4.0, there is a customizable role system, with default roles created for `Moderator`, `Admin`, and `Owner`. Names of custom roles are case-sensitive.
{{</hint>}}
### From the command line {#admin-cli}
You can create a new account using the command-line interface.
@ -42,7 +46,7 @@ After logging in, navigate to the **Site settings** page. While there are no tec
| Instance description | Why did you start this server? Who is it for? What makes it different? |
| Custom extended information | You can put all sorts of information in here but a **code of conduct** is recommended |
After you fill these in, simply hit “Save changes”.
After you fill these in, click “Save changes”.
## Running periodic cleanup tasks {#cleanup}

File diff suppressed because it is too large Load Diff

@ -9,7 +9,7 @@ menu:
## **I see an error page that says something went wrong. How do I find out whats wrong?**
All error messages with stack traces are written to the system log. When using systemd, the logs of each systemd service can be browsed with `journalctl -u mastodon-web` \(substitute with the correct service name\). When using Docker, its similar: `docker logs mastodon_web_1` \(substitute with the correct container name\).
All error messages with stack traces are written to the system log. When using systemd, the logs of each systemd service can be browsed with `journalctl -u mastodon-web` (substitute with the correct service name). When using Docker, its similar: `docker logs mastodon_web_1` (substitute with the correct container name).
Specific details of server-side errors are _never_ displayed to the public, as they can reveal what your setup looks like internally and give attackers clues how to get in, or how to abuse the system more efficiently.

@ -11,7 +11,7 @@ A somewhat common configuration issue can lead to index corruption throughout th
## Locale data and collation {#explanation}
Textual values in the database, such as usernames, or toot identifiers, are compared using so-called collation rules defining how characters are ordered and how to change their case.
Textual values in the database, such as usernames, or status identifiers, are compared using so-called collation rules defining how characters are ordered and how to change their case.
When setting up a database, Mastodon will use the database server's default locale settings, including the default collation rules, which often is defined by the operating system's settings.
Unfortunately, in late 2018, a `glibc` update changed the collation rules for many locales, which means databases using an affected locale would now order textual values differently.

@ -7,10 +7,10 @@ menu:
---
{{< hint style="info" >}}
When a new version of Mastodon comes out, it appears on the [GitHub releases page](https://github.com/tootsuite/mastodon/releases). Please mind that running unreleased code from the `master` branch, while possible, is not recommended.
When a new version of Mastodon comes out, it appears on the [GitHub releases page](https://github.com/mastodon/mastodon/releases). Please mind that running unreleased code from the `master` branch, while possible, is not recommended.
{{< /hint >}}
Mastodon releases correspond to git tags. Before attempting an upgrade, look up the desired release on the [GitHub releases page](https://github.com/tootsuite/mastodon/releases). The page will contain a **changelog** describing everything you need to know about what's different, as well as **specific upgrade instructions**.
Mastodon releases correspond to git tags. Before attempting an upgrade, look up the desired release on the [GitHub releases page](https://github.com/mastodon/mastodon/releases). The page will contain a **changelog** describing everything you need to know about what's different, as well as **specific upgrade instructions**.
To begin, switch to the `mastodon` user:

@ -0,0 +1,134 @@
---
title: Guidelines and best practices
description: Things to keep in mind when implementing a Mastodon app.
menu:
docs:
weight: 10
parent: api
---
## Login {#login}
**The user must be able to login to any Mastodon server from the app**. This means you must ask for the server's domain and use the app registrations API to dynamically obtain OAuth2 credentials.
{{< page-ref page="client/authorized" >}}
{{< page-relref ref="methods/oauth" caption="oauth methods" >}}
{{< page-relref ref="api/oauth-scopes" caption="OAuth scopes" >}}
## Usernames {#username}
**Decentralization must be transparent to the user**. It should be possible to see that a given user is from another server, for example, by displaying their `acct` somewhere. Note that `acct` is equal to `username` for local users, and equal to `username@domain` for remote users.
## Handling and sorting IDs {#id}
Vanilla Mastodon entity IDs are generated as integers and cast to string. However, this does not mean that IDs _are_ integers, nor should they be cast to integer! Doing so can lead to broken client apps due to integer overflow, so **always treat IDs as strings.**
With that said, because IDs are string representations of numbers, they can still be sorted chronologically very easily by doing the following:
1. Sort by size. Newer statuses will have longer IDs.
2. Sort lexically. Newer statuses will have at least one digit that is higher when compared positionally.
## Paginating through API responses {#pagination}
Many API methods allow you to paginate for more information, using parameters such as `limit`, `max_id`, `min_id`, and `since_id`. However, some of these API methods operate on entity IDs that are not publicly exposed in the API response, and are only known to the backend and the database. (This is usually the case for entities that reference other entities, such as Follow entities which reference Accounts, or Favourite entities which reference Statuses, etc.)
To get around this, Mastodon may return links to a "prev" and "next" page. These links are made available via the HTTP `Link` header on the response. Consider the following fictitious API call:
```http
GET https://mastodon.social/api/v1/endpoint HTTP/1.1
Authorization: Bearer token
Link: <https://mastodon.social/api/v1/endpoint?max_id=7163058>; rel="next", <https://mastodon.social/api/v1/endpoint?since_id=7275607>; rel="prev"
[
{
// some Entity
},
// more Entities in an Array
]
```
In this case, you may retrieve the `Link` header and parse it for links to the older or newer page. Keep in mind the following rules:
- The links will be returned all via one `Link` header, separated by a comma and a space (`, `)
- Each link consists of a URL and a link relation, separated by a semicolon and a space (`; `)
- The URL will be surrounded by angle brackets (`<>`), and the link relation will be surrounded by double quotes (`""`) and prefixed with `rel=`.
- The value of the link relation will be either `prev` or `next`.
Following the `next` link should show you older results. Following the `prev` link should show you newer results.
## Formatting {#formatting}
Plain text is not available for content from remote servers, and plain text syntax rules may vary wildly between Mastodon and other fediverse applications. For certain attributes, such as the content of statuses, **Mastodon provides sanitized HTML**. See [HTML Sanitization]({{< relref "spec/activitypub#sanitization" >}}) for more details. You may expect these tags to appear in the content:
* `<p>`
* `<br>`
* `<span>`
* `<a>`
{{< page-relref ref="spec/activitypub#sanitization" caption="ActivityPub > HTML Sanitization" >}}
### Mentions, hashtags, and custom emoji {#tags}
Mentions and hashtags are `<a>` tags. Custom emoji remain in their plain text shortcode form. To give those entities their semantic meaning and add special handling, such as opening a mentioned profile within your app instead of as a web page, metadata is included with the [Status]({{< relref "entities/Status" >}}), which can be matched to a particular tag.
{{< page-relref ref="entities/Status" caption="Status entity" >}}
{{< page-relref ref="entities/Status#mentions" caption="Status#mentions" >}}
{{< page-relref ref="entities/Status#tags" caption="Status#tags" >}}
{{< page-relref ref="entities/Status#emojis" caption="Status#emojis" >}}
### Link shortening {#links}
Links in Mastodon are not shortened using URL shorteners, and the usage of URL shorteners is heavily discouraged. URLs in text always count for 23 characters, and are intended to be shortened visually. For that purpose, a link is marked up like this:
```html
<a href="https://example.com/page/that/is/very/long">
<span class="invisible">https://</span>
<span class="ellipsis">example.com/page</span>
<span class="invisible">/that/is/very/long</span>
</a>
```
The spans with the `invisible` class can be hidden. The middle span is intended to remain visible. It may have no class if the URL is not very long; otherwise it will have an `ellipsis` class. No ellipsis (`…`) character is inserted in the markup; instead, you are expected to insert it yourself if you need it in your app.
## Filters {#filters}
### Server-side filtering (v2, Mastodon 4.0 and above) {#server-filtered}
If a filter applies to a Status, a corresponding FilterResult will be included in the `filtered` attribute. Clients should check this attribute for any matches and use them to apply the intended filter action.
However, client implementations may still want to perform their own rule matching client-side, as this would allow retroactively apply filter changes without re-fetching posts from the server. When doing so, they should take care to not ignore `filtered` entries for which there are other attributes than `keyword_matches`, so as to handle extensions of the filtering system (e.g. `status_matches`).
Matched filters need to be filtered based on context (`home`, `notifications`, `public`, `thread` or `profile`) and expiration date.
When at least one active matched filter has `hide` for `filter_action`, the post should not be shown at all. Otherwise, if at least one active matched filter has `warn` for `filter_action`, the post should be hidden with a warning, and the user should be able to reveal the post after being informed of which filters matched (identified by `title` rather than the exact matched keywords).
For extension purposes, unknown values for `filter_action` should be treated as `warn`.
### Client-side filtering (v1, prior to Mastodon 4.0) {#client-filtered}
Clients must do their own text filtering based on filters returned from the API. The server will apply `irreversible` filters for `home` and `notifications` contexts, but **anything else is still up to the client to filter**! If a status is somehow not removed by an `irreversible` filter, the client should still filter it.
Expired filters are not deleted by the server. They should no longer be applied, but they are still stored by the server, as users may update the expiry time to re-enable the filter. It is up to users to delete those filters eventually, if they wish to do so.
If `whole_word` is true, the client app should do the following:
* Define word constituent characters for your app. In the official implementation, its `[A-Za-z0-9_]` in JavaScript, and `[[:word:]]` in Ruby. Ruby uses the POSIX character class (Letter | Mark | Decimal_Number | Connector_Punctuation).
* If the phrase starts with a word character, and if the previous character before matched range is a word character, its matched range should be treated to not match.
* If the phrase ends with a word character, and if the next character after matched range is a word character, its matched range should be treated to not match.
Please check `app/javascript/mastodon/selectors/index.js` and `app/lib/feed_manager.rb` in the Mastodon source code for more details.
{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/app/javascript/mastodon/selectors/index.js" caption="app/javascript/mastodon/selectors/index.js" >}}
{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/app/lib/feed_manager.rb" caption="app/lib/feed_manager.rb" >}}
## Focal points for cropping media thumbnails {#focal-points}
Server-side preview images are never cropped, to support a variety of apps and user interfaces. Therefore, the cropping must be done by those apps. To crop intelligently, focal points can be used to ensure a certain section of the image is always within the cropped viewport. See this [guide on how focal points are defined](https://github.com/jonom/jquery-focuspoint#1-calculate-your-images-focus-point). In summary, floating points range from -1.0 to 1.0, left-to-right or bottom-to-top. (0,0) is the center of the image. (0.5, 0.5) would be in the center of the upper-right quadrant. (-0.5, -0.5) would be in the center of the lower-left quadrant. For reference, thumbnails in the Mastodon frontend are most commonly 16:9.
{{< figure src="/assets/focal-points.jpg" caption="A demonstration of various focal points and their coordinates." >}}

@ -3,7 +3,7 @@ title: OAuth Scopes
description: Defining what you have permission to do with the API
menu:
docs:
weight: 10
weight: 20
parent: api
---
@ -11,7 +11,7 @@ menu:
The API is divided up into access scopes. The scopes are hierarchical, i.e. if you have access to `read`, you automatically have access to `read:accounts`. **It is recommended that you request as little as possible for your application.**
Multiple scopes can be requested at the same time: During app creation with the `scopes` param, and during the authorization phase with the `scope` query param \(space-separate the scopes\).
Multiple scopes can be requested at the same time: During app creation with the `scopes` param, and during the authorization phase with the `scope` query param (space-separate the scopes).
{{< hint style="info" >}}
Mind the `scope` vs `scopes` difference. This is because `scope` is a standard OAuth parameter name, so it is used in the OAuth methods. Mastodons own REST API uses the more appropriate `scopes`.
@ -25,11 +25,12 @@ The set of scopes saved during app creation must include all the scopes that you
- 0.9.0 - read, write, follow
- 2.4.0 - push
- 2.4.3 - granular scopes [https://github.com/tootsuite/mastodon/pull/7929](https://github.com/tootsuite/mastodon/pull/7929)
- 2.6.0 - read:reports deprecated \(unused stub\) [https://github.com/tootsuite/mastodon/pull/8736/commits/adcf23f1d00c8ff6877ca2ee2af258f326ae4e1f](https://github.com/tootsuite/mastodon/pull/8736/commits/adcf23f1d00c8ff6877ca2ee2af258f326ae4e1f)
- 2.6.0 - write:conversations added [https://github.com/tootsuite/mastodon/pull/9009](https://github.com/tootsuite/mastodon/pull/9009)
- 2.9.1 - Admin scopes added [https://github.com/tootsuite/mastodon/pull/9387](https://github.com/tootsuite/mastodon/pull/9387)
- 3.1.0 - Bookmark scopes added
- 2.4.3 - granular scopes [#7929](https://github.com/mastodon/mastodon/pull/7929)
- 2.6.0 - read:reports deprecated (unused stub) [#8736/adcf23f](https://github.com/mastodon/mastodon/pull/8736/commits/adcf23f1d00c8ff6877ca2ee2af258f326ae4e1f)
- 2.6.0 - write:conversations added [#9009](https://github.com/mastodon/mastodon/pull/9009)
- 2.9.1 - Admin scopes added [#9387](https://github.com/mastodon/mastodon/pull/9387)
- 3.1.0 - Bookmark scopes added [#7107](https://github.com/mastodon/mastodon/pull/7107)
- 4.0.3 - Added admin scopes for blocks and allows [#20918](https://github.com/mastodon/mastodon/pull/20918)
## List of scopes
@ -37,12 +38,45 @@ The set of scopes saved during app creation must include all the scopes that you
Grants access to read data. Requesting `read` will also grant child scopes shown in the left column of the table below.
* `read`
* `read:accounts`
* `read:blocks`
* `read:bookmarks`
* `read:favourites`
* `read:filters`
* `read:follows`
* `read:lists`
* `read:mutes`
* `read:notifications`