diff --git a/.github/workflows/rebase-needed.yml b/.github/workflows/rebase-needed.yml new file mode 100644 index 00000000..05a1deab --- /dev/null +++ b/.github/workflows/rebase-needed.yml @@ -0,0 +1,27 @@ +name: PR Needs Rebase + +on: + schedule: + - cron: '0 * * * *' + +permissions: + pull-requests: write + +jobs: + label-rebase-needed: + runs-on: ubuntu-latest + + concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + + steps: + - name: Check for merge conflicts + uses: eps1lon/actions-label-merge-conflict@releases/2.x + with: + dirtyLabel: 'rebase needed :construction:' + repoToken: '${{ secrets.GITHUB_TOKEN }}' + commentOnClean: This pull request has resolved merge conflicts and is ready for review. + commentOnDirty: This pull request has merge conflicts that must be resolved before it can be merged. + retryMax: 30 + continueOnMissingPermissions: false \ No newline at end of file diff --git a/README.md b/README.md index 07de0046..53666116 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,9 @@ -![Mastodon](https://i.imgur.com/NhZc40l.png) -==== +

+ + + Mastodon +

+ +The documentation currently uses Hugo to generate a static site from Markdown. View the documentation at diff --git a/archetypes/entities.md b/archetypes/entities.md index f8774130..07bebffa 100644 --- a/archetypes/entities.md +++ b/archetypes/entities.md @@ -4,6 +4,10 @@ description: menu: docs: parent: entities +aliases: [ + "/api/entities/SOMETHING", + "/api/entities/something", +] --- ## Example diff --git a/archetypes/methods.md b/archetypes/methods.md index 83f6795b..ba3ea7f7 100644 --- a/archetypes/methods.md +++ b/archetypes/methods.md @@ -4,6 +4,10 @@ description: menu: docs: parent: methods +aliases: [ + "/api/methods/SOMETHING", + "/api/methods/something", +] --- ## What the method does {#anchor} @@ -34,14 +38,14 @@ Authorization **Internal parameter.** Use HTTP `Link` header for pagination. -max_id -: String. Return results older than ID. +max_id +: String. All results returned will be lesser than this ID. In effect, sets an upper bound on results. since_id -: String. Return results newer than ID. +: String. All results returned will be greater than this ID. In effect, sets a lower bound on results. min_id -: String. Return results immediately newer than ID. +: String. Returns results immediately newer than this ID. In effect, sets a cursor at this ID and paginates forward. limit : Integer. Maximum number of results to return. Defaults to 20 statuses or 40 accounts. Max twice the default limit. diff --git a/content/en/admin/config.md b/content/en/admin/config.md index ff908491..8e42dea0 100644 --- a/content/en/admin/config.md +++ b/content/en/admin/config.md @@ -202,7 +202,18 @@ Determines the amount of logs generated by Mastodon. Defaults to `info`, which g #### `TRUSTED_PROXY_IP` -If your Mastodon web process is on the same machine as your reverse proxy (e.g. nginx), then you don't need this setting. Otherwise, you need to set it to the IP from which your reverse proxy sends requests to Mastodon's web process, otherwise Mastodon will record the reverse proxy's own IP as the IP of all requests, which would be bad because IP addresses are used for important rate limits and security functions. +Tells the Mastodon web and streaming processes which IPs act as your trusted reverse proxy (e.g. nginx, Cloudflare). It affects how Mastodon determines the source IP of each request, which is used for important rate limits and security functions. If the value is set incorrectly then Mastodon could use the IP of the reverse proxy instead of the actual source. + +By default the loopback and private network address ranges are trusted. Specifically: + +- `127.0.0.1/8` +- `::1/128` +- `10.0.0.0/8` +- `172.16.0.0/12` +- `192.168.0.0/16` +- `fc00::/7` + +If you're using a single reverse proxy and it runs on the same machine or is in the same private network as your Mastodon web and streaming processes then you most likely don't need to modify this setting and can use the default. Or if you're using multiple reverse proxy servers and they're all in the same private network as your Mastodon web and streaming processes then, again, the default should be fine. However, if you're using a reverse proxy server that reaches your Mastodon web and streaming servers via a public IP address (for example if you're using Cloudflare or a similar proxy) then you'll need to set this variable. It should be the IPs of all reverse proxies in use, as a comma-separated list of IPs or IP ranges using [CIDR notation](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing#CIDR_notation). Note that when this variable is set the default ranges (mentioned above) will no longer be trusted, so if you have both an external reverse proxy _and_ a proxy on localhost then you must include the IPs (or IP ranges) of both. #### `SOCKET` @@ -240,6 +251,10 @@ This variable cannot be defined in dotenv (`.env`) files as it's used before the {{< page-ref page="admin/scaling" >}} +#### `SIDEKIQ_CONCURRENCY` + +Added in 4.1. Specific to Sidekiq, this variable determines how many different processes Sidekiq forks into. Defaults to `5`. + #### `WEB_CONCURRENCY` Specific to Puma, this variable determines how many different processes Puma forks into. Defaults to `2`. @@ -262,7 +277,11 @@ The streaming API can be deployed to a different domain/subdomain. This may impr Example value: `wss://streaming.example.com` -#### `STREAMING_CLUSTER_NUM` +#### `STREAMING_CLUSTER_NUM` (deprecated) {#streaming_cluster_num} + +{{< hint style="danger" >}} +Deprecated: The streaming server process now only uses a single node.js process, to scale it further, you'll need to follow the documentation in the [scaling guide](/admin/scaling#streaming) +{{< /hint >}} Specific to the streaming API, this variable determines how many different processes the streaming API forks into. Defaults to the number of CPU cores minus one. @@ -346,13 +365,25 @@ Defaults to value of `REDIS_NAMESPACE`. #### `SIDEKIQ_REDIS_URL` -### ElasticSearch {#elasticsearch} +### Elasticsearch {#elasticsearch} -{{< page-ref page="admin/optional/elasticsearch" >}} +{{< page-ref page="admin/elasticsearch" >}} #### `ES_ENABLED` -If set to `true`, Mastodon will use ElasticSearch for its search functions. +If set to `true`, Mastodon will use Elasticsearch for its search functions. + +#### `ES_PRESET` + +It controls the ElasticSearch indices configuration (number of shards and replica). + +Possible values are: + +- `single_node_cluster` (default) +- `small_cluster` +- `large_cluster` + +See the [ElasticSearch setup page for details on each setting](../elasticsearch#choosing-the-correct-preset). #### `ES_HOST` @@ -360,19 +391,19 @@ Host of the ElasticSearch server. Defaults to `localhost`. If using TLS, prepend #### `ES_PORT` -Port of the ElasticSearch server. Defaults to `9200` +Port of the Elasticsearch server. Defaults to `9200` #### `ES_USER` -Used for optionally authenticating with ElasticSearch +Used for optionally authenticating with Elasticsearch #### `ES_PASS` -Used for optionally authenticating with ElasticSearch +Used for optionally authenticating with Elasticsearch #### `ES_PREFIX` -Useful if the ElasticSearch server is shared between multiple projects or different Mastodon servers. Defaults to value of `REDIS_NAMESPACE`. +Useful if the Elasticsearch server is shared between multiple projects or different Mastodon servers. Defaults to value of `REDIS_NAMESPACE`. ### StatsD {#statsd} @@ -386,19 +417,34 @@ Example value: `localhost:8125` If set, all StatsD keys will be prefixed with this. Defaults to `Mastodon.production` when `RAILS_ENV` is `production`, `Mastodon.development` when it's `development`, etc. +#### `STATSD_SIDEKIQ` + +If set to `true`, Mastodon will log some Sidekiq metrics into StatsD. Defaults to `false`. + ### SMTP email delivery {#smtp} #### `SMTP_SERVER` + #### `SMTP_PORT` + #### `SMTP_LOGIN` + #### `SMTP_PASSWORD` + #### `SMTP_FROM_ADDRESS` + #### `SMTP_DOMAIN` + #### `SMTP_DELIVERY_METHOD` + #### `SMTP_AUTH_METHOD` + #### `SMTP_CA_FILE` + #### `SMTP_OPENSSL_VERIFY_MODE` + #### `SMTP_ENABLE_STARTTLS_AUTO` + #### `SMTP_ENABLE_STARTTLS` Set to `auto` (default), `always`, or `never`. @@ -407,11 +453,39 @@ Set to `auto` (default), `always`, or `never`. 4.0.0 - added #### `SMTP_TLS` + #### `SMTP_SSL` +E-mail configuration is based on the *action_mailer* component of the *Ruby on Rails* framework that Mastodon is built on. Complete documentation on action_mailer is available [here](https://guides.rubyonrails.org/action_mailer_basics.html#action-mailer-configuration). The client uses SMTP or derivatives: StartTLS + SMTP or SMTPS (SMTP over TLS). + +### Basic configuration {#basic} + +* `SMTP_SERVER`: Specify the server to use. For example `sub.domain.tld`. +* `SMTP_PORT`: By default, the value is `25` (usual port for SMTP). If StartTLS is detected, it may be switched to port 587. +* `SMTP_DOMAIN`: Only required if a HELO domain is needed. Will be set to the `SMTP_SERVER` domain by default. +* `SMTP_FROM_ADDRESS`: Specify a sender address. +* `SMTP_DELIVERY_METHOD`: By default, the value is `smtp` (can also be `sendmail`). + +### Authentication for the SMTP server {#smtpauthentication} + +* `SMTP_LOGIN`: Login for the SMTP user. +* `SMTP_PASSWORD`: Password for the SMTP user. +* `SMTP_AUTH_METHOD`: Either `plain` (default; password is transmitted in the clear), `login` (password will be base64 encoded) or `cram_md5`. + +### Secured SMTP +By default, a StartTLS connection will be attempted to the specified SMTP server. + +* `SMTP_ENABLE_STARTTLS_AUTO`: Default `true`. +* `SMTP_CA_FILE`: A value may be specified, but on many Linux distros (e.g. Debian-based) this will be `/etc/ssl/certs/ca-certificates.crt`. +* `SMTP_OPENSSL_VERIFY_MODE`: `none` or `peer`. When using TLS, it may be useful to accept connections with a self-signed certificate. +* `SMTP_TLS`: `true` or `false` (default `false`) +* `SMTP_SSL`: `true` or `false` (default `false`) + +Note that `TLSv1.3` and `TLSv1.2` are the only SSL/TLS protocols currently considered to be secure. + ## File storage {#files} -### CDN {cdn} +### CDN {#cdn} #### `CDN_HOST` @@ -425,9 +499,9 @@ You must serve the files with CORS headers, otherwise some functions of Mastodon #### `S3_ALIAS_HOST` -Similar to `CDN_HOST`, you may serve *user-uploaded* files from a separate host. In fact, if you are using external storage like Amazon S3, Minio or Google Cloud, you will by default be serving files from those services' URLs. +Similar to `CDN_HOST`, you may serve _user-uploaded_ files from a separate host. In fact, if you are using external storage like Amazon S3, Minio or Google Cloud, you will by default be serving files from those services' URLs. -It is *extremely recommended* to use your own host instead, for a few reasons: +It is _extremely recommended_ to use your own host instead, for a few reasons: 1. Bandwidth on external storage providers is metered and expensive 2. You may want to switch to a different provider later without breaking old links @@ -443,36 +517,59 @@ You must serve the files with CORS headers, otherwise some functions of Mastodon ### Local file storage {#paperclip} #### `PAPERCLIP_ROOT_PATH` + #### `PAPERCLIP_ROOT_URL` ### Amazon S3 and compatible {#s3} #### `S3_ENABLED` + #### `S3_BUCKET` + #### `AWS_ACCESS_KEY_ID` + #### `AWS_SECRET_ACCESS_KEY` + #### `S3_REGION` + #### `S3_PROTOCOL` + #### `S3_HOSTNAME` + #### `S3_ENDPOINT` + #### `S3_SIGNATURE_VERSION` + #### `S3_OVERRIDE_PATH_STYLE` + #### `S3_OPEN_TIMEOUT` + #### `S3_READ_TIMEOUT` + #### `S3_FORCE_SINGLE_REQUEST` ### Swift {#swift} #### `SWIFT_ENABLED` + #### `SWIFT_USERNAME` + #### `SWIFT_TENANT` + #### `SWIFT_PASSWORD` + #### `SWIFT_PROJECT_ID` + #### `SWIFT_AUTH_URL` + #### `SWIFT_CONTAINER` + #### `SWIFT_OBJECT_URL` + #### `SWIFT_REGION` + #### `SWIFT_DOMAIN_NAME` + #### `SWIFT_CACHE_TTL` ## External authentication {#external-authentication} @@ -484,71 +581,125 @@ You must serve the files with CORS headers, otherwise some functions of Mastodon ### LDAP {#ldap} #### `LDAP_ENABLED` + #### `LDAP_HOST` + #### `LDAP_PORT` + #### `LDAP_METHOD` + #### `LDAP_BASE` + #### `LDAP_BIND_DN` + #### `LDAP_PASSWORD` + #### `LDAP_UID` + #### `LDAP_SEARCH_FILTER` + #### `LDAP_MAIL` -#### `LDAP_UID_CONVERSTION_ENABLED` + +#### `LDAP_UID_CONVERSION_ENABLED` ### PAM {#pam} #### `PAM_ENABLED` + #### `PAM_EMAIL_DOMAIN` + #### `PAM_DEFAULT_SERVICE` + #### `PAM_CONTROLLED_SERVICE` ### CAS {#cas} #### `CAS_ENABLED` + #### `CAS_DISPLAY_NAME` + #### `CAS_URL` + #### `CAS_HOST` + #### `CAS_PORT` + #### `CAS_SSL` + #### `CAS_VALIDATE_URL` + #### `CAS_CALLBACK_URL` + #### `CAS_LOGOUT_URL` + #### `CAS_LOGIN_URL` + #### `CAS_UID_FIELD` + #### `CAS_CA_PATH` + #### `CAS_DISABLE_SSL_VERIFICATION` + #### `CAS_UID_KEY` + #### `CAS_NAME_KEY` + #### `CAS_EMAIL_KEY` + #### `CAS_NICKNAME_KEY` + #### `CAS_FIRST_NAME_KEY` + #### `CAS_LAST_NAME_KEY` + #### `CAS_LOCATION_KEY` + #### `CAS_IMAGE_KEY` + #### `CAS_PHONE_KEY` + #### `CAS_SECURITY_ASSUME_EMAIL_IS_VERIFIED` ### SAML {#saml} #### `SAML_ENABLED` + #### `SAML_ACS_URL` + #### `SAML_ISSUER` + #### `SAML_IDP_SSO_TARGET_URL` + #### `SAML_IDP_CERT` + #### `SAML_IDP_CERT_FINGERPRINT` + #### `SAML_NAME_IDENTIFIER_FORMAT` + #### `SAML_CERT` + #### `SAML_PRIVATE_KEY` + #### `SAML_SECURITY_WANT_ASSERTION_SIGNED` + #### `SAML_SECURITY_WANT_ASSERTION_ENCRYPTED` + #### `SAML_SECURITY_ASSUME_EMAIL_IS_VERIFIED` + #### `SAML_ATTRIBUTES_STATEMENTS_UID` + #### `SAML_ATTRIBUTES_STATEMENTS_EMAIL` + #### `SAML_ATTRIBUTES_STATEMENTS_FULL_NAME` + #### `SAML_ATTRIBUTES_STATEMENTS_FIRST_NAME` + #### `SAML_ATTRIBUTES_STATEMENTS_LAST_NAME` + #### `SAML_UID_ATTRIBUTE` + #### `SAML_ATTRIBUTES_STATEMENTS_VERIFIED` + #### `SAML_ATTRIBUTES_STATEMENTS_VERIFIED_EMAIL` ## Hidden services {#hidden-services} @@ -558,7 +709,9 @@ You must serve the files with CORS headers, otherwise some functions of Mastodon {{< page-ref page="admin/optional/tor" >}} #### `http_proxy` + #### `http_hidden_proxy` + #### `ALLOW_ACCESS_TO_HIDDEN_SERVICE` ## Limits {#limits} @@ -606,13 +759,21 @@ This variable only has any effect when running `rake db:migrate` and it is extre ### Uncategorized or unsorted #### `BUNDLE_GEMFILE` + #### `DEEPL_API_KEY` + #### `DEEPL_PLAN` + #### `LIBRE_TRANSLATE_ENDPOINT` + #### `LIBRE_TRANSLATE_API_KEY` + #### `CACHE_BUSTER_ENABLED` + #### `CACHE_BUSTER_SECRET_HEADER` + #### `CACHE_BUSTER_SECRET` + #### `GITHUB_REPOSITORY` Defaults to `mastodon/mastodon` @@ -622,8 +783,11 @@ Defaults to `mastodon/mastodon` Defaults to `https://github.com/$GITHUB_REPOSITORY` #### `FFMPEG_BINARY` + #### `LOCAL_HTTPS` + #### `PATH` + #### `MAX_FOLLOWS_THRESHOLD` Defaults to `7500` @@ -656,4 +820,5 @@ Defaults to `512`. #### `GITHUB_API_TOKEN` -Used in a rake task for generating AUTHORS.md from Github commit history. +Used in a rake task for generating AUTHORS.md from GitHub commit history. + diff --git a/content/en/admin/elasticsearch.md b/content/en/admin/elasticsearch.md new file mode 100644 index 00000000..cf33bda3 --- /dev/null +++ b/content/en/admin/elasticsearch.md @@ -0,0 +1,229 @@ +--- +title: Configuring full-text search +description: Setting up Elasticsearch to search for statuses (authored, favourited, or mentioned), public indexable status, and accounts +aliases: +- /admin/optional/elasticsearch +menu: + docs: + weight: 40 + parent: admin +--- + +Mastodon supports full-text search when Elasticsearch is available. It is strongly recommended to configure this feature. + +Mastodon’s full-text search allows logged in users to find results from: +- public statuses from account that opted into appearing in search results +- their own statuses +- their mentions +- their favourites +- their bookmarks +- accounts (display name, usernames and bios) + +It deliberately does not allow searching for arbitrary strings in the entire database. + +## Installing Elasticsearch {#install} + +{{< hint style="info" >}} +Mastodon is tested with Elasticsearch version 7. It should support OpenSearch, as well as Elasticsearch versions 6 and 8, but those setups are not officially supported. +{{< /hint >}} + +Elasticsearch requires a Java runtime. If you don’t have Java already installed, do it now. Assuming you are logged in as `root`: + +```bash +apt install openjdk-17-jre-headless +``` + +Add the official Elasticsearch repository to apt: + +```bash +wget -O /usr/share/keyrings/elasticsearch.asc https://artifacts.elastic.co/GPG-KEY-elasticsearch +echo "deb [signed-by=/usr/share/keyrings/elasticsearch.asc] https://artifacts.elastic.co/packages/7.x/apt stable main" > /etc/apt/sources.list.d/elastic-7.x.list +``` + +Now you can install Elasticsearch: + +```bash +apt update +apt install elasticsearch +``` + +{{< hint style="warning" >}} +**Security warning:** By default, Elasticsearch is supposed to bind to localhost only, i.e. be inaccessible from the outside network. You can check which address Elasticsearch binds to by looking at `network.host` within `/etc/elasticsearch/elasticsearch.yml`. Consider that anyone who can access Elasticsearch can access and modify any data within it, as there is no authentication layer. So it’s really important that the access is secured. Having a firewall that only exposes the 22, 80 and 443 ports is advisable, as outlined in the [main installation instructions](../../prerequisites/#install-a-firewall-and-only-whitelist-ssh-http-and-https-ports). If you have a multi-host setup, you must know how to secure internal traffic. +{{< /hint >}} + +To start Elasticsearch: + +```bash +systemctl daemon-reload +systemctl enable --now elasticsearch +``` + +## Configuring Mastodon {#config} + +Edit `.env.production` to add the following variables: + +```bash +ES_ENABLED=true +ES_HOST=localhost +ES_PORT=9200 +ES_PRESET= # single_node_cluster, small_cluster or large_cluster +# ES_USER= +# ES_PASS= +``` + +### Choosing the correct preset + +The value for `ES_PRESET` depends on the size of your Elasticsearch and will be used to set the number of shards and replica for your indices to the best value for your setup: +- `single_node_cluster` if you only have one node in your Elasticsearch cluster. Indices will be configured without any replica +- `small_cluster` if you have less than 6 nodes in your cluster. Indices will be configured with 1 replica +- `large_cluster` if you have 6 or more nodes in your cluster. Indices will be configured with more shards than with the `small_cluster` setting, to allow them to be distributed over more nodes + +If you have multiple Mastodon servers on the same machine, and you are planning to use the same Elasticsearch installation for all of them, make sure that all of them have unique `REDIS_NAMESPACE` in their configurations, to differentiate the indices. If you need to override the prefix of the Elasticsearch indices, you can set `ES_PREFIX` directly. + +### Security + +By default, Elasticsearch does not handle any authentication and every request is made with full admin permission. We strongly advise you to configure Elasticsearch security features on your cluster. + +To configure it, please refer [to the official documentation](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/security-minimal-setup.html). It will guide you through: +- Enabling the security features (`xpack.security.enabled: true`) +- Creating password for built-in users + +Once done, you can create a custom role for Mastodon to connect. + +For example (please adapt this snippet to use your Elastic admin password): + +```sh +curl -X POST -u elastic:admin_password "localhost:9200/_security/role/mastodon_full_access?pretty" -H 'Content-Type: application/json' -d' +{ + "cluster": ["monitor"], + "indices": [{ + "names": ["*"], + "privileges": ["read", "monitor", "write", "manage"] + }] +} +' +``` + +[Elasticsearch documentation for role creation](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/security-api-put-role.html) + +Once the role is created, you can create a user for the Mastodon server to use, and assign it the role. + +For example (please adapt this snippet to use your Elastic admin password, and customize your new user `mastodon` user password): + +```sh +curl -X POST -u elastic:admin_password "localhost:9200/_security/user/mastodon?pretty" -H 'Content-Type: application/json' -d' +{ + "password" : "l0ng-r4nd0m-p@ssw0rd", + "roles" : ["mastodon_full_access"] +} +' +``` + +[Elasticsearch documentation for user creation](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/security-api-put-user.html) + +Once this is done, you need to configure Mastodon to use the credentials for your newly created user. + +In `.env.production`, adjust your configuration: + +```bash +ES_USER=mastodon +ES_PASS=l0ng-r4nd0m-p@ssw0rd +``` + +You are all set, and your Elasticsearch server should be much more secure! + +### Populate the indices + +After saving the new configuration, restart Mastodon processes for it to take effect: + +```bash +systemctl restart mastodon-sidekiq +systemctl reload mastodon-web +``` + +Now it's time to create the Elasticsearch indices and fill them with data: + +```bash +su - mastodon +cd live +RAILS_ENV=production bin/tootctl search deploy +``` + +## Search optimization for other languages +### Chinese search optimization {#chinese-search-optimization} + +The default analyzer of the Elasticsearch is the standard analyzer, which may not be the best especially for Chinese. To improve search experience, you can install a language specific analyzer. Before creating the indices in Elasticsearch, install the following Elasticsearch extensions: + +- [elasticsearch-analysis-ik](https://github.com/medcl/elasticsearch-analysis-ik) +- [elasticsearch-analysis-stconvert](https://github.com/medcl/elasticsearch-analysis-stconvert) + +And then modify Mastodon's index definition as follows: + +```diff +diff --git a/app/chewy/accounts_index.rb b/app/chewy/accounts_index.rb +--- a/app/chewy/accounts_index.rb ++++ b/app/chewy/accounts_index.rb +@@ -4,7 +4,7 @@ class AccountsIndex < Chewy::Index + settings index: { refresh_interval: '5m' }, analysis: { + analyzer: { + content: { +- tokenizer: 'whitespace', ++ tokenizer: 'ik_max_word', + filter: %w(lowercase asciifolding cjk_width), + }, + +diff --git a/app/chewy/statuses_index.rb b/app/chewy/statuses_index.rb +--- a/app/chewy/statuses_index.rb ++++ b/app/chewy/statuses_index.rb +@@ -16,9 +16,17 @@ class StatusesIndex < Chewy::Index + language: 'possessive_english', + }, + }, ++ char_filter: { ++ tsconvert: { ++ type: 'stconvert', ++ keep_both: false, ++ delimiter: '#', ++ convert_type: 't2s', ++ }, ++ }, + analyzer: { + content: { +- tokenizer: 'uax_url_email', ++ tokenizer: 'ik_max_word', + filter: %w( + english_possessive_stemmer + lowercase +@@ -27,6 +35,7 @@ class StatusesIndex < Chewy::Index + english_stop + english_stemmer + ), ++ char_filter: %w(tsconvert), + }, + }, + } +diff --git a/app/chewy/tags_index.rb b/app/chewy/tags_index.rb +--- a/app/chewy/tags_index.rb ++++ b/app/chewy/tags_index.rb +@@ -2,10 +2,19 @@ + + class TagsIndex < Chewy::Index + settings index: { refresh_interval: '15m' }, analysis: { ++ char_filter: { ++ tsconvert: { ++ type: 'stconvert', ++ keep_both: false, ++ delimiter: '#', ++ convert_type: 't2s', ++ }, ++ }, + analyzer: { + content: { +- tokenizer: 'keyword', ++ tokenizer: 'ik_max_word', + filter: %w(lowercase asciifolding cjk_width), ++ char_filter: %w(tsconvert), + }, + + edge_ngram: { +``` diff --git a/content/en/admin/install.md b/content/en/admin/install.md index 2105f536..db9ca626 100644 --- a/content/en/admin/install.md +++ b/content/en/admin/install.md @@ -84,8 +84,8 @@ git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build Once this is done, we can install the correct Ruby version: ```bash -RUBY_CONFIGURE_OPTS=--with-jemalloc rbenv install 3.0.4 -rbenv global 3.0.4 +RUBY_CONFIGURE_OPTS=--with-jemalloc rbenv install 3.2.2 +rbenv global 3.2.2 ``` We’ll also need to install bundler: @@ -106,7 +106,7 @@ exit #### Performance configuration (optional) {#performance-configuration-optional} -For optimal performance, you may use [pgTune](https://pgtune.leopard.in.ua/#/) to generate an appropriate configuration and edit values in `/etc/postgresql/15/main/postgresql.conf` before restarting PostgreSQL with `systemctl restart postgresql` +For optimal performance, you may use [pgTune](https://pgtune.leopard.in.ua/#/) to generate an appropriate configuration and edit values in `/etc/postgresql/16/main/postgresql.conf` before restarting PostgreSQL with `systemctl restart postgresql` #### Creating a user {#creating-a-user} @@ -141,7 +141,7 @@ Use git to download the latest stable release of Mastodon: ```bash git clone https://github.com/mastodon/mastodon.git live && cd live -git checkout $(git tag -l | grep -v 'rc[0-9]*$' | sort -V | tail -n 1) +git checkout $(git tag -l | grep '^v[0-9.]*$' | sort -V | tail -n 1) ``` #### Installing the last dependencies {#installing-the-last-dependencies} @@ -181,6 +181,16 @@ You’re done with the mastodon user for now, so switch back to root: exit ``` +### Acquiring a SSL certificate {#acquiring-a-ssl-certificate} + +We’ll use Let’s Encrypt to get a free SSL certificate: + +```bash +certbot certonly --nginx -d example.com +``` + +This will obtain the certificate, and save it in the directory `/etc/letsencrypt/live/example.com/`. + ### Setting up nginx {#setting-up-nginx} Copy the configuration template for nginx from the Mastodon directory: @@ -192,23 +202,14 @@ ln -s /etc/nginx/sites-available/mastodon /etc/nginx/sites-enabled/mastodon Then edit `/etc/nginx/sites-available/mastodon` to replace `example.com` with your own domain name, and make any other adjustments you might need. -Reload nginx for the changes to take effect: +Un-comment the lines starting with `ssl_certificate` and `ssl_certificate_key`, updating the path with the correct domain name. +Reload nginx for the changes to take effect: ```bash systemctl reload nginx ``` -### Acquiring a SSL certificate {#acquiring-a-ssl-certificate} - -We’ll use Let’s Encrypt to get a free SSL certificate: - -```bash -certbot --nginx -d example.com -``` - -This will obtain the certificate, automatically update `/etc/nginx/sites-available/mastodon` to use the new certificate, and reload nginx for the changes to take effect. - At this point you should be able to visit your domain in the browser and see the elephant hitting the computer screen error page. This is because we haven’t started the Mastodon process yet. ### Setting up systemd services {#setting-up-systemd-services} diff --git a/content/en/admin/moderation.md b/content/en/admin/moderation.md index ebf15e52..ed7c1565 100644 --- a/content/en/admin/moderation.md +++ b/content/en/admin/moderation.md @@ -41,6 +41,8 @@ If the account is reinstated within the 30 day period, all data is once again ac Once the data has been deleted, whether that is after the 30 day period, or if an admin has force deleted it, the account can still be un-suspended. However, the account will have no data (statuses, profile information, avatar or header image) associated with it. +For remote accounts, suspending will make them unfollow any local account. Those relationships are not restored in case the remote account is un-suspended, even within the 30-day time window. + ## Moderating entire websites {#server-wide-moderation} Because individually moderating a large volume of users from a misbehaving server can be exhausting, it is possible to pre-emptively moderate against all users from that particular server using a so-called **domain block**, which comes with several different levels of severity. @@ -57,6 +59,8 @@ Equivalent to [limiting](#limit-user) all past and future accounts from the serv Equivalent to [suspending](#suspend-user) all past and future accounts from the server. No content from the server will be stored locally except for usernames. +Suspending a server will remove all existing follow relationships between local accounts and accounts on the suspended server. They will not be restored in case the remote server is un-suspended later. + ## Spam-fighting measures {#spam-fighting-measures} There are a few baseline measures for preventing spam in Mastodon: diff --git a/content/en/admin/optional.md b/content/en/admin/optional.md index 3439f0a1..2b15bae7 100644 --- a/content/en/admin/optional.md +++ b/content/en/admin/optional.md @@ -9,6 +9,6 @@ menu: Mastodon offers a few optional features that can be used if needed. -- [Full-text search](./elasticsearch/) +- [Object storage](./object-storage/) - [Hidden services](./tor/) - [Single Sign On](./sso/) diff --git a/content/en/admin/optional/elasticsearch.md b/content/en/admin/optional/elasticsearch.md deleted file mode 100644 index e6c52bf6..00000000 --- a/content/en/admin/optional/elasticsearch.md +++ /dev/null @@ -1,154 +0,0 @@ ---- -title: Full-text search -description: Setting up ElasticSearch to search for statuses authored, favourited, or mentioned in. -menu: - docs: - weight: 10 - parent: admin-optional ---- - -Mastodon supports full-text search when ElasticSearch is available. Mastodon’s full-text search allows logged in users to find results from their own statuses, their mentions, their favourites, and their bookmarks. It deliberately does not allow searching for arbitrary strings in the entire database. - -## Installing ElasticSearch {#install} - -ElasticSearch requires a Java runtime. If you don’t have Java already installed, do it now. Assuming you are logged in as `root`: - -```bash -apt install openjdk-17-jre-headless -``` - -Add the official ElasticSearch repository to apt: - -```bash -wget -O /usr/share/keyrings/elasticsearch.asc https://artifacts.elastic.co/GPG-KEY-elasticsearch -echo "deb [signed-by=/usr/share/keyrings/elasticsearch.asc] https://artifacts.elastic.co/packages/7.x/apt stable main" > /etc/apt/sources.list.d/elastic-7.x.list -``` - -Now you can install ElasticSearch: - -```bash -apt update -apt install elasticsearch -``` - -{{< hint style="warning" >}} -**Security warning:** By default, ElasticSearch is supposed to bind to localhost only, i.e. be inaccessible from the outside network. You can check which address ElasticSearch binds to by looking at `network.host` within `/etc/elasticsearch/elasticsearch.yml`. Consider that anyone who can access ElasticSearch can access and modify any data within it, as there is no authentication layer. So it’s really important that the access is secured. Having a firewall that only exposes the 22, 80 and 443 ports is advisable, as outlined in the [main installation instructions](../../prerequisites/#install-a-firewall-and-only-whitelist-ssh-http-and-https-ports). If you have a multi-host setup, you must know how to secure internal traffic. -{{< /hint >}} - -{{< hint style="danger" >}} -**Security warning:** ElasticSearch versions between `2.0` and `2.14.1` are affected by an [exploit](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44228) in the `log4j` library. If affected, please refer to the [temporary mitigation](https://github.com/elastic/elasticsearch/issues/81618#issuecomment-991000240) from the ElasticSearch issue tracker. -{{< /hint >}} - -To start ElasticSearch: - -```bash -systemctl daemon-reload -systemctl enable --now elasticsearch -``` - -## Configuring Mastodon {#config} - -Edit `.env.production` to add the following variables: - -```bash -ES_ENABLED=true -ES_HOST=localhost -ES_PORT=9200 -``` - -If you have multiple Mastodon servers on the same machine, and you are planning to use the same ElasticSearch installation for all of them, make sure that all of them have unique `REDIS_NAMESPACE` in their configurations, to differentiate the indices. If you need to override the prefix of the ElasticSearch indices, you can set `ES_PREFIX` directly. - -After saving the new configuration, restart Mastodon processes for it to take effect: - -```bash -systemctl restart mastodon-sidekiq -systemctl reload mastodon-web -``` - -Now it's time to create the ElasticSearch indices and fill them with data: - -```bash -su - mastodon -cd live -RAILS_ENV=production bin/tootctl search deploy -``` - -## Search optimization for other languages -### Chinese search optimization {#chinese-search-optimization} - -The default analyzer of the ElasticSearch is the standard analyzer, which may not be the best especially for Chinese. To improve search experience, you can install a language specific analyzer. Before creating the indices in ElasticSearch, install the following ElasticSearch extensions: - -- [elasticsearch-analysis-ik](https://github.com/medcl/elasticsearch-analysis-ik) -- [elasticsearch-analysis-stconvert](https://github.com/medcl/elasticsearch-analysis-stconvert) - -And then modify Mastodon's index definition as follows: - -```diff -diff --git a/app/chewy/accounts_index.rb b/app/chewy/accounts_index.rb ---- a/app/chewy/accounts_index.rb -+++ b/app/chewy/accounts_index.rb -@@ -4,7 +4,7 @@ class AccountsIndex < Chewy::Index - settings index: { refresh_interval: '5m' }, analysis: { - analyzer: { - content: { -- tokenizer: 'whitespace', -+ tokenizer: 'ik_max_word', - filter: %w(lowercase asciifolding cjk_width), - }, - -diff --git a/app/chewy/statuses_index.rb b/app/chewy/statuses_index.rb ---- a/app/chewy/statuses_index.rb -+++ b/app/chewy/statuses_index.rb -@@ -16,9 +16,17 @@ class StatusesIndex < Chewy::Index - language: 'possessive_english', - }, - }, -+ char_filter: { -+ tsconvert: { -+ type: 'stconvert', -+ keep_both: false, -+ delimiter: '#', -+ convert_type: 't2s', -+ }, -+ }, - analyzer: { - content: { -- tokenizer: 'uax_url_email', -+ tokenizer: 'ik_max_word', - filter: %w( - english_possessive_stemmer - lowercase -@@ -27,6 +35,7 @@ class StatusesIndex < Chewy::Index - english_stop - english_stemmer - ), -+ char_filter: %w(tsconvert), - }, - }, - } -diff --git a/app/chewy/tags_index.rb b/app/chewy/tags_index.rb ---- a/app/chewy/tags_index.rb -+++ b/app/chewy/tags_index.rb -@@ -2,10 +2,19 @@ - - class TagsIndex < Chewy::Index - settings index: { refresh_interval: '15m' }, analysis: { -+ char_filter: { -+ tsconvert: { -+ type: 'stconvert', -+ keep_both: false, -+ delimiter: '#', -+ convert_type: 't2s', -+ }, -+ }, - analyzer: { - content: { -- tokenizer: 'keyword', -+ tokenizer: 'ik_max_word', - filter: %w(lowercase asciifolding cjk_width), -+ char_filter: %w(tsconvert), - }, - - edge_ngram: { -``` - diff --git a/content/en/admin/optional/object-storage-proxy.md b/content/en/admin/optional/object-storage-proxy.md index b29842cc..8bc00bf9 100644 --- a/content/en/admin/optional/object-storage-proxy.md +++ b/content/en/admin/optional/object-storage-proxy.md @@ -37,7 +37,7 @@ server { } resolver 8.8.8.8; - proxy_set_header Host YOUR_S3_HOSTNAME; + proxy_set_header Host YOUR_BUCKET_NAME.YOUR_S3_HOSTNAME; proxy_set_header Connection ''; proxy_set_header Authorization ''; proxy_hide_header Set-Cookie; @@ -63,6 +63,8 @@ server { add_header Cache-Control public; add_header 'Access-Control-Allow-Origin' '*'; add_header X-Cache-Status $upstream_cache_status; + add_header X-Content-Type-Options nosniff; + add_header Content-Security-Policy "default-src 'none'; form-action 'none'"; } } ``` diff --git a/content/en/admin/optional/object-storage.md b/content/en/admin/optional/object-storage.md new file mode 100644 index 00000000..668a2a28 --- /dev/null +++ b/content/en/admin/optional/object-storage.md @@ -0,0 +1,232 @@ +--- +title: Configuring object storage +description: Serving user-uploaded files in Mastodon using external object storage +menu: + docs: + weight: 15 + parent: admin-optional +--- + +User-uploaded files can be stored on the main server's file system, or using an external object storage server, which can be required for scaling. + +## Using the filesystem {#FS} + +The simplest way to store user uploads is by using the server's file system. This is how it works by default and is suitable for small servers. + +By default, Mastodon will store file uploads under `public/system` in its installation directory, but that can be overridden using the `PAPERCLIP_ROOT_PATH` environment variable. + +By default, the files are served at `https://your-domain/system`, which can be overridden using `PAPERCLIP_ROOT_URL` and `CDN_HOST`. + +{{< hint style="info" >}} +While using the server's file system is perfectly serviceable for small servers, using external object storage is more scalable. +{{}} + +{{< hint style="danger" >}} +The web server must be configured to serve those files but not allow listing them (that is, `https://your-domain/system/` should not return a file list). This should be the case if you use the configuration files distributed with Mastodon, but it is worth double-checking. +{{}} + +## S3-compatible object storage backends {#S3} + +Mastodon can use S3-compatible object storage backends. ACL support is recommended as it allows Mastodon to quickly make the content of temporarily suspended users unavailable, or marginally improve security of private data. + +On Mastodon's end, you need to configure the following environment variables: +- `S3_ENABLED=true` +- `S3_BUCKET=mastodata` (replacing `mastodata` with the name of your bucket) +- `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` need to be set to your credentials +- `S3_ALIAS_HOST` is optional but highly recommended in order to set up a caching proxy and not lock you to a specific provider +- `S3_REGION` +- `S3_HOSTNAME` (optional if you use Amazon AWS) +- `S3_PERMISSION` (optional, if you use a provider that does not support ACLs or want to use custom ACLs) +- `S3_FORCE_SINGLE_REQUEST=true` (optional, if you run in trouble processing large files) + +{{< page-ref page="admin/optional/object-storage-proxy.md" >}} + +{{< hint style="info" >}} +You must serve the files with CORS headers, otherwise some functions of Mastodon's web UI will not work. For example, `Access-Control-Allow-Origin: *` +{{}} + +{{< hint style="danger" >}} +In any case, your S3 bucket must be configured so that -- ACL configuration nonwithstanding -- all objects are publicly readable but neither writable or listable, while Mastodon itself can write to it. The configuration should be similar for all S3 providers, but common ones have been highlighted below. +{{}} + +### MinIO + +MinIO is an open source implementation of an S3 object provider. This section does not cover how to install it, but how to configure a bucket for use in Mastodon. + +You need to set a policy for anonymous access that allows read-only access to objects contained by the bucket without allowing listing them. + +To do this, you need to set a custom policy (replace `mastodata` by the actual name of your S3 bucket): +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "AWS": "*" + }, + "Action": "s3:GetObject", + "Resource": "arn:aws:s3:::mastodata/*" + } + ] +} +``` + +Mastodon itself needs to be able to write to the bucket, so either use your admin MinIO account (discouraged) or an account specific to Mastodon (recommended) with the following policy attached (replace `mastodata` by the actual name of your S3 bucket): +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": "s3:*", + "Resource": "arn:aws:s3:::mastodata/*" + } + ] +} +``` + +You can set those policies from the MinIO Console (web-based user interface) or the command-line client (`mcli` / `mc`). + +#### Using the MinIO Console + +Connect to the MinIO Console web interface and create a new bucket (or navigate to your existing bucket): +![](/assets/object-storage/minio-bucket.png) + +Then, configure the “Access Policy” to a custom one that allows read access (`s3:GetObject`) without write access or ability to list objects (see above): +![](/assets/object-storage/minio-access-policy.png) + +{{< hint style="info" >}} +If the MinIO Console does not allow you to set a “Custom” policy, you will likely need to update MinIO. If you are using MinIO in *standalone* or *filesystem* mode, [`RELEASE.2022-10-24T18-35-07Z`](https://github.com/minio/minio/releases/tag/RELEASE.2022-10-24T18-35-07Z) should be a safe version to update to that does not require [an involved migration procedure](https://min.io/docs/minio/linux/operations/install-deploy-manage/migrate-fs-gateway.html#migrate-from-gateway-or-filesystem-mode). +{{< /hint >}} + +Create a new `mastodon-readwrite` policy (see above): +![](/assets/object-storage/minio-mastodon-readwrite.png) + +Finally, create a new `mastodon` user with the `mastodon-readwrite` policy: +![](/assets/object-storage/minio-mastodon-user.png) + +#### Using the command-line utility + +The same can be achieved using the [MinIO Client](https://min.io/docs/minio/linux/reference/minio-mc.html) command-line utility (can be called `mc` or `mcli` depending on where it is installed from). + +Create a new bucket: +`mc mb myminio/mastodata` + +Save the anonymous access policy from above as `anonymous-readonly-policy.json` and the Mastodon user access policy as `mastodon-readwrite.json` (make sure to replace `mastodata` with the name of your newly-created bucket). + +Set the anonymous access policy for your bucket: +`mc anonymous set-json anonymous-readonly-policy.json myminio/mastodata` + +Add a `mastodon-readwrite` policy: +`mc admin policy add myminio mastodon-readwrite mastodon-readwrite.json` + +Add the `mastodon` user (replace the password): +`mc admin user add myminio mastodon SECRET_PASSWORD` + +Apply the `mastodon-readwrite` policy to the `mastodon` user: +`mc admin policy set myminio mastodon-readwrite user=mastodon` + +### Wasabi Object Storage + +Create a new bucket and define its policy to allow objects to be anonymously readable but not listable: +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "AWS": "*" + }, + "Action": "s3:GetObject", + "Resource": "arn:aws:s3:::mastodata/*" + } + ] +} +``` + +![](/assets/object-storage/wasabi-access-policy.png) + +{{< hint style="info" >}} +If you are using an old bucket, ensure you are not giving “Everyone” read access to objects through Wasabi's legacy Access Control settings, as that allows listing objects and take precedence over the IAM policy defined above. + +![](/assets/object-storage/wasabi-access-control.png) +{{< /hint >}} + +Then, create a `mastodon-readwrite` policy to grant read and write access to your bucket: +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": "s3:*", + "Resource": "arn:aws:s3:::mastodata/*" + } + ] +} +``` + +![](/assets/object-storage/wasabi-mastodon-readwrite.png) + +Finally, create a new `mastodon` user and don't forget to enable the `mastodon-readwrite` policy: +![](/assets/object-storage/wasabi-mastodon-user.png) + +On Mastodon's side, you need to set `S3_FORCE_SINGLE_REQUEST=true` to properly handle large uploads. + +### DigitalOcean Spaces + +In your DigitalOcean Spaces Bucket, make sure that “File Listing” is “Restricted” to users with access keys. + +![](/assets/object-storage/do-spaces.png) + +### Scaleway + +If you want to use Scaleway Object Storage, we strongly recommend you to create a Scaleway project dedicaced to your Mastodon instance assets and to use a custom IAM policy. + +First, create a new Scaleway project, in which you create your object storage bucket. You need to set your bucket visibility to "Private" to not allow objects to be listed. + +![](/assets/object-storage/scaleway-bucket.png) + +Now that your bucket is created, you need to create API keys to be used in your Mastodon instance configuration. + +Head to the IAM settings (in your organisation menu, top right of the screen), and create a new IAM policy (eg `mastodon-media-access`) + +![](/assets/object-storage/scaleway-policy.jpg) + +This policy needs to have one rule, allowing it to read, write and delete objects in the Scaleway project you created above (the scope). + +![](/assets/object-storage/scaleway-policy-rules.jpg) + +Then head to the IAM Applications page, and a create a new one (eg `my-mastodon-instance`) and select the policy you created above. + +Finally, click on the application you just created, then "API Keys", and create a new API key to use in your instance configuration. You should use the "Yes, set up preferred Project" option and select the project you created above as the default project for this key. + +![](/assets/object-storage/scaleway-api-key.png) + +Copy the Access Key ID and Secret, and use them for your `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` Mastodon config variables. + +### Exoscale + +In Exoscale, your bucket should not have any read ACLs (Mastodon will set the ACLs on the object themselves as appropriate). + +You need to create an API Key for the Mastodon app, restricted to the Object Storage (`sos`) service, restricted to your bucket, and with unrestricted operations. + +![](/assets/object-storage/exoscale.png) + +On Mastodon's side, you need to set `S3_FORCE_SINGLE_REQUEST=true` to properly handle large uploads. + +### Cloudflare R2 + +Cloudflare R2 does not support ACLs, so Mastodon needs to be instructed not to try setting them. To do that, set the `S3_PERMISSION` environment variable to an empty string. + +{{< hint style="warning" >}} +Without support for ACLs, media files from temporarily-suspended users will remain accessible. +{{< /hint >}} + +To get credentials for use in Mastodon, selecte “Manage R2 API Tokens” and create a new API token with “Edit” permissions. + +{{< hint style="warning" >}} +This section is currently under construction. +{{< /hint >}} diff --git a/content/en/admin/optional/tor.md b/content/en/admin/optional/tor.md index 67c1d795..f49847fd 100644 --- a/content/en/admin/optional/tor.md +++ b/content/en/admin/optional/tor.md @@ -1,6 +1,6 @@ --- -title: Hidden services -description: Serving Mastodon through TOR hidden services. +title: Onion services +description: Serving Mastodon through Tor onion services. menu: docs: weight: 20 @@ -36,7 +36,7 @@ apt install tor deb.torproject.org-keyring Edit the file at `/etc/tor/torrc` and add the following configuration. ```text -HiddenServiceDir /var/lib/tor/hidden_service/ +HiddenServiceDir /var/lib/tor/onion_service/ HiddenServiceVersion 3 HiddenServicePort 80 127.0.0.1:80 ``` @@ -47,7 +47,7 @@ Restart tor. sudo service tor restart ``` -Your tor hostname can now be found at `/var/lib/tor/hidden_service/hostname`. +Your tor hostname can now be found at `/var/lib/tor/onion_service/hostname`. ## Move your Mastodon configuration {#nginx} @@ -134,7 +134,7 @@ server { } ``` -Replace the long hash provided here with your Tor domain located in the file at `/var/lib/tor/hidden_service/hostname`. +Replace the long hash provided here with your Tor domain located in the file at `/var/lib/tor/onion_service/hostname`. Note that the onion hostname has been prefixed with “mastodon.”. Your Tor address acts a wildcard domain. All subdomains will be routed through, and you can configure Nginx to respond to any subdomain you wish. If you do not wish to host any other services on your tor address you can omit the subdomain, or choose a different subdomain. diff --git a/content/en/admin/scaling.md b/content/en/admin/scaling.md index 3494d69c..15e63854 100644 --- a/content/en/admin/scaling.md +++ b/content/en/admin/scaling.md @@ -11,16 +11,16 @@ menu: Mastodon has three types of processes: -* Web (Puma) -* Streaming API -* Background processing (Sidekiq) +- Web (Puma) +- Streaming API +- Background processing (Sidekiq) ### Web (Puma) {#web} The web process serves short-lived HTTP requests for most of the application. The following environment variables control it: -* `WEB_CONCURRENCY` controls the number of worker processes -* `MAX_THREADS` controls the number of threads per process +- `WEB_CONCURRENCY` controls the number of worker processes +- `MAX_THREADS` controls the number of threads per process Threads share the memory of their parent process. Different processes allocate their own memory, though they share some memory via copy-on-write. A larger number of threads maxes out your CPU first, a larger number of processes maxes out your RAM first. @@ -32,10 +32,36 @@ In terms of throughput, more processes are better than more threads. The streaming API handles long-lived HTTP and WebSockets connections, through which clients receive real-time updates. The following environment variables control it: -* `STREAMING_CLUSTER_NUM` controls the number of worker processes -* `STREAMING_API_BASE_URL` controls the base URL of the streaming API +- `STREAMING_API_BASE_URL` controls the base URL of the streaming API +- `PORT` controls the port the streaming server will listen on, by default 4000. The `BIND` and `SOCKET` environment variables are also able to be used. +- Additionally the shared [database](/admin/config#postgresql) and [redis](/admin/config#redis) environment variables are used. -One process can handle a reasonably high number of connections. The streaming API can be hosted on a different subdomain if you want to e.g. avoid the overhead of nginx proxying the connections. +The streaming API can be use a different subdomain if you want to by setting `STREAMING_API_BASE_URL`, this allows you to have one load balancer for streaming and one for web/API requests. + +{{< hint style="warning" >}} +Previous versions of Mastodon had a `STREAMING_CLUSTER_NUM` environment variable that made the streaming server use clustering, which started mulitple processes (workers) and used node.js to load balance them. + +This interacted with the other settings in ways which made capacity planning difficult, especially when it comes to database connections and CPU resources. By default the streaming server would consume resources on all available CPUs which could cause contention with other software running on that server. Another common issue was that misconfiguring the `STREAMING_CLUSTER_NUM` would exhaust your database connections by opening up a connection pool per cluster worker process, so a `STREAMING_CLUSTER_NUM` of `5` and `DB_POOL` of `10` would potentially consume 50 database connections. + +Now a single streaming server process will only use at maximum `DB_POOL` PostgreSQL connections, and scaling is handled by running more instances of the streaming server. +{{< /hint >}} + +One process can handle a reasonably high number of connections and throughput, but if you find that a single streaming server process isn't handling your instance's load, you can run multiple processes by varying the `PORT` number of each, and then using nginx to load balance traffic to each of those instances. + +{{< hint style="info" >}} +The more streaming server processes that you run, the more database connections will be consumed on PostgreSQL, so you'll likely want to use PgBouncer, as documented below. +{{< /hint >}} + +An example nginx configuration to route traffic to three different processes on `PORT` 4000, 4001, and 4002 is as follows: + +``` +upstream streaming { + least_conn; + server 127.0.0.1:4000 fail_timeout=0; + server 127.0.0.1:4001 fail_timeout=0; + server 127.0.0.1:4002 fail_timeout=0; +} +``` ### Background processing (Sidekiq) {#sidekiq} @@ -57,14 +83,14 @@ Would start the sidekiq process with 15 threads. Please mind that each threads n Sidekiq uses different queues for tasks of varying importance, where importance is defined by how much it would impact the user experience of your server’s local users if the queue wasn’t working, in order of descending importance: -| Queue | Significance | -| :--- | :--- | -| `default` | All tasks that affect local users | -| `push` | Delivery of payloads to other servers | -| `mailers` | Delivery of e-mails | -| `pull` | Lower priority tasks such as handling imports, backups, resolving threads, deleting users, forwarding replies | -| `scheduler` | Doing cron jobs like refreshing trending hashtags and cleaning up logs | -| `ingress` | Incoming remote activities. Lower priority than the default queue so local users still see their posts when the server is under load | +| Queue | Significance | +| :---------- | :----------------------------------------------------------------------------------------------------------------------------------- | +| `default` | All tasks that affect local users | +| `push` | Delivery of payloads to other servers | +| `mailers` | Delivery of e-mails | +| `pull` | Lower priority tasks such as handling imports, backups, resolving threads, deleting users, forwarding replies | +| `scheduler` | Doing cron jobs like refreshing trending hashtags and cleaning up logs | +| `ingress` | Incoming remote activities. Lower priority than the default queue so local users still see their posts when the server is under load | The default queues and their priorities are stored in [config/sidekiq.yml](https://github.com/mastodon/mastodon/blob/main/config/sidekiq.yml), but can be overridden by the command-line invocation of Sidekiq, e.g.: @@ -80,7 +106,6 @@ As a solution, it is possible to start different Sidekiq processes for the queue **Make sure you only have one `scheduler` queue running!!** - ## Transaction pooling with pgBouncer {#pgbouncer} ### Why you might need PgBouncer {#pgbouncer-why} @@ -167,10 +192,22 @@ listen_port = 6432 Put `md5` as the `auth_type` (assuming you’re using the md5 format in `userlist.txt`): +```ini +auth_type = md5 +``` + Make sure the `pgbouncer` user is an admin: +```ini +admin_users = pgbouncer +``` + **This next part is very important!** The default pooling mode is session-based, but for Mastodon we want transaction-based. In other words, a Postgres connection is created when a transaction is created and dropped when the transaction is done. So you’ll want to change the `pool_mode` from `session` to `transaction`: +```ini +pool_mode = transaction +``` + Next up, `max_client_conn` defines how many connections PgBouncer itself will accept, and `default_pool_size` puts a limit on how many Postgres connections will be opened under the hood. (In PgHero the number of connections reported will correspond to `default_pool_size` because it has no knowledge of PgBouncer.) The defaults are fine to start, and you can always increase them later: @@ -258,10 +295,14 @@ As far as configuring the Redis database goes, basically you can get rid of back To reduce the load on your Postgresql server, you may wish to setup hot streaming replication (read replica). [See this guide for an example](https://cloud.google.com/community/tutorials/setting-up-postgres-hot-standby). You can make use of the replica in Mastodon in these ways: -* The streaming API server does not issue writes at all, so you can connect it straight to the replica. But it’s not querying the database very often anyway so the impact of this is little. -* Use the Makara driver in the web and sidekiq processes, so that writes go to the master database, while reads go to the replica. Let’s talk about that. +- The streaming API server does not issue writes at all, so you can connect it straight to the replica. But it’s not querying the database very often anyway so the impact of this is little. +- Use the Makara driver in the web processes, so that writes go to the primary database, while reads go to the replica. Let’s talk about that. -You will have to edit the `config/database.yml` file and replace the `production` section as follows: +{{< hint style="warning" >}} +Read replicas are currently not supported for the Sidekiq processes, and using them will lead to failing jobs and data loss. +{{< /hint >}} + +You will have to use a separate `config/database.yml` file for the web processes and edit it to replace the `production` section as follows: ```yaml production: @@ -279,9 +320,8 @@ production: url: postgresql://db_user:db_password@db_host:db_port/db_name ``` -Make sure the URLs point to wherever your PostgreSQL servers are. You can add multiple replicas. You could have a locally installed pgBouncer with configuration to connect to two different servers based on database name, e.g. “mastodon” going to master, “mastodon_replica” going to the replica, so in the file above both URLs would point to the local pgBouncer with the same user, password, host and port, but different database name. There are many possibilities how this could be setup! For more information on Makara, [see their documentation](https://github.com/taskrabbit/makara#databaseyml). +Make sure the URLs point to wherever your PostgreSQL servers are. You can add multiple replicas. You could have a locally installed pgBouncer with configuration to connect to two different servers based on database name, e.g. “mastodon” going to the primary, “mastodon_replica” going to the replica, so in the file above both URLs would point to the local pgBouncer with the same user, password, host and port, but different database name. There are many possibilities how this could be setup! For more information on Makara, [see their documentation](https://github.com/taskrabbit/makara#databaseyml). {{< hint style="warning" >}} -Sidekiq cannot reliably use read-replicas because even the tiniest replication lag leads to failing jobs due to queued up records not being found. +Make sure the sidekiq processes run with the stock `config/database.yml` to avoid failing jobs and data loss! {{< /hint >}} - diff --git a/content/en/admin/tootctl.md b/content/en/admin/tootctl.md index 49cd6b3e..cddc70a8 100644 --- a/content/en/admin/tootctl.md +++ b/content/en/admin/tootctl.md @@ -22,7 +22,7 @@ RAILS_ENV=production bin/tootctl help ## Base CLI -{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/lib/cli.rb" caption="lib/cli.rb" >}} +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/lib/mastodon/cli/base.rb" caption="lib/mastodon/cli/base.rb" >}} --- @@ -61,7 +61,7 @@ Show the version of the currently running Mastodon instance. ## Accounts CLI {#accounts} -{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/lib/mastodon/accounts_cli.rb" caption="lib/mastodon/accounts_cli.rb" >}} +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/lib/mastodon/cli/accounts.rb" caption="lib/mastodon/cli/accounts.rb" >}} --- @@ -127,6 +127,9 @@ Modify a user account's role, email, active status, approval mode, or 2FA requir `--role ROLE` : Define the existing account's custom role by providing the `name` of that [Role]({{< relref "entities/Role" >}}). Default roles include `Owner`, `Admin`, and `Moderator` (case-sensitive). +`--remove-role` +: Removes the current role from the user. + `--email EMAIL` : Update the user's email address to `EMAIL`. @@ -154,7 +157,7 @@ Modify a user account's role, email, active status, approval mode, or 2FA requir **Version history:**\ 2.6.0 - added\ 3.1.2 - added `--reset-password`\ -4.0.0 - `--role` no longer takes hard-coded `user`, `moderator`, or `admin` roles. Specify the name of the custom Role instead (case-sensitive). +4.0.0 - `--role` no longer takes hard-coded `user`, `moderator`, or `admin` roles. Specify the name of the custom Role instead (case-sensitive). Remove the current role with `--remove-role`. --- @@ -328,7 +331,7 @@ Approve new registrations when instance is in approval mode. : Local username. `--number N` -: Approve the N most recent registrations. +: Approve the N earliest pending registrations. `--all` : Approve all pending registrations. @@ -342,7 +345,7 @@ Approve new registrations when instance is in approval mode. ## Cache CLI {#cache} -{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/lib/mastodon/cache_cli.rb" caption="lib/mastodon/cache_cli.rb" >}} +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/lib/mastodon/cli/cache.rb" caption="lib/mastodon/cli/cache.rb" >}} --- @@ -381,7 +384,7 @@ Update hard-cached counters of TYPE by counting referenced records from scratch. ## Domains CLI {#domains} -{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/lib/mastodon/domains_cli.rb" caption="lib/mastodon/domains_cli.rb" >}} +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/lib/mastodon/cli/domains.rb" caption="lib/mastodon/cli/domains.rb" >}} --- @@ -447,7 +450,7 @@ Crawl the known fediverse by using Mastodon REST API endpoints that expose all k ## Email domain blocks CLI {#email-domain-blocks} -{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/lib/mastodon/email_domain_blocks_cli.rb" caption="lib/mastodon/email_domain_blocks_cli.rb" >}} +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/lib/mastodon/cli/email_domain_blocks.rb" caption="lib/mastodon/cli/email_domain_blocks.rb" >}} --- @@ -497,7 +500,7 @@ Remove entries from the e-mail domain blocklist. ## Emoji CLI {#emoji} -{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/lib/mastodon/emoji_cli.rb" caption="lib/mastodon/emoji_cli.rb" >}} +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/lib/mastodon/cli/emoji.rb" caption="lib/mastodon/cli/emoji.rb" >}} --- @@ -569,7 +572,7 @@ Remove all custom emoji. ## Feeds CLI {#feeds} -{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/lib/mastodon/feeds_cli.rb" caption="lib/mastodon/feeds_cli.rb" >}} +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/lib/mastodon/cli/feeds.rb" caption="lib/mastodon/cli/feeds.rb" >}} --- @@ -614,7 +617,7 @@ Remove all home and list feeds from Redis. ## Maintenance CLI {#maintenance} -{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/lib/mastodon/maintenance_cli.rb" caption="lib/mastodon/maintenance_cli.rb" >}} +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/lib/mastodon/cli/maintenance.rb" caption="lib/mastodon/cli/maintenance.rb" >}} --- @@ -633,7 +636,7 @@ Fix corrupted database indexes that may have been caused due to changing collati ## Media CLI {#media} -{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/lib/mastodon/media_cli.rb" caption="lib/mastodon/media_cli.rb" >}} +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/lib/mastodon/cli/media.rb" caption="lib/mastodon/cli/media.rb" >}} --- @@ -641,14 +644,23 @@ Fix corrupted database indexes that may have been caused due to changing collati ### `tootctl media remove` {#media-remove} -Remove locally cached copies of media attachments from other servers. +Removes locally cached copies of media attachments, avatars or profile headers from other servers. By default, only media attachments are removed. `--days N` -: How old media attachments have to be before they are removed. Defaults to 7. +: How old media attachments have to be before they are removed. In case of avatars and headers, how old the last webfinger request and update to the user has to be before they are removed. Defaults to 7. `--concurrency N` : The number of workers to use for this task. Defaults to N=5. +`--prune-profiles` +: Instead of media attachments, remove locally cached copies of avatars and headers from other servers. Cannot be combined with `--remove-headers`. + +`--remove-headers` +: Instead of media attachments, remove locally cached copies of headers from other servers. Cannot be combined with `--prune-profiles`. + +`--include-follows` +: Override the default behavior of `--prune-profiles` and `--remove-headers` to remove locally cached copies of avatars (and headers) from other servers, irrespective of follow status (by default, they are only removed from accounts that are not followed by or following anyone locally). Can only be used with `--prune-profiles` or `--remove-headers`. + `--verbose` : Print additional information while task is processing. @@ -658,6 +670,7 @@ Remove locally cached copies of media attachments from other servers. **Version history:**\ 2.5.0 - added\ 2.6.2 - show freed disk space +4.1.0 - added --prune-profiles, --remove-headers, and --include-follows. --- @@ -690,7 +703,7 @@ Scans for files that do not belong to existing media attachments, and remove the ### `tootctl media refresh` {#media-refresh} -Refetch remote media attachments from other servers. You must specify the source of media attachments with either `--status`, `--account`, or `--domain`. If an attachment already exists in the database, it will not be overwritten unless you use `--force`. +Refetch remote media attachments from other servers. You must specify the source of media attachments with either `--status`, `--account`, `--domain`, or `--days`. If an attachment already exists in the database, it will not be overwritten unless you use `--force`. `--account ACCT` : String `username@domain` handle of the account @@ -701,6 +714,9 @@ Refetch remote media attachments from other servers. You must specify the source `--status ID` : Local numeric ID of the status in the database. +`--days N` +: The number of days to limit this task to. + `--concurrency N` : The number of workers to use for this task. Defaults to 5. @@ -715,7 +731,8 @@ Refetch remote media attachments from other servers. You must specify the source **Version history:**\ 3.0.0 - added\ -3.0.1 - add `--force` and skip already downloaded attachments by default +3.0.1 - add `--force` and skip already downloaded attachments by default\ +4.0.0 - add `--days` --- @@ -745,7 +762,7 @@ Prompts for a media URL, then looks up the status where the media is displayed. ## Preview Cards CLI {#preview_cards} -{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/lib/mastodon/preview_cards_cli.rb" caption="lib/mastodon/preview_cards_cli.rb" >}} +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/lib/mastodon/cli/preview_cards.rb" caption="lib/mastodon/cli/preview_cards.rb" >}} --- @@ -779,7 +796,7 @@ Remove local thumbnails for preview cards. ## Search CLI {#search} -{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/lib/mastodon/search_cli.rb" caption="lib/mastodon/search_cli.rb" >}} +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/lib/mastodon/cli/search.rb" caption="lib/mastodon/cli/search.rb" >}} --- @@ -787,10 +804,10 @@ Remove local thumbnails for preview cards. ### `tootctl search deploy` {#search-deploy} -Create or update an ElasticSearch index and populate it. If ElasticSearch is empty, this command will create the necessary indices and then import data from the database into those indices. This command will also upgrade indices if the underlying schema has been changed since the last run. +Create or update an Elasticsearch index and populate it. If Elasticsearch is empty, this command will create the necessary indices and then import data from the database into those indices. This command will also upgrade indices if the underlying schema has been changed since the last run. `--batch-size` -: Defaults to 1000. A lower batch size can make ElasticSearch process records more quickly. +: Defaults to 100. A higher batch size can make Elasticsearch process records more quickly, with less load on the PostgreSQL database, but can increase memory pressure on the Elasticsearch nodes during indexing. `--only INDEX` : Specify an index name [`accounts`, `tags`, `statuses`] to create or update only that index. @@ -810,7 +827,7 @@ Create or update an ElasticSearch index and populate it. If ElasticSearch is emp ## Settings CLI {#settings} -{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/lib/mastodon/settings_cli.rb" caption="lib/mastodon/settings_cli.rb" >}} +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/lib/mastodon/cli/settings.rb" caption="lib/mastodon/cli/settings.rb" >}} --- @@ -854,7 +871,7 @@ Set registration to require approval. ## Statuses CLI {#statuses} -{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/lib/mastodon/statuses_cli.rb" caption="lib/mastodon/statuses_cli.rb" >}} +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/lib/mastodon/cli/statuses.rb" caption="lib/mastodon/cli/statuses.rb" >}} --- @@ -883,7 +900,7 @@ This is a computationally heavy procedure that creates extra database indices be ## Upgrade CLI {#upgrade} -{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/lib/mastodon/upgrade_cli.rb" caption="lib/mastodon/upgrade_cli.rb" >}} +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/lib/mastodon/cli/upgrade.rb" caption="lib/mastodon/cli/upgrade.rb" >}} --- @@ -900,4 +917,4 @@ Upgrade the storage schema to store all non-local media resources in a top-level : Print expected results only, without performing any actions. **Version history:**\ -3.1.4 - added \ No newline at end of file +3.1.4 - added diff --git a/content/en/admin/troubleshooting.md b/content/en/admin/troubleshooting.md index 3099d97e..06178075 100644 --- a/content/en/admin/troubleshooting.md +++ b/content/en/admin/troubleshooting.md @@ -30,3 +30,11 @@ Check that you are specifying the correct environment with `RAILS_ENV=production ## **I encountered a compilation error while executing `RAILS_ENV=production bundle exec rails assets:precompile`, but no more information is given. How to fix it?** Usually it's because your server ran out of memory while compiling assets. Use a swapfile or increase the swap space to increase the memory capacity. Run `RAILS_ENV=production bundle exec rake tmp:cache:clear` to clear cache, then execute `RAILS_ENV=production bundle exec rails assets:precompile` to compile again. Make sure you clear the cache after a compilation error, or it will show "Everything's OK" but leave the assets unchanged. + +## **I am getting this error: `Read-only file system @ dir_s_mkdir`. Why?** + +By default, Mastodon makes use of [systemd's sandboxing capabilities](https://www.freedesktop.org/software/systemd/man/systemd.exec.html#Sandboxing) in a way that disallows writing outside of `/home/mastodon`. If Mastodon is installed elsewhere, you may need to allow `mastodon-sidekiq` and `mastodon-web` to write to a custom directory: +1. Add parameter `ReadWritePaths` to files `/etc/systemd/system/mastodon-sidekiq.service` and `/etc/systemd/system/mastodon-web.service`. Example - `ReadWritePaths=/example/mastodon/live`. +2. run `systemctl stop mastodon-sidekiq mastodon-web` +3. run `systemctl daemon-reload` +4. run `systemctl start mastodon-sidekiq mastodon-web` diff --git a/content/en/admin/upgrading.md b/content/en/admin/upgrading.md index 2497f1ba..296f3e4f 100644 --- a/content/en/admin/upgrading.md +++ b/content/en/admin/upgrading.md @@ -7,9 +7,19 @@ menu: --- {{< hint style="info" >}} -When a new version of Mastodon comes out, it appears on the [GitHub releases page](https://github.com/mastodon/mastodon/releases). Please mind that running unreleased code from the `master` branch, while possible, is not recommended. +When a new version of Mastodon comes out, it appears on the [GitHub releases page](https://github.com/mastodon/mastodon/releases). Please mind that running unreleased code from the `main` branch, while possible, is not recommended. {{< /hint >}} +### Automatic update verification {#automated_checks} + +Since v4.2.0, Mastodon will automatically check for available updates and notify the users of your server that have the `DevOps` permission. + +This happens by fetching `https://api.joinmastodon.org/update-check?version=` in the background every 30 minutes. `current_version` omits the build metadata (everything after the first `+`, if there is one, in the version string). For instance, if your version is `4.3.0-beta2+my-fork`, Mastodon will query `https://api.joinmastodon.org/update-check?version=4.3.0-beta2`. + +You can change which URL Mastodon queries by setting the `UPDATE_CHECK_URL` environment variable. You can also completely disable this behavior by setting this environment variable to an empty string, although we strongly recommend against doing that unless you are keeping up with Mastodon updates in another way, as Mastodon occasionally releases critical security updates that must be applied in a timely fashion. + +### Upgrade steps + Mastodon releases correspond to git tags. Before attempting an upgrade, look up the desired release on the [GitHub releases page](https://github.com/mastodon/mastodon/releases). The page will contain a **changelog** describing everything you need to know about what's different, as well as **specific upgrade instructions**. To begin, switch to the `mastodon` user: @@ -59,14 +69,14 @@ systemctl reload mastodon-web The `reload` operation is a zero-downtime restart, also called "phased restart". As such, Mastodon upgrades usually do not require any advance notice to users about planned downtime. In rare cases, you can use the `restart` operation instead, but there will be a (short) felt interruption of service for your users. {{< /hint >}} -Rarely, the **streaming API** server is also updated and requires a restart: +The **streaming API** server is also updated and requires a restart, doing so will result in all connected clients being disconnected, which can increase load on your server: ```bash systemctl restart mastodon-streaming ``` {{< hint style="danger" >}} -The streaming API server is updated very rarely, and in most releases, does *not* require a restart. Restarting the streaming API leads to an increased load on your server as disconnected clients attempt to reconnect or poll the REST API instead, so avoid it whenever you can. +Restarting the streaming API leads to an increased load on your server as disconnected clients attempt to reconnect or poll the REST API instead, so avoid it whenever you can. {{< /hint >}} {{< hint style="success" >}} diff --git a/content/en/api/guidelines.md b/content/en/api/guidelines.md index c39cec82..a3a526a0 100644 --- a/content/en/api/guidelines.md +++ b/content/en/api/guidelines.md @@ -32,7 +32,27 @@ With that said, because IDs are string representations of numbers, they can stil ## Paginating through API responses {#pagination} -Many API methods allow you to paginate for more information, using parameters such as `limit`, `max_id`, `min_id`, and `since_id`. However, some of these API methods operate on entity IDs that are not publicly exposed in the API response, and are only known to the backend and the database. (This is usually the case for entities that reference other entities, such as Follow entities which reference Accounts, or Favourite entities which reference Statuses, etc.) +Many API methods allow you to paginate for more information, using parameters such as `limit`, `max_id`, `min_id`, and `since_id`. + +limit +: The maximum number of results to return. Usually, there is a default limit and a maximum limit; these will vary according to the API method. + +max_id +: String. All results returned will be lesser than this ID. In effect, sets an upper bound on results. + +since_id +: String. All results returned will be greater than this ID. In effect, sets a lower bound on results. + +min_id +: String. Returns results immediately newer than this ID. In effect, sets a cursor at this ID and paginates forward. (Available since v2.6.0.) + +For example, we might fetch `https://mastodon.example/api/v1/accounts/1/statuses` with certain parameters, and we will get the following results in the following cases: + +- Setting `?max_id=1` will return no statuses, since there are no statuses with an ID earlier than `1`. +- Setting `?since_id=1` will return the latest statuses, since there have been many statuses since `1`. +- Setting `?min_id=1` will return the oldest statuses, as `min_id` sets the cursor. + +Some API methods operate on entity IDs that are not publicly exposed in the API response, and are only known to the backend and the database. (This is usually the case for entities that reference other entities, such as Follow entities which reference Accounts, or Favourite entities which reference Statuses, etc.) To get around this, Mastodon may return links to a "prev" and "next" page. These links are made available via the HTTP `Link` header on the response. Consider the following fictitious API call: @@ -40,7 +60,7 @@ To get around this, Mastodon may return links to a "prev" and "next" page. These GET https://mastodon.example/api/v1/endpoint HTTP/1.1 Authorization: Bearer token -Link: ; rel="next", ; rel="prev" +Link: ; rel="next", ; rel="prev" [ { // some Entity @@ -123,9 +143,9 @@ If `whole_word` is true, the client app should do the following: Please check `app/javascript/mastodon/selectors/index.js` and `app/lib/feed_manager.rb` in the Mastodon source code for more details. -{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/app/javascript/mastodon/selectors/index.js" caption="app/javascript/mastodon/selectors/index.js" >}} +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/app/javascript/mastodon/selectors/index.js" caption="app/javascript/mastodon/selectors/index.js" >}} -{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/app/lib/feed_manager.rb" caption="app/lib/feed_manager.rb" >}} +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/app/lib/feed_manager.rb" caption="app/lib/feed_manager.rb" >}} ## Focal points for cropping media thumbnails {#focal-points} diff --git a/content/en/client/authorized.md b/content/en/client/authorized.md index cbe9ef56..b4ec71e0 100644 --- a/content/en/client/authorized.md +++ b/content/en/client/authorized.md @@ -109,7 +109,7 @@ With our OAuth token for the authorized user, we can now perform any action as t * See [/api/v2/search]({{< relref "methods/search#v2" >}}) for querying resources. * See [/api/v1/suggestions]({{< relref "methods/suggestions" >}}) for suggested accounts to follow. -### Use safety features {#safety} +### User safety features {#safety} * See [/api/v1/filters]({{< relref "methods/filters" >}}) for managing filtered keywords. * See [/api/v1/domain_blocks]({{< relref "methods/domain_blocks" >}}) for managing blocked domains. diff --git a/content/en/client/libraries.md b/content/en/client/libraries.md index 42f36c18..7fc3f124 100644 --- a/content/en/client/libraries.md +++ b/content/en/client/libraries.md @@ -1,18 +1,22 @@ --- title: Libraries and implementations -description: Interface with the Mastodon API in the programming language of your choice. +description: Code, libraries and SDKs for the Mastodon API in a range of programming languages. menu: docs: weight: 60 parent: client --- -## Apex (Salesforce) {#apex-salesforce} +Thank you to our awesome developer community, for supporting the project with a wide range of implementations of the API. If you have built a library or SDK for the Mastodon API, [let us know](https://github.com/mastodon/mastodon/discussions) about it, and it may be included below in a future update. -* [apex-mastodon](https://github.com/tzmfreedom/apex-mastodon) +Remember to check how recently the library was updated, and whether it includes the API features you may want to use. +## Arduino / ESP32 / IoT {#arduino-iot} + +* [lyuba](https://github.com/ringtailsoftware/lyuba) ## C# (.NET Standard) {#c-net-standard} +* [MastodonAPI](https://github.com/golf1052/MastodonAPI) * [Mastodot](https://github.com/yamachu/Mastodot) * [Mastonet](https://github.com/glacasa/Mastonet) * [TootNet](https://github.com/cucmberium/TootNet) @@ -23,26 +27,35 @@ menu: * [mastodonpp](https://schlomp.space/tastytea/mastodonpp) -## Crystal {#crystal} - -* [mastodon.cr](https://github.com/decors/mastodon.cr) - ## Common Lisp {#common-lisp} +* [mastodon-cl](https://github.com/compufox/mastodon-cl) * [tooter](https://github.com/Shinmera/tooter) +## Crystal {#crystal} + +* [mastodon-api-crystal](https://github.com/renatolond/mastodon-api-crystal) + ## Dart -* [mastodon](https://pub.dev/packages/mastodon) +* [mastodon-api](https://github.com/mastodon-dart/mastodon-api) +* [mastodon-oauth](https://github.com/mastodon-dart/mastodon-oauth2) +* [mastodon](https://github.com/mykdavies/Mastodon) +* [dartodon](https://github.com/darkcl/dartodon) ## Elixir {#elixir} * [hunter](https://github.com/milmazz/hunter) +## Erlang {#erlang} + +* [masterldon](https://github.com/igb/masterldon) + ## Go {#go} * [go-mastodon](https://github.com/mattn/go-mastodon) * [madon](https://github.com/McKael/madon) +* [go-mastodon-api](https://github.com/aaronland/go-mastodon-api) ## Haskell {#haskell} @@ -50,10 +63,13 @@ menu: ## Java {#java} -* [mastodon4j](https://github.com/sys1yagi/mastodon4j) +* [BigBone](https://github.com/andregasser/bigbone) +* [Mastodon4J](https://github.com/Mastodon4J/Mastodon4J) +* [mastodon-jfx](https://github.com/wakingrufus/mastodon-jfx) ## JavaScript {#javascript} +* [megalodon](https://github.com/h3poteto/megalodon) * [masto.js](https://github.com/neet/masto.js) * [libodonjs](https://github.com/Zatnosk/libodonjs) @@ -63,29 +79,51 @@ menu: ## JavaScript (Node.js) {#javascript-node-js} -* [node-mastodon](https://github.com/jessicahayley/node-mastodon) * [mastodon-api](https://github.com/vanita5/mastodon-api) +## Kotlin {#kotlin} + +* [BigBone](https://github.com/andregasser/bigbone) +* [mastodonk](https://github.com/outadoc/mastodonk) + +## Nim {#nim} + +* [Mastonim](https://github.com/matrix07012/Mastonim) + +## Objective-C {#objective-c} + +* [Cocotodon](https://github.com/shibafu528/Cocotodon) + ## Perl {#perl} * [Mastodon::Client](https://metacpan.org/pod/Mastodon::Client) ## PHP {#php} -* [Mastodon API for Laravel](https://github.com/kawax/laravel-mastodon-api) -* [Composer based php API wrapper](https://github.com/r-daneelolivaw/mastodon-api-php) -* [MastodonOAuthPHP](https://github.com/TheCodingCompany/MastodonOAuthPHP) +* [mastodon-api-client](https://github.com/vazaha-nl/mastodon-api-client) * [Phediverse Mastodon REST Client](https://github.com/phediverse/mastodon-rest) * [TootoPHP](https://framagit.org/MaxKoder/TootoPHP) * [oauth2-mastodon](https://github.com/lrf141/oauth2-mastodon) +* [MastodonBotPHP](https://github.com/Eleirbag89/MastodonBotPHP) +* [mastodon-api-php-oauth](https://github.com/yks118/Mastodon-api-php-oauth) +* [mastodon-api-php](https://github.com/colorfield/mastodon-api-php) +* [Mastodon API for Laravel](https://github.com/kawax/laravel-mastodon-api) +* [Mastodon for Drupal](https://www.drupal.org/project/mastodon) +* [Mastodon for Socialite](https://github.com/kawax/socialite-mastodon) + +## PowerShell {#powershell} + +* [Mastodon](https://github.com/JB405/Mastodon) ## Python {#python} * [Mastodon.py](https://github.com/halcy/Mastodon.py) +* [mastopy](https://gitlab.com/spla/mastopy) ## R {#r} * [mastodon](https://github.com/ThomasChln/mastodon) +* [rtoot](https://github.com/schochastics/rtoot) ## Ruby {#ruby} @@ -93,8 +131,8 @@ menu: ## Rust {#rust} -* [mammut](https://github.com/Aaronepower/mammut) (unmaintained) -* [elefren](https://github.com/DeeUnderscore/elefren) (unmaintained) +* [megalodon-rs](https://github.com/h3poteto/megalodon-rs) +* [mastodon-async](https://github.com/dscottboggs/mastodon-async) ## Scala {#scala} @@ -104,9 +142,17 @@ menu: ### Guile {#guile} -* [elefan](https://codeberg.org/WammKD/Guile-Mastodon) +* [Guile-Mastodon](https://codeberg.org/WammKD/Guile-Mastodon) ## Swift {#swift} -* [MastodonKit](https://github.com/ornithocoder/MastodonKit) +* [Mastodon.swift](https://github.com/Swiftodon/Mastodon.swift) +* [MastodonKit](https://github.com/MastodonKit/MastodonKit) +* [tootsdk](https://github.com/tootsdk/tootsdk) +* [MastodonAPI](https://github.com/li-bei/MastodonAPI) + +## TypeScript {#typescript} + +* [tsl-mastodon](https://github.com/typescriptlibs/tsl-mastodon-api) + diff --git a/content/en/dev/code.md b/content/en/dev/code.md index 16a43490..56927084 100644 --- a/content/en/dev/code.md +++ b/content/en/dev/code.md @@ -70,5 +70,4 @@ The following overview should not be seen as complete or authoritative, but as a All locale files are normalized to ensure consistent formatting and key order, which minimizes changesets in version control. - Run `bundle exec i18n-tasks normalize` to normalize server-side translations -- Run `yarn run manage:translations` to normalize client-side translations - +- Run `yarn run i18n:extract` to extract and normalize client-side translations into `en.json` diff --git a/content/en/dev/disclosure.md b/content/en/dev/disclosure.md index 3342a5b2..b6caaa33 100644 --- a/content/en/dev/disclosure.md +++ b/content/en/dev/disclosure.md @@ -7,7 +7,7 @@ menu: parent: dev --- -If you believe you've identified a security vulnerability in Mastodon (a bug that allows something to happen that shouldn't be possible), you should send the report to **hello@joinmastodon.org**. We will gladly reward such reports in proportion to the severity of the issue through our OpenCollective fund. +If you believe you've identified a security vulnerability in Mastodon (a bug that allows something to happen that shouldn't be possible), you should send the report to **security@joinmastodon.org**. We will gladly reward such reports in proportion to the severity of the issue through our OpenCollective fund. You should *not* report such issues on GitHub or in other public spaces to give us time to publish a fix for the issue without exposing Mastodon's users to increased risk. diff --git a/content/en/dev/routes.md b/content/en/dev/routes.md index 17e5de8f..17ef4395 100644 --- a/content/en/dev/routes.md +++ b/content/en/dev/routes.md @@ -7,7 +7,7 @@ menu: parent: dev --- -{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/config/routes.rb" caption="config/routes.rb" >}} +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/config/routes.rb" caption="config/routes.rb" >}} ## Explanation of routes {#routes} diff --git a/content/en/dev/setup.md b/content/en/dev/setup.md index 6c52096a..9477bda7 100644 --- a/content/en/dev/setup.md +++ b/content/en/dev/setup.md @@ -65,7 +65,7 @@ You can now create the databases `mastodon_development` and `mastodon_test`, loa rails db:setup ``` -You can now launch `http://localhost:3000` in your browser and log in with the default admin user (`admin@localhost:3000` / `mastodonadmin`). +You can now launch `http://localhost:3000` in your browser and log in with the default admin user (`admin@localhost` / `mastodonadmin`). {{}} By default, Mastodon will run on port 3000. If you configure a different port for it, the generated admin account will use that number as well. @@ -82,6 +82,14 @@ foreman start This will start processes defined in `Procfile.dev`, which will give you: A Rails server, a Webpack server, a streaming API server, and Sidekiq. Of course, you can run any of those things stand-alone depending on your needs. +## Working with emails in development + +In development mode, Mastodon will use a gem called [Letter Opener](https://github.com/ryanb/letter_opener) for "sending" emails, which allows you to debug emails in your browser, without actually having to send emails via an SMTP server. + +In order to work with emails, you'll need Sidekiq, Redis and PostgreSQL running, and then emails can be viewed by visiting: `http://localhost:3000/letter_opener/` + +If you're developing in docker, you'll need to set the `REMOTE_DEV=true` environment variable. + ## Useful commands for testing {#testing} `rspec` @@ -102,4 +110,4 @@ This will start processes defined in `Procfile.dev`, which will give you: A Rail : Update Javascript packages and install any new dependencies `RAILS_ENV=development rails db:migrate` -: Run new database migrations for your development instance's database \ No newline at end of file +: Run new database migrations for your development instance's database diff --git a/content/en/entities/Account.md b/content/en/entities/Account.md index 967dd51a..eee5aea2 100644 --- a/content/en/entities/Account.md +++ b/content/en/entities/Account.md @@ -48,7 +48,7 @@ aliases: [ "followers_count": 547, "following_count": 404, "statuses_count": 28468, - "last_status_at": "2019-11-17T00:02:23.693Z", + "last_status_at": "2019-11-17", "emojis": [ { "shortcode": "ms_rainbow_flag", @@ -442,8 +442,8 @@ aliases: [ {{< page-relref ref="methods/accounts" caption="accounts API methods" >}} -{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/app/serializers/rest/account_serializer.rb" caption="app/serializers/rest/account_serializer.rb" >}} +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/app/serializers/rest/account_serializer.rb" caption="app/serializers/rest/account_serializer.rb" >}} -{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/app/serializers/rest/credential_account_serializer.rb" caption="app/serializers/rest/credential_account_serializer.rb" >}} +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/app/serializers/rest/credential_account_serializer.rb" caption="app/serializers/rest/credential_account_serializer.rb" >}} -{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/app/serializers/rest/muted_account_serializer.rb" caption="app/serializers/rest/muted_account_serializer.rb" >}} \ No newline at end of file +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/app/serializers/rest/muted_account_serializer.rb" caption="app/serializers/rest/muted_account_serializer.rb" >}} diff --git a/content/en/entities/Admin_Account.md b/content/en/entities/Admin_Account.md index 23ab6af5..a70293be 100644 --- a/content/en/entities/Admin_Account.md +++ b/content/en/entities/Admin_Account.md @@ -210,4 +210,4 @@ aliases: [ {{< page-relref ref="methods/admin/accounts" caption="admin/accounts API methods" >}} -{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/app/serializers/rest/admin/account_serializer.rb" caption="app/serializers/rest/admin/account_serializer.rb" >}} \ No newline at end of file +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/app/serializers/rest/admin/account_serializer.rb" caption="app/serializers/rest/admin/account_serializer.rb" >}} \ No newline at end of file diff --git a/content/en/entities/Admin_Cohort.md b/content/en/entities/Admin_Cohort.md index 6a79d665..739f4e32 100644 --- a/content/en/entities/Admin_Cohort.md +++ b/content/en/entities/Admin_Cohort.md @@ -77,7 +77,7 @@ Daily retention data for the week between 2022-09-08 and 2022-09-14, given that "value": "1" } ] -}, +} ``` ## Attributes diff --git a/content/en/entities/Admin_Dimension.md b/content/en/entities/Admin_Dimension.md index 1b4cc919..4d67e6cb 100644 --- a/content/en/entities/Admin_Dimension.md +++ b/content/en/entities/Admin_Dimension.md @@ -165,7 +165,7 @@ Show dimensional data about how much space is used by each software in your serv "human_value": "0 Bytes" } ] -}, +} ``` ### `software_versions` {#software_versions} diff --git a/content/en/entities/Admin_Report.md b/content/en/entities/Admin_Report.md index 894d7a6a..22f8cf11 100644 --- a/content/en/entities/Admin_Report.md +++ b/content/en/entities/Admin_Report.md @@ -170,7 +170,7 @@ aliases: [ {{< page-relref page="methods/admin/reports" caption="admin/reports API methods">}} -{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/app/serializers/rest/admin/report_serializer.rb" caption="app/serializers/rest/admin/report_serializer.rb" >}} +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/app/serializers/rest/admin/report_serializer.rb" caption="app/serializers/rest/admin/report_serializer.rb" >}} diff --git a/content/en/entities/Application.md b/content/en/entities/Application.md index 5425466c..5ce71c8e 100644 --- a/content/en/entities/Application.md +++ b/content/en/entities/Application.md @@ -39,13 +39,6 @@ aliases: [ 0.9.9 - added\ 3.5.1 - this property is now nullable -### `vapid_key` {#vapid_key} - -**Description:** Used for Push Streaming API. Returned with [POST /api/v1/apps]({{< relref "methods/apps#create" >}}). Equivalent to [WebPushSubscription#server_key]({{< relref "entities/WebPushSubscription#server_key" >}})\ -**Type:** String\ -**Version history:**\ -2.8.0 - added - ### `client_id` {{%optional%}} {#client_id} **Description:** Client ID key, to be used for obtaining OAuth tokens\ @@ -60,10 +53,20 @@ aliases: [ **Version history:**\ 0.9.9 - added +## Deprecated attributes + +### `vapid_key` {#vapid_key} + +**Description:** Used for Push Streaming API. Returned with [POST /api/v1/apps]({{< relref "methods/apps#create" >}}). Equivalent to [WebPushSubscription#server_key]({{< relref "entities/WebPushSubscription#server_key" >}}) and [Instance#vapid_public_key]({{< relref "entities/Instance#vapid_public_key" >}})\ +**Type:** String\ +**Version history:**\ +2.8.0 - added +4.3.0 - deprecated pending removal + ## See also {{< page-relref ref="methods/apps" caption="apps API methods" >}} {{< page-relref ref="entities/Status#application" caption="Status (`application` attribute)" >}} -{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/app/serializers/rest/application_serializer.rb" caption="app/serializers/rest/application_serializer.rb" >}} \ No newline at end of file +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/app/serializers/rest/application_serializer.rb" caption="app/serializers/rest/application_serializer.rb" >}} diff --git a/content/en/entities/Error.md b/content/en/entities/Error.md index 67c29a09..6caa2126 100644 --- a/content/en/entities/Error.md +++ b/content/en/entities/Error.md @@ -117,7 +117,7 @@ Error: There was a temporary problem serving your request, please try again. App ## See also -{{< caption-link url="https://github.com/mastodon/mastodon/blob/master/app/controllers/api/base_controller.rb" caption="app/controllers/api/base_controller.rb" >}} +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/app/controllers/api/base_controller.rb" caption="app/controllers/api/base_controller.rb" >}} diff --git a/content/en/entities/FeaturedTag.md b/content/en/entities/FeaturedTag.md index 8c245e2d..44c93a9e 100644 --- a/content/en/entities/FeaturedTag.md +++ b/content/en/entities/FeaturedTag.md @@ -50,7 +50,7 @@ aliases: [ ### `statuses_count` {#statuses_count} **Description:** The number of authored statuses containing this hashtag.\ -**Type:** Number\ +**Type:** Integer\ **Version history:**\ 3.0.0 - added diff --git a/content/en/entities/FilterResult.md b/content/en/entities/FilterResult.md index d2323fbc..6b5546d1 100644 --- a/content/en/entities/FilterResult.md +++ b/content/en/entities/FilterResult.md @@ -53,7 +53,7 @@ aliases: [ ### `status_matches` {#status_matches} **Description:** The status ID within the filter that was matched.\ -**Type:** {{}} String, or null\ +**Type:** {{}} Array of String, or null\ **Version history:**\ 4.0.0 - added diff --git a/content/en/entities/FilterStatus.md b/content/en/entities/FilterStatus.md index b6402e40..3224b188 100644 --- a/content/en/entities/FilterStatus.md +++ b/content/en/entities/FilterStatus.md @@ -32,7 +32,7 @@ aliases: [ ### `status_id` {#keyword} -**Description:** The ID of the filtered Status in the database.\ +**Description:** The ID of the Status that will be filtered.\ **Type:** String (cast from an integer, but not guaranteed to be a number)\ **Version history:**\ 4.0.0 - added diff --git a/content/en/entities/Instance.md b/content/en/entities/Instance.md index 18a9e23e..ab17c154 100644 --- a/content/en/entities/Instance.md +++ b/content/en/entities/Instance.md @@ -41,6 +41,9 @@ aliases: [ "urls": { "streaming": "wss://mastodon.social" }, + "vapid": { + "public_key": "BCkMmVdKDnKYwzVCDC99Iuc9GvId-x7-kKtuHnLgfF98ENiZp_aj-UNthbCdI70DqN1zUVis-x0Wrot2sBagkMc=" + }, "accounts": { "max_featured_tags": 10 }, @@ -290,6 +293,12 @@ aliases: [ **Version history:**\ 4.0.0 - added +### `configuration[vapid][public_key]` (#vapid_public_key) +**Description:** The instances VAPID public key, used for push notifications, the same as [WebPushSubscription#server_key]({{< relref "entities/WebPushSubscription#server_key" >}}).\ +**Type:** String\ +**Version history:**\ +4.3.0 - added + #### `configuration[accounts]` {#accounts} **Description:** Limits related to accounts.\ diff --git a/content/en/entities/Reaction.md b/content/en/entities/Reaction.md index 455cec85..e45f970b 100644 --- a/content/en/entities/Reaction.md +++ b/content/en/entities/Reaction.md @@ -19,18 +19,20 @@ aliases: [ ## Example ```json -{ - "name": "bongoCat", - "count": 9, - "me": false, - "url": "https://files.mastodon.social/custom_emojis/images/000/067/715/original/fdba57dff7576d53.png", - "static_url": "https://files.mastodon.social/custom_emojis/images/000/067/715/static/fdba57dff7576d53.png" -}, -{ - "name": "🤔", - "count": 1, - "me": true -} +[ + { + "name": "bongoCat", + "count": 9, + "me": false, + "url": "https://files.mastodon.social/custom_emojis/images/000/067/715/original/fdba57dff7576d53.png", + "static_url": "https://files.mastodon.social/custom_emojis/images/000/067/715/static/fdba57dff7576d53.png" + }, + { + "name": "🤔", + "count": 1, + "me": true + } +] ``` ## Attributes diff --git a/content/en/entities/Relationship.md b/content/en/entities/Relationship.md index 592248fe..d513e5ed 100644 --- a/content/en/entities/Relationship.md +++ b/content/en/entities/Relationship.md @@ -134,6 +134,6 @@ aliases: [ ## See also -{{< page-relref ref="methods/accounts#relationships" caption="POST /api/v1/accounts/relationships" >}} +{{< page-relref ref="methods/accounts#relationships" caption="GET /api/v1/accounts/relationships" >}} {{< caption-link url="https://github.com/mastodon/mastodon/blob/main/app/serializers/rest/relationship_serializer.rb" caption="app/serializers/rest/relationship_serializer.rb" >}} diff --git a/content/en/entities/Report.md b/content/en/entities/Report.md index 00fec01c..314fe7c8 100644 --- a/content/en/entities/Report.md +++ b/content/en/entities/Report.md @@ -109,14 +109,14 @@ aliases: [ ### `status_ids` {#status_ids} -**Description:** The domain name of the instance.\ +**Description:** IDs of statuses that have been attached to this report for additional context.\ **Type:** {{}} Array of String (cast from integer), or null\ **Version history:**\ 4.0.0 - added ### `rule_ids` {#rule_ids} -**Description:** The domain name of the instance.\ +**Description:** IDs of the rules that have been cited as a violation by this report.\ **Type:** {{}} Array of String (cast from integer), or null\ **Version history:**\ 4.0.0 - added diff --git a/content/en/entities/Role.md b/content/en/entities/Role.md index 381dde8d..679fe02a 100644 --- a/content/en/entities/Role.md +++ b/content/en/entities/Role.md @@ -19,12 +19,9 @@ aliases: [ "id": 3, "name": "Owner", "color": "#ff3838", - "position": 1000, - "permissions": 1, - "highlighted": true, - "created_at": "2022-09-08T22:48:07.983Z", - "updated_at": "2022-09-08T22:48:07.983Z" -}, + "permissions": 1048575, + "highlighted": true +} ``` ## Attributes @@ -50,13 +47,6 @@ aliases: [ **Version history:**\ 4.0.0 - added -### `position` {#position} - -**Description:** An index for the role's position. The higher the position, the more priority the role has over other roles.\ -**Type:** Integer\ -**Version history:**\ -4.0.0 - added - ### `permissions` {#permissions} **Description:** A bitmask that represents the sum of all permissions granted to the role.\ @@ -71,20 +61,6 @@ aliases: [ **Version history:**\ 4.0.0 - added -### `created_at` {#created_at} - -**Description:** The date that the role was created.\ -**Type:** String (ISO 8601 Datetime)\ -**Version history:**\ -4.0.0 - added - -### `updated_at` {#created_at} - -**Description:** The date that the role was updated.\ -**Type:** String (ISO 8601 Datetime)\ -**Version history:**\ -4.0.0 - added - ## Permission flags To determine the permissions available to a certain role, convert the `permissions` attribute to binary and compare from the least significant bit upwards. For convenience (and to prevent the terms from growing too long), permissions will be presented below using hexadecimal values. diff --git a/content/en/entities/ScheduledStatus.md b/content/en/entities/ScheduledStatus.md index 16399848..93171ec3 100644 --- a/content/en/entities/ScheduledStatus.md +++ b/content/en/entities/ScheduledStatus.md @@ -73,7 +73,7 @@ Returned from `GET /api/v1/scheduled_statuses`: ### `scheduled_at` {#scheduled_at} -**Description:** ID of the status in the database.\ +**Description:** The timestamp for when the status will be posted.\ **Type:** String (ISO 8601 Datetime)\ **Version history:**\ 2.7.0 - added @@ -150,8 +150,12 @@ Returned from `GET /api/v1/scheduled_statuses`: #### `params[visibility]` {#params-visibility} -**Description:** The language that will be used for the status.\ -**Type:** {{}} String (ISO 639-1 two-letter language code)\ +**Description:** The visibility that the status will have once it is posted.\ +**Type:** String (Enumerable oneOf)\ +`public` = Visible to everyone, shown in public timelines.\ +`unlisted` = Visible to public, but not included in public timelines.\ +`private` = Visible to followers only, and to any mentioned users.\ +`direct` = Visible only to mentioned users.\ **Version history:**\ 2.7.0 - added diff --git a/content/en/entities/Status.md b/content/en/entities/Status.md index 3e941367..c603e209 100644 --- a/content/en/entities/Status.md +++ b/content/en/entities/Status.md @@ -334,7 +334,7 @@ aliases: [ ### `filtered` {{%optional%}} {#filtered} **Description:** If the current token has an authorized user: The filter and keywords that matched this status.\ -**Type:** Array of [FilterResult]({{ relref "entities/FilterResult" }})\ +**Type:** Array of [FilterResult]({{< relref "entities/FilterResult" >}})\ **Version history:**\ 4.0.0 - added diff --git a/content/en/entities/StatusEdit.md b/content/en/entities/StatusEdit.md index bc8f8453..f58f34d3 100644 --- a/content/en/entities/StatusEdit.md +++ b/content/en/entities/StatusEdit.md @@ -78,7 +78,7 @@ aliases: [ ### `account` {#account} **Description:** The account that published this revision.\ -**Type:** Account\ +**Type:** [Account]({{}})\ **Version history:**\ 3.5.0 - added diff --git a/content/en/entities/Translation.md b/content/en/entities/Translation.md new file mode 100644 index 00000000..67d0c475 --- /dev/null +++ b/content/en/entities/Translation.md @@ -0,0 +1,50 @@ +--- +title: Translation +description: Represents the result of machine translating some status content +menu: + docs: + parent: entities +aliases: [ + "/api/entities/Translation", + "/api/entities/translation", +] +--- + +## Example + +```json +{ + "content": "

Hola mundo

", + "detected_source_language": "en", + "provider": "DeepL.com" +} +``` + +## Attributes + +### `content` {#content} + +**Description:** The translated text of the status.\ +**Type:** String (HTML)\ +**Version history:**\ +4.0.0 - added + +### `detected_source_language` {#detected_source_language} + +**Description:** The language of the source text, as auto-detected by the machine translation provider.\ +**Type:** String (ISO 639 language code)\ +**Version history:**\ +4.0.0 - added + +### `provider` {#provider} + +**Description:** The service that provided the machine translation. +**Type:** String\ +**Version history:**\ +4.0.0 - added + +## See also + +{{< page-relref ref="methods/statuses#translate" caption="POST /api/v1/statuses/:id/translate" >}} + +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/app/serializers/rest/translation_serializer.rb" caption="app/serializers/rest/translation_serializer.rb" >}} \ No newline at end of file diff --git a/content/en/methods/accounts.md b/content/en/methods/accounts.md index 5803a249..85204fe9 100644 --- a/content/en/methods/accounts.md +++ b/content/en/methods/accounts.md @@ -745,14 +745,14 @@ Authorization ##### Query parameters -max_id -: String. Return results older than this ID +max_id +: String. All results returned will be lesser than this ID. In effect, sets an upper bound on results. since_id -: String. Return results newer than this ID +: String. All results returned will be greater than this ID. In effect, sets a lower bound on results. min_id -: String. Return results immediately newer than this ID +: String. Returns results immediately newer than this ID. In effect, sets a cursor at this ID and paginates forward. limit : Integer. Maximum number of results to return. Defaults to 20 statuses. Max 40 statuses. @@ -767,7 +767,7 @@ exclude_reblogs : Boolean. Filter out boosts from the response. pinned -: Boolean. Filter for pinned statuses only. +: Boolean. Filter for pinned statuses only. Defaults to false, which includes all statuses. Pinned statuses do not receive special priority in the order of the returned results. tagged : String. Filter for statuses using a specific hashtag. @@ -1940,7 +1940,8 @@ Find out whether a given account is followed, blocked, muted, etc. **Returns:** Array of [Relationship]({{< relref "entities/Relationship">}})\ **OAuth:** User token + `read:follows`\ **Version history:**\ -0.0.0 - added +0.0.0 - added\ +4.3.0 - added `with_suspended` parameter #### Request ##### Headers @@ -1953,6 +1954,9 @@ Authorization id[] : Array. Check relationships for the provided account IDs. +with_suspended +: Boolean. Whether relationships should be returned for suspended users, defaults to false. + #### Response ##### 200: OK @@ -2322,4 +2326,4 @@ Token does not have an authorized user {{< caption-link url="https://github.com/mastodon/mastodon/blob/main/app/controllers/api/v1/accounts/statuses_controller.rb" caption="app/controllers/api/v1/accounts/statuses_controller.rb" >}} -{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/app/models/account_statuses_filter.rb" caption="app/models/account_statuses_filter.rb" >}} \ No newline at end of file +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/app/models/account_statuses_filter.rb" caption="app/models/account_statuses_filter.rb" >}} diff --git a/content/en/methods/admin/accounts.md b/content/en/methods/admin/accounts.md index e15a1419..7f53ba32 100644 --- a/content/en/methods/admin/accounts.md +++ b/content/en/methods/admin/accounts.md @@ -83,14 +83,14 @@ ip staff : Boolean. Filter for staff accounts? -max_id -: String. Return results older than ID. +max_id +: String. All results returned will be lesser than this ID. In effect, sets an upper bound on results. since_id -: String. Return results newer than ID. +: String. All results returned will be greater than this ID. In effect, sets a lower bound on results. min_id -: String. Return results immediately newer than ID. +: String. Returns results immediately newer than this ID. In effect, sets a cursor at this ID and paginates forward. limit : Integer. Maximum number of results to return. Defaults to 100 accounts. Max 200 accounts. @@ -233,14 +233,14 @@ email ip : String. Lookup users with this IP address. -max_id -: String. Return results older than ID. +max_id +: String. All results returned will be lesser than this ID. In effect, sets an upper bound on results. since_id -: String. Return results newer than ID. +: String. All results returned will be greater than this ID. In effect, sets a lower bound on results. min_id -: String. Return results immediately newer than ID. +: String. Returns results immediately newer than this ID. In effect, sets a cursor at this ID and paginates forward. limit : Integer. Maximum number of results to return. Defaults to 100 accounts. Max 200 accounts. diff --git a/content/en/methods/admin/canonical_email_blocks.md b/content/en/methods/admin/canonical_email_blocks.md index c8cacb95..cdf44a7d 100644 --- a/content/en/methods/admin/canonical_email_blocks.md +++ b/content/en/methods/admin/canonical_email_blocks.md @@ -94,6 +94,11 @@ GET /api/v1/admin/canonical_email_blocks/:id HTTP/1.1 #### Request +##### Path parameters + +:id +: {{}} String. The ID of the Admin::CanonicalEmailBlock in the database. + ##### Headers Authorization @@ -262,6 +267,11 @@ DELETE /api/v1/admin/canonical_email_blocks/:id HTTP/1.1 #### Request +##### Path parameters + +:id +: {{}} String. The ID of the Admin::CanonicalEmailBlock in the database. + ##### Headers Authorization diff --git a/content/en/methods/admin/reports.md b/content/en/methods/admin/reports.md index 0e8913c3..f174ea80 100644 --- a/content/en/methods/admin/reports.md +++ b/content/en/methods/admin/reports.md @@ -505,7 +505,7 @@ Reopen a currently closed report, if it is closed. ##### Path parameters :id -: {{}} String. The ID of the SOMETHING in the database. +: {{}} String. The ID of the Report in the database. ##### Headers diff --git a/content/en/methods/emails.md b/content/en/methods/emails.md index 747662ab..b73db712 100644 --- a/content/en/methods/emails.md +++ b/content/en/methods/emails.md @@ -21,7 +21,7 @@ aliases: [ ## Resend confirmation email {#confirmation} ```http -POST /api/v1/emails/confirmation HTTP/1.1 +POST /api/v1/emails/confirmations HTTP/1.1 ``` **Returns:** Empty object\ @@ -73,4 +73,4 @@ Alternatively, the user has already confirmed their email. {{< page-relref ref="methods/accounts#create" caption="POST /api/v1/accounts" >}} -{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/app/controllers/api/v1/emails/confirmations_controller.rb" caption="app/controllers/api/v1/emails/confirmations_controller.rb" >}} \ No newline at end of file +{{< caption-link url="https://github.com/mastodon/mastodon/blob/main/app/controllers/api/v1/emails/confirmations_controller.rb" caption="app/controllers/api/v1/emails/confirmations_controller.rb" >}} diff --git a/content/en/methods/endorsements.md b/content/en/methods/endorsements.md index bbedf350..902a313f 100644 --- a/content/en/methods/endorsements.md +++ b/content/en/methods/endorsements.md @@ -29,8 +29,7 @@ Accounts that the user is currently featuring on their profile. **Returns:** Array of [Account]({{< relref "entities/account" >}})\ **OAuth:** User token + `read:accounts`\ **Version history:**\ -2.5.0 - added\ -3.3.0 - both `min_id` and `max_id` can be used at the same time now +2.5.0 - added #### Request diff --git a/content/en/methods/filters.md b/content/en/methods/filters.md index 329ff278..501b394a 100644 --- a/content/en/methods/filters.md +++ b/content/en/methods/filters.md @@ -16,7 +16,7 @@ aliases: [ ## Server-side (v2) methods {#v2} -Since Mastodon 3.6, filters can contain multiple keywords and are matched server-side. Clients apply the filter action based on [the status's `filtered` attribute]({{< relref "entities/Status#filtered" >}}). +Since Mastodon 4.0, filters can contain multiple keywords and are matched server-side. Clients apply the filter action based on [the status's `filtered` attribute]({{< relref "entities/Status#filtered" >}}). --- @@ -106,7 +106,7 @@ Obtain a single filter group owned by the current user. ##### Path parameters :id -: {{}} String. The ID of the SOMETHING in the database. +: {{}} String. The ID of the Filter in the database. ##### Headers @@ -206,7 +206,7 @@ keywords_attributes[][keyword] : String. A keyword to be added to the newly-created filter group. keywords_attributes[][whole_word] -: String. Whether the keyword should consider word boundaries. +: Boolean. Whether the keyword should consider word boundaries.