diff --git a/docs/gen_reference_doc.py b/docs/gen_reference_doc.py index a453e5a12..f05bf0ce6 100644 --- a/docs/gen_reference_doc.py +++ b/docs/gen_reference_doc.py @@ -43,6 +43,7 @@ global orphaned_export preprocess_rst = \ { 'manual.rst': 'manual-ref.rst', + 'tuning.rst': 'tuning-ref.rst', 'upgrade_to_1.2.rst': 'upgrade_to_1.2-ref.rst', 'settings.rst': 'settings-ref.rst' } diff --git a/docs/hacking.rst b/docs/hacking.rst index 975d322af..1533cd19f 100644 --- a/docs/hacking.rst +++ b/docs/hacking.rst @@ -118,7 +118,7 @@ libtorrent starts 3 to 5 threads. * The third thread is the SHA-1 hash thread. By default there's only one hash thread, but on multi-core machines downloading at very high rates, libtorrent can be configured to start any number of hashing threads, to take full use of multi core systems. - (see ``settings_pack::aio_threads``). + (see settings_pack::aio_threads). * The fourth and fifth threads are spawned by asio on systems that don't support asynchronous host name resolution, in order to simulate non-blocking ``getaddrinfo()``. diff --git a/docs/index.rst b/docs/index.rst index ee2e3444b..ff40244e3 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -70,7 +70,7 @@ libtorrent .. _documentation: reference.html .. _`libtorrent 1.2`: upgrade_to_1.2-ref.html .. _troubleshooting: troubleshooting.html -.. _`tuning`: tuning.html +.. _`tuning`: tuning-ref.html .. _fuzzing: fuzzing.html .. _screenshot: client_test.png .. _`uTP`: utp.html diff --git a/docs/makefile b/docs/makefile index bfe16c70c..47e2796c6 100644 --- a/docs/makefile +++ b/docs/makefile @@ -15,6 +15,7 @@ endif REFERENCE_TARGETS = \ manual-ref \ + tuning-ref \ upgrade_to_1.2-ref \ reference \ reference-Core \ @@ -49,7 +50,6 @@ MANUAL_TARGETS = index \ python_binding \ projects \ utp \ - tuning \ hacking \ streaming \ tutorial \ @@ -103,7 +103,7 @@ ifneq ($(STAGE),) cp $@ $(WEB_PATH)/$@ endif -$(REFERENCE_TARGETS:=.rst) plain_text_out.txt:gen_reference_doc.py ../include/libtorrent/*.hpp ../include/libtorrent/kademlia/*.hpp manual.rst settings.rst stats_counters.rst hunspell/settings.dic +$(REFERENCE_TARGETS:=.rst) plain_text_out.txt:gen_reference_doc.py ../include/libtorrent/*.hpp ../include/libtorrent/kademlia/*.hpp manual.rst tuning.rst settings.rst stats_counters.rst hunspell/settings.dic python gen_reference_doc.py --plain-output spell-check:plain_text_out.txt $(MANUAL_TARGETS:=.html) manual.rst settings.rst diff --git a/docs/manual.rst b/docs/manual.rst index 6841873e1..d044a9516 100644 --- a/docs/manual.rst +++ b/docs/manual.rst @@ -363,7 +363,7 @@ settings_pack::active_dht_limit and settings_pack::active_lsd_limit respectively. Specifically, announcing to a tracker is typically cheaper than -announcing to the DHT. ``active_dht_limit`` will limit the number of +announcing to the DHT. settings_pack::active_dht_limit will limit the number of torrents that are allowed to announce to the DHT. The highest priority ones will, and the lower priority ones won't. The will still be considered started though, and any incoming peers will still be accepted. diff --git a/docs/tuning.rst b/docs/tuning.rst index 1ef1d3e67..375af5ad5 100644 --- a/docs/tuning.rst +++ b/docs/tuning.rst @@ -12,7 +12,7 @@ tuning libtorrent ================= libtorrent expose most parameters used in the bittorrent engine for -customization through the ``settings_pack``. This makes it possible to +customization through the settings_pack. This makes it possible to test and tweak the parameters for certain algorithms to make a client that fits a wide range of needs. From low memory embedded devices to servers seeding thousands of torrents. The default settings in libtorrent @@ -45,20 +45,20 @@ Then print alerts to a file:: If you want to separate generic alerts from session stats, you can filter on the alert category in the alert, ``alert::category()``. -The alerts with data will have the type ``session_stats_alert`` and there is one -``session_log_alert`` that will be posted on startup containing the column names +The alerts with data will have the type session_stats_alert and there is one +session_log_alert that will be posted on startup containing the column names for all metrics. Logging this line will greatly simplify interpreting the output. The python scrip in ``tools/parse_session_stats.py`` can parse the resulting -file and produce graphs of relevant stats. It requires gnuplot__. +file and produce graphs of relevant stats. It requires gnuplot_. -__ http://www.gnuplot.info +.. _gnuplot: http://www.gnuplot.info reducing memory footprint ========================= These are things you can do to reduce the memory footprint of libtorrent. You get -some of this by basing your default ``settings_pack`` on the ``min_memory_usage()`` +some of this by basing your default settings_pack on the min_memory_usage() setting preset function. Keep in mind that lowering memory usage will affect performance, always profile @@ -89,9 +89,9 @@ disable disk cache The bulk of the memory libtorrent will use is used for the disk cache. To save the absolute most amount of memory, you can disable the cache by setting -``settings_pack::cache_size`` to 0. You might want to consider using the cache +settings_pack::cache_size to 0. You might want to consider using the cache but just disable caching read operations. You do this by settings -``settings_pack::use_read_cache`` to false. This is the main factor in how much +settings_pack::use_read_cache to false. This is the main factor in how much memory will be used by the client. Keep in mind that you will degrade performance by disabling the cache. You should benchmark the disk access in order to make an informed trade-off. @@ -125,8 +125,8 @@ connection, and might be worth considering if you have a very large number of peer connections. This memory will not be visible in your process, this sets the amount of kernel memory is used for your sockets. -Change this by setting ``settings_pack::recv_socket_buffer_size`` and -``settings_pack::send_socket_buffer_size``. +Change this by setting settings_pack::recv_socket_buffer_size and +settings_pack::send_socket_buffer_size. peer list size -------------- @@ -140,7 +140,7 @@ large number of paused torrents (that are popular) it will be even more significant. If you're short of memory, you should consider lowering the limit. 500 is probably -enough. You can do this by setting ``settings_pack::max_peerlist_size`` to +enough. You can do this by setting settings_pack::max_peerlist_size to the max number of peers you want in a torrent's peer list. This limit applies per torrent. For 5 torrents, the total number of peers in peer lists will be 5 times the setting. @@ -148,7 +148,7 @@ the setting. You should also lower the same limit but for paused torrents. It might even make sense to set that even lower, since you only need a few peers to start up while waiting for the tracker and DHT to give you fresh ones. The max peer list size for paused -torrents is set by ``settings_pack::max_paused_peerlist_size``. +torrents is set by settings_pack::max_paused_peerlist_size. The drawback of lowering this number is that if you end up in a position where the tracker is down for an extended period of time, your only hope of finding live @@ -163,7 +163,7 @@ The send buffer watermark controls when libtorrent will ask the disk I/O thread to read blocks from disk, and append it to a peer's send buffer. When the send buffer has fewer than or equal number of bytes as -``settings_pack::send_buffer_watermark``, the peer will ask the disk I/O thread +settings_pack::send_buffer_watermark, the peer will ask the disk I/O thread for more data to send. The trade-off here is between wasting memory by having too much data in the send buffer, and hurting send rate by starving out the socket, waiting for the disk read operation to complete. @@ -215,7 +215,7 @@ purpose of this is because of anti-virus software that hooks on file-open and fi scan the file. Anti-virus software that does that will significantly increase the cost of opening and closing files. However, for a high performance seed, the file open/close might be so frequent that it becomes a significant cost. It might therefore be a good idea to allow -a large file descriptor cache. Adjust this though ``settings_pack::file_pool_size``. +a large file descriptor cache. Adjust this though settings_pack::file_pool_size. Don't forget to set a high rlimit for file descriptors in your process as well. This limit must be high enough to keep all connections and files open. @@ -224,26 +224,26 @@ disk cache ---------- You typically want to set the cache size to as high as possible. The -``settings_pack::cache_size`` is specified in 16 kiB blocks. Since you're seeding, -the cache would be useless unless you also set ``settings_pack::use_read_cache`` +settings_pack::cache_size is specified in 16 kiB blocks. Since you're seeding, +the cache would be useless unless you also set settings_pack::use_read_cache to true. In order to increase the possibility of read cache hits, set the -``settings_pack::cache_expiry`` to a large number. This won't degrade anything as +settings_pack::cache_expiry to a large number. This won't degrade anything as long as the client is only seeding, and not downloading any torrents. There's a *guided cache* mode. This means the size of the read cache line that's stored in the cache is determined based on the upload rate to the peer that triggered the read operation. The idea being that slow peers don't use up a disproportional amount of space in the cache. This is enabled through -``settings_pack::guided_read_cache``. +settings_pack::guided_read_cache. In cases where the assumption is that the cache is only used as a read-ahead, and that no other peer will ever request the same block while it's still in the cache, the read cache can be set to be *volatile*. This means that every block that is requested out of the read cache is removed immediately. This saves a significant amount of cache space which can be used as read-ahead for other peers. To enable volatile read cache, set -``settings_pack::volatile_read_cache`` to true. +settings_pack::volatile_read_cache to true. uTP-TCP mixed mode ------------------ @@ -255,7 +255,7 @@ throttle TCP to avoid it taking over all bandwidth. This balances the bandwidth between the two protocols. When running on a network where the bandwidth is in such an abundance that it's virtually infinite, this algorithm is no longer necessary, and might even be harmful to throughput. It is advised to experiment with the -``session_setting::mixed_mode_algorithm``, setting it to ``settings_pack::prefer_tcp``. +settings_pack::mixed_mode_algorithm, setting it to settings_pack::prefer_tcp. This setting entirely disables the balancing and un-throttles all connections. On a typical home connection, this would mean that none of the benefits of uTP would be preserved (the modem's send buffer would be full at all times) and uTP connections would for the most @@ -274,36 +274,36 @@ enough to not draining the socket's send buffer before the disk operation comple The watermark is bound to a max value, to avoid buffer sizes growing out of control. The default max send buffer size might not be enough to sustain very high upload rates, and you might have to increase it. It's specified in bytes in -``settings_pack::send_buffer_watermark``. +settings_pack::send_buffer_watermark. peers ----- First of all, in order to allow many connections, set the global connection limit -high, ``settings_pack::connections_limit``. Also set the upload rate limit to -infinite, ``settings_pack::upload_rate_limit``, 0 means infinite. +high, settings_pack::connections_limit. Also set the upload rate limit to +infinite, settings_pack::upload_rate_limit, 0 means infinite. When dealing with a large number of peers, it might be a good idea to have slightly stricter timeouts, to get rid of lingering connections as soon as possible. -There are a couple of relevant settings: ``settings_pack::request_timeout``, -``settings_pack::peer_timeout`` and ``settings_pack::inactivity_timeout``. +There are a couple of relevant settings: settings_pack::request_timeout, +settings_pack::peer_timeout and settings_pack::inactivity_timeout. For seeds that are critical for a delivery system, you most likely want to allow multiple connections from the same IP. That way two people from behind the same NAT can use the service simultaneously. This is controlled by -``settings_pack::allow_multiple_connections_per_ip``. +settings_pack::allow_multiple_connections_per_ip. In order to always unchoke peers, turn off automatic unchoke by setting -``settings_pack::choking_algorithm`` to ``fixed_slot_choker`` and set the number -of upload slots to a large number via ``settings_pack::unchoke_slots_limit``, +settings_pack::choking_algorithm to settings_pack::fixed_slot_choker and set the number +of upload slots to a large number via settings_pack::unchoke_slots_limit, or use -1 (which means infinite). torrent limits -------------- -To seed thousands of torrents, you need to increase the ``settings_pack::active_limit`` -and ``settings_pack::active_seeds``. +To seed thousands of torrents, you need to increase the settings_pack::active_limit +and settings_pack::active_seeds. SHA-1 hashing ------------- @@ -311,7 +311,7 @@ SHA-1 hashing When downloading at very high rates, it is possible to have the CPU be the bottleneck for passing every downloaded byte through SHA-1. In order to enable calculating SHA-1 hashes in parallel, on multi-core systems, set -``settings_pack::aio_threads`` to the number of threads libtorrent should +settings_pack::aio_threads to the number of threads libtorrent should perform I/O and do SHA-1 hashing in. Only if that thread is close to saturating one core does it make sense to increase the number of threads. @@ -331,7 +331,7 @@ This will actually loop through all torrents and run a provided predicate functi determine whether or not to include it in the returned vector. To use ``session::post_torrent_updates()`` torrents need to have the ``flag_update_subscribe`` -flag set. When post_torrent_updates() is called, a ``state_update_alert`` alert +flag set. When post_torrent_updates() is called, a state_update_alert alert is posted, with all the torrents that have updated since the last time this function was called. The client have to keep its own state of all torrents, and update it based on this alert.