diff --git a/.travis.yml b/.travis.yml index 65fba8eb1..c4f95afb2 100644 --- a/.travis.yml +++ b/.travis.yml @@ -47,6 +47,8 @@ before_install: echo ''import site; site.addsitedir("/usr/local/lib/python2.7/site-packages")'' >> /Users/travis/Library/Python/2.7/lib/python/site-packages/homebrew.pth; sudo easy_install Pygments; sudo easy_install -U aafigure; + brew install --quiet graphviz; + brew install --quiet Homebrew/python/pillow; fi' # disable simulations on OSX for now. It hangs on travis diff --git a/docs/building.html b/docs/building.html deleted file mode 100644 index 1f1cf37f8..000000000 --- a/docs/building.html +++ /dev/null @@ -1,765 +0,0 @@ - - - - - - -libtorrent manual - - - - - - - - -
-
- - - - -
-

libtorrent manual

- --- - - - - - -
Author:Arvid Norberg, arvid@libtorrent.org
Version:1.1.0
- -
-

downloading and building

-

To acquire the latest version of libtorrent, you'll have to grab it from SVN. -You'll find instructions on how to do this here (see subversion access).

-

The build systems supported "out of the box" in libtorrent are boost-build v2 -(BBv2) and autotools (for unix-like systems). If you still can't build after -following these instructions, you can usually get help in the #libtorrent -IRC channel on irc.freenode.net.

-
-

Warning

-

A common mistake when building and linking against libtorrent is -to build with one set of configuration options (#defines) and -link against it using a different set of configuration options. Since -libtorrent has some code in header files, that code will not be -compatible with the built library if they see different configurations.

-

Always make sure that the same TORRENT_* macros are defined when you -link against libtorrent as when you build it.

-

Boost-build supports propagating configuration options to dependencies. -When building using the makefiles, this is handled by setting the -configuration options in the pkg-config file. Always use pkg-config -when linking against libtorrent.

-
-
-

building from svn

-

To build libtorrent from svn you need to check out the libtorrent sources from -sourceforge. If you downloaded a release tarball, you can skip this section.

-

To check out libtorrent follow these instructions.

-
-
-

building with BBv2

-

The primary reason to use boost-build is that it will automatically build the -dependent boost libraries with the correct compiler settings, in order to -ensure that the build targets are link compatible (see boost guidelines -for some details on this issue).

-

Since BBv2 will build the boost libraries for you, you need the full boost -source package. Having boost installed via some package system is usually not -enough (and even if it is enough, the necessary environment variables are -usually not set by the package installer).

-

If you want to build against an installed copy of boost, you can skip directly -to step 3 (assuming you also have boost build installed).

-
-

Step 1: Download boost

-

You'll find boost here.

-

Extract the archive to some directory where you want it. For the sake of this -guide, let's assume you extract the package to c:\boost_1_55_0 (I'm using -a windows path in this example since if you're on linux/unix you're more likely -to use the autotools). You'll need at least version 1.49 of the boost library -in order to build libtorrent.

-
-
-

Step 2: Setup BBv2

-

First you need to build bjam. You do this by opening a terminal (In -windows, run cmd). Change directory to -c:\boost_1_55_0\tools\jam\src. Then run the script called -build.bat or build.sh on a unix system. This will build bjam and -place it in a directory starting with bin. and then have the name of your -platform. Copy the bjam.exe (or bjam on a unix system) to a place -that's in you shell's PATH. On linux systems a place commonly used may be -/usr/local/bin or on windows c:\windows (you can also add directories -to the search paths by modifying the environment variable called PATH).

-

Now you have bjam installed. bjam can be considered an interpreter -that the boost-build system is implemented on. So boost-build uses bjam. -So, to complete the installation you need to make two more things. You need to -set the environment variable BOOST_BUILD_PATH. This is the path that tells -bjam where it can find boost-build, your configuration file and all the -toolsets (descriptions used by boost-build to know how to use different -compilers on different platforms). Assuming the boost install path above, set -it to c:\boost_1_55_0\tools\build\v2.

-

To set an environment variable in windows, type for example:

-
-set BOOST_BUILD_PATH=c:\boost_1_55_0\tools\build\v2
-
-

In a terminal window.

-

The last thing to do to complete the setup of BBv2 is to modify your -user-config.jam file. It is located in c:\boost_1_55_0\tools\build\v2. -Depending on your platform and which compiler you're using, you should add a -line for each compiler and compiler version you have installed on your system -that you want to be able to use with BBv2. For example, if you're using -Microsoft Visual Studio 12 (2013), just add a line:

-
-using msvc : 12.0 ;
-
-

If you use GCC, add the line:

-
-using gcc ;
-
-

If you have more than one version of GCC installed, you can add the -commandline used to invoke g++ after the version number, like this:

-
-using gcc : 3.3 : g++-3.3 ;
-using gcc : 4.0 : g++-4.0 ;
-
-

Another toolset worth mentioning is the darwin toolset (For MacOS X). -From Tiger (10.4) MacOS X comes with both GCC 3.3 and GCC 4.0. Then you can -use the following toolsets:

-
-using darwin : 3.3 : g++-3.3 ;
-using darwin : 4.0 : g++-4.0 ;
-
-

Note that the spaces around the semi-colons and colons are important!

-

Also see the official installation instructions.

-
-
-

Step 3: Building libtorrent

-

When building libtorrent, the Jamfile expects the environment variable -BOOST_ROOT to be set to the boost installation directory. It uses this to -find the boost libraries it depends on, so they can be built and their headers -files found. So, set this to c:\boost_1_55_0. You only need this if you're -building against a source distribution of boost.

-

Then the only thing left is simply to invoke bjam. If you want to specify -a specific toolset to use (compiler) you can just add that to the commandline. -For example:

-
-bjam msvc-7.1
-bjam gcc-3.3
-bjam darwin-4.0
-
-
-

Note

-

If the environment variable BOOST_ROOT is not set, the jamfile will -attempt to link against "installed" boost libraries. i.e. assume the headers -and libraries are available in default search paths.

-
-

To build different versions you can also just add the name of the build -variant. Some default build variants in BBv2 are release, debug, -profile.

-

You can build libtorrent as a dll too, by typing link=shared, or -link=static to build a static library.

-

If you want to explicitly say how to link against the runtime library, you -can set the runtime-link feature on the commandline, either to shared -or static. Most operating systems will only allow linking shared against -the runtime, but on windows you can do both. Example:

-
-bjam msvc-7.1 link=static runtime-link=static
-
-
-

Note

-

When building on windows, the path boost-build puts targets in may be too -long. If you get an error message like: "The input line is long", try to -pass --abbreviate-paths on the bjam command line.

-
-
-

Warning

-

If you link statically to the runtime library, you cannot build libtorrent -as a shared library (DLL), since you will get separate heaps in the library -and in the client application. It will result in crashes and possibly link -errors.

-
-
-

Note

-

With boost-build V2 (Milestone 11), the darwin toolset uses the -s linker -option to strip debug symbols. This option is buggy in Apple's GCC, and -will make the executable crash on startup. On Mac OS X, instead build -your release executables with the debug-symbols=on option, and -later strip your executable with strip.

-
-
-

Note

-

Some linux systems requires linking against librt in order to access -the POSIX clock functions. If you get an error complaining about a missing -symbol clock_gettime, you have to give need-librt=yes on the -bjam command line. This will make libtorrent link against librt.

-
-
-

Note

-

When building on Solaris, you might have to specify stdlib=sun-stlport -on the bjam command line.

-
-

The build targets are put in a directory called bin, and under it they are -sorted in directories depending on the toolset and build variant used.

-

To build the examples, just change directory to the examples directory and -invoke bjam from there. To build and run the tests, go to the test -directory and run bjam.

-

Note that if you're building on windows using the msvc toolset, you cannot run it -from a cygwin terminal, you'll have to run it from a cmd terminal. The same goes for -cygwin, if you're building with gcc in cygwin you'll have to run it from a cygwin terminal. -Also, make sure the paths are correct in the different environments. In cygwin, the paths -(BOOST_BUILD_PATH and BOOST_ROOT) should be in the typical unix-format (e.g. -/cygdrive/c/boost_1_55_0). In the windows environment, they should have the typical -windows format (c:/boost_1_55_0).

-
-

Note

-

In Jamfiles, spaces are separators. It's typically easiest to avoid spaces -in path names. If you want spaces in your paths, make sure to quote them -with double quotes (").

-
-

The Jamfile will define NDEBUG when it's building a release build. -For more build configuration flags see Build configurations.

-

Build features:

- ---- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
boost build featurevalues
boost-link
    -
  • static - links statically against the boost -libraries.
  • -
  • shared - links dynamically against the boost -libraries.
  • -
-
logging
    -
  • off - logging alerts disabled.
  • -
  • on - default. logging alerts available, -still need to be enabled by the alert mask. The -reason to disable logging is to keep the binary -size down.
  • -
-
dht
    -
  • on - build with support for tracker less -torrents and DHT support.
  • -
  • logging - build with DHT support and verbose -logging of the DHT protocol traffic.
  • -
  • off - build without DHT support.
  • -
-
asserts
    -
  • auto - asserts are on if in debug mode
  • -
  • on - asserts are on, even in release mode
  • -
  • off - asserts are disabled
  • -
  • production - assertion failures are logged -to asserts.log in the current working -directory, but won't abort the process. -The file they are logged to can be customized -by setting the global pointer extern char -const* libtorrent_assert_log to a different -filename.
  • -
  • system use the libc assert macro
  • -
-
encryption
    -
  • on - encrypted bittorrent connections -enabled. (Message Stream encryption).
  • -
  • off - turns off support for encrypted -connections. The shipped public domain SHA-1 -implementation is used.
  • -
-
mutable-torrents
    -
  • on - mutable torrents are supported -(BEP 38) (default).
  • -
  • off - mutable torrents are not supported.
  • -
-
crypto
    -
  • built-in - (default) uses built-in SHA-1 -implementation.
  • -
  • openssl - links against openssl and -libcrypto to use for SHA-1 hashing.
  • -
  • gcrypt - links against libgcrypt to use for -SHA-1 hashing.
  • -
-
allocator
    -
  • pool - default, uses pool allocators for -send buffers.
  • -
  • system - uses malloc() and free() -instead. Might be useful to debug buffer issues -with tools like electric fence or libgmalloc.
  • -
  • debug - instruments buffer usage to catch -bugs in libtorrent.
  • -
-
link
    -
  • static - builds libtorrent as a static -library (.a / .lib)
  • -
  • shared - builds libtorrent as a shared -library (.so / .dll).
  • -
-
runtime-link
    -
  • static - links statically against the -run-time library (if available on your -platform).
  • -
  • shared - link dynamically against the -run-time library (default).
  • -
-
variant
    -
  • debug - builds libtorrent with debug -information and invariant checks.
  • -
  • release - builds libtorrent in release mode -without invariant checks and with optimization.
  • -
  • profile - builds libtorrent with profile -information.
  • -
-
character-set

This setting will only have an affect on windows. -Other platforms are expected to support UTF-8.

-
    -
  • unicode - The unicode version of the win32 -API is used. This is default.
  • -
  • ansi - The ansi version of the win32 API is -used.
  • -
-
invariant-checks

This setting only affects debug builds (where -NDEBUG is not defined). It defaults to on.

-
    -
  • on - internal invariant checks are enabled.
  • -
  • off - internal invariant checks are -disabled. The resulting executable will run -faster than a regular debug build.
  • -
  • full - turns on extra expensive invariant -checks.
  • -
-
debug-symbols
    -
  • on - default for debug builds. This setting -is useful for building release builds with -symbols.
  • -
  • off - default for release builds.
  • -
-
deprecated-functions
    -
  • on - default. Includes deprecated functions -of the API (might produce warnings during build -when deprecated functions are used).
  • -
  • off - excludes deprecated functions from the -API. Generates build errors when deprecated -functions are used.
  • -
-
iconv
    -
  • auto - use iconv for string conversions for -linux and mingw and other posix platforms.
  • -
  • on - force use of iconv
  • -
  • off - force not using iconv (disables locale -awareness except on windows).
  • -
-
i2p
    -
  • on - build with I2P support
  • -
  • off - build without I2P support
  • -
-
profile-calls
    -
  • off - default. No additional call profiling.
  • -
  • on - Enable logging of stack traces of -calls into libtorrent that are blocking. On -session shutdown, a file blocking_calls.txt -is written with stack traces of blocking calls -ordered by the number of them.
  • -
-
-

The variant feature is implicit, which means you don't need to specify -the name of the feature, just the value.

-

The logs created when building vlog or log mode are put in a directory called -libtorrent_logs in the current working directory.

-

When building the example client on windows, you need to build with -link=static otherwise you may get unresolved external symbols for some -boost.program-options symbols.

-

For more information, see the Boost build v2 documentation, or more -specifically the section on builtin features.

-
-
-
-

building with autotools

-

First of all, you need to install automake and autoconf. Many -unix/linux systems comes with these preinstalled.

-

The prerequisites for building libtorrent are boost.thread, boost.date_time -and boost.filesystem. Those are the compiled boost libraries needed. The -headers-only libraries needed include (but is not necessarily limited to) -boost.bind, boost.ref, boost.multi_index, boost.optional, boost.lexical_cast, -boost.integer, boost.iterator, boost.tuple, boost.array, boost.function, -boost.smart_ptr, boost.preprocessor, boost.static_assert.

-

If you want to build the client_test example, you'll also need boost.regex -and boost.program_options.

-
-

Step 1: Generating the build system

-

No build system is present if libtorrent is checked out from CVS - it -needs to be generated first. If you're building from a released tarball, -you may skip directly to Step 2: Running configure.

-

Execute the following command to generate the build system:

-
-./autotool.sh
-
-
-
-

Step 2: Running configure

-

In your shell, change directory to the libtorrent directory and run -./configure. This will look for libraries and C++ features that libtorrent -is dependent on. If something is missing or can't be found it will print an -error telling you what failed.

-

The most likely problem you may encounter is that the configure script won't -find the boost libraries. Make sure you have boost installed on your system. -The easiest way to install boost is usually to use the preferred package -system on your platform. Usually libraries and headers are installed in -standard directories where the compiler will find them, but sometimes that -may not be the case. For example when installing boost on darwin using -darwinports (the package system based on BSD ports) all libraries are -installed to /opt/local/lib and headers are installed to -/opt/local/include. By default the compiler will not look in these -directories. You have to set the enviornment variables LDFLAGS and -CXXFLAGS in order to make the compiler find those libs. In this example -you'd set them like this:

-
-export LDFLAGS=-L/opt/local/lib
-export CXXFLAGS=-I/opt/local/include
-
-

It was observed on FreeBSD (release 6.0) that one needs to add '-lpthread' to -LDFLAGS, as Boost::Thread detection will fail without it, even if -Boost::Thread is installed.

-

If you need to set these variables, it may be a good idea to add those lines -to your ~/.profile or ~/.tcshrc depending on your shell.

-

If the boost libraries are named with a suffix on your platform, you may use -the --with-boost-thread= option to specify the suffix used for the thread -library in this case. For more information about these options, run:

-
-./configure --help
-
-

On gentoo the boost libraries that are built with multi-threading support have -the suffix mt.

-

You know that the boost libraries were found if you see the following output -from the configure script:

-
-checking whether the Boost::DateTime library is available... yes
-checking for main in -lboost_date_time... yes
-checking whether the Boost::Filesystem library is available... yes
-checking for main in -lboost_filesystem... yes
-checking whether the Boost::Thread library is available... yes
-checking for main in -lboost_thread... yes
-
-

Another possible source of problems may be if the path to your libtorrent -directory contains spaces. Make sure you either rename the directories with -spaces in their names to remove the spaces or move the libtorrent directory.

-
-
-

Creating a debug build

-

To tell configure to build a debug version (with debug info, asserts -and invariant checks enabled), you have to run the configure script -with the following option:

-
-./configure --enable-debug=yes
-
-
-
-

Creating a release build

-

To tell the configure to build a release version (without debug info, -asserts and invariant checks), you have to run the configure script -with the following option:

-
-./configure --enable-debug=no
-
-

The above option make use of -DNDEBUG, which is used throughout libtorrent.

-
-
-

Step 3: Building libtorrent

-

Once the configure script is run successfully, you just type make and -libtorrent, the examples and the tests will be built.

-

When libtorrent is built it may be a good idea to run the tests, you do this -by running make check.

-

If you want to build a release version (without debug info, asserts and -invariant checks), you have to rerun the configure script and rebuild, like this:

-
-./configure --disable-debug
-make clean
-make
-
-
-
-
-

building with other build systems

-

If you're building in MS Visual Studio, you may have to set the compiler -options "force conformance in for loop scope", "treat wchar_t as built-in -type" and "Enable Run-Time Type Info" to Yes.

-
-
-

build configurations

-

By default libtorrent is built In debug mode, and will have pretty expensive -invariant checks and asserts built into it. If you want to disable such checks -(you want to do that in a release build) you can see the table below for which -defines you can use to control the build.

- ---- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
macrodescription
NDEBUGIf you define this macro, all asserts, -invariant checks and general debug code will be -removed. Since there is quite a lot of code in -in header files in libtorrent, it may be -important to define the symbol consistently -across compilation units, including the clients -files. Potential problems is different -compilation units having different views of -structs and class layouts and sizes.
TORRENT_DISABLE_LOGGINGThis macro will disable support for logging -alerts, like log_alert, torrent_log_alert and -peer_log_alert. With this build flag, you -cannot enable those alerts.
TORRENT_STORAGE_DEBUGThis will enable extra expensive invariant -checks in the storage, including logging of -piece sorting.
TORRENT_DISK_STATSThis will create a log of all disk activity -which later can parsed and graphed using -parse_disk_log.py.
UNICODEIf building on windows this will make sure the -UTF-8 strings in pathnames are converted into -UTF-16 before they are passed to the file -operations.
TORRENT_DISABLE_POOL_ALLOCATORDisables use of boost::pool<>.
TORRENT_DISABLE_MUTABLE_TORRENTSDisables mutable torrent support (BEP 38)
TORRENT_LINKING_SHAREDIf this is defined when including the -libtorrent headers, the classes and functions -will be tagged with __declspec(dllimport) -on msvc and default visibility on GCC 4 and -later. Set this in your project if you're -linking against libtorrent as a shared library. -(This is set by the Jamfile when -link=shared is set).
TORRENT_BUILDING_SHAREDIf this is defined, the functions and classes -in libtorrent are marked with -__declspec(dllexport) on msvc, or with -default visibility on GCC 4 and later. This -should be defined when building libtorrent as -a shared library. (This is set by the Jamfile -when link=shared is set).
TORRENT_DISABLE_DHTIf this is defined, the support for trackerless -torrents will be disabled.
TORRENT_DISABLE_ENCRYPTIONThis will disable any encryption support and -the dependencies of a crypto library. -Encryption support is the peer connection -encrypted supported by clients such as -uTorrent, Azureus and KTorrent. -If this is not defined, either -TORRENT_USE_OPENSSL or -TORRENT_USE_GCRYPT must be defined.
TORRENT_DISABLE_EXTENSIONSWhen defined, libtorrent plugin support is -disabled along with support for the extension -handskake (BEP 10).
_UNICODEOn windows, this will cause the file IO -use wide character API, to properly support -non-ansi characters.
TORRENT_DISABLE_RESOLVE_COUNTRIESDefining this will disable the ability to -resolve countries of origin for peer IPs.
TORRENT_DISABLE_INVARIANT_CHECKSThis will disable internal invariant checks in -libtorrent. The invariant checks can sometime -be quite expensive, they typically don't scale -very well. This option can be used to still -build in debug mode, with asserts enabled, but -make the resulting executable faster.
TORRENT_EXPENSIVE_INVARIANT_CHECKSThis will enable extra expensive invariant -checks. Useful for finding particular bugs -or for running before releases.
TORRENT_NO_DEPRECATEThis will exclude all deprecated functions from -the header files and cpp files.
TORRENT_PRODUCTION_ASSERTSDefine to either 0 or 1. Enables assert logging -in release builds.
TORRENT_USE_ASSERTSDefine as 0 to disable asserts unconditionally.
TORRENT_USE_SYSTEM_ASSERTSUses the libc assert macro rather then the -custom one.
-

If you experience that libtorrent uses unreasonable amounts of cpu, it will -definitely help to define NDEBUG, since it will remove the invariant checks -within the library.

-
-
-

building openssl for windows

-

To build openssl for windows with Visual Studio 7.1 (2003) execute the following commands -in a command shell:

-
-perl Configure VC-WIN32 --prefix="c:/openssl
-call ms\do_nasm
-call "C:\Program Files\Microsoft Visual Studio .NET 2003\vc7\bin\vcvars32.bat"
-nmake -f ms\nt.mak
-copy inc32\openssl "C:\Program Files\Microsoft Visual Studio .NET 2003\vc7\include\"
-copy out32\libeay32.lib "C:\Program Files\Microsoft Visual Studio .NET 2003\vc7\lib"
-copy out32\ssleay32.lib "C:\Program Files\Microsoft Visual Studio .NET 2003\vc7\lib"
-
-

This will also install the headers and library files in the visual studio directories to -be picked up by libtorrent.

-
-
- -
-
-
- -
- -
- - diff --git a/docs/client_test.html b/docs/client_test.html deleted file mode 100644 index 454354383..000000000 --- a/docs/client_test.html +++ /dev/null @@ -1,112 +0,0 @@ - - - - - - -client_test example program - - - - - - - -
-
- - - - -
-

client_test example program

- -

Client test is a, more or less, complete bittorrent client. It lacks most -settings and you can't start or stop torrents once you've started it. All -the settings are hardcoded. The commandline arguments are:

-
-client_test <filename1.torrent> <filename2.torrent> ...
-
-

You can start any number of torrent downloads/seeds via the commandline. -If one argument starts with http:// it is interpreted as a tracker -announce url, and it expects an info-hash as the next argument. The info-hash -has to be hex-encoded. For example: 2410d4554d5ed856d69f426c38791673c59f4418. -If you pass an announce url and info-hash, a torrent-less download is started. -It relies on that at least one peer on the tracker is running a libtorrent based -client and has the metadata (.torrent file). The metadata extension in -libtorrent will then download it from that peer (or from those peers if more -than one).

-

While running, the client_test sample will look something like this:

-client_test.png -

The commands available in the client are:

-
    -
  • q quits the client (there will be a delay while the client waits -for tracker responses)
  • -
  • l toggle log. Will display the log at the bottom, informing about -tracker and peer events.
  • -
  • i toggles torrent info. Will show the peer list for each torrent.
  • -
  • d toggle download info. Will show the block list for each torrent, -showing downloaded and requested blocks.
  • -
  • p pause all torrents.
  • -
  • u unpause all torrents.
  • -
  • r force tracker reannounce for all torrents.
  • -
  • f toggle show file progress. Displays a list of all files and the -download progress for each file.
  • -
-

The list at the bottom (shown if you press d) shows which blocks has -been requested from which peer. The green background means that it has been -downloaded. It shows that fast peers will prefer to request whole pieces -instead of dowloading parts of pieces. It may make it easier to determine -which peer that sent the corrupt data if a piece fails the hash test.

- -
-
-
- -
- -
- - diff --git a/docs/contributing.html b/docs/contributing.html deleted file mode 100644 index 1df40233d..000000000 --- a/docs/contributing.html +++ /dev/null @@ -1,136 +0,0 @@ - - - - - - -libtorrent manual - - - - - - - - -
-
- - - - -
-

libtorrent manual

- --- - - - - - -
Author:Arvid Norberg, arvid@libtorrent.org
Version:1.1.0
-
-

Table of contents

- -
-
-

contributing to libtorrent

-

There are several ways to contribute to libtorrent at various levels. Any help is -much appreciated. If you're interested in something libtorrent related that's not -enumerated on this page, please contact arvid@libtorrent.org or the mailing list.

-
    -
  1. -
    Testing
    -

    This is not just limited to finding bugs and ways to reproduce crashes, but also -sub-optimal behavior is certain scenarios and finding ways to reproduce those. Please -report any issue to the bug tracker at github.

    -

    New features that need testing are streaming (set_piece_deadline()), the different -choking algorithms (especially the new BitTyrant choker), the disk cache options (such -as explicit_cache).

    -
    -
    -
  2. -
-
    -
  1. -
    Documentation
    -

    Finding typos or outdated sections in the documentation. Contributing documentation -based on your own experience and experimentation with the library or with BitTorrent -in general. Non-reference documentation is very much welcome as well, higher level -descriptions on how to configure libtorrent for various situations for instance. -The reference documentation for libtorrent is generated from the header files. -For updates, please submit a pull request.

    -
    -
    -
  2. -
  3. -
    Code
    -

    Contributing code for new features or bug-fixes is highly welcome. If you're interested -in adding a feature but not sure where to start, please contact the mailing list or -#libtorrent @ irc.freenode.net. For proposed fixes or udpates, please -submit a pull request.

    -

    New features might be better support for integrating with other services, new choking -algorithms, seeding policies, ports to new platforms etc.

    -
    -
    -
  4. -
-

For an overview of the internals of libtorrent, see the hacking page.

-

For outstanding things to do, see the todo list.

-
- -
-
-
- -
- -
- - diff --git a/docs/dht_extensions.html b/docs/dht_extensions.html deleted file mode 100644 index 679de2e8b..000000000 --- a/docs/dht_extensions.html +++ /dev/null @@ -1,148 +0,0 @@ - - - - - - - - - - - - - - - -
-
- - - - -
- - --- - - - -
Author:Arvid Norberg, arvid@libtorrent.org
-
-

Mainline DHT extensions

-

libtorrent implements a few extensions to the Mainline DHT protocol.

-
-

get_peers response

-

libtorrent always responds with nodes to a get_peers request. If it has -peers for the specified info-hash, it will return values as well. This is -because just because some peer announced to us, doesn't mean that we are -among the 8 closest nodes of the info hash. libtorrent also keeps traversing -nodes using get_peers until it has found the 8 closest ones, and then announces -to those nodes.

-
-
-

forward compatibility

-

In order to support future DHT messages, any message which is not recognized -but has either an info_hash or target argument is interpreted as -find node for that target. i.e. it returns nodes. This allows future messages -to be properly forwarded by clients that don't understand them instead of -being blocked.

-
-
-

client identification

-

In each DHT packet, an extra key is inserted named "v". This is a string -describing the client and version used. This can help alot when debugging -and finding errors in client implementations. The string is encoded as four -characters, two characters describing the client and two characters interpreted -as a binary number describing the client version.

-

Currently known clients:

- ---- - - - - - - - - - - - - - - -
uTorrentUT
libtorrentLT
MooPoliceMP
GetRightGR
-
-
-

IPv6 support

-

This extension is superseeded by BEP 32.

-

The DHT messages that don't support IPv6 are the nodes replies. -They encode all the contacts as 6 bytes packed together in sequence in a -string. The problem is that IPv6 endpoints cannot be encoded as 6 bytes, but -needs 18 bytes. The extension libtorrent applies is to add another key, called -nodes2.

-

nodes2 may be present in replies that contains a nodes key. It is encoded -as a list of strings. Each string represents one contact and is encoded as 20 -bytes node-id and then a variable length encoded IP address (6 bytes in IPv4 case -and 18 bytes in IPv6 case).

-

As an optimization, libtorrent does not include the extra key in case there are -only IPv4 nodes present.

-
-
- -
-
-
- -
- -
- - diff --git a/docs/dht_rss.html b/docs/dht_rss.html deleted file mode 100644 index 3d52252b4..000000000 --- a/docs/dht_rss.html +++ /dev/null @@ -1,435 +0,0 @@ - - - - - - -BitTorrent extension for DHT RSS feeds - - - - - - - - -
-
- - - - -
-

BitTorrent extension for DHT RSS feeds

- --- - - - - - -
Author:Arvid Norberg, arvid@libtorrent.org
Version:1.1.0
- -

This proposal has been superseded by the dht_put feature. This may -still be implemented on top of that.

-

This is a proposal for an extension to the BitTorrent DHT to allow -for decentralized RSS feed like functionality.

-

The intention is to allow the creation of repositories of torrents -where only a single identity has the authority to add new content. For -this repository to be robust against network failures and resilient -to attacks at the source.

-

The target ID under which the repository is stored in the DHT, is the -SHA-1 hash of a feed name and the 512 bit public key. This private key -in this pair MUST be used to sign every item stored in the repository. -Every message that contain signed items MUST also include this key, to -allow the receiver to verify the key itself against the target ID as well -as the validity of the signatures of the items. Every recipient of a -message with feed items in it MUST verify both the validity of the public -key against the target ID it is stored under, as well as the validity of -the signatures of each individual item.

-

As with normal DHT announces, the write-token mechanism is used to -prevent IP spoof attacks.

-
-

terminology

-

In this document, a storage node refers to the node in the DHT to which -an item is being announce. A subscribing node refers to a node which -makes look ups in the DHT to find the storage nodes, to request items -from them.

-
-
-

linked lists

-

Items are chained together in a geneal singly linked list. A linked -list does not necessarily contain RSS items, and no RSS related items -are mandatory. However, RSS items will be used as examples in this BEP:

-
-key = SHA1(name + key)
-+---------+
-| head    |           key = SHA1(bencode(item))
-| +---------+         +---------+
-| | next    |-------->| item    |          key = SHA1(bencode(item))
-| | key     |         | +---------+        +---------+
-| | name    |         | | next    |------->| item    |
-| | seq     |         | | key     |        | +---------+
-| | ...     |         | | ...     |        | | next    |--->0
-| +---------+         | +---------+        | | key     |
-| sig     |           | sig     |          | | ...     |
-+---------+           +---------+          | +---------+
-                                           | sig     |
-                                           +---------+
-
-

The next pointer is at least 20 byte ID in the DHT key space pointing to where the next -item in the list is announced. The list is terminated with an ID of all zeroes.

-

The ID an items is announced to is determined by the SHA1 hash of the bencoded representation -of the item iteself. This contains all fields in the item, except the signature. -The only mandatory fields in an item are next, key and sig.

-

The key field MUST match the public key of the list head node. The sig field -MUST be the signature of the bencoded representation of item or head (whichever -is included in the message).

-

All subscribers MUST verify that the item is announced under the correct DHT key -and MUST verify the signature is valid and MUST verify the public key is the same -as the list-head. If a node fails any of these checks, it must be ignored and the -chain of items considered terminated.

-

Each item holds a bencoded dictionary with arbitrary keys, except two mandatory keys: -next and key. The signature sig is transferred outside of this dictionary -and is the signature of all of it. An implementation should stora any arbitrary keys that -are announced to an item, within reasonable restriction such as nesting, size and numeric -range of integers.

-
-
-

skip lists

-

The next key stored in the list head and the items is a string of at least length -20 bytes, it may be any length divisible by 20. Each 20 bytes are the ID of the next -item in the list, the item 2 hops away, 4 hops away, 8 hops away, and so on. For -simplicity, only the first ID (1 hop) in the next field is illustrated above.

-

A publisher of an item SHOULD include as many IDs in the next field as the remaining -size of the list warrants, within reason.

-

These skip lists allow for parallelized lookups of items and also makes it more efficient -to search for specific items. It also mitigates breaking lists missing some items.

-

Figure of the skip list in the first list item:

-
-n      Item0  Item1  Item2  Item3  Item4  Item5  Item6  Item7  Item8  Item9  Item10
-0        O----->
-20       O------------>
-40       O-------------------------->
-60       O------------------------------------------------------>
-
-

n refers to the byte offset into the next field.

-
-
-

list-head

-

The list head item is special in that it can be updated, without changing its -DHT key. This is required to prepend new items to the linked list. To authenticate -that only the original publisher can update the head, the whole linked list head -is signed. In order to avoid a malicious node to overwrite the list head with an old -version, the sequence number seq must be monotonically increasing for each update, -and a node hosting the list node MUST not downgrade a list head from a higher sequence -number to a lower one, only upgrade.

-

The list head's DHT key (which it is announced to) MUST be the SHA1 hash of the name -(n) and key fields concatenated.

-

Any node MUST reject any list head which is announced under any other ID.

-
-
-

messages

-

These are the messages to deal with linked lists.

-

The id field in these messages has the same semantics as the standard DHT messages, -i.e. the node ID of the node sending the message, to maintain the structure of the DHT -network.

-

The token field also has the same semantics as the standard DHT message get_peers -and announce_peer, when requesting an item and to write an item respectively.

-

nodes and nodes6 has the same semantics as in its get_peers response.

-
-

requesting items

-

This message can be used to request both a list head and a list item. When requesting -a list head, the n (name) field MUST be specified. When requesting a list item the -n field is not required.

-
-{
-   "a":
-   {
-      "id": <20 byte ID of sending node>,
-      "key": <64 byte public curve25519 key for this list>,
-      "n": <list name>
-      "target": <target-id for 'head' or 'item'>
-   },
-   "q": "get_item",
-   "t": <transaction-id>,
-   "y": "q",
-}
-
-

When requesting a list-head the target MUST always be SHA-1(feed_name + public_key). -target is the target node ID the item was written to.

-

The n field is the name of the list. If specified, It MUST be UTF-8 encoded string -and it MUST match the name of the feed in the receiving node.

-
-
-

request item response

-

This is the format of a response of a list head:

-
-{
-   "r":
-   {
-      "head":
-      {
-         "key": <64 byte public curve25519 key for this list>,
-         "next": <20 bytes item ID>,
-         "n": <name of the linked list>,
-         "seq": <monotonically increasing sequence number>
-      },
-      "sig": <curve25519 signature of 'head' entry (in bencoded form)>,
-      "id": <20 byte id of sending node>,
-      "token": <write-token>,
-      "nodes": <n * compact IPv4-port pair>,
-      "nodes6": <n * compact IPv6-port pair>
-   },
-   "t": <transaction-id>,
-   "y": "r",
-}
-
-

This is the format of a response of a list item:

-
-{
-   "r":
-   {
-      "item":
-      {
-         "key": <64 byte public curve25519 key for this list>,
-         "next": <20 bytes item ID>,
-         ...
-      },
-      "sig": <curve25519 signature of 'item' entry (in bencoded form)>,
-      "id": <20 byte id of sending node>,
-      "token": <write-token>,
-      "nodes": <n * compact IPv4-port pair>,
-      "nodes6": <n * compact IPv6-port pair>
-   },
-   "t": <transaction-id>,
-   "y": "r",
-}
-
-

A client receiving a get_item response MUST verify the signature in the sig -field against the bencoded representation of the item field, using the key as -the public key. The key MUST match the public key of the feed.

-

The item dictionary MAY contain arbitrary keys, and all keys MUST be stored for -items.

-
-
-

announcing items

-

The message format for announcing a list head:

-
-{
-   "a":
-   {
-      "head":
-      {
-         "key": <64 byte public curve25519 key for this list>,
-         "next": <20 bytes item ID>,
-         "n": <name of the linked list>,
-         "seq": <monotonically increasing sequence number>
-      },
-      "sig": <curve25519 signature of 'head' entry (in bencoded form)>,
-      "id": <20 byte node-id of origin node>,
-      "target": <target-id as derived from public key and name>,
-      "token": <write-token as obtained by previous request>
-   },
-   "y": "q",
-   "q": "announce_item",
-   "t": <transaction-id>
-}
-
-

The message format for announcing a list item:

-
-{
-   "a":
-   {
-      "item":
-      {
-         "key": <64 byte public curve25519 key for this list>,
-         "next": <20 bytes item ID>,
-         ...
-      },
-      "sig": <curve25519 signature of 'item' entry (in bencoded form)>,
-      "id": <20 byte node-id of origin node>,
-      "target": <target-id as derived from item dict>,
-      "token": <write-token as obtained by previous request>
-   },
-   "y": "q",
-   "q": "announce_item",
-   "t": <transaction-id>
-}
-
-

A storage node MAY reject items and heads whose bencoded representation is -greater than 1024 bytes.

-
-
-
-

re-announcing

-

In order to keep feeds alive, subscriber nodes SHOULD help out in announcing -items they have downloaded to the DHT.

-

Every subscriber node SHOULD store items in long term storage, across sessions, -in order to keep items alive for as long as possible, with as few sources as possible.

-

Subscribers to a feed SHOULD also announce items that they know of, to the feed. -Since a feed may have many subscribers and many items, subscribers should re-announce -items according to the following algorithm.

-
-1. pick one random item (i) from the local repository (except
-   items already announced this round)
-2. If all items in the local repository have been announced
-  2.1 terminate
-3. look up item i in the DHT
-4. If fewer than 8 nodes returned the item
-  4.1 announce i to the DHT
-  4.2 goto 1
-
-

This ensures a balanced load on the DHT while still keeping items alive

-
-
-

timeouts

-

Items SHOULD be announced to the DHT every 30 minutes. A storage node MAY time -out an item after 60 minutes of no one announcing it.

-

A storing node MAY extend the timeout when it receives a request for it. Since -items are immutable, the data doesn't go stale. Therefore it doesn't matter if -the storing node no longer is in the set of the 8 closest nodes.

-
-
-

RSS feeds

-

For RSS feeds, following keys are mandatory in the list item's item dictionary.

-
-
ih
-
The torrent's info hash
-
size
-
The size (in bytes) of all files the torrent
-
n
-
name of the torrent
-
-
-

example

-

This is an example of an announce_item message:

-
-{
-   "a":
-   {
-      "item":
-      {
-         "key": "6bc1de5443d1a7c536cdf69433ac4a7163d3c63e2f9c92d
-            78f6011cf63dbcd5b638bbc2119cdad0c57e4c61bc69ba5e2c08
-            b918c2db8d1848cf514bd9958d307",
-         "info-hash": "7ea94c240691311dc0916a2a91eb7c3db2c6f3e4",
-         "size": 24315329,
-         "n": "my stuff",
-         "next": "c68f29156404e8e0aas8761ef5236bcagf7f8f2e"
-      }
-      "sig": <signature>
-      "id": "b46989156404e8e0acdb751ef553b210ef77822e",
-      "target": "b4692ef0005639e86d7165bf378474107bf3a762"
-      "token": "23ba"
-   },
-   "y": "q",
-   "q": "announce_item",
-"t": "a421"
-}
-
-

Strings are printed in hex for printability, but actual encoding is binary.

-

Note that target is in fact SHA1 hash of the same data the signature sig -is the signature of, i.e.:

-
-d9:info-hash20:7ea94c240691311dc0916a2a91eb7c3db2c6f3e43:key64:6bc1de5443d1
-a7c536cdf69433ac4a7163d3c63e2f9c92d78f6011cf63dbcd5b638bbc2119cdad0c57e4c61
-bc69ba5e2c08b918c2db8d1848cf514bd9958d3071:n8:my stuff4:next20:c68f29156404
-e8e0aas8761ef5236bcagf7f8f2e4:sizei24315329ee
-
-

(note that binary data is printed as hex)

-
-
-
-

RSS feed URI scheme

-

The proposed URI scheme for DHT feeds is:

-
-magnet:?xt=btfd:<base16-curve25519-public-key> &dn= <feed name>
-
-

Note that a difference from regular torrent magnet links is the btfd -versus btih used in regular magnet links to torrents.

-

The feed name is mandatory since it is used in the request and when -calculating the target ID.

-
-
-

rationale

-

The reason to use curve25519 instead of, for instance, RSA is compactness. According to -http://cr.yp.to/, curve25519 is free from patent claims and there are open implementations -in both C and Java.

-
- -
-
-
- -
- -
- - diff --git a/docs/dht_sec.html b/docs/dht_sec.html deleted file mode 100644 index 67fbd8c8c..000000000 --- a/docs/dht_sec.html +++ /dev/null @@ -1,305 +0,0 @@ - - - - - - -BitTorrent DHT security extension - - - - - - - - -
-
- - - - -
-

BitTorrent DHT security extension

- --- - - - - - -
Author:Arvid Norberg, arvid@libtorrent.org
Version:1.1.0
- -
-

BitTorrent DHT security extension

-

The purpose of this extension is to make it harder to launch a few -specific attacks against the BitTorrent DHT and also to make it harder -to snoop the network.

-

Specifically the attack this extension intends to make harder is launching -8 or more DHT nodes which node-IDs selected close to a specific target -info-hash, in order to become the main nodes hosting peers for it. Currently -this is very easy to do and lets the attacker not only see all the traffic -related to this specific info-hash but also block access to it by other -peers.

-

The proposed guard against this is to enforce restrictions on which node-ID -a node can choose, based on its external IP address.

-
-
-

considerations

-

One straight forward scheme to tie the node ID to an IP would be to hash -the IP and force the node ID to share the prefix of that hash. One main -draw back of this approach is that an entities control over the DHT key -space grows linearly with its control over the IP address space.

-

In order to successfully launch an attack, you just need to find 8 IPs -whose hash will be closest to the target info-hash. Given the current -size of the DHT, that is quite likely to be possible by anyone in control -of a /8 IP block.

-

The size of the DHT is approximately 8.4 million nodes. This is estmiated -by observing that a typical routing table typically has about 20 of its -top routing table buckets full. That means the key space is dense enough -to contain 8 nodes for every combination of the 20 top bits of node IDs.

-
-2^20 * 8 = 8388608
-

By controlling that many IP addresses, an attacker could snoop any info-hash. -By controlling 8 times that many IP addresses, an attacker could actually -take over any info-hash.

-

With IPv4, snooping would require a /8 IP block, giving access to 16.7 million -Ips.

-

Another problem with hashing the IP is that multiple users behind a NAT are -forced to run their DHT nodes on the same node ID.

-
-
-

Node ID restriction

-

In order to avoid the number node IDs controlled to grow linearly by the number -of IPs, as well as allowing more than one node ID per external IP, the node -ID can be restricted at each class level of the IP.

-

Another important property of the restriction put on node IDs is that the -distribution of the IDs remoain uniform. This is why CRC32C (Castagnoli) was -chosen as the hash function.

-

The expression to calculate a valid ID prefix (from an IPv4 address) is:

-
-crc32c((ip & 0x030f3fff) | (r << 29))
-
-

And for an IPv6 address (ip is the high 64 bits of the address):

-
-crc32c((ip & 0x0103070f1f3f7fff) | (r << 61))
-
-

r is a random number in the range [0, 7]. The resulting integer, -representing the masked IP address is supposed to be big-endian before -hashed. The "|" operator means bit-wise OR.

-

The details of implementing this is to evaluate the expression, store the -result in a big endian 64 bit integer and hash those 8 bytes with CRC32C.

-

The first (most significant) 21 bits of the node ID used in the DHT MUST -match the first 21 bits of the resulting hash. The last byte of the hash MUST -match the random number (r) used to generate the hash.

-ip_id_v4.png -ip_id_v6.png -

Example code code for calculating a valid node ID:

-
-uint8_t* ip; // our external IPv4 or IPv6 address (network byte order)
-int num_octets; // the number of octets to consider in ip (4 or 8)
-uint8_t node_id[20]; // resulting node ID
-
-uint8_t v4_mask[] = { 0x03, 0x0f, 0x3f, 0xff };
-uint8_t v6_mask[] = { 0x01, 0x03, 0x07, 0x0f, 0x1f, 0x3f, 0x7f, 0xff };
-uint8_t* mask = num_octets == 4 ? v4_mask : v6_mask;
-
-for (int i = 0; i < num_octets; ++i)
-        ip[i] &= mask[i];
-
-uint32_t rand = std::rand() & 0xff;
-uint8_t r = rand & 0x7;
-ip[0] |= r << 5;
-
-uint32_t crc = 0;
-crc = crc32c(crc, ip, num_octets);
-
-// only take the top 21 bits from crc
-node_id[0] = (crc >> 24) & 0xff;
-node_id[1] = (crc >> 16) & 0xff;
-node_id[2] = ((crc >> 8) & 0xf8) | (std::rand() & 0x7);
-for (int i = 3; i < 19; ++i) node_id[i] = std::rand();
-node_id[19] = rand;
-
-

test vectors:

-
-IP           rand  example node ID
-============ ===== ==========================================
-124.31.75.21   1   5fbfbf f10c5d6a4ec8a88e4c6ab4c28b95eee4 01
-21.75.31.124  86   5a3ce9 c14e7a08645677bbd1cfe7d8f956d532 56
-65.23.51.170  22   a5d432 20bc8f112a3d426c84764f8c2a1150e6 16
-84.124.73.14  65   1b0321 dd1bb1fe518101ceef99462b947a01ff 41
-43.213.53.83  90   e56f6c bf5b7c4be0237986d5243b87aa6d5130 5a
-
-

The bold parts of the node ID are the important parts. The rest are -random numbers. The last bold number of each row has only its most significant -bit pulled from the CRC32C function. The lower 3 bits are random.

-
-
-

bootstrapping

-

In order to set ones initial node ID, the external IP needs to be known. This -is not a trivial problem. With this extension, all DHT responses SHOULD include -a top-level field called ip, containing a compact binary representation of -the requestor's IP and port. That is big endian IP followed by 2 bytes of big endian -port.

-

The IP portion is the same byte sequence used to verify the node ID.

-

It is important that the ip field is in the top level dictionary. Nodes that -enforce the node-ID will respond with an error message ("y": "e", "e": { ... }), -whereas a node that supports this extension but without enforcing it will respond -with a normal reply ("y": "r", "r": { ... }).

-

A DHT node which receives an ip result in a request SHOULD consider restarting -its DHT node with a new node ID, taking this IP into account. Since a single node -can not be trusted, there should be some mechanism to determine whether or -not the node has a correct understanding of its external IP or not. This could -be done by voting, or only restart the DHT once at least a certain number of -nodes, from separate searches, tells you your node ID is incorrect.

-
-
-

rationale

-

The choice of using CRC32C instead of a more traditional cryptographic hash -function is justified primarily of these reasons:

-
    -
  1. it is a fast function
  2. -
  3. produces well distributed results
  4. -
  5. there is no need for the hash function to be one-way (the input set is -so small that any hash function could be reversed).
  6. -
  7. CRC32C (Castagnoli) is supported in hardware by SSE 4.2, which can -significantly speed up computation
  8. -
-

There are primarily two tests run on SHA-1 and CRC32C to establish the -distribution of results. The first one is the number of bits in the output -set that contain every possible combination of bits. The CRC32C function -has a longer such prefix in its output than SHA-1. This means nodes will still -have well uniformly distributed IDs, even when IP addresses in use are not -uniformly distributed.

-

The following graph illustrate a few different hash functions with regard -to this property.

-complete_bit_prefixes.png -

This test takes into account IP addresses that are not globally routable, i.e. -reserved for local networks, multicast and other things. It also takes into -account that some /8 blocks are not in use by end-users and exremely unlikely -to ever run a DHT node. This makes the results likely to be very similar to -what we would see in the wild.

-

These results indicate that CRC32C provides the best uniformity in the results -in terms of bit prefixes where all possibilities are represented, and that -no more than 21 bits should be used from the result. If more than 21 bits -were to be used, there would be certain node IDs that would be impossible to -have, which would make routing sub-optimal.

-

The second test is more of a sanity test for the uniform distribution property. -The target space (32 bit interger) is divided up into 1000 buckets. Every valid -IP and r input is run through the algorithm and the result is put in the -bucket it falls in. The expectation is that each bucket has roughly an equal -number of results falling into it. The following graph shows the resulting -histogram, comparing SHA-1 and CRC32C.

-hash_distribution.png -

The source code for these tests can be found here.

-

The reason to use CRC32C instead of the CRC32 implemented by zlib is that -Intel CPUs have hardware support for the CRC32C calculations. The input -being exactly 4 bytes is also deliberate, to make it fit in a single -instruction.

-
-
-

enforcement

-

Once enforced, write tokens from peers whose node ID does not match its external -IP should be considered dropped. In other words, a peer that uses a non-matching -ID MUST never be used to store information on, regardless of which request. In the -original DHT specification only announce_peer stores data in the network, -but any future extension which stores data in the network SHOULD use the same -restriction.

-

Any peer on a local network address is exempt from this node ID verification. -This includes the following IP blocks:

-
-
10.0.0.0/8
-
reserved for local networks
-
172.16.0.0/12
-
reserved for local networks
-
192.168.0.0/16
-
reserved for local networks
-
169.254.0.0/16
-
reserved for self-assigned IPs
-
127.0.0.0/8
-
reserved for loopback
-
-
-
-

backwards compatibility and transition

-

During some transition period, this restriction should not be enforced, and -peers whose node ID does not match this formula relative to their external IP -should not be blocked.

-

Requests from peers whose node ID does not match their external IP should -always be serviced, even after the transition period. The attack this protects -from is storing data on an attacker's node, not servicing an attackers request.

-
-
-

forward compatibility

-

If the total size of the DHT grows to the point where the inherent size limit -in this proposal is too small, the modulus constants can be updated in a new -proposal, and another transition period where both sets of modulus constants -are accepted.

-
- -
-
-
- -
- -
- - diff --git a/docs/dht_store.html b/docs/dht_store.html deleted file mode 100644 index c18096906..000000000 --- a/docs/dht_store.html +++ /dev/null @@ -1,538 +0,0 @@ - - - - - - -BitTorrent extension for arbitrary DHT store - - - - - - - - -
-
- - - - -
-

BitTorrent extension for arbitrary DHT store

- --- - - - - - -
Author:Arvid Norberg, arvid@libtorrent.org
Version:1.1.0
- -

This is a proposal for an extension to the BitTorrent DHT to allow -storing and retrieving of arbitrary data.

-

It supports both storing immutable items, where the key is -the SHA-1 hash of the data itself, and mutable items, where -the key is the public key of the key pair used to sign the data.

-

There are two new proposed messages, put and get.

-
-

terminology

-

In this document, a storage node refers to the node in the DHT to which -an item is being announced and stored on. A requesting node refers to -a node which makes look-ups in the DHT to find the storage nodes, to -request items from them, and possibly re-announce those items to keep them -alive.

-
-
-

messages

-

The proposed new messages get and put are similar to the existing -get_peers and announce_peer.

-

Responses to get should always include nodes and nodes6. Those -fields have the same semantics as in its get_peers response. It should also -include a write token, token, with the same semantics as int get_peers. -The write token MAY be tied specifically to the key which get requested. -i.e. the token can only be used to store values under that one key. It may -also be tied to the node ID and IP address of the requesting node.

-

The id field in these messages has the same semantics as the standard DHT -messages, i.e. the node ID of the node sending the message, to maintain the -structure of the DHT network.

-

The token field also has the same semantics as the standard DHT message -get_peers and announce_peer, when requesting an item and to write an -item respectively.

-

The k field is the 32 byte ed25519 public key, which the signature can be -authenticated with. When looking up a mutable item, the target field MUST be -the SHA-1 hash of this key concatenated with the salt, if present.

-

The distinction between storing mutable and immutable items is the inclusion of -a public key, a sequence number, signature and an optional salt (k, seq, -sig and salt).

-

get requests for mutable items and immutable items cannot be distinguished -from eachother. An implementation can either store mutable and immutable items -in the same hash table internally, or in separate ones and potentially do two -lookups for get requests.

-

The v field is the value to be stored. It is allowed to be any bencoded -type (list, dict, string or integer). When it's being hashed (for verifying its -signature or to calculate its key), its flattened, bencoded, form is used. It is -important to use the verbatim bencoded representation as it appeared in the -message. decoding and then re-encoding bencoded structures is not necessarily an -identity operation.

-

Storing nodes MAY reject put requests where the bencoded form of v is -longer than 1000 bytes. In other words, it's not safe to assume storing more -than 1000 bytes will succeed.

-
-
-

immutable items

-

Immutable items are stored under their SHA-1 hash, and since they cannot be -modified, there is no need to authenticate the origin of them. This makes -immutable items simple.

-

A node making a lookup SHOULD verify the data it receives from the network, to -verify that its hash matches the target that was looked up.

-
-

put message

-

Request:

-
-{
-        "a":
-        {
-                "id": <20 byte id of sending node (string)>,
-                "v": <any bencoded type, whose encoded size <= 1000>
-        },
-        "t": <transaction-id (string)>,
-        "y": "q",
-        "q": "put"
-}
-
-

Response:

-
-{
-        "r": { "id": <20 byte id of sending node (string)> },
-        "t": <transaction-id (string)>,
-        "y": "r",
-}
-
-
-
-

get message

-

Request:

-
-{
-        "a":
-        {
-                "id": <20 byte id of sending node (string)>,
-                "target": <SHA-1 hash of item (string)>,
-        },
-        "t": <transaction-id (string)>,
-        "y": "q",
-        "q": "get"
-}
-
-

Response:

-
-{
-        "r":
-        {
-                "id": <20 byte id of sending node (string)>,
-                "token": <write token (string)>,
-                "v": <any bencoded type whose SHA-1 hash matches 'target'>,
-                "nodes": <IPv4 nodes close to 'target'>,
-                "nodes6": <IPv6 nodes close to 'target'>
-        },
-        "t": <transaction-id>,
-        "y": "r",
-}
-
-
-
-
-

mutable items

-

Mutable items can be updated, without changing their DHT keys. To authenticate -that only the original publisher can update an item, it is signed by a private -key generated by the original publisher. The target ID mutable items are stored -under is the SHA-1 hash of the public key (as it appears in the put -message).

-

In order to avoid a malicious node to overwrite the list head with an old -version, the sequence number seq must be monotonically increasing for each -update, and a node hosting the list node MUST not downgrade a list head from a -higher sequence number to a lower one, only upgrade. The sequence number SHOULD -not exceed MAX_INT64, (i.e. 0x7fffffffffffffff. A client MAY reject any -message with a sequence number exceeding this. A client MAY also reject any -message with a negative sequence number.

-

The signature is a 64 byte ed25519 signature of the bencoded sequence number -concatenated with the v key. e.g. something like this:

-
-3:seqi4e1:v12:Hello world!
-
-

If the salt key is present and non-empty, the salt string must be included -in what's signed. Note that if salt is specified and an empty string, it is -as if it was not specified and nothing in addition to the sequence number and -the data is signed. The salt string may not be longer than 64 bytes.

-

When a salt is included in what is signed, the key salt with the value of -the key is prepended in its bencoded form. For example, if salt is "foobar", -the buffer to be signed is:

-
-4:salt6:foobar3:seqi4e1:v12:Hello world!
-
-
-

put message

-

Request:

-
-{
-        "a":
-        {
-                "cas": <optional expected seq-nr (int)>,
-                "id": <20 byte id of sending node (string)>,
-                "k": <ed25519 public key (32 bytes string)>,
-                "salt": <optional salt to be appended to "k" when hashing (string)>
-                "seq": <monotonically increasing sequence number (integer)>,
-                "sig": <ed25519 signature (64 bytes string)>,
-                "token": <write-token (string)>,
-                "v": <any bencoded type, whose encoded size < 1000>
-        },
-        "t": <transaction-id (string)>,
-        "y": "q",
-        "q": "put"
-}
-
-

Storing nodes receiving a put request where seq is lower than or equal -to what's already stored on the node, MUST reject the request. If the sequence -number is equal, and the value is also the same, the node SHOULD reset its -timeout counter.

-

If the sequence number in the put message is lower than the sequence number -associated with the currently stored value, the storing node MAY return an error -message with code 302 (see error codes below).

-

Note that this request does not contain a target hash. The target hash under -which this blob is stored is implied by the k argument. The key is the SHA-1 -hash of the key (k).

-

In order to support a single key being used to store separate items in the DHT, -an optional salt can be specified in the put request of mutable items. -If the salt entry is not present, it can be assumed to be an empty string, and -its semantics should be identical as specifying a salt key with an empty string. -The salt can be any binary string (but probably most conveniently a hash of -something). This string is appended to the key, as specified in the k field, -when calculating the key to store the blob under (i.e. the key get requests -specify to retrieve this data).

-

This lets a single entity, with a single key, publish any number of unrelated -items, with a single key that readers can verify. This is useful if the -publisher doesn't know ahead of time how many different items are to be -published. It can distribute a single public key for users to authenticate the -published blobs.

-

Note that the salt is not returned in the response to a get request. This -is intentional. When issuing a get request for an item is expected to -know what the salt is (because it is part of what the target ID that is being -looked up is derived from). There is no need to repeat it back for bystanders -to see.

-
-
-

CAS

-

CAS is short for compare and swap, it has similar semantics as CAS CPU -instructions. It is used to avoid race conditions when multiple nodes are -writing to the same slot in the DHT.

-

The cas field is optional. If present it specifies the sequence number of -the data blob being overwritten by the put. When present, the storing node -MUST compare this number to the current sequence number it has stored under -this key. Only if the cas matches the stored sequence number is the put -performed. If it mismatches, the store fails and an error is returned. -See errors below.

-

The cas field only applies to mutable puts. If there is no current -value, the cas field SHOULD be ignored.

-

When sending a put request to a node that did not return any data for the -get, the cas field SHOULD NOT be included.

-
-
-

response

-

Response:

-
-{
-        "r": { "id": <20 byte id of sending node (string)> },
-        "t": <transaction-id (string)>,
-        "y": "r",
-}
-
-
-
-

errors

-

If the store fails for any reason an error message is returned instead of the -message template above, i.e. one where "y" is "e" and "e" is a tuple of -[error-code, message]). Failures include cas mismatches and the sequence -number is outdated.

-

The error message (as specified by BEP5) looks like this:

-
-{
-        "e": [ <error-code (integer)>, <error-string (string)> ],
-        "t": <transaction-id (string)>,
-        "y": "e",
-}
-
-

In addition to the error codes defined in BEP5, this specification defines -some additional error codes.

- ---- - - - - - - - - - - - - - - - - - - - - - - -
error-codedescription
205message (v field) -too big.
206invalid signature
207salt (salt field) -too big.
301the CAS hash mismatched, -re-read value and try -again.
302sequence number less than -current.
-

An implementation MUST emit 301 errors if the cas mismatches. This is a -critical feature in synchronization of multiple agents sharing an immutable -item.

-
-
-

get message

-

Request:

-
-{
-        "a":
-        {
-                "id": <20 byte id of sending node (string)>,
-                "target:" <20 byte SHA-1 hash of public key and salt (string)>
-        },
-        "t": <transaction-id (string)>,
-        "y": "q",
-        "q": "get"
-}
-
-

Response:

-
-{
-        "r":
-        {
-                "id": <20 byte id of sending node (string)>,
-                "k": <ed25519 public key (32 bytes string)>,
-                "nodes": <IPv4 nodes close to 'target'>,
-                "nodes6": <IPv6 nodes close to 'target'>,
-                "seq": <monotonically increasing sequence number (integer)>,
-                "sig": <ed25519 signature (64 bytes string)>,
-                "token": <write-token (string)>,
-                "v": <any bencoded type, whose encoded size <= 1000>
-        },
-        "t": <transaction-id (string)>,
-        "y": "r",
-}
-
-
-
-
-

signature verification

-

In order to make it maximally difficult to attack the bencoding parser, signing -and verification of the value and sequence number should be done as follows:

-
    -
  1. encode value and sequence number separately
  2. -
  3. concatenate ("4:salt" length-of-salt ":" salt) "3:seqi" seq -"e1:v" len ":" and the encoded value. -sequence number 1 of value "Hello World!" would be converted to: -"3:seqi1e1:v12:Hello World!". In this way it is not possible to convince a -node that part of the length is actually part of the sequence number even if -the parser contains certain bugs. Furthermore it is not possible to have a -verification failure if a bencoding serializer alters the order of entries in -the dictionary. The salt is in parenthesis because it is optional. It is only -prepended if a non-empty salt is specified in the put request.
  4. -
  5. sign or verify the concatenated string
  6. -
-

On the storage node, the signature MUST be verified before accepting the store -command. The data MUST be stored under the SHA-1 hash of the public key (as it -appears in the bencoded dict) and the salt (if present).

-

On the requesting nodes, the key they get back from a get request MUST be -verified to hash to the target ID the lookup was made for, as well as verifying -the signature. If any of these fail, the response SHOULD be considered invalid.

-
-
-

expiration

-

Without re-announcement, these items MAY expire in 2 hours. In order -to keep items alive, they SHOULD be re-announced once an hour.

-

Any node that's interested in keeping a blob in the DHT alive may announce it. -It would simply repeat the signature for a mutable put without having the -private key.

-
-
-

test vectors

-
-

test 1 (mutable)

-

value:

-
-12:Hello World!
-
-

buffer being signed:

-
-3:seqi1e1:v12:Hello World!
-
-

public key:

-
-77ff84905a91936367c01360803104f92432fcd904a43511876df5cdf3e7e548
-
-

private key:

-
-e06d3183d14159228433ed599221b80bd0a5ce8352e4bdf0262f76786ef1c74d
-b7e7a9fea2c0eb269d61e3b38e450a22e754941ac78479d6c54e1faf6037881d
-
-

target ID:

-
-4a533d47ec9c7d95b1ad75f576cffc641853b750
-
-

signature:

-
-305ac8aeb6c9c151fa120f120ea2cfb923564e11552d06a5d856091e5e853cff
-1260d3f39e4999684aa92eb73ffd136e6f4f3ecbfda0ce53a1608ecd7ae21f01
-
-
-
-

test 2 (mutable with salt)

-

value:

-
-12:Hello World!
-
-

salt:

-
-foobar
-
-

buffer being signed:

-
-4:salt6:foobar3:seqi1e1:v12:Hello World!
-
-

public key:

-
-77ff84905a91936367c01360803104f92432fcd904a43511876df5cdf3e7e548
-
-

private key:

-
-e06d3183d14159228433ed599221b80bd0a5ce8352e4bdf0262f76786ef1c74d
-b7e7a9fea2c0eb269d61e3b38e450a22e754941ac78479d6c54e1faf6037881d
-
-

target ID:

-
-411eba73b6f087ca51a3795d9c8c938d365e32c1
-
-

signature:

-
-6834284b6b24c3204eb2fea824d82f88883a3d95e8b4a21b8c0ded553d17d17d
-df9a8a7104b1258f30bed3787e6cb896fca78c58f8e03b5f18f14951a87d9a08
-
-
-
-

test 3 (immutable)

-

value:

-
-12:Hello World!
-
-

target ID:

-
-e5f96f6f38320f0f33959cb4d3d656452117aadb
-
-
-
-
-

resources

-

Libraries that implement ed25519 DSA:

- -
- -
-
-
- -
- -
- - diff --git a/docs/examples.html b/docs/examples.html deleted file mode 100644 index ac327bd23..000000000 --- a/docs/examples.html +++ /dev/null @@ -1,763 +0,0 @@ - - - - - - -libtorrent Examples - - - - - - - - -
-
- - - - -
-

libtorrent Examples

- --- - - - - - -
Author:Arvid Norberg, arvid@libtorrent.org
Version:1.1.0
-
-

Table of contents

- -
-
-

examples

-

Except for the example programs in this manual, there's also a bigger example -of a (little bit) more complete client, client_test. There are separate -instructions for how to use it here if you'd like to try it. Note that building -client_test also requires boost.regex and boost.program_options library.

-
-

simple client

-

This is a simple client. It doesn't have much output to keep it simple:

-
-#include <stdlib.h>
-#include <boost/make_shared.hpp>
-#include "libtorrent/entry.hpp"
-#include "libtorrent/bencode.hpp"
-#include "libtorrent/session.hpp"
-#include "libtorrent/torrent_info.hpp"
-
-int main(int argc, char* argv[])
-{
-  using namespace libtorrent;
-  namespace lt = libtorrent;
-
-  if (argc != 2)
-  {
-    fputs("usage: ./simple_client torrent-file\n"
-      "to stop the client, press return.\n", stderr);
-    return 1;
-  }
-
-  settings_pack sett;
-  sett.set_str(settings_pack::listen_interfaces, "0.0.0.0:6881");
-  lt::session s(sett);
-  error_code ec;
-  if (ec)
-  {
-    fprintf(stderr, "failed to open listen socket: %s\n", ec.message().c_str());
-    return 1;
-  }
-  add_torrent_params p;
-  p.save_path = "./";
-  p.ti = boost::make_shared<torrent_info>(std::string(argv[1]), boost::ref(ec), 0);
-  if (ec)
-  {
-    fprintf(stderr, "%s\n", ec.message().c_str());
-    return 1;
-  }
-  s.add_torrent(p, ec);
-  if (ec)
-  {
-    fprintf(stderr, "%s\n", ec.message().c_str());
-    return 1;
-  }
-
-  // wait for the user to end
-  char a;
-  scanf("%c\n", &a);
-  return 0;
-}
-
-
-
-

make_torrent

-

Shows how to create a torrent from a directory tree:

-
-#include "libtorrent/entry.hpp"
-#include "libtorrent/bencode.hpp"
-#include "libtorrent/torrent_info.hpp"
-#include "libtorrent/file.hpp"
-#include "libtorrent/storage.hpp"
-#include "libtorrent/hasher.hpp"
-#include "libtorrent/create_torrent.hpp"
-#include "libtorrent/file.hpp"
-#include "libtorrent/file_pool.hpp"
-#include "libtorrent/hex.hpp" // for from_hex
-
-#include <boost/bind.hpp>
-
-#ifdef TORRENT_WINDOWS
-#include <direct.h> // for _getcwd
-#endif
-
-using namespace libtorrent;
-
-int load_file(std::string const& filename, std::vector<char>& v, libtorrent::error_code& ec, int limit = 8000000)
-{
-  ec.clear();
-  FILE* f = fopen(filename.c_str(), "rb");
-  if (f == NULL)
-  {
-    ec.assign(errno, boost::system::system_category());
-    return -1;
-  }
-
-  int r = fseek(f, 0, SEEK_END);
-  if (r != 0)
-  {
-    ec.assign(errno, boost::system::system_category());
-    fclose(f);
-    return -1;
-  }
-  long s = ftell(f);
-  if (s < 0)
-  {
-    ec.assign(errno, boost::system::system_category());
-    fclose(f);
-    return -1;
-  }
-
-  if (s > limit)
-  {
-    fclose(f);
-    return -2;
-  }
-
-  r = fseek(f, 0, SEEK_SET);
-  if (r != 0)
-  {
-    ec.assign(errno, boost::system::system_category());
-    fclose(f);
-    return -1;
-  }
-
-  v.resize(s);
-  if (s == 0)
-  {
-    fclose(f);
-    return 0;
-  }
-
-  r = fread(&v[0], 1, v.size(), f);
-  if (r < 0)
-  {
-    ec.assign(errno, boost::system::system_category());
-    fclose(f);
-    return -1;
-  }
-
-  fclose(f);
-
-  if (r != s) return -3;
-
-  return 0;
-}
-
-std::string branch_path(std::string const& f)
-{
-  if (f.empty()) return f;
-
-#ifdef TORRENT_WINDOWS
-  if (f == "\\\\") return "";
-#endif
-  if (f == "/") return "";
-
-  int len = f.size();
-  // if the last character is / or \ ignore it
-  if (f[len-1] == '/' || f[len-1] == '\\') --len;
-  while (len > 0)
-  {
-    --len;
-    if (f[len] == '/' || f[len] == '\\')
-      break;
-  }
-
-  if (f[len] == '/' || f[len] == '\\') ++len;
-  return std::string(f.c_str(), len);
-}
-
-// do not include files and folders whose
-// name starts with a .
-bool file_filter(std::string const& f)
-{
-  if (f.empty()) return false;
-
-  char const* first = f.c_str();
-  char const* sep = strrchr(first, '/');
-#if defined(TORRENT_WINDOWS) || defined(TORRENT_OS2)
-  char const* altsep = strrchr(first, '\\');
-  if (sep == NULL || altsep > sep) sep = altsep;
-#endif
-  // if there is no parent path, just set 'sep'
-  // to point to the filename.
-  // if there is a parent path, skip the '/' character
-  if (sep == NULL) sep = first;
-  else ++sep;
-
-  // return false if the first character of the filename is a .
-  if (sep[0] == '.') return false;
-
-  fprintf(stderr, "%s\n", f.c_str());
-  return true;
-}
-
-void print_progress(int i, int num)
-{
-  fprintf(stderr, "\r%d/%d", i+1, num);
-}
-
-void print_usage()
-{
-  fputs("usage: make_torrent FILE [OPTIONS]\n"
-    "\n"
-    "Generates a torrent file from the specified file\n"
-    "or directory and writes it to standard out\n\n"
-    "OPTIONS:\n"
-    "-m file       generate a merkle hash tree torrent.\n"
-    "              merkle torrents require client support\n"
-    "              the resulting full merkle tree is written to\n"
-    "              the specified file\n"
-    "-w url        adds a web seed to the torrent with\n"
-    "              the specified url\n"
-    "-t url        adds the specified tracker to the\n"
-    "              torrent. For multiple trackers, specify more\n"
-    "              -t options\n"
-    "-c comment    sets the comment to the specified string\n"
-    "-C creator    sets the created-by field to the specified string\n"
-    "-p bytes      enables padding files. Files larger\n"
-    "              than bytes will be piece-aligned\n"
-    "-s bytes      specifies a piece size for the torrent\n"
-    "              This has to be a multiple of 16 kiB\n"
-    "-l            Don't follow symlinks, instead encode them as\n"
-    "              links in the torrent file\n"
-    "-o file       specifies the output filename of the torrent file\n"
-    "              If this is not specified, the torrent file is\n"
-    "              printed to the standard out, except on windows\n"
-    "              where the filename defaults to a.torrent\n"
-    "-r file       add root certificate to the torrent, to verify\n"
-    "              the HTTPS tracker\n"
-    "-S info-hash  add a similar torrent by info-hash. The similar\n"
-    "              torrent is expected to share some files with this one\n"
-    "-L collection add a collection name to this torrent. Other torrents\n"
-    "              in the same collection is expected to share files\n"
-    "              with this one.\n"
-    "-M            make the torrent compatible with mutable torrents\n"
-    "              this means aligning large files and pad them in order\n"
-    "              for piece hashes to uniquely indentify a file without\n"
-    "              overlap\n"
-    , stderr);
-}
-
-int main(int argc, char* argv[])
-{
-  using namespace libtorrent;
-
-  std::string creator_str = "libtorrent";
-  std::string comment_str;
-
-  if (argc < 2)
-  {
-    print_usage();
-    return 1;
-  }
-
-#ifndef BOOST_NO_EXCEPTIONS
-  try
-  {
-#endif
-    std::vector<std::string> web_seeds;
-    std::vector<std::string> trackers;
-    std::vector<std::string> collections;
-    std::vector<sha1_hash> similar;
-    int pad_file_limit = -1;
-    int piece_size = 0;
-    int flags = 0;
-    std::string root_cert;
-
-    std::string outfile;
-    std::string merklefile;
-#ifdef TORRENT_WINDOWS
-    // don't ever write binary data to the console on windows
-    // it will just be interpreted as text and corrupted
-    outfile = "a.torrent";
-#endif
-
-    for (int i = 2; i < argc; ++i)
-    {
-      if (argv[i][0] != '-')
-      {
-        print_usage();
-        return 1;
-      }
-
-      switch (argv[i][1])
-      {
-        case 'w':
-          ++i;
-          web_seeds.push_back(argv[i]);
-          break;
-        case 't':
-          ++i;
-          trackers.push_back(argv[i]);
-          break;
-        case 'M':
-          flags |= create_torrent::mutable_torrent_support;
-          pad_file_limit = 0x4000;
-          break;
-        case 'p':
-          ++i;
-          pad_file_limit = atoi(argv[i]);
-          flags |= create_torrent::optimize;
-          break;
-        case 's':
-          ++i;
-          piece_size = atoi(argv[i]);
-          break;
-        case 'm':
-          ++i;
-          merklefile = argv[i];
-          flags |= create_torrent::merkle;
-          break;
-        case 'o':
-          ++i;
-          outfile = argv[i];
-          break;
-        case 'l':
-          flags |= create_torrent::symlinks;
-          break;
-        case 'C':
-          ++i;
-          creator_str = argv[i];
-          break;
-        case 'c':
-          ++i;
-          comment_str = argv[i];
-          break;
-        case 'r':
-          ++i;
-          root_cert = argv[i];
-          break;
-        case 'S':
-          {
-          ++i;
-          if (strlen(argv[i]) != 40)
-          {
-            fprintf(stderr, "invalid info-hash for -S. "
-              "Expected 40 hex characters\n");
-            print_usage();
-            return 1;
-          }
-          sha1_hash ih;
-          if (!from_hex(argv[i], 40, (char*)&ih[0]))
-          {
-            fprintf(stderr, "invalid info-hash for -S\n");
-            print_usage();
-            return 1;
-          }
-          similar.push_back(ih);
-          }
-          break;
-        case 'L':
-          ++i;
-          collections.push_back(argv[i]);
-          break;
-        default:
-          print_usage();
-          return 1;
-      }
-    }
-
-    file_storage fs;
-    std::string full_path = argv[1];
-#ifdef TORRENT_WINDOWS
-    if (full_path[1] != ':')
-#else
-    if (full_path[0] != '/')
-#endif
-    {
-      char cwd[TORRENT_MAX_PATH];
-#ifdef TORRENT_WINDOWS
-      _getcwd(cwd, sizeof(cwd));
-      full_path = cwd + ("\\" + full_path);
-#else
-      getcwd(cwd, sizeof(cwd));
-      full_path = cwd + ("/" + full_path);
-#endif
-    }
-
-    add_files(fs, full_path, file_filter, flags);
-    if (fs.num_files() == 0)
-    {
-      fputs("no files specified.\n", stderr);
-      return 1;
-    }
-
-    create_torrent t(fs, piece_size, pad_file_limit, flags);
-    int tier = 0;
-    for (std::vector<std::string>::iterator i = trackers.begin()
-      , end(trackers.end()); i != end; ++i, ++tier)
-      t.add_tracker(*i, tier);
-
-    for (std::vector<std::string>::iterator i = web_seeds.begin()
-      , end(web_seeds.end()); i != end; ++i)
-      t.add_url_seed(*i);
-
-    for (std::vector<std::string>::iterator i = collections.begin()
-      , end(collections.end()); i != end; ++i)
-      t.add_collection(*i);
-
-    for (std::vector<sha1_hash>::iterator i = similar.begin()
-      , end(similar.end()); i != end; ++i)
-      t.add_similar_torrent(*i);
-
-    error_code ec;
-    set_piece_hashes(t, branch_path(full_path)
-      , boost::bind(&print_progress, _1, t.num_pieces()), ec);
-    if (ec)
-    {
-      fprintf(stderr, "%s\n", ec.message().c_str());
-      return 1;
-    }
-
-    fprintf(stderr, "\n");
-    t.set_creator(creator_str.c_str());
-    if (!comment_str.empty())
-      t.set_comment(comment_str.c_str());
-
-    if (!root_cert.empty())
-    {
-      std::vector<char> pem;
-      load_file(root_cert, pem, ec, 10000);
-      if (ec)
-      {
-        fprintf(stderr, "failed to load root certificate for tracker: %s\n", ec.message().c_str());
-      }
-      else
-      {
-        t.set_root_cert(std::string(&pem[0], pem.size()));
-      }
-    }
-
-    // create the torrent and print it to stdout
-    std::vector<char> torrent;
-    bencode(back_inserter(torrent), t.generate());
-    FILE* output = stdout;
-    if (!outfile.empty())
-      output = fopen(outfile.c_str(), "wb+");
-    if (output == NULL)
-    {
-      fprintf(stderr, "failed to open file \"%s\": (%d) %s\n"
-        , outfile.c_str(), errno, strerror(errno));
-      return 1;
-    }
-    fwrite(&torrent[0], 1, torrent.size(), output);
-
-    if (output != stdout)
-      fclose(output);
-
-    if (!merklefile.empty())
-    {
-      output = fopen(merklefile.c_str(), "wb+");
-      if (output == NULL)
-      {
-        fprintf(stderr, "failed to open file \"%s\": (%d) %s\n"
-          , merklefile.c_str(), errno, strerror(errno));
-        return 1;
-      }
-      int ret = fwrite(&t.merkle_tree()[0], 20, t.merkle_tree().size(), output);
-      if (ret != int(t.merkle_tree().size()))
-      {
-        fprintf(stderr, "failed to write %s: (%d) %s\n"
-          , merklefile.c_str(), errno, strerror(errno));
-      }
-      fclose(output);
-    }
-
-#ifndef BOOST_NO_EXCEPTIONS
-  }
-  catch (std::exception& e)
-  {
-    fprintf(stderr, "%s\n", e.what());
-  }
-#endif
-
-  return 0;
-}
-
-
-
-

dump_torrent

-

This is an example of a program that will take a torrent-file as a parameter and -print information about it to std out:

-
-#include "libtorrent/entry.hpp"
-#include "libtorrent/bencode.hpp"
-#include "libtorrent/torrent_info.hpp"
-#include "libtorrent/announce_entry.hpp"
-#include "libtorrent/bdecode.hpp"
-#include "libtorrent/magnet_uri.hpp"
-
-int load_file(std::string const& filename, std::vector<char>& v
-  , libtorrent::error_code& ec, int limit = 8000000)
-{
-  ec.clear();
-  FILE* f = fopen(filename.c_str(), "rb");
-  if (f == NULL)
-  {
-    ec.assign(errno, boost::system::system_category());
-    return -1;
-  }
-
-  int r = fseek(f, 0, SEEK_END);
-  if (r != 0)
-  {
-    ec.assign(errno, boost::system::system_category());
-    fclose(f);
-    return -1;
-  }
-  long s = ftell(f);
-  if (s < 0)
-  {
-    ec.assign(errno, boost::system::system_category());
-    fclose(f);
-    return -1;
-  }
-
-  if (s > limit)
-  {
-    fclose(f);
-    return -2;
-  }
-
-  r = fseek(f, 0, SEEK_SET);
-  if (r != 0)
-  {
-    ec.assign(errno, boost::system::system_category());
-    fclose(f);
-    return -1;
-  }
-
-  v.resize(s);
-  if (s == 0)
-  {
-    fclose(f);
-    return 0;
-  }
-
-  r = fread(&v[0], 1, v.size(), f);
-  if (r < 0)
-  {
-    ec.assign(errno, boost::system::system_category());
-    fclose(f);
-    return -1;
-  }
-
-  fclose(f);
-
-  if (r != s) return -3;
-
-  return 0;
-}
-
-int main(int argc, char* argv[])
-{
-  using namespace libtorrent;
-
-  if (argc < 2 || argc > 4)
-  {
-    fputs("usage: dump_torrent torrent-file [total-items-limit] [recursion-limit]\n", stderr);
-    return 1;
-  }
-
-  int item_limit = 1000000;
-  int depth_limit = 1000;
-
-  if (argc > 2) item_limit = atoi(argv[2]);
-  if (argc > 3) depth_limit = atoi(argv[3]);
-
-  std::vector<char> buf;
-  error_code ec;
-  int ret = load_file(argv[1], buf, ec, 40 * 1000000);
-  if (ret == -1)
-  {
-    fprintf(stderr, "file too big, aborting\n");
-    return 1;
-  }
-
-  if (ret != 0)
-  {
-    fprintf(stderr, "failed to load file: %s\n", ec.message().c_str());
-    return 1;
-  }
-  bdecode_node e;
-  int pos = -1;
-  printf("decoding. recursion limit: %d total item count limit: %d\n"
-    , depth_limit, item_limit);
-  ret = bdecode(&buf[0], &buf[0] + buf.size(), e, ec, &pos
-    , depth_limit, item_limit);
-
-  printf("\n\n----- raw info -----\n\n%s\n", print_entry(e).c_str());
-
-  if (ret != 0)
-  {
-    fprintf(stderr, "failed to decode: '%s' at character: %d\n", ec.message().c_str(), pos);
-    return 1;
-  }
-
-  torrent_info t(e, ec);
-  if (ec)
-  {
-    fprintf(stderr, "%s\n", ec.message().c_str());
-    return 1;
-  }
-  e.clear();
-  std::vector<char>().swap(buf);
-
-  // print info about torrent
-  printf("\n\n----- torrent file info -----\n\n"
-    "nodes:\n");
-
-  typedef std::vector<std::pair<std::string, int> > node_vec;
-  node_vec const& nodes = t.nodes();
-  for (node_vec::const_iterator i = nodes.begin(), end(nodes.end());
-    i != end; ++i)
-  {
-    printf("%s: %d\n", i->first.c_str(), i->second);
-  }
-  puts("trackers:\n");
-  for (std::vector<announce_entry>::const_iterator i = t.trackers().begin();
-    i != t.trackers().end(); ++i)
-  {
-    printf("%2d: %s\n", i->tier, i->url.c_str());
-  }
-
-  char ih[41];
-  to_hex((char const*)&t.info_hash()[0], 20, ih);
-  printf("number of pieces: %d\n"
-    "piece length: %d\n"
-    "info hash: %s\n"
-    "comment: %s\n"
-    "created by: %s\n"
-    "magnet link: %s\n"
-    "name: %s\n"
-    "number of files: %d\n"
-    "files:\n"
-    , t.num_pieces()
-    , t.piece_length()
-    , ih
-    , t.comment().c_str()
-    , t.creator().c_str()
-    , make_magnet_uri(t).c_str()
-    , t.name().c_str()
-    , t.num_files());
-  file_storage const& st = t.files();
-  for (int i = 0; i < st.num_files(); ++i)
-  {
-    int first = st.map_file(i, 0, 0).piece;
-    int last = st.map_file(i, (std::max)(boost::int64_t(st.file_size(i))-1, boost::int64_t(0)), 0).piece;
-    int flags = st.file_flags(i);
-    printf(" %8" PRIx64 " %11" PRId64 " %c%c%c%c [ %5d, %5d ] %7u %s %s %s%s\n"
-      , st.file_offset(i)
-      , st.file_size(i)
-      , ((flags & file_storage::flag_pad_file)?'p':'-')
-      , ((flags & file_storage::flag_executable)?'x':'-')
-      , ((flags & file_storage::flag_hidden)?'h':'-')
-      , ((flags & file_storage::flag_symlink)?'l':'-')
-      , first, last
-      , boost::uint32_t(st.mtime(i))
-      , st.hash(i) != sha1_hash(0) ? to_hex(st.hash(i).to_string()).c_str() : ""
-      , st.file_path(i).c_str()
-      , (flags & file_storage::flag_symlink) ? "-> " : ""
-      , (flags & file_storage::flag_symlink) ? st.symlink(i).c_str() : "");
-  }
-
-  return 0;
-}
-
-
-
- -
-
-
- -
- -
- - diff --git a/docs/extension_protocol.html b/docs/extension_protocol.html deleted file mode 100644 index ce554488a..000000000 --- a/docs/extension_protocol.html +++ /dev/null @@ -1,498 +0,0 @@ - - - - - - - - - - - - - - - -
-
- - - - -
- - --- - - - -
Author:Arvid Norberg, arvid@libtorrent.org -Ludvig Strigeus, ludde@utorrent.com
-
-

extension protocol for bittorrent

-

The intention of this protocol is to provide a simple and thin transport -for extensions to the bittorrent protocol. Supporting this protocol makes -it easy to add new extensions without interfering with the standard -bittorrent protocol or clients that don't support this extension or the -one you want to add.

-

To advertise to other clients that you support, one bit from the reserved -bytes is used.

-

The bit selected for the extension protocol is bit 20 from the right (counting -starts at 0). So (reserved_byte[5] & 0x10) is the expression to use for checking -if the client supports extended messaging.

-

Once support for the protocol is established, the client is supposed to -support 1 new message:

- ---- - - - - - - - - - - -
nameid
extended20
-

This message is sent as any other bittorrent message, with a 4 byte length -prefix and a single byte identifying the message (the single byte being 20 -in this case). At the start of the payload of the message, is a single byte -message identifier. This identifier can refer to different extension messages -and only one ID is specified, 0. If the ID is 0, the message is a handshake -message which is described below. The layout of a general extended message -follows (including the message headers used by the bittorrent protocol):

- ---- - - - - - - - - - - - - - - - - -
sizedescription
uint32_tlength prefix. Specifies the number of bytes for the -entire message. (Big endian)
uint8_tbittorrent message ID, = 20
uint8_textended message ID. 0 = handshake, >0 = extended -message as specified by the handshake.
-
-

handshake message

-

The payload of the handshake message is a bencoded dictionary. All items -in the dictionary are optional. Any unknown names should be ignored -by the client. All parts of the dictionary are case sensitive. -This is the defined item in the dictionary:

- ---- - - - - - - - - - - -
namedescription
m

Dictionary of supported extension messages which maps -names of extensions to an extended message ID for each -extension message. The only requirement on these IDs -is that no extension message share the same one. Setting -an extension number to zero means that the extension is -not supported/disabled. The client should ignore any -extension names it doesn't recognize.

-

The extension message IDs are the IDs used to send the -extension messages to the peer sending this handshake. -i.e. The IDs are local to this particular peer.

-
-

Here are some other items that an implementation may choose to support:

- ---- - - - - - - - - - - - - - - - - - - - - - - - - - -
namedescription
pLocal TCP listen port. Allows each side to learn about -the TCP port number of the other side. Note that there is -no need for the receiving side of the connection to send -this extension message, since its port number is already -known.
vClient name and version (as a utf-8 string). -This is a much more reliable way of identifying the -client than relying on the peer id encoding.
youripA string containing the compact representation of the ip -address this peer sees you as. i.e. this is the -receiver's external ip address (no port is included). -This may be either an IPv4 (4 bytes) or an IPv6 -(16 bytes) address.
ipv6If this peer has an IPv6 interface, this is the compact -representation of that address (16 bytes). The client may -prefer to connect back via the IPv6 address.
ipv4If this peer has an IPv4 interface, this is the compact -representation of that address (4 bytes). The client may -prefer to connect back via this interface.
reqqAn integer, the number of outstanding request messages -this client supports without dropping any. The default in -in libtorrent is 250.
-

The handshake dictionary could also include extended handshake -information, such as support for encrypted headers or anything -imaginable.

-

An example of what the payload of a handshake message could look like:

- ---- - - - - - - - - - - - - - - - -
Dictionary
m ---- - - - - - - - - - - - - -
Dictionary
LT_metadata1
ut_pex2
-
p6881
v"uTorrent 1.2"
-

and in the encoded form:

-

d1:md11:LT_metadatai1e6:ut_pexi2ee1:pi6881e1:v12:uTorrent 1.2e

-

To make sure the extension names do not collide by mistake, they should be -prefixed with the two (or one) character code that is used to identify the -client that introduced the extension. This applies for both the names of -extension messages, and for any additional information put inside the -top-level dictionary. All one and two byte identifiers are invalid to use -unless defined by this specification.

-

This message should be sent immediately after the standard bittorrent handshake -to any peer that supports this extension protocol. It is valid to send the -handshake message more than once during the lifetime of a connection, -the sending client should not be disconnected. An implementation may choose -to ignore the subsequent handshake messages (or parts of them).

-

Subsequent handshake messages can be used to enable/disable extensions -without restarting the connection. If a peer supports changing extensions -at run time, it should note that the m dictionary is additive. -It's enough that it contains the actual CHANGES to the extension list. -To disable the support for LT_metadata at run-time, without affecting -any other extensions, this message should be sent: -d11:LT_metadatai0ee. -As specified above, the value 0 is used to turn off an extension.

-

The extension IDs must be stored for every peer, becuase every peer may have -different IDs for the same extension.

-

This specification, deliberately, does not specify any extensions such as -peer-exchange or metadata exchange. This protocol is merely a transport -for the actual extensions to the bittorrent protocol and the extensions -named in the example above (such as p) are just examples of possible -extensions.

-
-
-

rationale

-

The reason why the extension messages' IDs would be defined in the handshake -is to avoid having a global registry of message IDs. Instead the names of the -extension messages requires unique names, which is much easier to do without -a global registry. The convention is to use a two letter prefix on the -extension message names, the prefix would identify the client first -implementing the extension message. e.g. LT_metadata is implemented by -libtorrent, and hence it has the LT prefix.

-

If the client supporting the extensions can decide which numbers the messages -it receives will have, it means they are constants within that client. i.e. -they can be used in switch statements. It's easy for the other end to -store an array with the ID's we expect for each message and use that for -lookups each time it sends an extension message.

-

The reason for having a dictionary instead of having an array (using -implicitly assigned index numbers to the extensions) is that if a client -want to disable some extensions, the ID numbers would change, and it wouldn't -be able to use constants (and hence, not use them in a switch). If the -messages IDs would map directly to bittorrent message IDs, It would also make -it possible to map extensions in the handshake to existing extensions with -fixed message IDs.

-

The reasoning behind having a single byte as extended message identifier is -to follow the the bittorrent spec. with its single byte message identifiers. -It is also considered to be enough. It won't limit the total number of -extensions, only the number of extensions used simultaneously.

-

The reason for using single byte identifiers for the standardized handshake -identifiers is 1) The mainline DHT uses single byte identifiers. 2) Saves -bandwidth. The only advantage of longer messages is that it makes the -protocol more readable for a human, but the BT protocol wasn't designed to -be a human readable protocol, so why bother.

-
-
-
-

extensions

-

These extensions all operates within the extension protocol. The name of the -extension is the name used in the extension-list packets, and the payload is -the data in the extended message (not counting the length-prefix, message-id -nor extension-id).

-

Note that since this protocol relies on one of the reserved bits in the -handshake, it may be incompatible with future versions of the mainline -bittorrent client.

-

These are the extensions that are currently implemented.

-
-

metadata from peers

-

Extension name: "LT_metadata"

-
-

Note

-

This extension is deprecated in favor of the more widely supported -ut_metadata extension, see BEP 9.

-
-

The point with this extension is that you don't have to distribute the -metadata (.torrent-file) separately. The metadata can be distributed -through the bittorrent swarm. The only thing you need to download such -a torrent is the tracker url and the info-hash of the torrent.

-

It works by assuming that the initial seeder has the metadata and that the -metadata will propagate through the network as more peers join.

-

There are three kinds of messages in the metadata extension. These packets are -put as payload to the extension message. The three packets are:

-
-
    -
  • request metadata
  • -
  • metadata
  • -
  • don't have metadata
  • -
-
-

request metadata:

- ----- - - - - - - - - - - - - - - - - - - - - -
sizenamedescription
uint8_tmsg_typeDetermines the kind of message this is -0 means 'request metadata'
uint8_tstartThe start of the metadata block that -is requested. It is given in 256:ths -of the total size of the metadata, -since the requesting client don't know -the size of the metadata.
uint8_tsizeThe size of the metadata block that is -requested. This is also given in -256:ths of the total size of the -metadata. The size is given as size-1. -That means that if this field is set -0, the request wants one 256:th of the -metadata.
-

metadata:

- ----- - - - - - - - - - - - - - - - - - - - - - - - - -
sizenamedescription
uint8_tmsg_type1 means 'metadata'
int32_ttotal_sizeThe total size of the metadata, given -in number of bytes.
int32_toffsetThe offset of where the metadata block -in this message belongs in the final -metadata. This is given in bytes.
uint8_t[]metadataThe actual metadata block. The size of -this part is given implicit by the -length prefix in the bittorrent -protocol packet.
-

Don't have metadata:

- ----- - - - - - - - - - - - - -
sizenamedescription
uint8_tmsg_type2 means 'I don't have metadata'. -This message is sent as a reply to a -metadata request if the the client -doesn't have any metadata.
-
-
-

dont_have

-

Extension name: "lt_donthave"

-

The dont_have extension message is used to tell peers that the client no -longer has a specific piece. The extension message should be advertised in the -m dictionary as lt_donthave. The message format mimics the regular -HAVE bittorrent message.

-

Just like all extension messages, the first 2 bytes in the mssage itself are 20 -(the bittorrent extension message) and the message ID assigned to this -extension in the m dictionary in the handshake.

- ----- - - - - - - - - - - - - -
sizenamedescription
uint32_tpieceindex of the piece the peer no longer -has.
-

The length of this message (including the extension message prefix) is 6 bytes, -i.e. one byte longer than the normal HAVE message, because of the extension -message wrapping.

-
-
- -
-
-
- -
- -
- - diff --git a/docs/features.html b/docs/features.html deleted file mode 100644 index efe362179..000000000 --- a/docs/features.html +++ /dev/null @@ -1,415 +0,0 @@ - - - - - - -libtorrent manual - - - - - - - - -
-
- - - - -
-

libtorrent manual

- --- - - - - - -
Author:Arvid Norberg, arvid@libtorrent.org
Version:1.1.0
- -
-

introduction

-

libtorrent is a feature complete C++ bittorrent implementation focusing -on efficiency and scalability. It runs on embedded devices as well as -desktops. It boasts a well documented library interface that is easy to -use. It comes with a simple bittorrent client demonstrating the use of -the library.

-
-
-

features

-

libtorrent is an ongoing project under active development. Its -current state supports and includes the following features:

-
-

extensions

-
    -
  • plugin interface for implementing custom bittorrent extensions -without having to modify libtorrent
  • -
  • supports trackerless torrents (using the Mainline kademlia DHT protocol) with -some DHT extensions. BEP 5.
  • -
  • supports the bittorrent extension protocol. See extensions. BEP 10.
  • -
  • supports the uTorrent metadata transfer protocol BEP 9 (i.e. magnet links).
  • -
  • supports the uTorrent peer exchange protocol (PEX).
  • -
  • supports local peer discovery (multicasts for peers on the same local network)
  • -
  • multitracker extension support (supports both strict BEP 12 and the -uTorrent interpretation).
  • -
  • tracker scrapes
  • -
  • supports lt_trackers extension, to exchange trackers between peers
  • -
  • HTTP seeding, as specified in BEP 17 and BEP 19.
  • -
  • supports the udp-tracker protocol. (BEP 15).
  • -
  • supports the no_peer_id=1 extension that will ease the load off trackers.
  • -
  • supports the compact=1 tracker parameter.
  • -
  • super seeding/initial seeding (BEP 16).
  • -
  • private torrents (BEP 27).
  • -
  • upload-only extension (BEP 21).
  • -
  • support for IPv6, including BEP 7 and BEP 24.
  • -
  • support for merkle hash tree torrents. This makes the size of torrent files -scale well with the size of the content.
  • -
  • share-mode. This is a special mode torrents can be put in to optimize share -ratio rather than downloading the torrent.
  • -
-
-
-

disk management

-
    -
  • can use multipled disk I/O threads to not have the disk block network or -client interaction.
  • -
  • supports verifying the SHA-1 hash of pieces in multiple threads, to take -advantage of multi core machines.
  • -
  • supports files > 2 gigabytes.
  • -
  • fast resume support, a way to avoid the costly piece check at the -start of a resumed torrent. Saves the storage state, piece_picker state -as well as all local peers in a fast-resume file.
  • -
  • has an adjustable read and write disk cache for improved disk throughput.
  • -
  • queues torrents for file check, instead of checking all of them in parallel.
  • -
  • does not have any requirements on the piece order in a torrent that it -resumes. This means it can resume a torrent downloaded by any client.
  • -
  • seed mode, where the files on disk are assumed to be complete, and each -piece's hash is verified the first time it is requested.
  • -
  • implements an ARC disk cache, tuned for performing well under bittorrent work -loads
  • -
-
-
-

network

-
    -
  • a high quality uTP implementation (BEP 29). A transport protocol with -delay based congestion control. See separate article.
  • -
  • adjusts the length of the request queue depending on download rate.
  • -
  • serves multiple torrents on a single port and in a single thread
  • -
  • piece picking on block-level (as opposed to piece-level). -This means it can download parts of the same piece from different peers. -It will also prefer to download whole pieces from single peers if the -download speed is high enough from that particular peer.
  • -
  • supports http proxies and basic proxy authentication
  • -
  • supports gzipped tracker-responses
  • -
  • can limit the upload and download bandwidth usage and the maximum number of -unchoked peers
  • -
  • possibility to limit the number of connections.
  • -
  • delays have messages if there's no other outgoing traffic to the peer, and -doesn't send have messages to peers that already has the piece. This saves -bandwidth.
  • -
  • selective downloading. The ability to select which parts of a torrent you -want to download.
  • -
  • ip filter to disallow ip addresses and ip ranges from connecting and -being connected.
  • -
  • NAT-PMP and UPnP support (automatic port mapping on routers that supports it)
  • -
  • implements automatic upload slots, to optimize download rate without spreading -upload capacity too thin. The number of upload slots is adjusted based on the -peers' download capacity to work even for connections that are orders of -magnitude faster than others.
  • -
-
-
-
-

highlighted features

-
-

disk caching

-

All disk I/O in libtorrent is done asynchronously to the network thread, by the -disk io thread. When a block is read, the disk io thread reads all subsequent -blocks from that piece into the read cache, assuming that the peer requesting -the block will also request more blocks from the same piece. This decreases the -number of syscalls for reading data. It also decreases delay from seeking.

-

Similarly, for write requests, blocks are cached and flushed to disk once one full -piece is complete or the piece is the least recently updated one when more cache -space is needed. The cache dynamically allocates space between the write and read -cache. The write cache is strictly prioritized over the read cache.

-

The cache blocks that are in used, are locked into physical memory to avoid it -being paged out to disk. Allowing the disk cache to be paged out to disk means -that it would become extremely inefficient to flush it, since it would have to be -read back into physical memory only to be flushed back out to disk again.

-

In order to conserve memory, and system calls, iovec file operations are -used to flush multiple cache blocks in a single call.

-

On low-memory systems, the disk cache can be disabled altogether or set to smaller -limit, to save memory.

-

The disk caching algorithm is configurable between 'LRU' and 'largest contiguous'. -The largest contiguous algorithm is the default and flushes the largest contiguous -block of buffers, instead of flushing all blocks belonging to the piece which was -written to least recently.

-

For version 0.15 a lot of work went into optimizing the cache algorithm, trying -to increase the cache hit rate and utilization. The graph to the left shows the -memory utilization in 0.14. This cache is a straight forward, fairly naive, implementation. -Every block read will also read all subsequent blocks in that piece into the cache. -Whenever we need more space, the entire oldest piece is evicted from the cache. Caching -writes always takes presedence over the read cache. Whenever a piece is fully downloaded, -it is flushed to disk.

-disk_buffer_before_optimization.png -disk_buffer.png -

The left graph shows the problem of evicting entire pieces at a time, and waiting until -an entire piece is downloaded until flushing it. These graphs were generated for a torrent -with fairly large pieces. This means that granularity was poor in 0.14, since it only -dealt with entire pieces. In 0.15, the granularity problem has been fixed by evicting one -block at a time from the read cache. This maximizes the read cache utilization. The write -cache is also flushed when a sufficient number of contiguous blocks have been downloaded -for a piece, which is not tied to the piece size anymore. This way the cache scales a lot -better with piece sizes.

-

The graph to the right shows the same download but with the new optimized disk cache -algorithm. It clearly shows an increased utilization, which means higher read hit rates -or smaller caches with maintained hit rate.

-
-
-

high performance disk subsystem

-

In some circumstances, the disk cache may not suffice to provide maximum performance. -One such example is high performance seeding, to a large number of peers, over a fast -up-link. In such a case, the amount of RAM may simply not be enough to cache disk -reads. When there's not enough RAM to cache disk reads, the disk throughput would -typically degrade to perform as poorly as with no cache at all, with the majority -of the time spent waiting for the disk head to seek.

-

To solve this problem, libtorrent sorts read requests by their physical offset on the -disk. They are processed by having the disk read head sweep back and forth over the drive.

-

This makes libtorrent very suitable for large scale, high-throughput seeding.

-disk_access_no_elevator.png -disk_access_elevator.png -

These plots illustrates the physical disk offset for reads over time. The left plot -is of a run where disk operation re-ordering is turned off and the righ is when it's -turned on. The right one has a relatively smooth sine wave shape whereas the left -one is more random and involves much longer seeks back and forth over the disk.

-

True physical disk offset queries are only supported on newer linux kernels, Mac OS X and -Windows 2000 and up.

-
-
-

network buffers

-

On CPUs with small L2 caches, copying memory can be expensive operations. It is important -to keep copying to a minimum on such machines. This mostly applies to embedded systems.

-

In order to minimize the number of times received data is copied, the receive buffer -for payload data is received directly into a page aligned disk buffer. If the connection -is encrypted, the buffer is decrypted in-place. The buffer is then moved into the disk -cache without being copied. Once all the blocks for a piece have been received, or the -cache needs to be flushed, all the blocks are passed directly to writev() to flush -them in a single syscall. This means a single copy into user space memory, and a single -copy back into kernel memory, as illustrated by this figure:

-write_disk_buffers.png -

When seeding and uploading in general, unnecessary copying is avoided by caching blocks -in aligned buffers, that are copied once into the peer's send buffer. The peer's send buffer -is not guaranteed to be aligned, even though it is most of the time. The send buffer is -then encrypted with the peer specific key and chained onto the iovec for sending. -This means there is one user space copy in order to allow unaligned peer requests and -peer-specific encryption. This is illustrated by the following figure:

-read_disk_buffers.png -
-
-

piece picker

-

The piece picker is a central component in a bittorrent implementation. The piece picker -in libtorrent is optimized for quickly finding the rarest pieces. It keeps a list of all -available pieces sorted by rarity, and pieces with the same rarity, shuffled. The rarest -first mode is the dominant piece picker mode. Other modes are supported as well, and -used by peers in specific situations.

-

The piece picker allows to combine the availability of a piece with a priority. Together -they determine the sort order of the piece list. Pieces with priority 0 will never be -picked, which is used for the selective download feature.

-

In order to have as few partially finished pieces as possible, peers have an affinity -towards picking blocks from the same pieces as other peers in the same speed category. -The speed category is a coarse categorization of peers based on their download rate. This -makes slow peers pick blocks from the same piece, and fast peers pick from the same piece, -and hence decreasing the likelihood of slow peers blocking the completion of pieces.

-

The piece picker can also be set to download pieces in sequential order.

-
-
-

share mode

-

The share mode feature in libtorrent is intended for users who are only interested in -helping out swarms, not downloading the torrents.

-

It works by predicting the demand for pieces, and only download pieces if there is enough -demand. New pieces will only be downloaded once the share ratio has hit a certain target.

-

This feature is especially useful when combined with RSS, so that a client can be set up -to provide additional bandwidth to an entire feed.

-
-
-

merkle hash tree torrents

-merkle_tree.png -

Merkle hash tree torrents is an extension that lets a torrent file only contain the -root hash of the hash tree forming the piece hashes. The main benefit of this feature -is that regardless of how many pieces there is in a torrent, the .torrent file will -always be the same size. It will only grow with the number of files (since it still -has to contain the file names).

-

With regular torrents, clients have to request multiple blocks for pieces, typically -from different peers, before the data can be verified against the piece hash. The -larger the pieces are, the longer it will take to download a complete piece and verify -it. Before the piece is verified, it cannot be shared with the swarm, which means the -larger piece sizes, the slower turnaround data has when it is downloaded by peers. -Since on average the data has to sit around, waiting, in client buffers before it has -been verified and can be uploaded again.

-

Another problem with large piece sizes is that it is harder for a client to pinpoint -the malicious or buggy peer when a piece fails, and it will take longer to re-download -it and take more tries before the piece succeeds the larger the pieces are.

-

The piece size in regular torrents is a tradeoff between the size of the .torrent file -itself and the piece size. Often, for files that are 4 GB, the piece size is 2 or 4 MB, -just to avoid making the .torrent file too big.

-

Merkle torrents solves these problems by removing the tradeoff between .torrent size and -piece size. With merkle torrents, the piece size can be the minimum block size (16 kB), -which lets peers verify every block of data received from peers, immediately. This -gives a minimum turnaround time and completely removes the problem of identifying malicious -peers.

-

The root hash is built by hashing all the piece hashes pair-wise, until they all collapse -down to the root.

-
-
-

customizable file storage

-storage.png -

libtorrent's storage implementation is customizable. That means a special purpose bittorrent -client can replace the default way to store files on disk.

-

When implementing a bittorrent cache, it doesn't matter how the data is stored on disk, as -long as it can be retrieved and seeded. In that case a new storage class can be implemented -(inheriting from the storage_interface class) that avoids the unnecessary step of mapping -slots to files and offsets. The storage can ignore the file boundaries and just store the -entire torrent in a single file (which will end up being all the files concatenated). The main -advantage of this, other than a slight cpu performance gain, is that all file operations would -be page (and sector) aligned. This enables efficient unbuffered I/O, and can potentially -lead to more efficient read caching (using the built in disk cache rather than relying on the -operating system's disk cache).

-

The storage interface supports operating systems where you can ask for sparse regions -(such as Windows and Solaris). The advantage of this is that when checking files, the regions -that are known to be sparse can be skipped, which can reduce the time to check a torrent -significantly.

-
-
-

easy to use API

-

One of the design goals of the libtorrent API is to make common operations simple, but still -have it possible to do complicated and advanced operations. This is best illustrated by example -code to implement a simple bittorrent client:

-
-#include <iostream>
-#include "libtorrent/session.hpp"
-
-// usage a.out [torrent-file]
-int main(int argc, char* argv[]) try
-{
-        using namespace libtorrent;
-
-        session s;
-        s.listen_on(std::make_pair(6881, 6889));
-        add_torrent_params p;
-        p.save_path = "./";
-        p.ti = new torrent_info(argv[1]);
-        s.add_torrent(p);
-
-        // wait for the user to end
-        char a;
-        std::cin.unsetf(std::ios_base::skipws);
-        std::cin >> a;
-        return 0;
-}
-catch (std::exception& e)
-{
-        std::cerr << ec.what() << std::endl;
-        return 1;
-}
-
-

This client doesn't give the user any status information or progress about the torrent, but -it is fully functional.

-

libtorrent also comes with python bindings for easy access for python developers.

-
-
-
-

portability

-

libtorrent runs on most major operating systems, including Windows, -MacOS X, Linux, BSD and Solaris. -It uses Boost.Thread, Boost.Filesystem, Boost.Date_time and various other -boost libraries. At least version 1.46.1 of boost is required.

-

libtorrent uses asio, hence it will take full advantage of high performance -network APIs on the most popular platforms. I/O completion ports on windows, -epoll on linux and kqueue on MacOS X and BSD.

-

libtorrent does not build with the following compilers:

-
    -
  • GCC 2.95.4
  • -
  • Visual Studio 6, 7.0, 7.1
  • -
-
- -
-
-
- -
- -
- - diff --git a/docs/index.html b/docs/index.html deleted file mode 100644 index 205b158bb..000000000 --- a/docs/index.html +++ /dev/null @@ -1,171 +0,0 @@ - - - - - - - - - - - - - - - -
-
- - - - -
- - --- - - - - - -
Author:Arvid Norberg, arvid@libtorrent.org
Version:1.1.0
- -
-

libtorrent

-

libtorrent is a feature complete C++ bittorrent implementation focusing -on efficiency and scalability. It runs on embedded devices as well as -desktops. It boasts a well documented library interface that is easy to -use. It comes with a simple bittorrent client demonstrating the use of -the library.

-

The main goals of libtorrent are:

-
    -
  • to be cpu efficient
  • -
  • to be memory efficient
  • -
  • to be very easy to use
  • -
- -
-

Feedback

-

There's a mailing list, general libtorrent discussion.

-

You can usually find me as hydri in #libtorrent on irc.freenode.net.

-
-
-

license

-

libtorrent is released under the BSD-license.

-

This means that you can use the library in your project without having to -release its source code. The only requirement is that you give credit -to the author of the library by including the libtorrent license in your -software or documentation.

-

It is however greatly appreciated if additional features are contributed -back to the open source project. Patches can be emailed to the mailing -list or posted to the bug tracker.

-
-
-

Acknowledgements

-

Written by Arvid Norberg. Copyright © 2003-2016

-

Contributions by Steven Siloti, Magnus Jonsson, Daniel Wallin and Cory Nelson

-

Thanks to Reimond Retz for bugfixes, suggestions and testing

-

Thanks to Umeå University for providing development and test hardware.

-

Project is hosted by github.

-
-
- -
-
-
- -
- -
- - diff --git a/docs/projects.html b/docs/projects.html deleted file mode 100644 index 922db4b07..000000000 --- a/docs/projects.html +++ /dev/null @@ -1,222 +0,0 @@ - - - - - - -projects using libtorrent - - - - - - - -
-
- - - - -
-

projects using libtorrent

- -

These are some of the public projects that uses libtorrent. If you want your -project listed here, let me know.

-
-

Wyzo

-

wyzo is a media browser with built-in bittorrent support.

-
-
-

deluge

-

deluge Torrent is a more full-featured yet still lightweight bittorrent -client. It has the ability to automatically resume partial downloads and -background to the system tray.

-
-
-

qBittorrent

-

qBittorrent is a QT bittorrent client available for linux (likely portable to -most other desktops as well). Written by Christophe Dumez.

-
-
-

tonidoplug

-

Tonidoplug is a tiny, low-power, low-cost home server and -NAS device powered by Tonido software that allows you to access -your apps, files, music and media from anywhere.

-
-
-

Folx

-

Folx is a torrent client and download manager for Mac OS X. -The Free version of Folx has all the basic functionality of the torrent -client, which allows users to download and create torrent files. -Folx PRO (available for a small fee) features the possibility to search -for torrent files just from Folx interface. So there is no need to -browse through multiple torrent trackers searching for particular file.

-
-
-

Miro

-

Miro is a free application for channels of internet video (also known as -video podcasts and video rss). Miro is designed to be easy to use and to give -you an elegant fullscreen viewing experience.

-
-
-

MooPolice

-

MooPolice is a windows bittorrent client with a unique look.

-
-
-

LeechCraft

-

LeechCraft LeechCraft is a free open source cross-platform extensible -software, which primary goal is support of file sharing networks and protocols -like HTTP and FTP

-
-
-

Free download manager

-

FDM is a powerful, easy-to-use and absolutely free download accelerator and -manager. Moreover, FDM is 100% safe, open-source software distributed under -GPL License.

-
-
-

btg

-

btg is a unix bittorrent client which is run as a daemon. It has multiple user -interfaces which connects to the daemon. One GUI (Gtkmm), one terminal -interface (ncurses) and one web interface (accessable through a web browser). -Written by Michael Wojciechowski and Johan Strom.

-
-
-

electric sheep

-

electric sheep is a screensaver which collectively generates animations and -lets the users vote which one to live on.

-
-
-

Tvitty

-

tvitty is a bittorrent client for Vista Media Center, which allows -searching and downloading of torrents directly on your TV.

-
-
-

hrktorrent

-

hrktorrent hrktorrent is a light console torrent client written in C++.

-
-
-

FatRat

-

FatRat is an open source download manager for Linux/Unix systems written in -C++ with the help of the Trolltech Qt 4 library. It's simple to use and -install.

-
-
-

halite BitTorrent

-

Halite is a windows bittorrent client controllabel via an xml-rpc -interface.

-
-
-

Arctic Torrent

-

Arctic Torrent is a light-weight -bittorrent client for windows. -Written by Cory Nelson.

-
-
-

bubba

-

Bubba is a mini-sized server, designed to fit your home better than -an always running PC. Boasting Torrent downloader, DAAP streaming, -Web, E-mail, printer and FTP server etc.

-
-
-

tvblob

-

The BLOBbox represents the ability to harness all of the content available -on the web, without any filtering or pre-selection by a third party just -like surfing the web.

-

This means that anyone will have the ability to reach viewers via the Internet -directly on TV, without them having to connect a PC.

-
-
-

Flush

-

Flush is a GTK-based BitTorrent client.

-
-
-

Lince

-

Lince is a bittorrent client using libtorrent to handle bittorrent protocol -and gtkmm for the interface, it has been designed to be a light and full -featured client.

-
-
-

Linkage

-

Linkage is a gtkmm client that aims to be middle weight.

-
-
-

Bitfox

-

Bitfox is a firefox plugin integrating bittorrent downloads in firefox.

-
-
-

BitSlug

-

BitSlug is a MacOSX cocoa client.

-
-
-

DelCo

-

DelCo is a research project at Tampere university of technology, finland.

-
-
-

Torrent2Exe

-

Torrent2Exe Torrent2exe is a small BitTorrent client. Its basic idea is to -let users download a custom-built EXE program with the torrent file -integrated into it.

-
-
-

ZyXEL NSA-220

-

ZyXEL NSA220 makes it easy to store, protect and share files between users -on your home network. The built-in DLNA server works with many set top boxes -to allow you to play back music, watch video files, or view photos on your -home theater system, while the built in download manager can automatically -download video and audio podcasts as well as allow you to download bittorrent -files without needing to leave your computer on.

-
- -
-
-
- -
- -
- - diff --git a/docs/python_binding.html b/docs/python_binding.html deleted file mode 100644 index 633a60bb3..000000000 --- a/docs/python_binding.html +++ /dev/null @@ -1,209 +0,0 @@ - - - - - - -libtorrent python binding - - - - - - - - -
-
- - - - -
-

libtorrent python binding

- --- - - - -
Author:Arvid Norberg, arvid@libtorrent.org
- -
-

building

-

Building the libtorrent python bindings will produce a shared library (DLL) -which is a python module that can be imported in a python program.

-
-

building using setup.py

-

There is a setup.py shipped with libtorrent that can be used on windows. -On windows the setup.py will invoke bjam and assume that you have boost -sources at $BOOST_PATH. The resulting executable is self-contained, it does -not depend any boost or libtorrent dlls.

-

On other systems, the setup.py is generated by running -./configure --enable-python-binding.

-

To build the Python bindings do:

-
    -
  1. Run:

    -
    -python setup.py build
    -
    -
  2. -
  3. As root, run:

    -
    -python setup.py install
    -
    -
  4. -
-
-
-

building using boost build

-

To set up your build environment, you need to add some settings to your -$BOOST_BUILD_PATH/user-config.jam.

-

Make sure your user config contains the following line:

-
-using python : 2.3 ;
-
-

Set the version to the version of python you have installed or want to use. If -you've installed python in a non-standard location, you have to add the prefix -path used when you installed python as a second option. Like this:

-
-using python : 2.6 : /usr/bin/python2.6 : /usr/include/python2.6 : /usr/lib/python2.6 ;
-
-

The bindings require at least python version 2.2.

-

For more information on how to install and set up boost-build, see the -building libtorrent section.

-

Once you have boost-build set up, you cd to the bindings/python -directory and invoke bjam with the apropriate settings. For the available -build variants, see libtorrent build options.

-

For example:

-
-$ bjam dht-support=on link=static
-
-

On Mac OS X, this will produce the following python module:

-
-bin/darwin-4.0/release/dht-support-on/link-static/logging-none/threading-multi/libtorrent.so
-
-
-
-
-

using libtorrent in python

-

The python interface is nearly identical to the C++ interface. Please refer to -the library reference. The main differences are:

-
-
asio::tcp::endpoint
-
The endpoint type is represented as a tuple of a string (as the address) and an int for -the port number. E.g. ('127.0.0.1', 6881) represents the localhost port 6881.
-
libtorrent::time_duration
-
The time duration is represented as a number of seconds in a regular integer.
-
-

The following functions takes a reference to a container that is filled with -entries by the function. The python equivalent of these functions instead returns -a list of entries.

-
    -
  • torrent_handle::get_peer_info
  • -
  • torrent_handle::file_progress
  • -
  • torrent_handle::get_download_queue
  • -
  • torrent_handle::piece_availability
  • -
-

create_torrent::add_node() takes two arguments, one string and one integer, -instead of a pair. The string is the address and the integer is the port.

-

session::set_settings() not only accepts a session_settings object, but also -a dictionary with keys matching the names of the members of the session_settings struct. -When calling set_settings, the dictionary does not need to have every settings set, -keys that are not present, are set to their default value.

-

For backwards compatibility, session::settings() still returns a session_settings -struct. To get a python dictionary of the settings, call session::get_settings.

-

For an example python program, see client.py in the bindings/python -directory.

-

A very simple example usage of the module would be something like this:

-
-import libtorrent as lt
-import time
-
-ses = lt.session()
-ses.listen_on(6881, 6891)
-
-e = lt.bdecode(open("test.torrent", 'rb').read())
-info = lt.torrent_info(e)
-
-params = { 'save_path': '.', \
-        'storage_mode': lt.storage_mode_t.storage_mode_sparse, \
-        'ti': info }
-h = ses.add_torrent(params)
-
-s = h.status()
-while (not s.is_seeding):
-        s = h.status()
-
-        state_str = ['queued', 'checking', 'downloading metadata', \
-                'downloading', 'finished', 'seeding', 'allocating']
-        print '%.2f%% complete (down: %.1f kb/s up: %.1f kB/s peers: %d) %s' % \
-                (s.progress * 100, s.download_rate / 1000, s.upload_rate / 1000, \
-                s.num_peers, state_str[s.state])
-
-        time.sleep(1)
-
-
- -
-
-
- -
- -
- - diff --git a/docs/settings.rst b/docs/settings.rst deleted file mode 100644 index 8893415ad..000000000 --- a/docs/settings.rst +++ /dev/null @@ -1,3268 +0,0 @@ -.. _user_agent: - -.. raw:: html - - - -+------------+--------+----------------------------------+ -| name | type | default | -+============+========+==================================+ -| user_agent | string | "libtorrent/" LIBTORRENT_VERSION | -+------------+--------+----------------------------------+ - -this is the client identification to the tracker. The recommended -format of this string is: "ClientName/ClientVersion -libtorrent/libtorrentVersion". This name will not only be used when -making HTTP requests, but also when sending extended headers to -peers that support that extension. It may not contain \r or \n - -.. _announce_ip: - -.. raw:: html - - - -+-------------+--------+---------+ -| name | type | default | -+=============+========+=========+ -| announce_ip | string | 0 | -+-------------+--------+---------+ - -``announce_ip`` is the ip address passed along to trackers as the -``&ip=`` parameter. If left as the default, that parameter is -omitted. - -.. _mmap_cache: - -.. raw:: html - - - -+------------+--------+---------+ -| name | type | default | -+============+========+=========+ -| mmap_cache | string | 0 | -+------------+--------+---------+ - -``mmap_cache`` may be set to a filename where the disk cache will -be mmapped to. This could be useful, for instance, to map the disk -cache from regular rotating hard drives onto an SSD drive. Doing -that effectively introduces a second layer of caching, allowing the -disk cache to be as big as can fit on an SSD drive (probably about -one order of magnitude more than the available RAM). The intention -of this setting is to set it up once at the start up and not change -it while running. The setting may not be changed as long as there -are any disk buffers in use. This default to the empty string, -which means use regular RAM allocations for the disk cache. The -file specified will be created and truncated to the disk cache size -(``cache_size``). Any existing file with the same name will be -replaced. - -Since this setting sets a hard upper limit on cache usage, it -cannot be combined with -``session_settings::contiguous_recv_buffer``, since that feature -treats the ``cache_size`` setting as a soft (but still pretty hard) -limit. The result of combining the two is peers being disconnected -after failing to allocate more disk buffers. - -This feature requires the ``mmap`` system call, on systems that -don't have ``mmap`` this setting is ignored. - -.. _handshake_client_version: - -.. raw:: html - - - -+--------------------------+--------+---------+ -| name | type | default | -+==========================+========+=========+ -| handshake_client_version | string | 0 | -+--------------------------+--------+---------+ - -this is the client name and version identifier sent to peers in the -handshake message. If this is an empty string, the user_agent is -used instead - -.. _outgoing_interfaces: - -.. raw:: html - - - -+---------------------+--------+---------+ -| name | type | default | -+=====================+========+=========+ -| outgoing_interfaces | string | "" | -+---------------------+--------+---------+ - -sets the network interface this session will use when it opens -outgoing connections. By default, it binds outgoing connections to -INADDR_ANY and port 0 (i.e. let the OS decide). Ths parameter must -be a string containing one or more, comma separated, adapter names. -Adapter names on unix systems are of the form "eth0", "eth1", -"tun0", etc. When specifying multiple interfaces, they will be -assigned in round-robin order. This may be useful for clients that -are multi-homed. Binding an outgoing connection to a local IP does -not necessarily make the connection via the associated NIC/Adapter. -Setting this to an empty string will disable binding of outgoing -connections. - -.. _listen_interfaces: - -.. raw:: html - - - -+-------------------+--------+----------------+ -| name | type | default | -+===================+========+================+ -| listen_interfaces | string | "0.0.0.0:6881" | -+-------------------+--------+----------------+ - -a comma-separated list of IP port-pairs. These -are the listen ports that will be opened for accepting incoming uTP -and TCP connections. It is possible to listen on multiple -IPs and multiple ports. Binding to port 0 will make the -operating system pick the port. The default is "0.0.0.0:6881", which -binds to all interfaces on port 6881. -if binding fails, the listen_failed_alert is posted, potentially -more than once. Once/if binding the listen socket(s) succeed, -listen_succeeded_alert is posted. -Each port will attempt to open both a UDP and a TCP listen socket, -to allow accepting uTP connections as well as TCP. If using the DHT, -this will also make the DHT use the same UDP ports. - -.. note:: - The current support for opening arbitrary UDP sockets is limited. - In this version of libtorrent, there will only ever be two UDP - sockets, one for IPv4 and one for IPv6. - -.. _proxy_hostname: - -.. raw:: html - - - -+----------------+--------+---------+ -| name | type | default | -+================+========+=========+ -| proxy_hostname | string | "" | -+----------------+--------+---------+ - -when using a poxy, this is the hostname where the proxy is running -see proxy_type. - -.. _proxy_username: - -.. _proxy_password: - -.. raw:: html - - - - -+----------------+--------+---------+ -| name | type | default | -+================+========+=========+ -| proxy_username | string | "" | -+----------------+--------+---------+ -| proxy_password | string | "" | -+----------------+--------+---------+ - -when using a proxy, these are the credentials (if any) to use whne -connecting to it. see proxy_type - -.. _i2p_hostname: - -.. raw:: html - - - -+--------------+--------+---------+ -| name | type | default | -+==============+========+=========+ -| i2p_hostname | string | "" | -+--------------+--------+---------+ - -sets the i2p_ SAM bridge to connect to. set the port with the -``i2p_port`` setting. - -.. _i2p: http://www.i2p2.de - -.. _peer_fingerprint: - -.. raw:: html - - - -+------------------+--------+------------+ -| name | type | default | -+==================+========+============+ -| peer_fingerprint | string | "-LT1100-" | -+------------------+--------+------------+ - -this is the fingerprint for the client. It will be used as the -prefix to the peer_id. If this is 20 bytes (or longer) it will be -used as the peer-id - -.. _allow_multiple_connections_per_ip: - -.. raw:: html - - - -+-----------------------------------+------+---------+ -| name | type | default | -+===================================+======+=========+ -| allow_multiple_connections_per_ip | bool | false | -+-----------------------------------+------+---------+ - -determines if connections from the same IP address as existing -connections should be rejected or not. Multiple connections from -the same IP address is not allowed by default, to prevent abusive -behavior by peers. It may be useful to allow such connections in -cases where simulations are run on the same machie, and all peers -in a swarm has the same IP address. - -.. _send_redundant_have: - -.. raw:: html - - - -+---------------------+------+---------+ -| name | type | default | -+=====================+======+=========+ -| send_redundant_have | bool | true | -+---------------------+------+---------+ - -if set to true, upload, download and unchoke limits are ignored for -peers on the local network. This option is *DEPRECATED*, please use -set_peer_class_filter() instead. -``send_redundant_have`` controls if have messages will be sent to -peers that already have the piece. This is typically not necessary, -but it might be necessary for collecting statistics in some cases. -Default is false. - -.. _lazy_bitfields: - -.. raw:: html - - - -+----------------+------+---------+ -| name | type | default | -+================+======+=========+ -| lazy_bitfields | bool | false | -+----------------+------+---------+ - -if this is true, outgoing bitfields will never be fuil. If the -client is seed, a few bits will be set to 0, and later filled in -with have messages. This is to prevent certain ISPs from stopping -people from seeding. - -.. _use_dht_as_fallback: - -.. raw:: html - - - -+---------------------+------+---------+ -| name | type | default | -+=====================+======+=========+ -| use_dht_as_fallback | bool | false | -+---------------------+------+---------+ - -``use_dht_as_fallback`` determines how the DHT is used. If this is -true, the DHT will only be used for torrents where all trackers in -its tracker list has failed. Either by an explicit error message or -a time out. This is false by default, which means the DHT is used -by default regardless of if the trackers fail or not. - -.. _upnp_ignore_nonrouters: - -.. raw:: html - - - -+------------------------+------+---------+ -| name | type | default | -+========================+======+=========+ -| upnp_ignore_nonrouters | bool | false | -+------------------------+------+---------+ - -``upnp_ignore_nonrouters`` indicates whether or not the UPnP -implementation should ignore any broadcast response from a device -whose address is not the configured router for this machine. i.e. -it's a way to not talk to other people's routers by mistake. - -.. _use_parole_mode: - -.. raw:: html - - - -+-----------------+------+---------+ -| name | type | default | -+=================+======+=========+ -| use_parole_mode | bool | true | -+-----------------+------+---------+ - -``use_parole_mode`` specifies if parole mode should be used. Parole -mode means that peers that participate in pieces that fail the hash -check are put in a mode where they are only allowed to download -whole pieces. If the whole piece a peer in parole mode fails the -hash check, it is banned. If a peer participates in a piece that -passes the hash check, it is taken out of parole mode. - -.. _use_read_cache: - -.. _use_write_cache: - -.. raw:: html - - - - -+-----------------+------+---------+ -| name | type | default | -+=================+======+=========+ -| use_read_cache | bool | true | -+-----------------+------+---------+ -| use_write_cache | bool | true | -+-----------------+------+---------+ - -enable and disable caching of read blocks and blocks to be written -to disk respsectively. the purpose of the read cache is partly -read-ahead of requests but also to avoid reading blocks back from -the disk multiple times for popular pieces. the write cache purpose -is to hold off writing blocks to disk until they have been hashed, -to avoid having to read them back in again. - -.. _dont_flush_write_cache: - -.. raw:: html - - - -+------------------------+------+---------+ -| name | type | default | -+========================+======+=========+ -| dont_flush_write_cache | bool | false | -+------------------------+------+---------+ - -this will make the disk cache never flush a write piece if it would -cause is to have to re-read it once we want to calculate the piece -hash - -.. _explicit_read_cache: - -.. raw:: html - - - -+---------------------+------+---------+ -| name | type | default | -+=====================+======+=========+ -| explicit_read_cache | bool | false | -+---------------------+------+---------+ - -``explicit_read_cache`` defaults to 0. If set to something greater -than 0, the disk read cache will not be evicted by cache misses and -will explicitly be controlled based on the rarity of pieces. Rare -pieces are more likely to be cached. This would typically be used -together with ``suggest_mode`` set to ``suggest_read_cache``. The -value is the number of pieces to keep in the read cache. If the -actual read cache can't fit as many, it will essentially be -clamped. - -.. _coalesce_reads: - -.. _coalesce_writes: - -.. raw:: html - - - - -+-----------------+------+---------+ -| name | type | default | -+=================+======+=========+ -| coalesce_reads | bool | false | -+-----------------+------+---------+ -| coalesce_writes | bool | false | -+-----------------+------+---------+ - -allocate separate, contiguous, buffers for read and write calls. -Only used where writev/readv cannot be used will use more RAM but -may improve performance - -.. _auto_manage_prefer_seeds: - -.. raw:: html - - - -+--------------------------+------+---------+ -| name | type | default | -+==========================+======+=========+ -| auto_manage_prefer_seeds | bool | false | -+--------------------------+------+---------+ - -prefer seeding torrents when determining which torrents to give -active slots to, the default is false which gives preference to -downloading torrents - -.. _dont_count_slow_torrents: - -.. raw:: html - - - -+--------------------------+------+---------+ -| name | type | default | -+==========================+======+=========+ -| dont_count_slow_torrents | bool | true | -+--------------------------+------+---------+ - -if ``dont_count_slow_torrents`` is true, torrents without any -payload transfers are not subject to the ``active_seeds`` and -``active_downloads`` limits. This is intended to make it more -likely to utilize all available bandwidth, and avoid having -torrents that don't transfer anything block the active slots. - -.. _close_redundant_connections: - -.. raw:: html - - - -+-----------------------------+------+---------+ -| name | type | default | -+=============================+======+=========+ -| close_redundant_connections | bool | true | -+-----------------------------+------+---------+ - -``close_redundant_connections`` specifies whether libtorrent should -close connections where both ends have no utility in keeping the -connection open. For instance if both ends have completed their -downloads, there's no point in keeping it open. - -.. _prioritize_partial_pieces: - -.. raw:: html - - - -+---------------------------+------+---------+ -| name | type | default | -+===========================+======+=========+ -| prioritize_partial_pieces | bool | false | -+---------------------------+------+---------+ - -If ``prioritize_partial_pieces`` is true, partial pieces are picked -before pieces that are more rare. If false, rare pieces are always -prioritized, unless the number of partial pieces is growing out of -proportion. - -.. _rate_limit_ip_overhead: - -.. raw:: html - - - -+------------------------+------+---------+ -| name | type | default | -+========================+======+=========+ -| rate_limit_ip_overhead | bool | true | -+------------------------+------+---------+ - -if set to true, the estimated TCP/IP overhead is drained from the -rate limiters, to avoid exceeding the limits with the total traffic - -.. _announce_to_all_tiers: - -.. _announce_to_all_trackers: - -.. raw:: html - - - - -+--------------------------+------+---------+ -| name | type | default | -+==========================+======+=========+ -| announce_to_all_tiers | bool | false | -+--------------------------+------+---------+ -| announce_to_all_trackers | bool | false | -+--------------------------+------+---------+ - -``announce_to_all_trackers`` controls how multi tracker torrents -are treated. If this is set to true, all trackers in the same tier -are announced to in parallel. If all trackers in tier 0 fails, all -trackers in tier 1 are announced as well. If it's set to false, the -behavior is as defined by the multi tracker specification. It -defaults to false, which is the same behavior previous versions of -libtorrent has had as well. - -``announce_to_all_tiers`` also controls how multi tracker torrents -are treated. When this is set to true, one tracker from each tier -is announced to. This is the uTorrent behavior. This is false by -default in order to comply with the multi-tracker specification. - -.. _prefer_udp_trackers: - -.. raw:: html - - - -+---------------------+------+---------+ -| name | type | default | -+=====================+======+=========+ -| prefer_udp_trackers | bool | true | -+---------------------+------+---------+ - -``prefer_udp_trackers`` is true by default. It means that trackers -may be rearranged in a way that udp trackers are always tried -before http trackers for the same hostname. Setting this to false -means that the trackers' tier is respected and there's no -preference of one protocol over another. - -.. _strict_super_seeding: - -.. raw:: html - - - -+----------------------+------+---------+ -| name | type | default | -+======================+======+=========+ -| strict_super_seeding | bool | false | -+----------------------+------+---------+ - -``strict_super_seeding`` when this is set to true, a piece has to -have been forwarded to a third peer before another one is handed -out. This is the traditional definition of super seeding. - -.. _lock_disk_cache: - -.. raw:: html - - - -+-----------------+------+---------+ -| name | type | default | -+=================+======+=========+ -| lock_disk_cache | bool | false | -+-----------------+------+---------+ - -if this is set to true, the memory allocated for the disk cache -will be locked in physical RAM, never to be swapped out. Every time -a disk buffer is allocated and freed, there will be the extra -overhead of a system call. - -.. _disable_hash_checks: - -.. raw:: html - - - -+---------------------+------+---------+ -| name | type | default | -+=====================+======+=========+ -| disable_hash_checks | bool | false | -+---------------------+------+---------+ - -when set to true, all data downloaded from peers will be assumed to -be correct, and not tested to match the hashes in the torrent this -is only useful for simulation and testing purposes (typically -combined with disabled_storage) - -.. _allow_i2p_mixed: - -.. raw:: html - - - -+-----------------+------+---------+ -| name | type | default | -+=================+======+=========+ -| allow_i2p_mixed | bool | false | -+-----------------+------+---------+ - -if this is true, i2p torrents are allowed to also get peers from -other sources than the tracker, and connect to regular IPs, not -providing any anonymization. This may be useful if the user is not -interested in the anonymization of i2p, but still wants to be able -to connect to i2p peers. - -.. _low_prio_disk: - -.. raw:: html - - - -+---------------+------+---------+ -| name | type | default | -+===============+======+=========+ -| low_prio_disk | bool | true | -+---------------+------+---------+ - -``low_prio_disk`` determines if the disk I/O should use a normal or -low priority policy. This defaults to true, which means that it's -low priority by default. Other processes doing disk I/O will -normally take priority in this mode. This is meant to improve the -overall responsiveness of the system while downloading in the -background. For high-performance server setups, this might not be -desirable. - -.. _volatile_read_cache: - -.. raw:: html - - - -+---------------------+------+---------+ -| name | type | default | -+=====================+======+=========+ -| volatile_read_cache | bool | false | -+---------------------+------+---------+ - -``volatile_read_cache``, if this is set to true, read cache blocks -that are hit by peer read requests are removed from the disk cache -to free up more space. This is useful if you don't expect the disk -cache to create any cache hits from other peers than the one who -triggered the cache line to be read into the cache in the first -place. - -.. _guided_read_cache: - -.. raw:: html - - - -+-------------------+------+---------+ -| name | type | default | -+===================+======+=========+ -| guided_read_cache | bool | false | -+-------------------+------+---------+ - -``guided_read_cache`` enables the disk cache to adjust the size of -a cache line generated by peers to depend on the upload rate you -are sending to that peer. The intention is to optimize the RAM -usage of the cache, to read ahead further for peers that you're -sending faster to. - -.. _no_atime_storage: - -.. raw:: html - - - -+------------------+------+---------+ -| name | type | default | -+==================+======+=========+ -| no_atime_storage | bool | true | -+------------------+------+---------+ - -``no_atime_storage`` this is a linux-only option and passes in the -``O_NOATIME`` to ``open()`` when opening files. This may lead to -some disk performance improvements. - -.. _incoming_starts_queued_torrents: - -.. raw:: html - - - -+---------------------------------+------+---------+ -| name | type | default | -+=================================+======+=========+ -| incoming_starts_queued_torrents | bool | false | -+---------------------------------+------+---------+ - -``incoming_starts_queued_torrents`` defaults to false. If a torrent -has been paused by the auto managed feature in libtorrent, i.e. the -torrent is paused and auto managed, this feature affects whether or -not it is automatically started on an incoming connection. The main -reason to queue torrents, is not to make them unavailable, but to -save on the overhead of announcing to the trackers, the DHT and to -avoid spreading one's unchoke slots too thin. If a peer managed to -find us, even though we're no in the torrent anymore, this setting -can make us start the torrent and serve it. - -.. _report_true_downloaded: - -.. raw:: html - - - -+------------------------+------+---------+ -| name | type | default | -+========================+======+=========+ -| report_true_downloaded | bool | false | -+------------------------+------+---------+ - -when set to true, the downloaded counter sent to trackers will -include the actual number of payload bytes downloaded including -redundant bytes. If set to false, it will not include any redundancy -bytes - -.. _strict_end_game_mode: - -.. raw:: html - - - -+----------------------+------+---------+ -| name | type | default | -+======================+======+=========+ -| strict_end_game_mode | bool | true | -+----------------------+------+---------+ - -``strict_end_game_mode`` defaults to true, and controls when a -block may be requested twice. If this is ``true``, a block may only -be requested twice when there's ay least one request to every piece -that's left to download in the torrent. This may slow down progress -on some pieces sometimes, but it may also avoid downloading a lot -of redundant bytes. If this is ``false``, libtorrent attempts to -use each peer connection to its max, by always requesting -something, even if it means requesting something that has been -requested from another peer already. - -.. _broadcast_lsd: - -.. raw:: html - - - -+---------------+------+---------+ -| name | type | default | -+===============+======+=========+ -| broadcast_lsd | bool | true | -+---------------+------+---------+ - -if ``broadcast_lsd`` is set to true, the local peer discovery (or -Local Service Discovery) will not only use IP multicast, but also -broadcast its messages. This can be useful when running on networks -that don't support multicast. Since broadcast messages might be -expensive and disruptive on networks, only every 8th announce uses -broadcast. - -.. _enable_outgoing_utp: - -.. _enable_incoming_utp: - -.. _enable_outgoing_tcp: - -.. _enable_incoming_tcp: - -.. raw:: html - - - - - - -+---------------------+------+---------+ -| name | type | default | -+=====================+======+=========+ -| enable_outgoing_utp | bool | true | -+---------------------+------+---------+ -| enable_incoming_utp | bool | true | -+---------------------+------+---------+ -| enable_outgoing_tcp | bool | true | -+---------------------+------+---------+ -| enable_incoming_tcp | bool | true | -+---------------------+------+---------+ - -when set to true, libtorrent will try to make outgoing utp -connections controls whether libtorrent will accept incoming -connections or make outgoing connections of specific type. - -.. _ignore_resume_timestamps: - -.. raw:: html - - - -+--------------------------+------+---------+ -| name | type | default | -+==========================+======+=========+ -| ignore_resume_timestamps | bool | false | -+--------------------------+------+---------+ - -``ignore_resume_timestamps`` determines if the storage, when -loading resume data files, should verify that the file modification -time with the timestamps in the resume data. This defaults to -false, which means timestamps are taken into account, and resume -data is less likely to accepted (torrents are more likely to be -fully checked when loaded). It might be useful to set this to true -if your network is faster than your disk, and it would be faster to -redownload potentially missed pieces than to go through the whole -storage to look for them. - -.. _no_recheck_incomplete_resume: - -.. raw:: html - - - -+------------------------------+------+---------+ -| name | type | default | -+==============================+======+=========+ -| no_recheck_incomplete_resume | bool | false | -+------------------------------+------+---------+ - -``no_recheck_incomplete_resume`` determines if the storage should -check the whole files when resume data is incomplete or missing or -whether it should simply assume we don't have any of the data. By -default, this is determined by the existence of any of the files. -By setting this setting to true, the files won't be checked, but -will go straight to download mode. - -.. _anonymous_mode: - -.. raw:: html - - - -+----------------+------+---------+ -| name | type | default | -+================+======+=========+ -| anonymous_mode | bool | false | -+----------------+------+---------+ - -``anonymous_mode`` defaults to false. When set to true, the client -tries to hide its identity to a certain degree. The peer-ID will no -longer include the client's fingerprint. The user-agent will be -reset to an empty string. Trackers will only be used if they are -using a proxy server. The listen sockets are closed, and incoming -connections will only be accepted through a SOCKS5 or I2P proxy (if -a peer proxy is set up and is run on the same machine as the -tracker proxy). Since no incoming connections are accepted, -NAT-PMP, UPnP, DHT and local peer discovery are all turned off when -this setting is enabled. - -If you're using I2P, it might make sense to enable anonymous mode -as well. - -.. _report_web_seed_downloads: - -.. raw:: html - - - -+---------------------------+------+---------+ -| name | type | default | -+===========================+======+=========+ -| report_web_seed_downloads | bool | true | -+---------------------------+------+---------+ - -specifies whether downloads from web seeds is reported to the -tracker or not. Defaults to on. Turning it off also excludes web -seed traffic from other stats and download rate reporting via the -libtorrent API. - -.. _announce_double_nat: - -.. raw:: html - - - -+---------------------+------+---------+ -| name | type | default | -+=====================+======+=========+ -| announce_double_nat | bool | false | -+---------------------+------+---------+ - -set to true if uTP connections should be rate limited This option -is *DEPRECATED*, please use set_peer_class_filter() instead. -if this is true, the ``&ip=`` argument in tracker requests (unless -otherwise specified) will be set to the intermediate IP address if -the user is double NATed. If the user is not double NATed, this -option does not have an affect - -.. _seeding_outgoing_connections: - -.. raw:: html - - - -+------------------------------+------+---------+ -| name | type | default | -+==============================+======+=========+ -| seeding_outgoing_connections | bool | true | -+------------------------------+------+---------+ - -``seeding_outgoing_connections`` determines if seeding (and -finished) torrents should attempt to make outgoing connections or -not. By default this is true. It may be set to false in very -specific applications where the cost of making outgoing connections -is high, and there are no or small benefits of doing so. For -instance, if no nodes are behind a firewall or a NAT, seeds don't -need to make outgoing connections. - -.. _no_connect_privileged_ports: - -.. raw:: html - - - -+-----------------------------+------+---------+ -| name | type | default | -+=============================+======+=========+ -| no_connect_privileged_ports | bool | false | -+-----------------------------+------+---------+ - -when this is true, libtorrent will not attempt to make outgoing -connections to peers whose port is < 1024. This is a safety -precaution to avoid being part of a DDoS attack - -.. _smooth_connects: - -.. raw:: html - - - -+-----------------+------+---------+ -| name | type | default | -+=================+======+=========+ -| smooth_connects | bool | true | -+-----------------+------+---------+ - -``smooth_connects`` is true by default, which means the number of -connection attempts per second may be limited to below the -``connection_speed``, in case we're close to bump up against the -limit of number of connections. The intention of this setting is to -more evenly distribute our connection attempts over time, instead -of attempting to connect in batches, and timing them out in -batches. - -.. _always_send_user_agent: - -.. raw:: html - - - -+------------------------+------+---------+ -| name | type | default | -+========================+======+=========+ -| always_send_user_agent | bool | false | -+------------------------+------+---------+ - -always send user-agent in every web seed request. If false, only -the first request per http connection will include the user agent - -.. _apply_ip_filter_to_trackers: - -.. raw:: html - - - -+-----------------------------+------+---------+ -| name | type | default | -+=============================+======+=========+ -| apply_ip_filter_to_trackers | bool | true | -+-----------------------------+------+---------+ - -``apply_ip_filter_to_trackers`` defaults to true. It determines -whether the IP filter applies to trackers as well as peers. If this -is set to false, trackers are exempt from the IP filter (if there -is one). If no IP filter is set, this setting is irrelevant. - -.. _use_disk_read_ahead: - -.. raw:: html - - - -+---------------------+------+---------+ -| name | type | default | -+=====================+======+=========+ -| use_disk_read_ahead | bool | true | -+---------------------+------+---------+ - -``use_disk_read_ahead`` defaults to true and will attempt to -optimize disk reads by giving the operating system heads up of disk -read requests as they are queued in the disk job queue. - -.. _lock_files: - -.. raw:: html - - - -+------------+------+---------+ -| name | type | default | -+============+======+=========+ -| lock_files | bool | false | -+------------+------+---------+ - -``lock_files`` determines whether or not to lock files which -libtorrent is downloading to or seeding from. This is implemented -using ``fcntl(F_SETLK)`` on unix systems and by not passing in -``SHARE_READ`` and ``SHARE_WRITE`` on windows. This might prevent -3rd party processes from corrupting the files under libtorrent's -feet. - -.. _contiguous_recv_buffer: - -.. raw:: html - - - -+------------------------+------+---------+ -| name | type | default | -+========================+======+=========+ -| contiguous_recv_buffer | bool | true | -+------------------------+------+---------+ - -``contiguous_recv_buffer`` determines whether or not libtorrent -should receive data from peers into a contiguous intermediate -buffer, to then copy blocks into disk buffers from, or to make many -smaller calls to ``read()``, each time passing in the specific -buffer the data belongs in. When downloading at high rates, the -latter may save some time copying data. When seeding at high rates, -all incoming traffic consists of a very large number of tiny -packets, and enabling ``contiguous_recv_buffer`` will provide -higher performance. When this is enabled, it will only be used when -seeding to peers, since that's when it provides performance -improvements. - -.. _ban_web_seeds: - -.. raw:: html - - - -+---------------+------+---------+ -| name | type | default | -+===============+======+=========+ -| ban_web_seeds | bool | true | -+---------------+------+---------+ - -when true, web seeds sending bad data will be banned - -.. _allow_partial_disk_writes: - -.. raw:: html - - - -+---------------------------+------+---------+ -| name | type | default | -+===========================+======+=========+ -| allow_partial_disk_writes | bool | true | -+---------------------------+------+---------+ - -when set to false, the ``write_cache_line_size`` will apply across -piece boundaries. this is a bad idea unless the piece picker also -is configured to have an affinity to pick pieces belonging to the -same write cache line as is configured in the disk cache. - -.. _force_proxy: - -.. raw:: html - - - -+-------------+------+---------+ -| name | type | default | -+=============+======+=========+ -| force_proxy | bool | false | -+-------------+------+---------+ - -If true, disables any communication that's not going over a proxy. -Enabling this requires a proxy to be configured as well, see -``set_proxy_settings``. The listen sockets are closed, and incoming -connections will only be accepted through a SOCKS5 or I2P proxy (if -a peer proxy is set up and is run on the same machine as the -tracker proxy). This setting also disabled peer country lookups, -since those are done via DNS lookups that aren't supported by -proxies. - -.. _support_share_mode: - -.. raw:: html - - - -+--------------------+------+---------+ -| name | type | default | -+====================+======+=========+ -| support_share_mode | bool | true | -+--------------------+------+---------+ - -if false, prevents libtorrent to advertise share-mode support - -.. _support_merkle_torrents: - -.. raw:: html - - - -+-------------------------+------+---------+ -| name | type | default | -+=========================+======+=========+ -| support_merkle_torrents | bool | true | -+-------------------------+------+---------+ - -if this is false, don't advertise support for the Tribler merkle -tree piece message - -.. _report_redundant_bytes: - -.. raw:: html - - - -+------------------------+------+---------+ -| name | type | default | -+========================+======+=========+ -| report_redundant_bytes | bool | true | -+------------------------+------+---------+ - -if this is true, the number of redundant bytes is sent to the -tracker - -.. _listen_system_port_fallback: - -.. raw:: html - - - -+-----------------------------+------+---------+ -| name | type | default | -+=============================+======+=========+ -| listen_system_port_fallback | bool | true | -+-----------------------------+------+---------+ - -if this is true, libtorrent will fall back to listening on a port -chosen by the operating system (i.e. binding to port 0). If a -failure is preferred, set this to false. - -.. _use_disk_cache_pool: - -.. raw:: html - - - -+---------------------+------+---------+ -| name | type | default | -+=====================+======+=========+ -| use_disk_cache_pool | bool | false | -+---------------------+------+---------+ - -``use_disk_cache_pool`` enables using a pool allocator for disk -cache blocks. Enabling it makes the cache perform better at high -throughput. It also makes the cache less likely and slower at -returning memory back to the system, once allocated. - -.. _announce_crypto_support: - -.. raw:: html - - - -+-------------------------+------+---------+ -| name | type | default | -+=========================+======+=========+ -| announce_crypto_support | bool | true | -+-------------------------+------+---------+ - -when this is true, and incoming encrypted connections are enabled, -&supportcrypt=1 is included in http tracker announces - -.. _enable_upnp: - -.. raw:: html - - - -+-------------+------+---------+ -| name | type | default | -+=============+======+=========+ -| enable_upnp | bool | true | -+-------------+------+---------+ - -Starts and stops the UPnP service. When started, the listen port -and the DHT port are attempted to be forwarded on local UPnP router -devices. - -The upnp object returned by ``start_upnp()`` can be used to add and -remove arbitrary port mappings. Mapping status is returned through -the portmap_alert and the portmap_error_alert. The object will be -valid until ``stop_upnp()`` is called. See upnp-and-nat-pmp_. - -.. _enable_natpmp: - -.. raw:: html - - - -+---------------+------+---------+ -| name | type | default | -+===============+======+=========+ -| enable_natpmp | bool | true | -+---------------+------+---------+ - -Starts and stops the NAT-PMP service. When started, the listen port -and the DHT port are attempted to be forwarded on the router -through NAT-PMP. - -The natpmp object returned by ``start_natpmp()`` can be used to add -and remove arbitrary port mappings. Mapping status is returned -through the portmap_alert and the portmap_error_alert. The object -will be valid until ``stop_natpmp()`` is called. See -upnp-and-nat-pmp_. - -.. _enable_lsd: - -.. raw:: html - - - -+------------+------+---------+ -| name | type | default | -+============+======+=========+ -| enable_lsd | bool | true | -+------------+------+---------+ - -Starts and stops Local Service Discovery. This service will -broadcast the infohashes of all the non-private torrents on the -local network to look for peers on the same swarm within multicast -reach. - -.. _enable_dht: - -.. raw:: html - - - -+------------+------+---------+ -| name | type | default | -+============+======+=========+ -| enable_dht | bool | true | -+------------+------+---------+ - -starts the dht node and makes the trackerless service available to -torrents. - -.. _prefer_rc4: - -.. raw:: html - - - -+------------+------+---------+ -| name | type | default | -+============+======+=========+ -| prefer_rc4 | bool | false | -+------------+------+---------+ - -if the allowed encryption level is both, setting this to true will -prefer rc4 if both methods are offered, plaintext otherwise - -.. _proxy_hostnames: - -.. raw:: html - - - -+-----------------+------+---------+ -| name | type | default | -+=================+======+=========+ -| proxy_hostnames | bool | true | -+-----------------+------+---------+ - -if true, hostname lookups are done via the configured proxy (if -any). This is only supported by SOCKS5 and HTTP. - -.. _proxy_peer_connections: - -.. raw:: html - - - -+------------------------+------+---------+ -| name | type | default | -+========================+======+=========+ -| proxy_peer_connections | bool | true | -+------------------------+------+---------+ - -if true, peer connections are made (and accepted) over the -configured proxy, if any. Web seeds as well as regular bittorrent -peer connections are considered "peer connections". Anything -transporting actual torrent payload (trackers and DHT traffic are -not considered peer connections). - -.. _auto_sequential: - -.. raw:: html - - - -+-----------------+------+---------+ -| name | type | default | -+=================+======+=========+ -| auto_sequential | bool | true | -+-----------------+------+---------+ - -if this setting is true, torrents with a very high availability of -pieces (and seeds) are downloaded sequentially. This is more -efficient for the disk I/O. With many seeds, the download order is -unlikely to matter anyway - -.. _proxy_tracker_connections: - -.. raw:: html - - - -+---------------------------+------+---------+ -| name | type | default | -+===========================+======+=========+ -| proxy_tracker_connections | bool | true | -+---------------------------+------+---------+ - -if true, tracker connections are made over the configured proxy, if -any. - -.. _tracker_completion_timeout: - -.. raw:: html - - - -+----------------------------+------+---------+ -| name | type | default | -+============================+======+=========+ -| tracker_completion_timeout | int | 30 | -+----------------------------+------+---------+ - -``tracker_completion_timeout`` is the number of seconds the tracker -connection will wait from when it sent the request until it -considers the tracker to have timed-out. Default value is 60 -seconds. - -.. _tracker_receive_timeout: - -.. raw:: html - - - -+-------------------------+------+---------+ -| name | type | default | -+=========================+======+=========+ -| tracker_receive_timeout | int | 10 | -+-------------------------+------+---------+ - -``tracker_receive_timeout`` is the number of seconds to wait to -receive any data from the tracker. If no data is received for this -number of seconds, the tracker will be considered as having timed -out. If a tracker is down, this is the kind of timeout that will -occur. - -.. _stop_tracker_timeout: - -.. raw:: html - - - -+----------------------+------+---------+ -| name | type | default | -+======================+======+=========+ -| stop_tracker_timeout | int | 5 | -+----------------------+------+---------+ - -the time to wait when sending a stopped message before considering -a tracker to have timed out. this is usually shorter, to make the -client quit faster - -.. _tracker_maximum_response_length: - -.. raw:: html - - - -+---------------------------------+------+-----------+ -| name | type | default | -+=================================+======+===========+ -| tracker_maximum_response_length | int | 1024*1024 | -+---------------------------------+------+-----------+ - -this is the maximum number of bytes in a tracker response. If a -response size passes this number of bytes it will be rejected and -the connection will be closed. On gzipped responses this size is -measured on the uncompressed data. So, if you get 20 bytes of gzip -response that'll expand to 2 megabytes, it will be interrupted -before the entire response has been uncompressed (assuming the -limit is lower than 2 megs). - -.. _piece_timeout: - -.. raw:: html - - - -+---------------+------+---------+ -| name | type | default | -+===============+======+=========+ -| piece_timeout | int | 20 | -+---------------+------+---------+ - -the number of seconds from a request is sent until it times out if -no piece response is returned. - -.. _request_timeout: - -.. raw:: html - - - -+-----------------+------+---------+ -| name | type | default | -+=================+======+=========+ -| request_timeout | int | 60 | -+-----------------+------+---------+ - -the number of seconds one block (16kB) is expected to be received -within. If it's not, the block is requested from a different peer - -.. _request_queue_time: - -.. raw:: html - - - -+--------------------+------+---------+ -| name | type | default | -+====================+======+=========+ -| request_queue_time | int | 3 | -+--------------------+------+---------+ - -the length of the request queue given in the number of seconds it -should take for the other end to send all the pieces. i.e. the -actual number of requests depends on the download rate and this -number. - -.. _max_allowed_in_request_queue: - -.. raw:: html - - - -+------------------------------+------+---------+ -| name | type | default | -+==============================+======+=========+ -| max_allowed_in_request_queue | int | 500 | -+------------------------------+------+---------+ - -the number of outstanding block requests a peer is allowed to queue -up in the client. If a peer sends more requests than this (before -the first one has been sent) the last request will be dropped. the -higher this is, the faster upload speeds the client can get to a -single peer. - -.. _max_out_request_queue: - -.. raw:: html - - - -+-----------------------+------+---------+ -| name | type | default | -+=======================+======+=========+ -| max_out_request_queue | int | 500 | -+-----------------------+------+---------+ - -``max_out_request_queue`` is the maximum number of outstanding -requests to send to a peer. This limit takes precedence over -``request_queue_time``. i.e. no matter the download speed, the -number of outstanding requests will never exceed this limit. - -.. _whole_pieces_threshold: - -.. raw:: html - - - -+------------------------+------+---------+ -| name | type | default | -+========================+======+=========+ -| whole_pieces_threshold | int | 20 | -+------------------------+------+---------+ - -if a whole piece can be downloaded in this number of seconds, or -less, the peer_connection will prefer to request whole pieces at a -time from this peer. The benefit of this is to better utilize disk -caches by doing localized accesses and also to make it easier to -identify bad peers if a piece fails the hash check. - -.. _peer_timeout: - -.. raw:: html - - - -+--------------+------+---------+ -| name | type | default | -+==============+======+=========+ -| peer_timeout | int | 120 | -+--------------+------+---------+ - -``peer_timeout`` is the number of seconds the peer connection -should wait (for any activity on the peer connection) before -closing it due to time out. This defaults to 120 seconds, since -that's what's specified in the protocol specification. After half -the time out, a keep alive message is sent. - -.. _urlseed_timeout: - -.. raw:: html - - - -+-----------------+------+---------+ -| name | type | default | -+=================+======+=========+ -| urlseed_timeout | int | 20 | -+-----------------+------+---------+ - -same as peer_timeout, but only applies to url-seeds. this is -usually set lower, because web servers are expected to be more -reliable. - -.. _urlseed_pipeline_size: - -.. raw:: html - - - -+-----------------------+------+---------+ -| name | type | default | -+=======================+======+=========+ -| urlseed_pipeline_size | int | 5 | -+-----------------------+------+---------+ - -controls the pipelining size of url-seeds. i.e. the number of HTTP -request to keep outstanding before waiting for the first one to -complete. It's common for web servers to limit this to a relatively -low number, like 5 - -.. _urlseed_wait_retry: - -.. raw:: html - - - -+--------------------+------+---------+ -| name | type | default | -+====================+======+=========+ -| urlseed_wait_retry | int | 30 | -+--------------------+------+---------+ - -time to wait until a new retry of a web seed takes place - -.. _file_pool_size: - -.. raw:: html - - - -+----------------+------+---------+ -| name | type | default | -+================+======+=========+ -| file_pool_size | int | 40 | -+----------------+------+---------+ - -sets the upper limit on the total number of files this session will -keep open. The reason why files are left open at all is that some -anti virus software hooks on every file close, and scans the file -for viruses. deferring the closing of the files will be the -difference between a usable system and a completely hogged down -system. Most operating systems also has a limit on the total number -of file descriptors a process may have open. It is usually a good -idea to find this limit and set the number of connections and the -number of files limits so their sum is slightly below it. - -.. _max_failcount: - -.. raw:: html - - - -+---------------+------+---------+ -| name | type | default | -+===============+======+=========+ -| max_failcount | int | 3 | -+---------------+------+---------+ - -``max_failcount`` is the maximum times we try to connect to a peer -before stop connecting again. If a peer succeeds, the failcounter -is reset. If a peer is retrieved from a peer source (other than -DHT) the failcount is decremented by one, allowing another try. - -.. _min_reconnect_time: - -.. raw:: html - - - -+--------------------+------+---------+ -| name | type | default | -+====================+======+=========+ -| min_reconnect_time | int | 60 | -+--------------------+------+---------+ - -the number of seconds to wait to reconnect to a peer. this time is -multiplied with the failcount. - -.. _peer_connect_timeout: - -.. raw:: html - - - -+----------------------+------+---------+ -| name | type | default | -+======================+======+=========+ -| peer_connect_timeout | int | 15 | -+----------------------+------+---------+ - -``peer_connect_timeout`` the number of seconds to wait after a -connection attempt is initiated to a peer until it is considered as -having timed out. This setting is especially important in case the -number of half-open connections are limited, since stale half-open -connection may delay the connection of other peers considerably. - -.. _connection_speed: - -.. raw:: html - - - -+------------------+------+---------+ -| name | type | default | -+==================+======+=========+ -| connection_speed | int | 6 | -+------------------+------+---------+ - -``connection_speed`` is the number of connection attempts that are -made per second. If a number < 0 is specified, it will default to -200 connections per second. If 0 is specified, it means don't make -outgoing connections at all. - -.. _inactivity_timeout: - -.. raw:: html - - - -+--------------------+------+---------+ -| name | type | default | -+====================+======+=========+ -| inactivity_timeout | int | 600 | -+--------------------+------+---------+ - -if a peer is uninteresting and uninterested for longer than this -number of seconds, it will be disconnected. default is 10 minutes - -.. _unchoke_interval: - -.. raw:: html - - - -+------------------+------+---------+ -| name | type | default | -+==================+======+=========+ -| unchoke_interval | int | 15 | -+------------------+------+---------+ - -``unchoke_interval`` is the number of seconds between -chokes/unchokes. On this interval, peers are re-evaluated for being -choked/unchoked. This is defined as 30 seconds in the protocol, and -it should be significantly longer than what it takes for TCP to -ramp up to it's max rate. - -.. _optimistic_unchoke_interval: - -.. raw:: html - - - -+-----------------------------+------+---------+ -| name | type | default | -+=============================+======+=========+ -| optimistic_unchoke_interval | int | 30 | -+-----------------------------+------+---------+ - -``optimistic_unchoke_interval`` is the number of seconds between -each *optimistic* unchoke. On this timer, the currently -optimistically unchoked peer will change. - -.. _num_want: - -.. raw:: html - - - -+----------+------+---------+ -| name | type | default | -+==========+======+=========+ -| num_want | int | 200 | -+----------+------+---------+ - -``num_want`` is the number of peers we want from each tracker -request. It defines what is sent as the ``&num_want=`` parameter to -the tracker. - -.. _initial_picker_threshold: - -.. raw:: html - - - -+--------------------------+------+---------+ -| name | type | default | -+==========================+======+=========+ -| initial_picker_threshold | int | 4 | -+--------------------------+------+---------+ - -``initial_picker_threshold`` specifies the number of pieces we need -before we switch to rarest first picking. This defaults to 4, which -means the 4 first pieces in any torrent are picked at random, the -following pieces are picked in rarest first order. - -.. _allowed_fast_set_size: - -.. raw:: html - - - -+-----------------------+------+---------+ -| name | type | default | -+=======================+======+=========+ -| allowed_fast_set_size | int | 10 | -+-----------------------+------+---------+ - -the number of allowed pieces to send to peers that supports the -fast extensions - -.. _suggest_mode: - -.. raw:: html - - - -+--------------+------+-------------------------------------+ -| name | type | default | -+==============+======+=====================================+ -| suggest_mode | int | settings_pack::no_piece_suggestions | -+--------------+------+-------------------------------------+ - -``suggest_mode`` controls whether or not libtorrent will send out -suggest messages to create a bias of its peers to request certain -pieces. The modes are: - -* ``no_piece_suggestsions`` which is the default and will not send - out suggest messages. -* ``suggest_read_cache`` which will send out suggest messages for - the most recent pieces that are in the read cache. - -.. _max_queued_disk_bytes: - -.. raw:: html - - - -+-----------------------+------+-------------+ -| name | type | default | -+=======================+======+=============+ -| max_queued_disk_bytes | int | 1024 * 1024 | -+-----------------------+------+-------------+ - -``max_queued_disk_bytes`` is the number maximum number of bytes, to -be written to disk, that can wait in the disk I/O thread queue. -This queue is only for waiting for the disk I/O thread to receive -the job and either write it to disk or insert it in the write -cache. When this limit is reached, the peer connections will stop -reading data from their sockets, until the disk thread catches up. -Setting this too low will severely limit your download rate. - -.. _handshake_timeout: - -.. raw:: html - - - -+-------------------+------+---------+ -| name | type | default | -+===================+======+=========+ -| handshake_timeout | int | 10 | -+-------------------+------+---------+ - -the number of seconds to wait for a handshake response from a peer. -If no response is received within this time, the peer is -disconnected. - -.. _send_buffer_low_watermark: - -.. _send_buffer_watermark: - -.. _send_buffer_watermark_factor: - -.. raw:: html - - - - - -+------------------------------+------+------------+ -| name | type | default | -+==============================+======+============+ -| send_buffer_low_watermark | int | 10 * 1024 | -+------------------------------+------+------------+ -| send_buffer_watermark | int | 500 * 1024 | -+------------------------------+------+------------+ -| send_buffer_watermark_factor | int | 50 | -+------------------------------+------+------------+ - -``send_buffer_low_watermark`` the minimum send buffer target size -(send buffer includes bytes pending being read from disk). For good -and snappy seeding performance, set this fairly high, to at least -fit a few blocks. This is essentially the initial window size which -will determine how fast we can ramp up the send rate - -if the send buffer has fewer bytes than ``send_buffer_watermark``, -we'll read another 16kB block onto it. If set too small, upload -rate capacity will suffer. If set too high, memory will be wasted. -The actual watermark may be lower than this in case the upload rate -is low, this is the upper limit. - -the current upload rate to a peer is multiplied by this factor to -get the send buffer watermark. The factor is specified as a -percentage. i.e. 50 -> 0.5 This product is clamped to the -``send_buffer_watermark`` setting to not exceed the max. For high -speed upload, this should be set to a greater value than 100. For -high capacity connections, setting this higher can improve upload -performance and disk throughput. Setting it too high may waste RAM -and create a bias towards read jobs over write jobs. - -.. _choking_algorithm: - -.. _seed_choking_algorithm: - -.. raw:: html - - - - -+------------------------+------+-----------------------------------+ -| name | type | default | -+========================+======+===================================+ -| choking_algorithm | int | settings_pack::fixed_slots_choker | -+------------------------+------+-----------------------------------+ -| seed_choking_algorithm | int | settings_pack::round_robin | -+------------------------+------+-----------------------------------+ - -``choking_algorithm`` specifies which algorithm to use to determine -which peers to unchoke. - -The options for choking algorithms are: - -* ``fixed_slots_choker`` is the traditional choker with a fixed - number of unchoke slots (as specified by - ``session::set_max_uploads()``). - -* ``rate_based_choker`` opens up unchoke slots based on the upload - rate achieved to peers. The more slots that are opened, the - marginal upload rate required to open up another slot increases. - -* ``bittyrant_choker`` attempts to optimize download rate by - finding the reciprocation rate of each peer individually and - prefers peers that gives the highest *return on investment*. It - still allocates all upload capacity, but shuffles it around to - the best peers first. For this choker to be efficient, you need - to set a global upload rate limit - (``session::set_upload_rate_limit()``). For more information - about this choker, see the paper_. This choker is not fully - implemented nor tested. - -.. _paper: http://bittyrant.cs.washington.edu/#papers - -``seed_choking_algorithm`` controls the seeding unchoke behavior. -The available options are: - -* ``round_robin`` which round-robins the peers that are unchoked - when seeding. This distributes the upload bandwidht uniformly and - fairly. It minimizes the ability for a peer to download everything - without redistributing it. - -* ``fastest_upload`` unchokes the peers we can send to the fastest. - This might be a bit more reliable in utilizing all available - capacity. - -* ``anti_leech`` prioritizes peers who have just started or are - just about to finish the download. The intention is to force - peers in the middle of the download to trade with each other. - -.. _cache_size: - -.. _cache_buffer_chunk_size: - -.. _cache_expiry: - -.. raw:: html - - - - - -+-------------------------+------+---------+ -| name | type | default | -+=========================+======+=========+ -| cache_size | int | 1024 | -+-------------------------+------+---------+ -| cache_buffer_chunk_size | int | 0 | -+-------------------------+------+---------+ -| cache_expiry | int | 300 | -+-------------------------+------+---------+ - -``cache_size`` is the disk write and read cache. It is specified -in units of 16 KiB blocks. Buffers that are part of a peer's send -or receive buffer also count against this limit. Send and receive -buffers will never be denied to be allocated, but they will cause -the actual cached blocks to be flushed or evicted. If this is set -to -1, the cache size is automatically set to the amount of -physical RAM available in the machine divided by 8. If the amount -of physical RAM cannot be determined, it's set to 1024 (= 16 MiB). - -Disk buffers are allocated using a pool allocator, the number of -blocks that are allocated at a time when the pool needs to grow can -be specified in ``cache_buffer_chunk_size``. Lower numbers saves -memory at the expense of more heap allocations. If it is set to 0, -the effective chunk size is proportional to the total cache size, -attempting to strike a good balance between performance and memory -usage. It defaults to 0. ``cache_expiry`` is the number of seconds -from the last cached write to a piece in the write cache, to when -it's forcefully flushed to disk. Default is 60 second. - -On 32 bit builds, the effective cache size will be limited to 3/4 of -2 GiB to avoid exceeding the virtual address space limit. - -.. _explicit_cache_interval: - -.. raw:: html - - - -+-------------------------+------+---------+ -| name | type | default | -+=========================+======+=========+ -| explicit_cache_interval | int | 30 | -+-------------------------+------+---------+ - -``explicit_cache_interval`` is the number of seconds in between -each refresh of a part of the explicit read cache. Torrents take -turns in refreshing and this is the time in between each torrent -refresh. Refreshing a torrent's explicit read cache means scanning -all pieces and picking a random set of the rarest ones. There is an -affinity to pick pieces that are already in the cache, so that -subsequent refreshes only swaps in pieces that are rarer than -whatever is in the cache at the time. - -.. _disk_io_write_mode: - -.. _disk_io_read_mode: - -.. raw:: html - - - - -+--------------------+------+--------------------------------+ -| name | type | default | -+====================+======+================================+ -| disk_io_write_mode | int | settings_pack::enable_os_cache | -+--------------------+------+--------------------------------+ -| disk_io_read_mode | int | settings_pack::enable_os_cache | -+--------------------+------+--------------------------------+ - -determines how files are opened when they're in read only mode -versus read and write mode. The options are: - -enable_os_cache - This is the default and files are opened normally, with the OS - caching reads and writes. -disable_os_cache - This opens all files in no-cache mode. This corresponds to the - OS not letting blocks for the files linger in the cache. This - makes sense in order to avoid the bittorrent client to - potentially evict all other processes' cache by simply handling - high throughput and large files. If libtorrent's read cache is - disabled, enabling this may reduce performance. - -One reason to disable caching is that it may help the operating -system from growing its file cache indefinitely. - -.. _outgoing_port: - -.. _num_outgoing_ports: - -.. raw:: html - - - - -+--------------------+------+---------+ -| name | type | default | -+====================+======+=========+ -| outgoing_port | int | 0 | -+--------------------+------+---------+ -| num_outgoing_ports | int | 0 | -+--------------------+------+---------+ - -this is the first port to use for binding outgoing connections to. -This is useful for users that have routers that allow QoS settings -based on local port. when binding outgoing connections to specific -ports, ``num_outgoing_ports`` is the size of the range. It should -be more than a few - -.. warning:: setting outgoing ports will limit the ability to keep - multiple connections to the same client, even for different - torrents. It is not recommended to change this setting. Its main - purpose is to use as an escape hatch for cheap routers with QoS - capability but can only classify flows based on port numbers. - -It is a range instead of a single port because of the problems with -failing to reconnect to peers if a previous socket to that peer and -port is in ``TIME_WAIT`` state. - -.. _peer_tos: - -.. raw:: html - - - -+----------+------+---------+ -| name | type | default | -+==========+======+=========+ -| peer_tos | int | 0 | -+----------+------+---------+ - -``peer_tos`` determines the TOS byte set in the IP header of every -packet sent to peers (including web seeds). The default value for -this is ``0x0`` (no marking). One potentially useful TOS mark is -``0x20``, this represents the *QBone scavenger service*. For more -details, see QBSS_. - -.. _`QBSS`: http://qbone.internet2.edu/qbss/ - -.. _active_downloads: - -.. _active_seeds: - -.. _active_checking: - -.. _active_dht_limit: - -.. _active_tracker_limit: - -.. _active_lsd_limit: - -.. _active_limit: - -.. _active_loaded_limit: - -.. raw:: html - - - - - - - - - - -+----------------------+------+---------+ -| name | type | default | -+======================+======+=========+ -| active_downloads | int | 3 | -+----------------------+------+---------+ -| active_seeds | int | 5 | -+----------------------+------+---------+ -| active_checking | int | 1 | -+----------------------+------+---------+ -| active_dht_limit | int | 88 | -+----------------------+------+---------+ -| active_tracker_limit | int | 1600 | -+----------------------+------+---------+ -| active_lsd_limit | int | 60 | -+----------------------+------+---------+ -| active_limit | int | 15 | -+----------------------+------+---------+ -| active_loaded_limit | int | 100 | -+----------------------+------+---------+ - -for auto managed torrents, these are the limits they are subject -to. If there are too many torrents some of the auto managed ones -will be paused until some slots free up. ``active_downloads`` and -``active_seeds`` controls how many active seeding and downloading -torrents the queuing mechanism allows. The target number of active -torrents is ``min(active_downloads + active_seeds, active_limit)``. -``active_downloads`` and ``active_seeds`` are upper limits on the -number of downloading torrents and seeding torrents respectively. -Setting the value to -1 means unlimited. -For example if there are 10 seeding torrents and 10 downloading -torrents, and ``active_downloads`` is 4 and ``active_seeds`` is 4, -there will be 4 seeds active and 4 downloading torrents. If the -settings are ``active_downloads`` = 2 and ``active_seeds`` = 4, -then there will be 2 downloading torrents and 4 seeding torrents -active. Torrents that are not auto managed are not counted against -these limits. - -``active_checking`` is the limit of number of simultaneous checking -torrents. - -``active_limit`` is a hard limit on the number of active (auto -managed) torrents. This limit also applies to slow torrents. - -``active_dht_limit`` is the max number of torrents to announce to -the DHT. By default this is set to 88, which is no more than one -DHT announce every 10 seconds. - -``active_tracker_limit`` is the max number of torrents to announce -to their trackers. By default this is 360, which is no more than -one announce every 5 seconds. - -``active_lsd_limit`` is the max number of torrents to announce to -the local network over the local service discovery protocol. By -default this is 80, which is no more than one announce every 5 -seconds (assuming the default announce interval of 5 minutes). - -You can have more torrents *active*, even though they are not -announced to the DHT, lsd or their tracker. If some peer knows -about you for any reason and tries to connect, it will still be -accepted, unless the torrent is paused, which means it won't accept -any connections. - -``active_loaded_limit`` is the number of torrents that are allowed -to be *loaded* at any given time. Note that a torrent can be active -even though it's not loaded. if an unloaded torrents finds a peer -that wants to access it, the torrent will be loaded on demand, -using a user-supplied callback function. If the feature of -unloading torrents is not enabled, this setting have no effect. If -this limit is set to 0, it means unlimited. For more information, -see dynamic-loading-of-torrent-files_. - -.. _auto_manage_interval: - -.. raw:: html - - - -+----------------------+------+---------+ -| name | type | default | -+======================+======+=========+ -| auto_manage_interval | int | 30 | -+----------------------+------+---------+ - -``auto_manage_interval`` is the number of seconds between the -torrent queue is updated, and rotated. - -.. _seed_time_limit: - -.. raw:: html - - - -+-----------------+------+--------------+ -| name | type | default | -+=================+======+==============+ -| seed_time_limit | int | 24 * 60 * 60 | -+-----------------+------+--------------+ - -this is the limit on the time a torrent has been an active seed -(specified in seconds) before it is considered having met the seed -limit criteria. See queuing_. - -.. _auto_scrape_interval: - -.. _auto_scrape_min_interval: - -.. raw:: html - - - - -+--------------------------+------+---------+ -| name | type | default | -+==========================+======+=========+ -| auto_scrape_interval | int | 1800 | -+--------------------------+------+---------+ -| auto_scrape_min_interval | int | 300 | -+--------------------------+------+---------+ - -``auto_scrape_interval`` is the number of seconds between scrapes -of queued torrents (auto managed and paused torrents). Auto managed -torrents that are paused, are scraped regularly in order to keep -track of their downloader/seed ratio. This ratio is used to -determine which torrents to seed and which to pause. - -``auto_scrape_min_interval`` is the minimum number of seconds -between any automatic scrape (regardless of torrent). In case there -are a large number of paused auto managed torrents, this puts a -limit on how often a scrape request is sent. - -.. _max_peerlist_size: - -.. _max_paused_peerlist_size: - -.. raw:: html - - - - -+--------------------------+------+---------+ -| name | type | default | -+==========================+======+=========+ -| max_peerlist_size | int | 3000 | -+--------------------------+------+---------+ -| max_paused_peerlist_size | int | 1000 | -+--------------------------+------+---------+ - -``max_peerlist_size`` is the maximum number of peers in the list of -known peers. These peers are not necessarily connected, so this -number should be much greater than the maximum number of connected -peers. Peers are evicted from the cache when the list grows passed -90% of this limit, and once the size hits the limit, peers are no -longer added to the list. If this limit is set to 0, there is no -limit on how many peers we'll keep in the peer list. - -``max_paused_peerlist_size`` is the max peer list size used for -torrents that are paused. This default to the same as -``max_peerlist_size``, but can be used to save memory for paused -torrents, since it's not as important for them to keep a large peer -list. - -.. _min_announce_interval: - -.. raw:: html - - - -+-----------------------+------+---------+ -| name | type | default | -+=======================+======+=========+ -| min_announce_interval | int | 5 * 60 | -+-----------------------+------+---------+ - -this is the minimum allowed announce interval for a tracker. This -is specified in seconds and is used as a sanity check on what is -returned from a tracker. It mitigates hammering misconfigured -trackers. - -.. _auto_manage_startup: - -.. raw:: html - - - -+---------------------+------+---------+ -| name | type | default | -+=====================+======+=========+ -| auto_manage_startup | int | 60 | -+---------------------+------+---------+ - -this is the number of seconds a torrent is considered active after -it was started, regardless of upload and download speed. This is so -that newly started torrents are not considered inactive until they -have a fair chance to start downloading. - -.. _seeding_piece_quota: - -.. raw:: html - - - -+---------------------+------+---------+ -| name | type | default | -+=====================+======+=========+ -| seeding_piece_quota | int | 20 | -+---------------------+------+---------+ - -``seeding_piece_quota`` is the number of pieces to send to a peer, -when seeding, before rotating in another peer to the unchoke set. -It defaults to 3 pieces, which means that when seeding, any peer -we've sent more than this number of pieces to will be unchoked in -favour of a choked peer. - -.. _max_rejects: - -.. raw:: html - - - -+-------------+------+---------+ -| name | type | default | -+=============+======+=========+ -| max_rejects | int | 50 | -+-------------+------+---------+ - -TODO: deprecate this -``max_rejects`` is the number of piece requests we will reject in a -row while a peer is choked before the peer is considered abusive -and is disconnected. - -.. _recv_socket_buffer_size: - -.. _send_socket_buffer_size: - -.. raw:: html - - - - -+-------------------------+------+---------+ -| name | type | default | -+=========================+======+=========+ -| recv_socket_buffer_size | int | 0 | -+-------------------------+------+---------+ -| send_socket_buffer_size | int | 0 | -+-------------------------+------+---------+ - -``recv_socket_buffer_size`` and ``send_socket_buffer_size`` -specifies the buffer sizes set on peer sockets. 0 (which is the -default) means the OS default (i.e. don't change the buffer sizes). -The socket buffer sizes are changed using setsockopt() with -SOL_SOCKET/SO_RCVBUF and SO_SNDBUFFER. - -.. _file_checks_delay_per_block: - -.. raw:: html - - - -+-----------------------------+------+---------+ -| name | type | default | -+=============================+======+=========+ -| file_checks_delay_per_block | int | 0 | -+-----------------------------+------+---------+ - -``file_checks_delay_per_block`` is the number of milliseconds to -sleep in between disk read operations when checking torrents. This -defaults to 0, but can be set to higher numbers to slow down the -rate at which data is read from the disk while checking. This may -be useful for background tasks that doesn't matter if they take a -bit longer, as long as they leave disk I/O time for other -processes. - -.. _read_cache_line_size: - -.. _write_cache_line_size: - -.. raw:: html - - - - -+-----------------------+------+---------+ -| name | type | default | -+=======================+======+=========+ -| read_cache_line_size | int | 32 | -+-----------------------+------+---------+ -| write_cache_line_size | int | 16 | -+-----------------------+------+---------+ - -``read_cache_line_size`` is the number of blocks to read into the -read cache when a read cache miss occurs. Setting this to 0 is -essentially the same thing as disabling read cache. The number of -blocks read into the read cache is always capped by the piece -boundary. - -When a piece in the write cache has ``write_cache_line_size`` -contiguous blocks in it, they will be flushed. Setting this to 1 -effectively disables the write cache. - -.. _optimistic_disk_retry: - -.. raw:: html - - - -+-----------------------+------+---------+ -| name | type | default | -+=======================+======+=========+ -| optimistic_disk_retry | int | 10 * 60 | -+-----------------------+------+---------+ - -``optimistic_disk_retry`` is the number of seconds from a disk -write errors occur on a torrent until libtorrent will take it out -of the upload mode, to test if the error condition has been fixed. - -libtorrent will only do this automatically for auto managed -torrents. - -You can explicitly take a torrent out of upload only mode using -set_upload_mode(). - -.. _max_suggest_pieces: - -.. raw:: html - - - -+--------------------+------+---------+ -| name | type | default | -+====================+======+=========+ -| max_suggest_pieces | int | 10 | -+--------------------+------+---------+ - -``max_suggest_pieces`` is the max number of suggested piece indices -received from a peer that's remembered. If a peer floods suggest -messages, this limit prevents libtorrent from using too much RAM. -It defaults to 10. - -.. _local_service_announce_interval: - -.. raw:: html - - - -+---------------------------------+------+---------+ -| name | type | default | -+=================================+======+=========+ -| local_service_announce_interval | int | 5 * 60 | -+---------------------------------+------+---------+ - -``local_service_announce_interval`` is the time between local -network announces for a torrent. By default, when local service -discovery is enabled a torrent announces itself every 5 minutes. -This interval is specified in seconds. - -.. _dht_announce_interval: - -.. raw:: html - - - -+-----------------------+------+---------+ -| name | type | default | -+=======================+======+=========+ -| dht_announce_interval | int | 15 * 60 | -+-----------------------+------+---------+ - -``dht_announce_interval`` is the number of seconds between -announcing torrents to the distributed hash table (DHT). - -.. _udp_tracker_token_expiry: - -.. raw:: html - - - -+--------------------------+------+---------+ -| name | type | default | -+==========================+======+=========+ -| udp_tracker_token_expiry | int | 60 | -+--------------------------+------+---------+ - -``udp_tracker_token_expiry`` is the number of seconds libtorrent -will keep UDP tracker connection tokens around for. This is -specified to be 60 seconds, and defaults to that. The higher this -value is, the fewer packets have to be sent to the UDP tracker. In -order for higher values to work, the tracker needs to be configured -to match the expiration time for tokens. - -.. _default_cache_min_age: - -.. raw:: html - - - -+-----------------------+------+---------+ -| name | type | default | -+=======================+======+=========+ -| default_cache_min_age | int | 1 | -+-----------------------+------+---------+ - -``default_cache_min_age`` is the minimum number of seconds any read -cache line is kept in the cache. This defaults to one second but -may be greater if ``guided_read_cache`` is enabled. Having a lower -bound on the time a cache line stays in the cache is an attempt -to avoid swapping the same pieces in and out of the cache in case -there is a shortage of spare cache space. - -.. _num_optimistic_unchoke_slots: - -.. raw:: html - - - -+------------------------------+------+---------+ -| name | type | default | -+==============================+======+=========+ -| num_optimistic_unchoke_slots | int | 0 | -+------------------------------+------+---------+ - -``num_optimistic_unchoke_slots`` is the number of optimistic -unchoke slots to use. It defaults to 0, which means automatic. -Having a higher number of optimistic unchoke slots mean you will -find the good peers faster but with the trade-off to use up more -bandwidth. When this is set to 0, libtorrent opens up 20% of your -allowed upload slots as optimistic unchoke slots. - -.. _default_est_reciprocation_rate: - -.. _increase_est_reciprocation_rate: - -.. _decrease_est_reciprocation_rate: - -.. raw:: html - - - - - -+---------------------------------+------+---------+ -| name | type | default | -+=================================+======+=========+ -| default_est_reciprocation_rate | int | 16000 | -+---------------------------------+------+---------+ -| increase_est_reciprocation_rate | int | 20 | -+---------------------------------+------+---------+ -| decrease_est_reciprocation_rate | int | 3 | -+---------------------------------+------+---------+ - -``default_est_reciprocation_rate`` is the assumed reciprocation -rate from peers when using the BitTyrant choker. This defaults to -14 kiB/s. If set too high, you will over-estimate your peers and be -more altruistic while finding the true reciprocation rate, if it's -set too low, you'll be too stingy and waste finding the true -reciprocation rate. - -``increase_est_reciprocation_rate`` specifies how many percent the -estimated reciprocation rate should be increased by each unchoke -interval a peer is still choking us back. This defaults to 20%. -This only applies to the BitTyrant choker. - -``decrease_est_reciprocation_rate`` specifies how many percent the -estimated reciprocation rate should be decreased by each unchoke -interval a peer unchokes us. This default to 3%. This only applies -to the BitTyrant choker. - -.. _max_pex_peers: - -.. raw:: html - - - -+---------------+------+---------+ -| name | type | default | -+===============+======+=========+ -| max_pex_peers | int | 50 | -+---------------+------+---------+ - -the max number of peers we accept from pex messages from a single -peer. this limits the number of concurrent peers any of our peers -claims to be connected to. If they claim to be connected to more -than this, we'll ignore any peer that exceeds this limit - -.. _tick_interval: - -.. raw:: html - - - -+---------------+------+---------+ -| name | type | default | -+===============+======+=========+ -| tick_interval | int | 500 | -+---------------+------+---------+ - -``tick_interval`` specifies the number of milliseconds between -internal ticks. This is the frequency with which bandwidth quota is -distributed to peers. It should not be more than one second (i.e. -1000 ms). Setting this to a low value (around 100) means higher -resolution bandwidth quota distribution, setting it to a higher -value saves CPU cycles. - -.. _share_mode_target: - -.. raw:: html - - - -+-------------------+------+---------+ -| name | type | default | -+===================+======+=========+ -| share_mode_target | int | 3 | -+-------------------+------+---------+ - -``share_mode_target`` specifies the target share ratio for share -mode torrents. This defaults to 3, meaning we'll try to upload 3 -times as much as we download. Setting this very high, will make it -very conservative and you might end up not downloading anything -ever (and not affecting your share ratio). It does not make any -sense to set this any lower than 2. For instance, if only 3 peers -need to download the rarest piece, it's impossible to download a -single piece and upload it more than 3 times. If the -share_mode_target is set to more than 3, nothing is downloaded. - -.. _upload_rate_limit: - -.. _download_rate_limit: - -.. raw:: html - - - - -+---------------------+------+---------+ -| name | type | default | -+=====================+======+=========+ -| upload_rate_limit | int | 0 | -+---------------------+------+---------+ -| download_rate_limit | int | 0 | -+---------------------+------+---------+ - -``upload_rate_limit``, ``download_rate_limit``, -``local_upload_rate_limit`` and ``local_download_rate_limit`` sets -the session-global limits of upload and download rate limits, in -bytes per second. The local rates refer to peers on the local -network. By default peers on the local network are not rate -limited. - -These rate limits are only used for local peers (peers within the -same subnet as the client itself) and it is only used when -``ignore_limits_on_local_network`` is set to true (which it is by -default). These rate limits default to unthrottled, but can be -useful in case you want to treat local peers preferentially, but -not quite unthrottled. - -A value of 0 means unlimited. - -.. _dht_upload_rate_limit: - -.. raw:: html - - - -+-----------------------+------+---------+ -| name | type | default | -+=======================+======+=========+ -| dht_upload_rate_limit | int | 4000 | -+-----------------------+------+---------+ - -``dht_upload_rate_limit`` sets the rate limit on the DHT. This is -specified in bytes per second and defaults to 4000. For busy boxes -with lots of torrents that requires more DHT traffic, this should -be raised. - -.. _unchoke_slots_limit: - -.. raw:: html - - - -+---------------------+------+---------+ -| name | type | default | -+=====================+======+=========+ -| unchoke_slots_limit | int | 8 | -+---------------------+------+---------+ - -``unchoke_slots_limit`` is the max number of unchoked peers in the -session. The number of unchoke slots may be ignored depending on -what ``choking_algorithm`` is set to. - -.. _connections_limit: - -.. raw:: html - - - -+-------------------+------+---------+ -| name | type | default | -+===================+======+=========+ -| connections_limit | int | 200 | -+-------------------+------+---------+ - -``connections_limit`` sets a global limit on the number of -connections opened. The number of connections is set to a hard -minimum of at least two per torrent, so if you set a too low -connections limit, and open too many torrents, the limit will not -be met. - -.. _connections_slack: - -.. raw:: html - - - -+-------------------+------+---------+ -| name | type | default | -+===================+======+=========+ -| connections_slack | int | 10 | -+-------------------+------+---------+ - -``connections_slack`` is the the number of incoming connections -exceeding the connection limit to accept in order to potentially -replace existing ones. - -.. _utp_target_delay: - -.. _utp_gain_factor: - -.. _utp_min_timeout: - -.. _utp_syn_resends: - -.. _utp_fin_resends: - -.. _utp_num_resends: - -.. _utp_connect_timeout: - -.. _utp_loss_multiplier: - -.. raw:: html - - - - - - - - - - -+---------------------+------+---------+ -| name | type | default | -+=====================+======+=========+ -| utp_target_delay | int | 100 | -+---------------------+------+---------+ -| utp_gain_factor | int | 3000 | -+---------------------+------+---------+ -| utp_min_timeout | int | 500 | -+---------------------+------+---------+ -| utp_syn_resends | int | 2 | -+---------------------+------+---------+ -| utp_fin_resends | int | 2 | -+---------------------+------+---------+ -| utp_num_resends | int | 3 | -+---------------------+------+---------+ -| utp_connect_timeout | int | 3000 | -+---------------------+------+---------+ -| utp_loss_multiplier | int | 50 | -+---------------------+------+---------+ - -``utp_target_delay`` is the target delay for uTP sockets in -milliseconds. A high value will make uTP connections more -aggressive and cause longer queues in the upload bottleneck. It -cannot be too low, since the noise in the measurements would cause -it to send too slow. The default is 50 milliseconds. -``utp_gain_factor`` is the number of bytes the uTP congestion -window can increase at the most in one RTT. This defaults to 300 -bytes. If this is set too high, the congestion controller reacts -too hard to noise and will not be stable, if it's set too low, it -will react slow to congestion and not back off as fast. -``utp_min_timeout`` is the shortest allowed uTP socket timeout, -specified in milliseconds. This defaults to 500 milliseconds. The -timeout depends on the RTT of the connection, but is never smaller -than this value. A connection times out when every packet in a -window is lost, or when a packet is lost twice in a row (i.e. the -resent packet is lost as well). - -The shorter the timeout is, the faster the connection will recover -from this situation, assuming the RTT is low enough. -``utp_syn_resends`` is the number of SYN packets that are sent (and -timed out) before giving up and closing the socket. -``utp_num_resends`` is the number of times a packet is sent (and -lossed or timed out) before giving up and closing the connection. -``utp_connect_timeout`` is the number of milliseconds of timeout -for the initial SYN packet for uTP connections. For each timed out -packet (in a row), the timeout is doubled. ``utp_loss_multiplier`` -controls how the congestion window is changed when a packet loss is -experienced. It's specified as a percentage multiplier for -``cwnd``. By default it's set to 50 (i.e. cut in half). Do not -change this value unless you know what you're doing. Never set it -higher than 100. - -.. _mixed_mode_algorithm: - -.. raw:: html - - - -+----------------------+------+----------------------------------+ -| name | type | default | -+======================+======+==================================+ -| mixed_mode_algorithm | int | settings_pack::peer_proportional | -+----------------------+------+----------------------------------+ - -The ``mixed_mode_algorithm`` determines how to treat TCP -connections when there are uTP connections. Since uTP is designed -to yield to TCP, there's an inherent problem when using swarms that -have both TCP and uTP connections. If nothing is done, uTP -connections would often be starved out for bandwidth by the TCP -connections. This mode is ``prefer_tcp``. The ``peer_proportional`` -mode simply looks at the current throughput and rate limits all TCP -connections to their proportional share based on how many of the -connections are TCP. This works best if uTP connections are not -rate limited by the global rate limiter (which they aren't by -default). - -.. _listen_queue_size: - -.. raw:: html - - - -+-------------------+------+---------+ -| name | type | default | -+===================+======+=========+ -| listen_queue_size | int | 5 | -+-------------------+------+---------+ - -``listen_queue_size`` is the value passed in to listen() for the -listen socket. It is the number of outstanding incoming connections -to queue up while we're not actively waiting for a connection to be -accepted. The default is 5 which should be sufficient for any -normal client. If this is a high performance server which expects -to receive a lot of connections, or used in a simulator or test, it -might make sense to raise this number. It will not take affect -until listen_on() is called again (or for the first time). - -.. _torrent_connect_boost: - -.. raw:: html - - - -+-----------------------+------+---------+ -| name | type | default | -+=======================+======+=========+ -| torrent_connect_boost | int | 10 | -+-----------------------+------+---------+ - -``torrent_connect_boost`` is the number of peers to try to connect -to immediately when the first tracker response is received for a -torrent. This is a boost to given to new torrents to accelerate -them starting up. The normal connect scheduler is run once every -second, this allows peers to be connected immediately instead of -waiting for the session tick to trigger connections. - -.. _alert_queue_size: - -.. raw:: html - - - -+------------------+------+---------+ -| name | type | default | -+==================+======+=========+ -| alert_queue_size | int | 1000 | -+------------------+------+---------+ - -``alert_queue_size`` is the maximum number of alerts queued up -internally. If alerts are not popped, the queue will eventually -fill up to this level. - -.. _max_metadata_size: - -.. raw:: html - - - -+-------------------+------+------------------+ -| name | type | default | -+===================+======+==================+ -| max_metadata_size | int | 3 * 1024 * 10240 | -+-------------------+------+------------------+ - -``max_metadata_size`` is the maximum allowed size (in bytes) to be -received by the metadata extension, i.e. magnet links. - -.. _hashing_threads: - -.. raw:: html - - - -+-----------------+------+---------+ -| name | type | default | -+=================+======+=========+ -| hashing_threads | int | 1 | -+-----------------+------+---------+ - -``hashing_threads`` is the number of threads to use for piece hash -verification. It defaults to 1. For very high download rates, on -machines with multiple cores, this could be incremented. Setting it -higher than the number of CPU cores would presumably not provide -any benefit of setting it to the number of cores. If it's set to 0, -hashing is done in the disk thread. - -.. _checking_mem_usage: - -.. raw:: html - - - -+--------------------+------+---------+ -| name | type | default | -+====================+======+=========+ -| checking_mem_usage | int | 256 | -+--------------------+------+---------+ - -the number of blocks to keep outstanding at any given time when -checking torrents. Higher numbers give faster re-checks but uses -more memory. Specified in number of 16 kiB blocks - -.. _predictive_piece_announce: - -.. raw:: html - - - -+---------------------------+------+---------+ -| name | type | default | -+===========================+======+=========+ -| predictive_piece_announce | int | 0 | -+---------------------------+------+---------+ - -if set to > 0, pieces will be announced to other peers before they -are fully downloaded (and before they are hash checked). The -intention is to gain 1.5 potential round trip times per downloaded -piece. When non-zero, this indicates how many milliseconds in -advance pieces should be announced, before they are expected to be -completed. - -.. _aio_threads: - -.. _aio_max: - -.. raw:: html - - - - -+-------------+------+---------+ -| name | type | default | -+=============+======+=========+ -| aio_threads | int | 4 | -+-------------+------+---------+ -| aio_max | int | 300 | -+-------------+------+---------+ - -for some aio back-ends, ``aio_threads`` specifies the number of -io-threads to use, and ``aio_max`` the max number of outstanding -jobs. - -.. _network_threads: - -.. raw:: html - - - -+-----------------+------+---------+ -| name | type | default | -+=================+======+=========+ -| network_threads | int | 0 | -+-----------------+------+---------+ - -``network_threads`` is the number of threads to use to call -``async_write_some`` (i.e. send) on peer connection sockets. When -seeding at extremely high rates, this may become a bottleneck, and -setting this to 2 or more may parallelize that cost. When using SSL -torrents, all encryption for outgoing traffic is done within the -socket send functions, and this will help parallelizing the cost of -SSL encryption as well. - -.. _ssl_listen: - -.. raw:: html - - - -+------------+------+---------+ -| name | type | default | -+============+======+=========+ -| ssl_listen | int | 4433 | -+------------+------+---------+ - -``ssl_listen`` sets the listen port for SSL connections. If this is -set to 0, no SSL listen port is opened. Otherwise a socket is -opened on this port. This setting is only taken into account when -opening the regular listen port, and won't re-open the listen -socket simply by changing this setting. - -.. _tracker_backoff: - -.. raw:: html - - - -+-----------------+------+---------+ -| name | type | default | -+=================+======+=========+ -| tracker_backoff | int | 250 | -+-----------------+------+---------+ - -``tracker_backoff`` determines how aggressively to back off from -retrying failing trackers. This value determines *x* in the -following formula, determining the number of seconds to wait until -the next retry: - - delay = 5 + 5 * x / 100 * fails^2 - -This setting may be useful to make libtorrent more or less -aggressive in hitting trackers. - -.. _share_ratio_limit: - -.. _seed_time_ratio_limit: - -.. raw:: html - - - - -+-----------------------+------+---------+ -| name | type | default | -+=======================+======+=========+ -| share_ratio_limit | int | 200 | -+-----------------------+------+---------+ -| seed_time_ratio_limit | int | 700 | -+-----------------------+------+---------+ - -when a seeding torrent reaches either the share ratio (bytes up / -bytes down) or the seed time ratio (seconds as seed / seconds as -downloader) or the seed time limit (seconds as seed) it is -considered done, and it will leave room for other torrents these -are specified as percentages - -.. _peer_turnover: - -.. _peer_turnover_cutoff: - -.. _peer_turnover_interval: - -.. raw:: html - - - - - -+------------------------+------+---------+ -| name | type | default | -+========================+======+=========+ -| peer_turnover | int | 4 | -+------------------------+------+---------+ -| peer_turnover_cutoff | int | 90 | -+------------------------+------+---------+ -| peer_turnover_interval | int | 300 | -+------------------------+------+---------+ - -peer_turnover is the percentage of peers to disconnect every -turnover peer_turnover_interval (if we're at the peer limit), this -is specified in percent when we are connected to more than limit * -peer_turnover_cutoff peers disconnect peer_turnover fraction of the -peers. It is specified in percent peer_turnover_interval is the -interval (in seconds) between optimistic disconnects if the -disconnects happen and how many peers are disconnected is -controlled by peer_turnover and peer_turnover_cutoff - -.. _connect_seed_every_n_download: - -.. raw:: html - - - -+-------------------------------+------+---------+ -| name | type | default | -+===============================+======+=========+ -| connect_seed_every_n_download | int | 10 | -+-------------------------------+------+---------+ - -this setting controls the priority of downloading torrents over -seeding or finished torrents when it comes to making peer -connections. Peer connections are throttled by the connection_speed -and the half-open connection limit. This makes peer connections a -limited resource. Torrents that still have pieces to download are -prioritized by default, to avoid having many seeding torrents use -most of the connection attempts and only give one peer every now -and then to the downloading torrent. libtorrent will loop over the -downloading torrents to connect a peer each, and every n:th -connection attempt, a finished torrent is picked to be allowed to -connect to a peer. This setting controls n. - -.. _max_http_recv_buffer_size: - -.. raw:: html - - - -+---------------------------+------+------------+ -| name | type | default | -+===========================+======+============+ -| max_http_recv_buffer_size | int | 4*1024*204 | -+---------------------------+------+------------+ - -the max number of bytes to allow an HTTP response to be when -announcing to trackers or downloading .torrent files via the -``url`` provided in ``add_torrent_params``. - -.. _max_retry_port_bind: - -.. raw:: html - - - -+---------------------+------+---------+ -| name | type | default | -+=====================+======+=========+ -| max_retry_port_bind | int | 10 | -+---------------------+------+---------+ - -if binding to a specific port fails, should the port be incremented -by one and tried again? This setting specifies how many times to -retry a failed port bind - -.. _alert_mask: - -.. raw:: html - - - -+------------+------+---------------------------+ -| name | type | default | -+============+======+===========================+ -| alert_mask | int | alert::error_notification | -+------------+------+---------------------------+ - -a bitmask combining flags from alert::category_t defining which -kinds of alerts to receive - -.. _out_enc_policy: - -.. _in_enc_policy: - -.. raw:: html - - - - -+----------------+------+---------------------------+ -| name | type | default | -+================+======+===========================+ -| out_enc_policy | int | settings_pack::pe_enabled | -+----------------+------+---------------------------+ -| in_enc_policy | int | settings_pack::pe_enabled | -+----------------+------+---------------------------+ - -control the settings for incoming and outgoing connections -respectively. see enc_policy enum for the available options. -Keep in mind that protocol encryption degrades performance in -several respects: - -1. It prevents "zero copy" disk buffers being sent to peers, since - each peer needs to mutate the data (i.e. encrypt it) the data - must be copied per peer connection rather than sending the same - buffer to multiple peers. -2. The encryption itself requires more CPU than plain bittorrent - protocol. The highest cost is the Diffie Hellman exchange on - connection setup. -3. The encryption handshake adds several round-trips to the - connection setup, and delays transferring data. - -.. _allowed_enc_level: - -.. raw:: html - - - -+-------------------+------+------------------------+ -| name | type | default | -+===================+======+========================+ -| allowed_enc_level | int | settings_pack::pe_both | -+-------------------+------+------------------------+ - -determines the encryption level of the connections. This setting -will adjust which encryption scheme is offered to the other peer, -as well as which encryption scheme is selected by the client. See -enc_level enum for options. - -.. _inactive_down_rate: - -.. _inactive_up_rate: - -.. raw:: html - - - - -+--------------------+------+---------+ -| name | type | default | -+====================+======+=========+ -| inactive_down_rate | int | 2048 | -+--------------------+------+---------+ -| inactive_up_rate | int | 2048 | -+--------------------+------+---------+ - -the download and upload rate limits for a torrent to be considered -active by the queuing mechanism. A torrent whose download rate is -less than ``inactive_down_rate`` and whose upload rate is less than -``inactive_up_rate`` for ``auto_manage_startup`` seconds, is -considered inactive, and another queued torrent may be started. -This logic is disabled if ``dont_count_slow_torrents`` is false. - -.. _proxy_type: - -.. raw:: html - - - -+------------+------+---------------------+ -| name | type | default | -+============+======+=====================+ -| proxy_type | int | settings_pack::none | -+------------+------+---------------------+ - -proxy to use, defaults to none. see proxy_type_t. - -.. _proxy_port: - -.. raw:: html - - - -+------------+------+---------+ -| name | type | default | -+============+======+=========+ -| proxy_port | int | 0 | -+------------+------+---------+ - -the port of the proxy server - -.. _i2p_port: - -.. raw:: html - - - -+----------+------+---------+ -| name | type | default | -+==========+======+=========+ -| i2p_port | int | 0 | -+----------+------+---------+ - -sets the i2p_ SAM bridge port to connect to. set the hostname with -the ``i2p_hostname`` setting. - -.. _i2p: http://www.i2p2.de - diff --git a/docs/stats_counters.rst b/docs/stats_counters.rst deleted file mode 100644 index f3f8bfc47..000000000 --- a/docs/stats_counters.rst +++ /dev/null @@ -1,2031 +0,0 @@ -.. _peer.error_peers: - -.. _peer.disconnected_peers: - -.. raw:: html - - - - -+-------------------------+---------+ -| name | type | -+=========================+=========+ -| peer.error_peers | counter | -+-------------------------+---------+ -| peer.disconnected_peers | counter | -+-------------------------+---------+ - - -``error_peers`` is the total number of peer disconnects -caused by an error (not initiated by this client) and -disconnected initiated by this client (``disconnected_peers``). - -.. _peer.eof_peers: - -.. _peer.connreset_peers: - -.. _peer.connrefused_peers: - -.. _peer.connaborted_peers: - -.. _peer.notconnected_peers: - -.. _peer.perm_peers: - -.. _peer.buffer_peers: - -.. _peer.unreachable_peers: - -.. _peer.broken_pipe_peers: - -.. _peer.addrinuse_peers: - -.. _peer.no_access_peers: - -.. _peer.invalid_arg_peers: - -.. _peer.aborted_peers: - -.. raw:: html - - - - - - - - - - - - - - - -+-------------------------+---------+ -| name | type | -+=========================+=========+ -| peer.eof_peers | counter | -+-------------------------+---------+ -| peer.connreset_peers | counter | -+-------------------------+---------+ -| peer.connrefused_peers | counter | -+-------------------------+---------+ -| peer.connaborted_peers | counter | -+-------------------------+---------+ -| peer.notconnected_peers | counter | -+-------------------------+---------+ -| peer.perm_peers | counter | -+-------------------------+---------+ -| peer.buffer_peers | counter | -+-------------------------+---------+ -| peer.unreachable_peers | counter | -+-------------------------+---------+ -| peer.broken_pipe_peers | counter | -+-------------------------+---------+ -| peer.addrinuse_peers | counter | -+-------------------------+---------+ -| peer.no_access_peers | counter | -+-------------------------+---------+ -| peer.invalid_arg_peers | counter | -+-------------------------+---------+ -| peer.aborted_peers | counter | -+-------------------------+---------+ - - -these counters break down the peer errors into more specific -categories. These errors are what the underlying transport -reported (i.e. TCP or uTP) - -.. _peer.piece_requests: - -.. _peer.max_piece_requests: - -.. _peer.invalid_piece_requests: - -.. _peer.choked_piece_requests: - -.. _peer.cancelled_piece_requests: - -.. _peer.piece_rejects: - -.. raw:: html - - - - - - - - -+-------------------------------+---------+ -| name | type | -+===============================+=========+ -| peer.piece_requests | counter | -+-------------------------------+---------+ -| peer.max_piece_requests | counter | -+-------------------------------+---------+ -| peer.invalid_piece_requests | counter | -+-------------------------------+---------+ -| peer.choked_piece_requests | counter | -+-------------------------------+---------+ -| peer.cancelled_piece_requests | counter | -+-------------------------------+---------+ -| peer.piece_rejects | counter | -+-------------------------------+---------+ - - -the total number of incoming piece requests we've received followed -by the number of rejected piece requests for various reasons. -max_piece_requests mean we already had too many outstanding requests -from this peer, so we rejected it. cancelled_piece_requests are ones -where the other end explicitly asked for the piece to be rejected. - -.. _peer.error_incoming_peers: - -.. _peer.error_outgoing_peers: - -.. raw:: html - - - - -+---------------------------+---------+ -| name | type | -+===========================+=========+ -| peer.error_incoming_peers | counter | -+---------------------------+---------+ -| peer.error_outgoing_peers | counter | -+---------------------------+---------+ - - -these counters break down the peer errors into -whether they happen on incoming or outgoing peers. - -.. _peer.error_rc4_peers: - -.. _peer.error_encrypted_peers: - -.. raw:: html - - - - -+----------------------------+---------+ -| name | type | -+============================+=========+ -| peer.error_rc4_peers | counter | -+----------------------------+---------+ -| peer.error_encrypted_peers | counter | -+----------------------------+---------+ - - -these counters break down the peer errors into -whether they happen on encrypted peers (just -encrypted handshake) and rc4 peers (full stream -encryption). These can indicate whether encrypted -peers are more or less likely to fail - -.. _peer.error_tcp_peers: - -.. _peer.error_utp_peers: - -.. raw:: html - - - - -+----------------------+---------+ -| name | type | -+======================+=========+ -| peer.error_tcp_peers | counter | -+----------------------+---------+ -| peer.error_utp_peers | counter | -+----------------------+---------+ - - -these counters break down the peer errors into -whether they happen on uTP peers or TCP peers. -these may indicate whether one protocol is -more error prone - -.. _peer.connect_timeouts: - -.. _peer.uninteresting_peers: - -.. _peer.timeout_peers: - -.. _peer.no_memory_peers: - -.. _peer.too_many_peers: - -.. _peer.transport_timeout_peers: - -.. _peer.num_banned_peers: - -.. _peer.banned_for_hash_failure: - -.. _peer.connection_attempts: - -.. _peer.connection_attempt_loops: - -.. _peer.incoming_connections: - -.. raw:: html - - - - - - - - - - - - - -+-------------------------------+---------+ -| name | type | -+===============================+=========+ -| peer.connect_timeouts | counter | -+-------------------------------+---------+ -| peer.uninteresting_peers | counter | -+-------------------------------+---------+ -| peer.timeout_peers | counter | -+-------------------------------+---------+ -| peer.no_memory_peers | counter | -+-------------------------------+---------+ -| peer.too_many_peers | counter | -+-------------------------------+---------+ -| peer.transport_timeout_peers | counter | -+-------------------------------+---------+ -| peer.num_banned_peers | counter | -+-------------------------------+---------+ -| peer.banned_for_hash_failure | counter | -+-------------------------------+---------+ -| peer.connection_attempts | counter | -+-------------------------------+---------+ -| peer.connection_attempt_loops | counter | -+-------------------------------+---------+ -| peer.incoming_connections | counter | -+-------------------------------+---------+ - - -these counters break down the reasons to -disconnect peers. - -.. _peer.num_tcp_peers: - -.. _peer.num_socks5_peers: - -.. _peer.num_http_proxy_peers: - -.. _peer.num_utp_peers: - -.. _peer.num_i2p_peers: - -.. _peer.num_ssl_peers: - -.. _peer.num_ssl_socks5_peers: - -.. _peer.num_ssl_http_proxy_peers: - -.. _peer.num_ssl_utp_peers: - -.. _peer.num_peers_half_open: - -.. _peer.num_peers_connected: - -.. _peer.num_peers_up_interested: - -.. _peer.num_peers_down_interested: - -.. _peer.num_peers_up_unchoked_all: - -.. _peer.num_peers_up_unchoked_optimistic: - -.. _peer.num_peers_up_unchoked: - -.. _peer.num_peers_down_unchoked: - -.. _peer.num_peers_up_requests: - -.. _peer.num_peers_down_requests: - -.. _peer.num_peers_end_game: - -.. _peer.num_peers_up_disk: - -.. _peer.num_peers_down_disk: - -.. raw:: html - - - - - - - - - - - - - - - - - - - - - - - - -+---------------------------------------+---------+ -| name | type | -+=======================================+=========+ -| peer.num_tcp_peers | counter | -+---------------------------------------+---------+ -| peer.num_socks5_peers | counter | -+---------------------------------------+---------+ -| peer.num_http_proxy_peers | counter | -+---------------------------------------+---------+ -| peer.num_utp_peers | counter | -+---------------------------------------+---------+ -| peer.num_i2p_peers | counter | -+---------------------------------------+---------+ -| peer.num_ssl_peers | counter | -+---------------------------------------+---------+ -| peer.num_ssl_socks5_peers | counter | -+---------------------------------------+---------+ -| peer.num_ssl_http_proxy_peers | counter | -+---------------------------------------+---------+ -| peer.num_ssl_utp_peers | counter | -+---------------------------------------+---------+ -| peer.num_peers_half_open | counter | -+---------------------------------------+---------+ -| peer.num_peers_connected | counter | -+---------------------------------------+---------+ -| peer.num_peers_up_interested | counter | -+---------------------------------------+---------+ -| peer.num_peers_down_interested | counter | -+---------------------------------------+---------+ -| peer.num_peers_up_unchoked_all | counter | -+---------------------------------------+---------+ -| peer.num_peers_up_unchoked_optimistic | counter | -+---------------------------------------+---------+ -| peer.num_peers_up_unchoked | counter | -+---------------------------------------+---------+ -| peer.num_peers_down_unchoked | counter | -+---------------------------------------+---------+ -| peer.num_peers_up_requests | counter | -+---------------------------------------+---------+ -| peer.num_peers_down_requests | counter | -+---------------------------------------+---------+ -| peer.num_peers_end_game | counter | -+---------------------------------------+---------+ -| peer.num_peers_up_disk | counter | -+---------------------------------------+---------+ -| peer.num_peers_down_disk | counter | -+---------------------------------------+---------+ - - -the number of peer connections for each kind of socket. -these counts include half-open (connecting) peers. -``num_peers_up_unchoked_all`` is the total number of unchoked peers, -whereas ``num_peers_up_unchoked`` only are unchoked peers that count -against the limit (i.e. excluding peers that are unchoked because the -limit doesn't apply to them). ``num_peers_up_unchoked_optimistic`` is -the number of optimistically unchoked peers. - -.. _net.on_read_counter: - -.. _net.on_write_counter: - -.. _net.on_tick_counter: - -.. _net.on_lsd_counter: - -.. _net.on_lsd_peer_counter: - -.. _net.on_udp_counter: - -.. _net.on_accept_counter: - -.. _net.on_disk_queue_counter: - -.. _net.on_disk_counter: - -.. raw:: html - - - - - - - - - - - -+---------------------------+---------+ -| name | type | -+===========================+=========+ -| net.on_read_counter | counter | -+---------------------------+---------+ -| net.on_write_counter | counter | -+---------------------------+---------+ -| net.on_tick_counter | counter | -+---------------------------+---------+ -| net.on_lsd_counter | counter | -+---------------------------+---------+ -| net.on_lsd_peer_counter | counter | -+---------------------------+---------+ -| net.on_udp_counter | counter | -+---------------------------+---------+ -| net.on_accept_counter | counter | -+---------------------------+---------+ -| net.on_disk_queue_counter | counter | -+---------------------------+---------+ -| net.on_disk_counter | counter | -+---------------------------+---------+ - - -These counters count the number of times the -network thread wakes up for each respective -reason. If these counters are very large, it -may indicate a performance issue, causing the -network thread to wake up too ofte, wasting CPU. -mitigate it by increasing buffers and limits -for the specific trigger that wakes up the -thread. - -.. _net.sent_payload_bytes: - -.. _net.sent_bytes: - -.. _net.sent_ip_overhead_bytes: - -.. _net.sent_tracker_bytes: - -.. _net.recv_payload_bytes: - -.. _net.recv_bytes: - -.. _net.recv_ip_overhead_bytes: - -.. _net.recv_tracker_bytes: - -.. raw:: html - - - - - - - - - - -+----------------------------+---------+ -| name | type | -+============================+=========+ -| net.sent_payload_bytes | counter | -+----------------------------+---------+ -| net.sent_bytes | counter | -+----------------------------+---------+ -| net.sent_ip_overhead_bytes | counter | -+----------------------------+---------+ -| net.sent_tracker_bytes | counter | -+----------------------------+---------+ -| net.recv_payload_bytes | counter | -+----------------------------+---------+ -| net.recv_bytes | counter | -+----------------------------+---------+ -| net.recv_ip_overhead_bytes | counter | -+----------------------------+---------+ -| net.recv_tracker_bytes | counter | -+----------------------------+---------+ - - -total number of bytes sent and received by the session - -.. _net.limiter_up_queue: - -.. _net.limiter_down_queue: - -.. raw:: html - - - - -+------------------------+---------+ -| name | type | -+========================+=========+ -| net.limiter_up_queue | counter | -+------------------------+---------+ -| net.limiter_down_queue | counter | -+------------------------+---------+ - - -the number of sockets currently waiting for upload and download -bandwidht from the rate limiter. - -.. _net.limiter_up_bytes: - -.. _net.limiter_down_bytes: - -.. raw:: html - - - - -+------------------------+---------+ -| name | type | -+========================+=========+ -| net.limiter_up_bytes | counter | -+------------------------+---------+ -| net.limiter_down_bytes | counter | -+------------------------+---------+ - - -the number of upload and download bytes waiting to be handed out from -the rate limiter. - -.. _net.recv_failed_bytes: - -.. raw:: html - - - -+-----------------------+---------+ -| name | type | -+=======================+=========+ -| net.recv_failed_bytes | counter | -+-----------------------+---------+ - - -the number of bytes downloaded that had to be discarded because they -failed the hash check - -.. _net.recv_redundant_bytes: - -.. raw:: html - - - -+--------------------------+---------+ -| name | type | -+==========================+=========+ -| net.recv_redundant_bytes | counter | -+--------------------------+---------+ - - -the number of downloaded bytes that were discarded because they -were downloaded multiple times (from different peers) - -.. _net.has_incoming_connections: - -.. raw:: html - - - -+------------------------------+---------+ -| name | type | -+==============================+=========+ -| net.has_incoming_connections | counter | -+------------------------------+---------+ - - -is false by default and set to true when -the first incoming connection is established -this is used to know if the client is behind -NAT or not. - -.. _ses.num_checking_torrents: - -.. _ses.num_stopped_torrents: - -.. _ses.num_upload_only_torrents: - -.. _ses.num_downloading_torrents: - -.. _ses.num_seeding_torrents: - -.. _ses.num_queued_seeding_torrents: - -.. _ses.num_queued_download_torrents: - -.. _ses.num_error_torrents: - -.. raw:: html - - - - - - - - - - -+----------------------------------+---------+ -| name | type | -+==================================+=========+ -| ses.num_checking_torrents | counter | -+----------------------------------+---------+ -| ses.num_stopped_torrents | counter | -+----------------------------------+---------+ -| ses.num_upload_only_torrents | counter | -+----------------------------------+---------+ -| ses.num_downloading_torrents | counter | -+----------------------------------+---------+ -| ses.num_seeding_torrents | counter | -+----------------------------------+---------+ -| ses.num_queued_seeding_torrents | counter | -+----------------------------------+---------+ -| ses.num_queued_download_torrents | counter | -+----------------------------------+---------+ -| ses.num_error_torrents | counter | -+----------------------------------+---------+ - - -these gauges count the number of torrents in -different states. Each torrent only belongs to -one of these states. For torrents that could -belong to multiple of these, the most prominent -in picked. For instance, a torrent with an error -counts as an error-torrent, regardless of its other -state. - -.. _ses.non_filter_torrents: - -.. raw:: html - - - -+-------------------------+---------+ -| name | type | -+=========================+=========+ -| ses.non_filter_torrents | counter | -+-------------------------+---------+ - - -the number of torrents that don't have the -IP filter applied to them. - -.. _ses.num_loaded_torrents: - -.. _ses.num_pinned_torrents: - -.. raw:: html - - - - -+-------------------------+---------+ -| name | type | -+=========================+=========+ -| ses.num_loaded_torrents | counter | -+-------------------------+---------+ -| ses.num_pinned_torrents | counter | -+-------------------------+---------+ - - -the number of torrents that are currently loaded - -.. _ses.num_piece_passed: - -.. _ses.num_piece_failed: - -.. _ses.num_have_pieces: - -.. _ses.num_total_pieces_added: - -.. raw:: html - - - - - - -+----------------------------+---------+ -| name | type | -+============================+=========+ -| ses.num_piece_passed | counter | -+----------------------------+---------+ -| ses.num_piece_failed | counter | -+----------------------------+---------+ -| ses.num_have_pieces | counter | -+----------------------------+---------+ -| ses.num_total_pieces_added | counter | -+----------------------------+---------+ - - -these count the number of times a piece has passed the -hash check, the number of times a piece was successfully -written to disk and the number of total possible pieces -added by adding torrents. e.g. when adding a torrent with -1000 piece, num_total_pieces_added is incremented by 1000. - -.. _ses.torrent_evicted_counter: - -.. raw:: html - - - -+-----------------------------+---------+ -| name | type | -+=============================+=========+ -| ses.torrent_evicted_counter | counter | -+-----------------------------+---------+ - - -this counts the number of times a torrent has been -evicted (only applies when `dynamic loading of torrent files`_ -is enabled). - -.. _ses.num_unchoke_slots: - -.. raw:: html - - - -+-----------------------+---------+ -| name | type | -+=======================+=========+ -| ses.num_unchoke_slots | counter | -+-----------------------+---------+ - - -the number of allowed unchoked peers - -.. _ses.num_incoming_choke: - -.. _ses.num_incoming_unchoke: - -.. _ses.num_incoming_interested: - -.. _ses.num_incoming_not_interested: - -.. _ses.num_incoming_have: - -.. _ses.num_incoming_bitfield: - -.. _ses.num_incoming_request: - -.. _ses.num_incoming_piece: - -.. _ses.num_incoming_cancel: - -.. _ses.num_incoming_dht_port: - -.. _ses.num_incoming_suggest: - -.. _ses.num_incoming_have_all: - -.. _ses.num_incoming_have_none: - -.. _ses.num_incoming_reject: - -.. _ses.num_incoming_allowed_fast: - -.. _ses.num_incoming_ext_handshake: - -.. _ses.num_incoming_pex: - -.. _ses.num_incoming_metadata: - -.. _ses.num_incoming_extended: - -.. _ses.num_outgoing_choke: - -.. _ses.num_outgoing_unchoke: - -.. _ses.num_outgoing_interested: - -.. _ses.num_outgoing_not_interested: - -.. _ses.num_outgoing_have: - -.. _ses.num_outgoing_bitfield: - -.. _ses.num_outgoing_request: - -.. _ses.num_outgoing_piece: - -.. _ses.num_outgoing_cancel: - -.. _ses.num_outgoing_dht_port: - -.. _ses.num_outgoing_suggest: - -.. _ses.num_outgoing_have_all: - -.. _ses.num_outgoing_have_none: - -.. _ses.num_outgoing_reject: - -.. _ses.num_outgoing_allowed_fast: - -.. _ses.num_outgoing_ext_handshake: - -.. _ses.num_outgoing_pex: - -.. _ses.num_outgoing_metadata: - -.. _ses.num_outgoing_extended: - -.. raw:: html - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -+---------------------------------+---------+ -| name | type | -+=================================+=========+ -| ses.num_incoming_choke | counter | -+---------------------------------+---------+ -| ses.num_incoming_unchoke | counter | -+---------------------------------+---------+ -| ses.num_incoming_interested | counter | -+---------------------------------+---------+ -| ses.num_incoming_not_interested | counter | -+---------------------------------+---------+ -| ses.num_incoming_have | counter | -+---------------------------------+---------+ -| ses.num_incoming_bitfield | counter | -+---------------------------------+---------+ -| ses.num_incoming_request | counter | -+---------------------------------+---------+ -| ses.num_incoming_piece | counter | -+---------------------------------+---------+ -| ses.num_incoming_cancel | counter | -+---------------------------------+---------+ -| ses.num_incoming_dht_port | counter | -+---------------------------------+---------+ -| ses.num_incoming_suggest | counter | -+---------------------------------+---------+ -| ses.num_incoming_have_all | counter | -+---------------------------------+---------+ -| ses.num_incoming_have_none | counter | -+---------------------------------+---------+ -| ses.num_incoming_reject | counter | -+---------------------------------+---------+ -| ses.num_incoming_allowed_fast | counter | -+---------------------------------+---------+ -| ses.num_incoming_ext_handshake | counter | -+---------------------------------+---------+ -| ses.num_incoming_pex | counter | -+---------------------------------+---------+ -| ses.num_incoming_metadata | counter | -+---------------------------------+---------+ -| ses.num_incoming_extended | counter | -+---------------------------------+---------+ -| ses.num_outgoing_choke | counter | -+---------------------------------+---------+ -| ses.num_outgoing_unchoke | counter | -+---------------------------------+---------+ -| ses.num_outgoing_interested | counter | -+---------------------------------+---------+ -| ses.num_outgoing_not_interested | counter | -+---------------------------------+---------+ -| ses.num_outgoing_have | counter | -+---------------------------------+---------+ -| ses.num_outgoing_bitfield | counter | -+---------------------------------+---------+ -| ses.num_outgoing_request | counter | -+---------------------------------+---------+ -| ses.num_outgoing_piece | counter | -+---------------------------------+---------+ -| ses.num_outgoing_cancel | counter | -+---------------------------------+---------+ -| ses.num_outgoing_dht_port | counter | -+---------------------------------+---------+ -| ses.num_outgoing_suggest | counter | -+---------------------------------+---------+ -| ses.num_outgoing_have_all | counter | -+---------------------------------+---------+ -| ses.num_outgoing_have_none | counter | -+---------------------------------+---------+ -| ses.num_outgoing_reject | counter | -+---------------------------------+---------+ -| ses.num_outgoing_allowed_fast | counter | -+---------------------------------+---------+ -| ses.num_outgoing_ext_handshake | counter | -+---------------------------------+---------+ -| ses.num_outgoing_pex | counter | -+---------------------------------+---------+ -| ses.num_outgoing_metadata | counter | -+---------------------------------+---------+ -| ses.num_outgoing_extended | counter | -+---------------------------------+---------+ - - -bittorrent message counters. These counters are incremented -every time a message of the corresponding type is received from -or sent to a bittorrent peer. - -.. _ses.waste_piece_timed_out: - -.. _ses.waste_piece_cancelled: - -.. _ses.waste_piece_unknown: - -.. _ses.waste_piece_seed: - -.. _ses.waste_piece_end_game: - -.. _ses.waste_piece_closing: - -.. raw:: html - - - - - - - - -+---------------------------+---------+ -| name | type | -+===========================+=========+ -| ses.waste_piece_timed_out | counter | -+---------------------------+---------+ -| ses.waste_piece_cancelled | counter | -+---------------------------+---------+ -| ses.waste_piece_unknown | counter | -+---------------------------+---------+ -| ses.waste_piece_seed | counter | -+---------------------------+---------+ -| ses.waste_piece_end_game | counter | -+---------------------------+---------+ -| ses.waste_piece_closing | counter | -+---------------------------+---------+ - - -the number of wasted downloaded bytes by reason of the bytes being -wasted. - -.. _picker.piece_picker_partial_loops: - -.. _picker.piece_picker_suggest_loops: - -.. _picker.piece_picker_sequential_loops: - -.. _picker.piece_picker_reverse_rare_loops: - -.. _picker.piece_picker_rare_loops: - -.. _picker.piece_picker_rand_start_loops: - -.. _picker.piece_picker_rand_loops: - -.. _picker.piece_picker_busy_loops: - -.. raw:: html - - - - - - - - - - -+----------------------------------------+---------+ -| name | type | -+========================================+=========+ -| picker.piece_picker_partial_loops | counter | -+----------------------------------------+---------+ -| picker.piece_picker_suggest_loops | counter | -+----------------------------------------+---------+ -| picker.piece_picker_sequential_loops | counter | -+----------------------------------------+---------+ -| picker.piece_picker_reverse_rare_loops | counter | -+----------------------------------------+---------+ -| picker.piece_picker_rare_loops | counter | -+----------------------------------------+---------+ -| picker.piece_picker_rand_start_loops | counter | -+----------------------------------------+---------+ -| picker.piece_picker_rand_loops | counter | -+----------------------------------------+---------+ -| picker.piece_picker_busy_loops | counter | -+----------------------------------------+---------+ - - -the number of pieces considered while picking pieces - -.. _picker.reject_piece_picks: - -.. _picker.unchoke_piece_picks: - -.. _picker.incoming_redundant_piece_picks: - -.. _picker.incoming_piece_picks: - -.. _picker.end_game_piece_picks: - -.. _picker.snubbed_piece_picks: - -.. _picker.interesting_piece_picks: - -.. _picker.hash_fail_piece_picks: - -.. _disk.write_cache_blocks: - -.. _disk.read_cache_blocks: - -.. raw:: html - - - - - - - - - - - - -+---------------------------------------+---------+ -| name | type | -+=======================================+=========+ -| picker.reject_piece_picks | counter | -+---------------------------------------+---------+ -| picker.unchoke_piece_picks | counter | -+---------------------------------------+---------+ -| picker.incoming_redundant_piece_picks | counter | -+---------------------------------------+---------+ -| picker.incoming_piece_picks | counter | -+---------------------------------------+---------+ -| picker.end_game_piece_picks | counter | -+---------------------------------------+---------+ -| picker.snubbed_piece_picks | counter | -+---------------------------------------+---------+ -| picker.interesting_piece_picks | counter | -+---------------------------------------+---------+ -| picker.hash_fail_piece_picks | counter | -+---------------------------------------+---------+ -| disk.write_cache_blocks | counter | -+---------------------------------------+---------+ -| disk.read_cache_blocks | counter | -+---------------------------------------+---------+ - - -This breaks down the piece picks into the event that -triggered it - -.. _disk.request_latency: - -.. _disk.pinned_blocks: - -.. _disk.disk_blocks_in_use: - -.. _disk.queued_disk_jobs: - -.. _disk.num_running_disk_jobs: - -.. _disk.num_read_jobs: - -.. _disk.num_write_jobs: - -.. _disk.num_jobs: - -.. _disk.num_writing_threads: - -.. _disk.num_running_threads: - -.. _disk.blocked_disk_jobs: - -.. raw:: html - - - - - - - - - - - - - -+----------------------------+---------+ -| name | type | -+============================+=========+ -| disk.request_latency | counter | -+----------------------------+---------+ -| disk.pinned_blocks | counter | -+----------------------------+---------+ -| disk.disk_blocks_in_use | counter | -+----------------------------+---------+ -| disk.queued_disk_jobs | counter | -+----------------------------+---------+ -| disk.num_running_disk_jobs | counter | -+----------------------------+---------+ -| disk.num_read_jobs | counter | -+----------------------------+---------+ -| disk.num_write_jobs | counter | -+----------------------------+---------+ -| disk.num_jobs | counter | -+----------------------------+---------+ -| disk.num_writing_threads | counter | -+----------------------------+---------+ -| disk.num_running_threads | counter | -+----------------------------+---------+ -| disk.blocked_disk_jobs | counter | -+----------------------------+---------+ - - -the number of microseconds it takes from receiving a request from a -peer until we're sending the response back on the socket. - -.. _disk.queued_write_bytes: - -.. _disk.arc_mru_size: - -.. _disk.arc_mru_ghost_size: - -.. _disk.arc_mfu_size: - -.. _disk.arc_mfu_ghost_size: - -.. _disk.arc_write_size: - -.. _disk.arc_volatile_size: - -.. raw:: html - - - - - - - - - -+-------------------------+---------+ -| name | type | -+=========================+=========+ -| disk.queued_write_bytes | counter | -+-------------------------+---------+ -| disk.arc_mru_size | counter | -+-------------------------+---------+ -| disk.arc_mru_ghost_size | counter | -+-------------------------+---------+ -| disk.arc_mfu_size | counter | -+-------------------------+---------+ -| disk.arc_mfu_ghost_size | counter | -+-------------------------+---------+ -| disk.arc_write_size | counter | -+-------------------------+---------+ -| disk.arc_volatile_size | counter | -+-------------------------+---------+ - - -the number of bytes we have sent to the disk I/O -thread for writing. Every time we hear back from -the disk I/O thread with a completed write job, this -is updated to the number of bytes the disk I/O thread -is actually waiting for to be written (as opposed to -bytes just hanging out in the cache) - -.. _disk.num_blocks_written: - -.. _disk.num_blocks_read: - -.. raw:: html - - - - -+-------------------------+---------+ -| name | type | -+=========================+=========+ -| disk.num_blocks_written | counter | -+-------------------------+---------+ -| disk.num_blocks_read | counter | -+-------------------------+---------+ - - -the number of blocks written and read from disk in total. A block is -16 kiB. - -.. _disk.num_blocks_hashed: - -.. raw:: html - - - -+------------------------+---------+ -| name | type | -+========================+=========+ -| disk.num_blocks_hashed | counter | -+------------------------+---------+ - - -the total number of blocks run through SHA-1 hashing - -.. _disk.num_blocks_cache_hits: - -.. raw:: html - - - -+----------------------------+---------+ -| name | type | -+============================+=========+ -| disk.num_blocks_cache_hits | counter | -+----------------------------+---------+ - - -the number of blocks read from the disk cache - -.. _disk.num_write_ops: - -.. _disk.num_read_ops: - -.. raw:: html - - - - -+--------------------+---------+ -| name | type | -+====================+=========+ -| disk.num_write_ops | counter | -+--------------------+---------+ -| disk.num_read_ops | counter | -+--------------------+---------+ - - -the number of disk I/O operation for reads and writes. One disk -operation may transfer more then one block. - -.. _disk.num_read_back: - -.. raw:: html - - - -+--------------------+---------+ -| name | type | -+====================+=========+ -| disk.num_read_back | counter | -+--------------------+---------+ - - -the number of blocks that had to be read back from disk in order to -hash a piece (when verifying against the piece hash) - -.. _disk.disk_read_time: - -.. _disk.disk_write_time: - -.. _disk.disk_hash_time: - -.. _disk.disk_job_time: - -.. raw:: html - - - - - - -+----------------------+---------+ -| name | type | -+======================+=========+ -| disk.disk_read_time | counter | -+----------------------+---------+ -| disk.disk_write_time | counter | -+----------------------+---------+ -| disk.disk_hash_time | counter | -+----------------------+---------+ -| disk.disk_job_time | counter | -+----------------------+---------+ - - -cumulative time spent in various disk jobs, as well -as total for all disk jobs. Measured in microseconds - -.. _disk.num_fenced_read: - -.. _disk.num_fenced_write: - -.. _disk.num_fenced_hash: - -.. _disk.num_fenced_move_storage: - -.. _disk.num_fenced_release_files: - -.. _disk.num_fenced_delete_files: - -.. _disk.num_fenced_check_fastresume: - -.. _disk.num_fenced_save_resume_data: - -.. _disk.num_fenced_rename_file: - -.. _disk.num_fenced_stop_torrent: - -.. _disk.num_fenced_cache_piece: - -.. _disk.num_fenced_flush_piece: - -.. _disk.num_fenced_flush_hashed: - -.. _disk.num_fenced_flush_storage: - -.. _disk.num_fenced_trim_cache: - -.. _disk.num_fenced_file_priority: - -.. _disk.num_fenced_load_torrent: - -.. _disk.num_fenced_clear_piece: - -.. _disk.num_fenced_tick_storage: - -.. raw:: html - - - - - - - - - - - - - - - - - - - - - -+----------------------------------+---------+ -| name | type | -+==================================+=========+ -| disk.num_fenced_read | counter | -+----------------------------------+---------+ -| disk.num_fenced_write | counter | -+----------------------------------+---------+ -| disk.num_fenced_hash | counter | -+----------------------------------+---------+ -| disk.num_fenced_move_storage | counter | -+----------------------------------+---------+ -| disk.num_fenced_release_files | counter | -+----------------------------------+---------+ -| disk.num_fenced_delete_files | counter | -+----------------------------------+---------+ -| disk.num_fenced_check_fastresume | counter | -+----------------------------------+---------+ -| disk.num_fenced_save_resume_data | counter | -+----------------------------------+---------+ -| disk.num_fenced_rename_file | counter | -+----------------------------------+---------+ -| disk.num_fenced_stop_torrent | counter | -+----------------------------------+---------+ -| disk.num_fenced_cache_piece | counter | -+----------------------------------+---------+ -| disk.num_fenced_flush_piece | counter | -+----------------------------------+---------+ -| disk.num_fenced_flush_hashed | counter | -+----------------------------------+---------+ -| disk.num_fenced_flush_storage | counter | -+----------------------------------+---------+ -| disk.num_fenced_trim_cache | counter | -+----------------------------------+---------+ -| disk.num_fenced_file_priority | counter | -+----------------------------------+---------+ -| disk.num_fenced_load_torrent | counter | -+----------------------------------+---------+ -| disk.num_fenced_clear_piece | counter | -+----------------------------------+---------+ -| disk.num_fenced_tick_storage | counter | -+----------------------------------+---------+ - - -for each kind of disk job, a counter of how many jobs of that kind -are currently blocked by a disk fence - -.. _dht.dht_nodes: - -.. raw:: html - - - -+---------------+---------+ -| name | type | -+===============+=========+ -| dht.dht_nodes | counter | -+---------------+---------+ - - -The number of nodes in the DHT routing table - -.. _dht.dht_node_cache: - -.. raw:: html - - - -+--------------------+---------+ -| name | type | -+====================+=========+ -| dht.dht_node_cache | counter | -+--------------------+---------+ - - -The number of replacement nodes in the DHT routing table - -.. _dht.dht_torrents: - -.. raw:: html - - - -+------------------+---------+ -| name | type | -+==================+=========+ -| dht.dht_torrents | counter | -+------------------+---------+ - - -the number of torrents currently tracked by our DHT node - -.. _dht.dht_peers: - -.. raw:: html - - - -+---------------+---------+ -| name | type | -+===============+=========+ -| dht.dht_peers | counter | -+---------------+---------+ - - -the number of peers currently tracked by our DHT node - -.. _dht.dht_immutable_data: - -.. raw:: html - - - -+------------------------+---------+ -| name | type | -+========================+=========+ -| dht.dht_immutable_data | counter | -+------------------------+---------+ - - -the number of immutable data items tracked by our DHT node - -.. _dht.dht_mutable_data: - -.. raw:: html - - - -+----------------------+---------+ -| name | type | -+======================+=========+ -| dht.dht_mutable_data | counter | -+----------------------+---------+ - - -the number of mutable data items tracked by our DHT node - -.. _dht.dht_allocated_observers: - -.. raw:: html - - - -+-----------------------------+---------+ -| name | type | -+=============================+=========+ -| dht.dht_allocated_observers | counter | -+-----------------------------+---------+ - - -the number of RPC observers currently allocated - -.. _dht.dht_messages_in: - -.. _dht.dht_messages_out: - -.. raw:: html - - - - -+----------------------+---------+ -| name | type | -+======================+=========+ -| dht.dht_messages_in | counter | -+----------------------+---------+ -| dht.dht_messages_out | counter | -+----------------------+---------+ - - -the total number of DHT messages sent and received - -.. _dht.dht_messages_out_dropped: - -.. raw:: html - - - -+------------------------------+---------+ -| name | type | -+==============================+=========+ -| dht.dht_messages_out_dropped | counter | -+------------------------------+---------+ - - -the number of outgoing messages that failed to be -sent - -.. _dht.dht_bytes_in: - -.. _dht.dht_bytes_out: - -.. raw:: html - - - - -+-------------------+---------+ -| name | type | -+===================+=========+ -| dht.dht_bytes_in | counter | -+-------------------+---------+ -| dht.dht_bytes_out | counter | -+-------------------+---------+ - - -the total number of bytes sent and received by the DHT - -.. _dht.dht_ping_in: - -.. _dht.dht_ping_out: - -.. _dht.dht_find_node_in: - -.. _dht.dht_find_node_out: - -.. _dht.dht_get_peers_in: - -.. _dht.dht_get_peers_out: - -.. _dht.dht_announce_peer_in: - -.. _dht.dht_announce_peer_out: - -.. _dht.dht_get_in: - -.. _dht.dht_get_out: - -.. _dht.dht_put_in: - -.. _dht.dht_put_out: - -.. raw:: html - - - - - - - - - - - - - - -+---------------------------+---------+ -| name | type | -+===========================+=========+ -| dht.dht_ping_in | counter | -+---------------------------+---------+ -| dht.dht_ping_out | counter | -+---------------------------+---------+ -| dht.dht_find_node_in | counter | -+---------------------------+---------+ -| dht.dht_find_node_out | counter | -+---------------------------+---------+ -| dht.dht_get_peers_in | counter | -+---------------------------+---------+ -| dht.dht_get_peers_out | counter | -+---------------------------+---------+ -| dht.dht_announce_peer_in | counter | -+---------------------------+---------+ -| dht.dht_announce_peer_out | counter | -+---------------------------+---------+ -| dht.dht_get_in | counter | -+---------------------------+---------+ -| dht.dht_get_out | counter | -+---------------------------+---------+ -| dht.dht_put_in | counter | -+---------------------------+---------+ -| dht.dht_put_out | counter | -+---------------------------+---------+ - - -the number of DHT messages we've sent and received -by kind. - -.. _dht.dht_invalid_announce: - -.. _dht.dht_invalid_get_peers: - -.. _dht.dht_invalid_put: - -.. _dht.dht_invalid_get: - -.. raw:: html - - - - - - -+---------------------------+---------+ -| name | type | -+===========================+=========+ -| dht.dht_invalid_announce | counter | -+---------------------------+---------+ -| dht.dht_invalid_get_peers | counter | -+---------------------------+---------+ -| dht.dht_invalid_put | counter | -+---------------------------+---------+ -| dht.dht_invalid_get | counter | -+---------------------------+---------+ - - -the number of failed incoming DHT requests by kind of request - -.. _utp.utp_packet_loss: - -.. _utp.utp_timeout: - -.. _utp.utp_packets_in: - -.. _utp.utp_packets_out: - -.. _utp.utp_fast_retransmit: - -.. _utp.utp_packet_resend: - -.. _utp.utp_samples_above_target: - -.. _utp.utp_samples_below_target: - -.. _utp.utp_payload_pkts_in: - -.. _utp.utp_payload_pkts_out: - -.. _utp.utp_invalid_pkts_in: - -.. _utp.utp_redundant_pkts_in: - -.. raw:: html - - - - - - - - - - - - - - -+------------------------------+---------+ -| name | type | -+==============================+=========+ -| utp.utp_packet_loss | counter | -+------------------------------+---------+ -| utp.utp_timeout | counter | -+------------------------------+---------+ -| utp.utp_packets_in | counter | -+------------------------------+---------+ -| utp.utp_packets_out | counter | -+------------------------------+---------+ -| utp.utp_fast_retransmit | counter | -+------------------------------+---------+ -| utp.utp_packet_resend | counter | -+------------------------------+---------+ -| utp.utp_samples_above_target | counter | -+------------------------------+---------+ -| utp.utp_samples_below_target | counter | -+------------------------------+---------+ -| utp.utp_payload_pkts_in | counter | -+------------------------------+---------+ -| utp.utp_payload_pkts_out | counter | -+------------------------------+---------+ -| utp.utp_invalid_pkts_in | counter | -+------------------------------+---------+ -| utp.utp_redundant_pkts_in | counter | -+------------------------------+---------+ - - -uTP counters. Each counter represents the number of time each event -has occurred. - -.. _utp.num_utp_idle: - -.. _utp.num_utp_syn_sent: - -.. _utp.num_utp_connected: - -.. _utp.num_utp_fin_sent: - -.. _utp.num_utp_close_wait: - -.. _utp.num_utp_deleted: - -.. raw:: html - - - - - - - - -+------------------------+---------+ -| name | type | -+========================+=========+ -| utp.num_utp_idle | counter | -+------------------------+---------+ -| utp.num_utp_syn_sent | counter | -+------------------------+---------+ -| utp.num_utp_connected | counter | -+------------------------+---------+ -| utp.num_utp_fin_sent | counter | -+------------------------+---------+ -| utp.num_utp_close_wait | counter | -+------------------------+---------+ -| utp.num_utp_deleted | counter | -+------------------------+---------+ - - -the number of uTP sockets in each respective state - -.. _sock_bufs.socket_send_size3: - -.. _sock_bufs.socket_send_size4: - -.. _sock_bufs.socket_send_size5: - -.. _sock_bufs.socket_send_size6: - -.. _sock_bufs.socket_send_size7: - -.. _sock_bufs.socket_send_size8: - -.. _sock_bufs.socket_send_size9: - -.. _sock_bufs.socket_send_size10: - -.. _sock_bufs.socket_send_size11: - -.. _sock_bufs.socket_send_size12: - -.. _sock_bufs.socket_send_size13: - -.. _sock_bufs.socket_send_size14: - -.. _sock_bufs.socket_send_size15: - -.. _sock_bufs.socket_send_size16: - -.. _sock_bufs.socket_send_size17: - -.. _sock_bufs.socket_send_size18: - -.. _sock_bufs.socket_send_size19: - -.. _sock_bufs.socket_send_size20: - -.. _sock_bufs.socket_recv_size3: - -.. _sock_bufs.socket_recv_size4: - -.. _sock_bufs.socket_recv_size5: - -.. _sock_bufs.socket_recv_size6: - -.. _sock_bufs.socket_recv_size7: - -.. _sock_bufs.socket_recv_size8: - -.. _sock_bufs.socket_recv_size9: - -.. _sock_bufs.socket_recv_size10: - -.. _sock_bufs.socket_recv_size11: - -.. _sock_bufs.socket_recv_size12: - -.. _sock_bufs.socket_recv_size13: - -.. _sock_bufs.socket_recv_size14: - -.. _sock_bufs.socket_recv_size15: - -.. _sock_bufs.socket_recv_size16: - -.. _sock_bufs.socket_recv_size17: - -.. _sock_bufs.socket_recv_size18: - -.. _sock_bufs.socket_recv_size19: - -.. _sock_bufs.socket_recv_size20: - -.. raw:: html - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -+------------------------------+---------+ -| name | type | -+==============================+=========+ -| sock_bufs.socket_send_size3 | counter | -+------------------------------+---------+ -| sock_bufs.socket_send_size4 | counter | -+------------------------------+---------+ -| sock_bufs.socket_send_size5 | counter | -+------------------------------+---------+ -| sock_bufs.socket_send_size6 | counter | -+------------------------------+---------+ -| sock_bufs.socket_send_size7 | counter | -+------------------------------+---------+ -| sock_bufs.socket_send_size8 | counter | -+------------------------------+---------+ -| sock_bufs.socket_send_size9 | counter | -+------------------------------+---------+ -| sock_bufs.socket_send_size10 | counter | -+------------------------------+---------+ -| sock_bufs.socket_send_size11 | counter | -+------------------------------+---------+ -| sock_bufs.socket_send_size12 | counter | -+------------------------------+---------+ -| sock_bufs.socket_send_size13 | counter | -+------------------------------+---------+ -| sock_bufs.socket_send_size14 | counter | -+------------------------------+---------+ -| sock_bufs.socket_send_size15 | counter | -+------------------------------+---------+ -| sock_bufs.socket_send_size16 | counter | -+------------------------------+---------+ -| sock_bufs.socket_send_size17 | counter | -+------------------------------+---------+ -| sock_bufs.socket_send_size18 | counter | -+------------------------------+---------+ -| sock_bufs.socket_send_size19 | counter | -+------------------------------+---------+ -| sock_bufs.socket_send_size20 | counter | -+------------------------------+---------+ -| sock_bufs.socket_recv_size3 | counter | -+------------------------------+---------+ -| sock_bufs.socket_recv_size4 | counter | -+------------------------------+---------+ -| sock_bufs.socket_recv_size5 | counter | -+------------------------------+---------+ -| sock_bufs.socket_recv_size6 | counter | -+------------------------------+---------+ -| sock_bufs.socket_recv_size7 | counter | -+------------------------------+---------+ -| sock_bufs.socket_recv_size8 | counter | -+------------------------------+---------+ -| sock_bufs.socket_recv_size9 | counter | -+------------------------------+---------+ -| sock_bufs.socket_recv_size10 | counter | -+------------------------------+---------+ -| sock_bufs.socket_recv_size11 | counter | -+------------------------------+---------+ -| sock_bufs.socket_recv_size12 | counter | -+------------------------------+---------+ -| sock_bufs.socket_recv_size13 | counter | -+------------------------------+---------+ -| sock_bufs.socket_recv_size14 | counter | -+------------------------------+---------+ -| sock_bufs.socket_recv_size15 | counter | -+------------------------------+---------+ -| sock_bufs.socket_recv_size16 | counter | -+------------------------------+---------+ -| sock_bufs.socket_recv_size17 | counter | -+------------------------------+---------+ -| sock_bufs.socket_recv_size18 | counter | -+------------------------------+---------+ -| sock_bufs.socket_recv_size19 | counter | -+------------------------------+---------+ -| sock_bufs.socket_recv_size20 | counter | -+------------------------------+---------+ - - -the buffer sizes accepted by -socket send and receive calls respectively. -The larger the buffers are, the more efficient, -because it reqire fewer system calls per byte. -The size is 1 << n, where n is the number -at the end of the counter name. i.e. -8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, 8192, -16384, 32768, 65536, 131072, 262144, 524288, 1048576 -bytes - diff --git a/docs/todo.html b/docs/todo.html deleted file mode 100644 index 4d759622a..000000000 --- a/docs/todo.html +++ /dev/null @@ -1,10465 +0,0 @@ - - - - -

libtorrent todo-list

-0 urgent -21 important -61 relevant -7 feasible -177 notes -
relevance 3../src/file.cpp:538find out what error code is reported when the filesystem does not support hard links.
relevance 3../src/peer_connection.cpp:2962instead of having to ask the torrent whether it's in graceful pause mode or not, the peers should keep that state (and the torrent should update them when it enters graceful pause). When a peer enters graceful pause mode, it should cancel all outstanding requests and clear its request queue.
relevance 3../src/peer_connection.cpp:3832once peers are properly put in graceful pause mode, they can cancel all outstanding requests and this test can be removed.
relevance 3../src/session_impl.cpp:2065port map SSL udp socket here
relevance 3../src/session_impl.cpp:3806it would probably make sense to have a separate list of peers that are eligible for optimistic unchoke, similar to the torrents perhaps this could even iterate over the pool allocators of torrent_peer objects. It could probably be done in a single pass and collect the n best candidates
relevance 3../src/session_impl.cpp:3828peers should know whether their torrent is paused or not, instead of having to ask it over and over again
relevance 3../src/session_impl.cpp:4105there should be a pre-calculated list of all peers eligible for unchoking
relevance 3../src/torrent.cpp:9611this really needs to be moved to do_async_save_resume_data. flags need to be passed on
relevance 3../src/upnp.cpp:94listen_interface is not used. It's meant to bind the broadcast socket. it would probably have to be changed to a vector of interfaces to bind to though, since the broadcast socket opens one socket per local interface by default
relevance 3../src/kademlia/node_id.cpp:54the XORing should be done at full words instead of bytes
relevance 3../src/kademlia/node_id.cpp:66the XORing should be done at full words instead of bytes
relevance 3../src/kademlia/node_id.cpp:82the xoring should be done at full words and _builtin_clz() could be used as the last step
relevance 3../src/kademlia/routing_table.cpp:698the call to compare_ip_cidr here is expensive. peel off some layers of abstraction here to make it quicker. Look at xoring and using _builtin_ctz()
relevance 3../src/kademlia/rpc_manager.cpp:87move this into it's own .cpp file
relevance 3../include/libtorrent/socks5_stream.hpp:114enable this assert and fix remaining causes of it triggering
relevance 3../include/libtorrent/torrent.hpp:1347factor out the links (as well as update_list() to a separate class that torrent can inherit)
relevance 3../include/libtorrent/torrent_handle.hpp:237consider replacing all the setters and getters for pause, resume, stop-when-ready, share-mode, upload-mode, super-seeding, apply-ip-filter, resolve-countries, pinned, sequential-download, seed-mode with just set_flags() and clear_flags() using the flags from add_torrent_params. Perhaps those flags should have a more generic name.
relevance 3../include/libtorrent/torrent_handle.hpp:484unify url_seed and http_seed with just web_seed, using the web_seed_entry.
relevance 3../include/libtorrent/web_peer_connection.hpp:131if we make this be a disk_buffer_holder instead we would save a copy use allocate_disk_receive_buffer and release_disk_receive_buffer
relevance 3../include/libtorrent/kademlia/routing_table.hpp:99to improve memory locality and scanning performance, turn the routing table into a single vector with boundaries for the nodes instead. Perhaps replacement nodes should be in a separate vector.
relevance 3../include/libtorrent/aux_/allocating_handler.hpp:77make sure the handlers we pass in are potentially movable!
relevance 2../test/test_dht.cpp:514split this test up into smaller test cases
relevance 2../test/test_dht.cpp:2141split this up into smaller test cases
relevance 2../test/test_piece_picker.cpp:281split this up into smaller tests (where we print_title)
relevance 2../test/test_storage.cpp:485split this test up into smaller parts
relevance 2../src/alert.cpp:1461the salt here is allocated on the heap. It would be nice to allocate in in the stack_allocator
relevance 2../src/alert_manager.cpp:90keep a count of the number of threads waiting. Only if it's > 0 notify them
relevance 2../src/block_cache.cpp:1723turn these return values into enums returns -1: block not in cache -2: out of memory
relevance 2../src/escape_string.cpp:209this should probably be moved into string_util.cpp
relevance 2../src/file.cpp:567test this on a FAT volume to see what error we get!
relevance 2../src/http_tracker_connection.cpp:379returning a bool here is redundant. Instead this function should return the peer_entry
relevance 2../src/instantiate_connection.cpp:43peer_connection and tracker_connection should probably be flags
relevance 2../src/instantiate_connection.cpp:44move this function into libtorrent::aux namespace
relevance 2../src/peer_connection.cpp:2382this should probably be based on time instead of number of request messages. For a very high throughput connection, 300 may be a legitimate number of requests to have in flight when getting choked
relevance 2../src/peer_connection.cpp:3116since we throw away the queue entry once we issue the disk job, this may happen. Instead, we should keep the queue entry around, mark it as having been requested from disk and once the disk job comes back, discard it if it has been cancelled. Maybe even be able to cancel disk jobs?
relevance 2../src/peer_connection.cpp:4791use a deadline_timer for timeouts. Don't rely on second_tick()! Hook this up to connect timeout as well. This would improve performance because of less work in second_tick(), and might let use remove ticking entirely eventually
relevance 2../src/peer_list.cpp:495it would be nice if there was a way to iterate over these torrent_peer objects in the order they are allocated in the pool instead. It would probably be more efficient
relevance 2../src/piece_picker.cpp:1982make the 2048 limit configurable
relevance 2../src/piece_picker.cpp:2614the first_block returned here is the largest free range, not the first-fit range, which would be better
relevance 2../src/piece_picker.cpp:3395it would be nice if this could be folded into lock_piece() the main distinction is that this also maintains the m_num_passed counter and the passed_hash_check member Is there ever a case where we call write filed without also locking the piece? Perhaps write_failed() should imply locking it.
relevance 2../src/session_impl.cpp:463is there a reason not to move all of this into init()? and just post it to the io_service?
relevance 2../src/session_impl.cpp:1946the udp socket(s) should be using the same generic mechanism and not be restricted to a single one we should open a one listen socket for each entry in the listen_interfaces list
relevance 2../src/session_impl.cpp:2049use bind_to_device in udp_socket
relevance 2../src/session_impl.cpp:2069use bind_to_device in udp_socket
relevance 2../src/session_impl.cpp:3576make a list for torrents that want to be announced on the DHT so we don't have to loop over all torrents, just to find the ones that want to announce
relevance 2../src/session_impl.cpp:6057this should be factored into the udp socket, so we only have the code once
relevance 2../src/session_impl.cpp:6761perhaps DHT logging should be disabled by TORRENT_DISABLE_LOGGING too
relevance 2../src/storage.cpp:1069we probably need to do this unconditionally in this function. Even if the resume data file appears stale, we need to create these hard links, right?
relevance 2../src/storage.cpp:1093is this risky? The upper layer will assume we have the whole file. Perhaps we should verify that at least the size of the file is correct
relevance 2../src/torrent.cpp:681post alert
relevance 2../src/torrent.cpp:1907add a unit test where we don't have metadata, connect to a peer that sends a bitfield that's too large, then we get the metadata
relevance 2../src/torrent.cpp:4941abort lookups this torrent has made via the session host resolver interface
relevance 2../src/torrent.cpp:8127if peer is a really good peer, maybe we shouldn't disconnect it perhaps this logic should be disabled if we have too many idle peers (with some definition of idle)
relevance 2../src/tracker_manager.cpp:200some of these arguments could probably be moved to the tracker request itself. like the ip_filter and settings
relevance 2../src/udp_socket.cpp:810the udp_socket should really just be a single socket, and the session should support having more than one, just like with TCP sockets for now, just make bind failures non-fatal
relevance 2../src/udp_tracker_connection.cpp:83support authentication here. tracker_req().auth
relevance 2../src/ut_metadata.cpp:123if we were to initialize m_metadata_size lazily instead, we would probably be more efficient initialize m_metadata_size
relevance 2../src/utp_socket_manager.cpp:235we may want to take ec into account here. possibly close connections quicker
relevance 2../src/utp_stream.cpp:389it would be nice if not everything would have to be public here
relevance 2../src/web_peer_connection.cpp:328do we really need a special case here? wouldn't the multi-file case handle single file torrents correctly too?
relevance 2../src/web_peer_connection.cpp:550just make this peer not have the pieces associated with the file we just requested. Only when it doesn't have any of the file do the following
relevance 2../src/web_peer_connection.cpp:603create a mapping of file-index to redirection URLs. Use that to form URLs instead. Support to reconnect to a new server without destructing this peer_connection
relevance 2../src/kademlia/dht_storage.cpp:110make this configurable in dht_settings
relevance 2../src/kademlia/node.cpp:656it would be nice to have a bias towards node-id prefixes that are missing in the bucket
relevance 2../src/kademlia/node.cpp:734use the non deprecated function instead of this one
relevance 2../src/kademlia/node.cpp:893find_node should write directly to the response entry
relevance 2../src/kademlia/routing_table.cpp:132use the non deprecated function instead of this one
relevance 2../src/kademlia/routing_table.cpp:991move the lowest priority nodes to the replacement bucket
relevance 2../include/libtorrent/alert_types.hpp:1416should the alert baseclass have this object instead?
relevance 2../include/libtorrent/build_config.hpp:40instead of using a dummy function to cause link errors when incompatible build configurations are used, make the namespace name depend on the configuration, and have a using declaration in the headers to pull it into libtorrent.
relevance 2../include/libtorrent/enum_net.hpp:151this could be done more efficiently by just looking up the interface with the given name, maybe even with if_nametoindex()
relevance 2../include/libtorrent/heterogeneous_queue.hpp:56add emplace_back() version
relevance 2../include/libtorrent/peer_connection.hpp:1116rename this target queue size
relevance 2../include/libtorrent/piece_picker.hpp:594having 8 priority levels is probably excessive. It should probably be changed to 3 levels + dont-download
relevance 2../include/libtorrent/proxy_base.hpp:260use the resolver interface that has a built-in cache
relevance 2../include/libtorrent/session_handle.hpp:78the ip filter should probably be saved here too
relevance 2../include/libtorrent/socks5_stream.hpp:144add async_connect() that takes a hostname and port as well
relevance 2../include/libtorrent/tracker_manager.hpp:290this class probably doesn't need to have virtual functions.
relevance 2../include/libtorrent/aux_/session_impl.hpp:1163the throttling of saving resume data could probably be factored out into a separate class
relevance 2../include/libtorrent/aux_/session_interface.hpp:137the IP voting mechanism should be factored out to its own class, not part of the session
relevance 2../include/libtorrent/aux_/session_interface.hpp:162remove this. There's already get_resolver()
relevance 2../include/libtorrent/aux_/session_interface.hpp:217factor out the thread pool for socket jobs into a separate class used to (potentially) issue socket write calls onto multiple threads
relevance 1../src/disk_io_thread.cpp:218it would be nice to have the number of threads be set dynamically
relevance 1../src/http_seed_connection.cpp:129in chunked encoding mode, this assert won't hold. the chunk headers should be subtracted from the receive_buffer_size
relevance 1../src/session_impl.cpp:5449report the proper address of the router as the source IP of this understanding of our external address, instead of the empty address
relevance 1../src/torrent.cpp:1240make this depend on the error and on the filesystem the files are being downloaded to. If the error is no_space_left_on_device and the filesystem doesn't support sparse files, only zero the priorities of the pieces that are at the tails of all files, leaving everything up to the highest written piece in each file
relevance 1../src/torrent.cpp:7227save the send_stats state instead of throwing them away it may pose an issue when downgrading though
relevance 1../src/torrent.cpp:8471should disconnect all peers that have the pieces we have not just seeds. It would be pretty expensive to check all pieces for all peers though
relevance 1../include/libtorrent/ip_voter.hpp:124instead, have one instance per possible subnet, global IPv4, global IPv6, loopback, 192.168.x.x, 10.x.x.x, etc.
relevance 0../test/test_block_cache.cpp:469test try_evict_blocks
relevance 0../test/test_block_cache.cpp:470test evicting volatile pieces, to see them be removed
relevance 0../test/test_block_cache.cpp:471test evicting dirty pieces
relevance 0../test/test_block_cache.cpp:472test free_piece
relevance 0../test/test_block_cache.cpp:473test abort_dirty
relevance 0../test/test_block_cache.cpp:474test unaligned reads
relevance 0../test/test_bloom_filter.cpp:130test size()
relevance 0../test/test_bloom_filter.cpp:131test clear()
relevance 0../test/test_dht.cpp:103ideally the mock_socket would contain this queue of packets, to make tests independent
relevance 0../test/test_dht.cpp:420check to make sure the "best" items are stored
relevance 0../test/test_dht.cpp:513test obfuscated_get_peers
relevance 0../test/test_fast_extension.cpp:880test sending invalid requests (out of bound piece index, offsets and sizes)
relevance 0../test/test_file_progress.cpp:109test the update function too
relevance 0../test/test_file_storage.cpp:214test file_storage::optimize
relevance 0../test/test_file_storage.cpp:215test map_block
relevance 0../test/test_file_storage.cpp:216test piece_size(int piece)
relevance 0../test/test_file_storage.cpp:217test file_index_at_offset
relevance 0../test/test_file_storage.cpp:218test file attributes
relevance 0../test/test_file_storage.cpp:219test symlinks
relevance 0../test/test_file_storage.cpp:220test pad_files
relevance 0../test/test_file_storage.cpp:221test reorder_file (make sure internal_file_entry::swap() is used)
relevance 0../test/test_peer_list.cpp:939test erasing peers
relevance 0../test/test_peer_list.cpp:940test update_peer_port with allow_multiple_connections_per_ip and without
relevance 0../test/test_peer_list.cpp:941test add i2p peers
relevance 0../test/test_peer_list.cpp:942test allow_i2p_mixed
relevance 0../test/test_peer_list.cpp:943test insert_peer failing with all error conditions
relevance 0../test/test_peer_list.cpp:944test IPv6
relevance 0../test/test_peer_list.cpp:945test connect_to_peer() failing
relevance 0../test/test_peer_list.cpp:946test connection_closed
relevance 0../test/test_peer_list.cpp:947connect candidates recalculation when incrementing failcount
relevance 0../test/test_resolve_links.cpp:80test files with different piece size (negative test)
relevance 0../test/test_resolve_links.cpp:83it would be nice to test resolving of more than just 2 files as well. like 3 single file torrents merged into one, resolving all 3 files.
relevance 0../test/test_resume.cpp:232test what happens when loading a resume file with both piece priorities and file priorities (file prio should take presedence)
relevance 0../test/test_resume.cpp:235make sure a resume file only ever contain file priorities OR piece priorities. Never both.
relevance 0../test/test_resume.cpp:238generally save
relevance 0../test/test_resume.cpp:695test all other resume flags here too. This would require returning more than just the torrent_status from test_resume_flags. Also http seeds and trackers for instance
relevance 0../test/test_settings_pack.cpp:140load_pack_from_dict
relevance 0../test/test_ssl.cpp:394test using a signed certificate with the wrong info-hash in DN
relevance 0../test/test_ssl.cpp:492also test using a hash that refers to a valid torrent but that differs from the SNI hash
relevance 0../test/test_timestamp_history.cpp:54test the case where we have > 120 samples (and have the base delay actually be updated)
relevance 0../test/test_timestamp_history.cpp:55test the case where a sample is lower than the history entry but not lower than the base
relevance 0../test/test_torrent.cpp:135wait for an alert rather than just waiting 10 seconds. This is kind of silly
relevance 0../test/test_torrent_info.cpp:156test remap_files
relevance 0../test/test_torrent_info.cpp:157merkle torrents. specifically torrent_info::add_merkle_nodes and torrent with "root hash"
relevance 0../test/test_torrent_info.cpp:158torrent with 'p' (padfile) attribute
relevance 0../test/test_torrent_info.cpp:159torrent with 'h' (hidden) attribute
relevance 0../test/test_torrent_info.cpp:160torrent with 'x' (executable) attribute
relevance 0../test/test_torrent_info.cpp:161torrent with 'l' (symlink) attribute
relevance 0../test/test_torrent_info.cpp:162creating a merkle torrent (torrent_info::build_merkle_list)
relevance 0../test/test_torrent_info.cpp:163torrent with multiple trackers in multiple tiers, making sure we shuffle them (how do you test shuffling?, load it multiple times and make sure it's in different order at least once)
relevance 0../test/test_torrent_info.cpp:164torrents with a missing name
relevance 0../test/test_torrent_info.cpp:165torrents with a zero-length name
relevance 0../test/test_torrent_info.cpp:166torrents with a merkle tree and add_merkle_nodes
relevance 0../test/test_torrent_info.cpp:167torrent with a non-dictionary info-section
relevance 0../test/test_torrent_info.cpp:168torrents with DHT nodes
relevance 0../test/test_torrent_info.cpp:169torrent with url-list as a single string
relevance 0../test/test_torrent_info.cpp:170torrent with http seed as a single string
relevance 0../test/test_torrent_info.cpp:171torrent with a comment
relevance 0../test/test_torrent_info.cpp:172torrent with an SSL cert
relevance 0../test/test_torrent_info.cpp:173torrent with attributes (executable and hidden)
relevance 0../test/test_torrent_info.cpp:174torrent_info::add_tracker
relevance 0../test/test_torrent_info.cpp:175torrent_info::unload
relevance 0../test/test_torrent_info.cpp:176torrent_info constructor that takes an invalid bencoded buffer
relevance 0../test/test_torrent_info.cpp:177verify_encoding with a string that triggers character replacement
relevance 0../test/test_tracker.cpp:53test scrape requests
relevance 0../test/test_tracker.cpp:54test parse peers6
relevance 0../test/test_tracker.cpp:55test parse tracker-id
relevance 0../test/test_tracker.cpp:56test parse failure-reason
relevance 0../test/test_tracker.cpp:57test all failure paths, including invalid bencoding not a dictionary no files entry in scrape response no info-hash entry in scrape response malformed peers in peer list of dictionaries uneven number of bytes in peers and peers6 string responses
relevance 0../test/test_transfer.cpp:215these settings_pack tests belong in their own test
relevance 0../test/test_transfer.cpp:294factor out the disk-full test into its own unit test
relevance 0../test/test_upnp.cpp:108store the log and verify that some key messages are there
relevance 0../src/block_cache.cpp:987it's somewhat expensive to iterate over this linked list. Presumably because of the random access of memory. It would be nice if pieces with no evictable blocks weren't in this list
relevance 0../src/block_cache.cpp:1051this should probably only be done every n:th time
relevance 0../src/block_cache.cpp:1810create a holder for refcounts that automatically decrement
relevance 0../src/bt_peer_connection.cpp:691this could be optimized using knuth morris pratt
relevance 0../src/bt_peer_connection.cpp:2261if we're finished, send upload_only message
relevance 0../src/choker.cpp:336optimize this using partial_sort or something. We don't need to sort the entire list
relevance 0../src/choker.cpp:339make the comparison function a free function and move it into this cpp file
relevance 0../src/choker.cpp:344make configurable
relevance 0../src/choker.cpp:358make configurable
relevance 0../src/create_torrent.cpp:287this should probably be optional
relevance 0../src/disk_buffer_pool.cpp:322perhaps we should sort the buffers here?
relevance 0../src/disk_io_thread.cpp:883it would be nice to optimize this by having the cache pieces also ordered by
relevance 0../src/disk_io_thread.cpp:926instead of doing a lookup each time through the loop, save cached_piece_entry pointers with piece_refcount incremented to pin them
relevance 0../src/disk_io_thread.cpp:1105instead of doing this. pass in the settings to each storage_interface call. Each disk thread could hold its most recent understanding of the settings in a shared_ptr, and update it every time it wakes up from a job. That way each access to the settings won't require a mutex to be held.
relevance 0../src/disk_io_thread.cpp:1133a potentially more efficient solution would be to have a special queue for retry jobs, that's only ever run when a job completes, in any thread. It would only work if counters::num_running_disk_jobs > 0
relevance 0../src/disk_io_thread.cpp:1147it should clear the hash state even when there's an error, right?
relevance 0../src/disk_io_thread.cpp:1848maybe the tailqueue_iterator should contain a pointer-pointer instead and have an unlink function
relevance 0../src/disk_io_thread.cpp:2113this is potentially very expensive. One way to solve it would be to have a fence for just this one piece.
relevance 0../src/disk_io_thread.cpp:2375we should probably just hang the job on the piece and make sure the hasher gets kicked
relevance 0../src/disk_io_thread.cpp:2442introduce a holder class that automatically increments and decrements the piece_refcount
relevance 0../src/disk_io_thread.cpp:2692it would be nice to not have to lock the mutex every turn through this loop
relevance 0../src/file_progress.cpp:137it would be nice to not depend on alert_manager here
relevance 0../src/http_tracker_connection.cpp:186support this somehow
relevance 0../src/metadata_transfer.cpp:365this is not safe. The torrent could be unloaded while we're still sending the metadata
relevance 0../src/packet_buffer.cpp:180use compare_less_wrap for this comparison as well
relevance 0../src/part_file.cpp:252what do we do if someone is currently reading from the disk from this piece? does it matter? Since we won't actively erase the data from disk, but it may be overwritten soon, it's probably not that big of a deal
relevance 0../src/part_file.cpp:353instead of rebuilding the whole file header and flushing it, update the slot entries as we go
relevance 0../src/peer_connection.cpp:524it would be neat to be able to print this straight into the alert's stack allocator
relevance 0../src/peer_connection.cpp:1031this should be the global download rate
relevance 0../src/peer_connection.cpp:3364sort the allowed fast set in priority order
relevance 0../src/peer_connection.cpp:6176The stats checks can not be honored when authenticated encryption is in use because we may have encrypted data which we cannot authenticate yet
relevance 0../src/piece_picker.cpp:2061this could probably be optimized by incrementally calling partial_sort to sort one more element in the list. Because chances are that we'll just need a single piece, and once we've picked from it we're done. Sorting the rest of the list in that case is a waste of time.
relevance 0../src/piece_picker.cpp:2584when expanding pieces for cache stripe reasons, the !downloading condition doesn't make much sense
relevance 0../src/session_impl.cpp:523there's no rule here to make uTP connections not have the global or local rate limits apply to it. This used to be the default.
relevance 0../src/session_impl.cpp:1625it would be nice to reserve() these vectors up front
relevance 0../src/session_impl.cpp:1862instead of having a special case for this, just make the default listen interfaces be "0.0.0.0:6881,[::]:6881" and use the generic path. That would even allow for not listening at all.
relevance 0../src/session_impl.cpp:2794should this function take a shared_ptr instead?
relevance 0../src/session_impl.cpp:3162have a separate list for these connections, instead of having to loop through all of them
relevance 0../src/session_impl.cpp:3195this should apply to all bandwidth channels
relevance 0../src/session_impl.cpp:3989use a lower limit than m_settings.connections_limit to allocate the to 10% or so of connection slots for incoming connections
relevance 0../src/session_impl.cpp:4141post a message to have this happen immediately instead of waiting for the next tick
relevance 0../src/session_impl.cpp:4525it might be a nice feature here to limit the number of torrents to send in a single update. By just posting the first n torrents, they would nicely be round-robined because the torrent lists are always pushed back. Perhaps the status_update_alert could even have a fixed array of n entries rather than a vector, to further improve memory locality.
relevance 0../src/session_impl.cpp:4738this logic could probably be less spaghetti looking by being moved to a function with early exits
relevance 0../src/session_impl.cpp:5414perhaps this function should not exist when logging is disabled
relevance 0../src/storage.cpp:895make this more generic to not just work if files have been renamed, but also if they have been merged into a single file for instance maybe use the same format as .torrent files and reuse some code from torrent_info
relevance 0../src/storage.cpp:1234if everything moves OK, except for the partfile we currently won't update the save path, which breaks things. it would probably make more sense to give up on the partfile
relevance 0../src/string_util.cpp:60warning C4146: unary minus operator applied to unsigned type, result still unsigned
relevance 0../src/torrent.cpp:101factor out cache_status to its own header
relevance 0../src/torrent.cpp:468if the existing torrent doesn't have metadata, insert the metadata we just downloaded into it.
relevance 0../src/torrent.cpp:580if the existing torrent doesn't have metadata, insert the metadata we just downloaded into it.
relevance 0../src/torrent.cpp:1562is verify_peer_cert called once per certificate in the chain, and this function just tells us which depth we're at right now? If so, the comment makes sense. any certificate that isn't the leaf (i.e. the one presented by the peer) should be accepted automatically, given preverified is true. The leaf certificate need to be verified to make sure its DN matches the info-hash
relevance 0../src/torrent.cpp:1984instead of creating the picker up front here, maybe this whole section should move to need_picker()
relevance 0../src/torrent.cpp:2058this could be optimized by looking up which files are complete and just look at those
relevance 0../src/torrent.cpp:2074this could be optimized by looking up which files are complete and just look at those
relevance 0../src/torrent.cpp:2240there may be peer extensions relying on the torrent extension still being alive. Only do this if there are no peers. And when the last peer is disconnected, if the torrent is unloaded, clear the extensions m_extensions.clear();
relevance 0../src/torrent.cpp:2932this pattern is repeated in a few places. Factor this into a function and generalize the concept of a torrent having a dedicated listen port
relevance 0../src/torrent.cpp:3779add one peer per IP the hostname resolves to
relevance 0../src/torrent.cpp:4719update suggest_piece?
relevance 0../src/torrent.cpp:4863really, we should just keep the picker around in this case to maintain the availability counters
relevance 0../src/torrent.cpp:6926make this more generic to not just work if files have been renamed, but also if they have been merged into a single file for instance maybe use the same format as .torrent files and reuse some code from torrent_info The mapped_files needs to be read both in the network thread and in the disk thread, since they both have their own mapped files structures which are kept in sync
relevance 0../src/torrent.cpp:7059if this is a merkle torrent and we can't restore the tree, we need to wipe all the bits in the have array, but not necessarily we might want to do a full check to see if we have all the pieces. This is low priority since almost no one uses merkle torrents
relevance 0../src/torrent.cpp:7303make this more generic to not just work if files have been renamed, but also if they have been merged into a single file for instance. using file_base
relevance 0../src/torrent.cpp:9559add a flag to ignore stats, and only care about resume data for content. For unchanged files, don't trigger a load of the metadata just to save an empty resume data file
relevance 0../src/torrent.cpp:11186instead of resorting the whole list, insert the peers directly into the right place
relevance 0../src/torrent_peer.cpp:188how do we deal with our external address changing?
relevance 0../src/udp_socket.cpp:320it would be nice to detect this on posix systems also
relevance 0../src/udp_socket.cpp:869use the system resolver_interface here
relevance 0../src/ut_metadata.cpp:320we really need to increment the refcounter on the torrent while this buffer is still in the peer's send buffer
relevance 0../src/utp_stream.cpp:1761this loop is not very efficient. It could be fixed by having a separate list of sequence numbers that need resending
relevance 0../src/web_connection_base.cpp:81introduce a web-seed default class which has a low download priority
relevance 0../src/kademlia/dht_storage.cpp:426c++11 use a lambda here instead
relevance 0../src/kademlia/node.cpp:721in the future, this function should update all the dht related counter. For now, it just update the storage related ones.
relevance 0../src/kademlia/put_data.cpp:97what if o is not an isntance of put_data_observer? This need to be redesigned for better type saftey.
relevance 0../include/libtorrent/announce_entry.hpp:97include the number of peers received from this tracker, at last announce
relevance 0../include/libtorrent/block_cache.hpp:223make this 32 bits and to count seconds since the block cache was created
relevance 0../include/libtorrent/config.hpp:362Make this count Unicode characters instead of bytes on windows
relevance 0../include/libtorrent/disk_buffer_pool.hpp:137try to remove the observers, only using the async_allocate handlers
relevance 0../include/libtorrent/file.hpp:173move this into a separate header file, TU pair
relevance 0../include/libtorrent/heterogeneous_queue.hpp:184if this throws, should we do anything?
relevance 0../include/libtorrent/identify_client.hpp:50hide these declarations when deprecaated functions are disabled, and expose them internally in a header under aux_.
relevance 0../include/libtorrent/peer_connection.hpp:209make this a raw pointer (to save size in the first cache line) and make the constructor take a raw pointer. torrent objects should always outlive their peers
relevance 0../include/libtorrent/peer_connection.hpp:1044factor this out into its own class with a virtual interface torrent and session should implement this interface
relevance 0../include/libtorrent/peer_connection_interface.hpp:47make this interface smaller!
relevance 0../include/libtorrent/performance_counters.hpp:139should keepalives be in here too? how about dont-have, share-mode, upload-only
relevance 0../include/libtorrent/performance_counters.hpp:450some space could be saved here by making gauges 32 bits
relevance 0../include/libtorrent/performance_counters.hpp:451restore these to regular integers. Instead have one copy of the counters per thread and collect them at convenient synchronization points
relevance 0../include/libtorrent/piece_picker.hpp:756should this be allocated lazily?
relevance 0../include/libtorrent/proxy_base.hpp:174it would be nice to remember the bind port and bind once we know where the proxy is m_sock.bind(endpoint, ec);
relevance 0../include/libtorrent/receive_buffer.hpp:258Detect when the start of the next crpyto packet is aligned with the start of piece data and the crpyto packet is at least as large as the piece data. With a little extra work we could receive directly into a disk buffer in that case.
relevance 0../include/libtorrent/session_handle.hpp:682add get_peer_class_type_filter() as well
relevance 0../include/libtorrent/settings_pack.hpp:1094deprecate this ``max_rejects`` is the number of piece requests we will reject in a row while a peer is choked before the peer is considered abusive and is disconnected.
relevance 0../include/libtorrent/torrent.hpp:195make this a raw pointer. perhaps keep the shared_ptr around further down the object to maintain an owner
relevance 0../include/libtorrent/torrent.hpp:1244this wastes 5 bits per file
relevance 0../include/libtorrent/torrent.hpp:1302These two bitfields should probably be coalesced into one
relevance 0../include/libtorrent/torrent_info.hpp:117there may be some opportunities to optimize the size if torrent_info. specifically to turn some std::string and std::vector into pointers
relevance 0../include/libtorrent/tracker_manager.hpp:392this should be unique_ptr in the future
relevance 0../include/libtorrent/upnp.hpp:132support using the windows API for UPnP operations as well
relevance 0../include/libtorrent/utp_stream.hpp:424implement blocking write. Low priority since it's not used (yet)
relevance 0../include/libtorrent/kademlia/item.hpp:61since this is a public function, it should probably be moved out of this header and into one with other public functions.
relevance 0../include/libtorrent/aux_/session_impl.hpp:859should this be renamed m_outgoing_interfaces?
relevance 0../include/libtorrent/aux_/session_impl.hpp:911replace this by a proper asio timer
relevance 0../include/libtorrent/aux_/session_impl.hpp:916replace this by a proper asio timer
relevance 0../include/libtorrent/aux_/session_impl.hpp:923replace this by a proper asio timer
relevance 0../include/libtorrent/aux_/session_interface.hpp:241it would be nice to not have this be part of session_interface
relevance 0../include/libtorrent/aux_/session_settings.hpp:78make this a bitfield
\ No newline at end of file diff --git a/docs/troubleshooting.html b/docs/troubleshooting.html deleted file mode 100644 index 92e1c4aeb..000000000 --- a/docs/troubleshooting.html +++ /dev/null @@ -1,90 +0,0 @@ - - - - - - -libtorrent manual - - - - - - - - -
-
- - - - -
-

libtorrent manual

- --- - - - - - -
Author:Arvid Norberg, arvid@libtorrent.org
Version:1.1.0
-

The following troubleshooting chart may help in finding out why torrents fail -to download. It is not complete, please submit suggestions via mail to -arvid@libtorrent.org or to the mailing list. Ideally in the form of patches -against docs/troubleshooting.dot.

-

thumb

- -
-
-
- -
- -
- - diff --git a/docs/troubleshooting.png b/docs/troubleshooting.png deleted file mode 100644 index 9318e4e45..000000000 Binary files a/docs/troubleshooting.png and /dev/null differ diff --git a/docs/tuning.html b/docs/tuning.html deleted file mode 100644 index 63e23f57f..000000000 --- a/docs/tuning.html +++ /dev/null @@ -1,628 +0,0 @@ - - - - - - -libtorrent manual - - - - - - - - -
-
- - - - -
-

libtorrent manual

- --- - - - - - -
Author:Arvid Norberg, arvid@libtorrent.org
Version:1.1.0
- -
-

tuning libtorrent

-

libtorrent expose most constants used in the bittorrent engine for -customization through the session_settings. This makes it possible to -test and tweak the parameters for certain algorithms to make a client -that fits a wide range of needs. From low memory embedded devices to -servers seeding thousands of torrents. The default settings in libtorrent -are tuned for an end-user bittorrent client running on a normal desktop -computer.

-

This document describes techniques to benchmark libtorrent performance -and how parameters are likely to affect it.

-
-
-

reducing memory footprint

-

These are things you can do to reduce the memory footprint of libtorrent. You get -some of this by basing your default session_settings on the min_memory_usage() -setting preset function.

-

Keep in mind that lowering memory usage will affect performance, always profile -and benchmark your settings to determine if it's worth the trade-off.

-

The typical buffer usage of libtorrent, for a single download, with the cache -size set to 256 blocks (256 * 16 kiB = 4 MiB) is:

-
-read cache:      128.6 (2058 kiB)
-write cache:     103.5 (1656 kiB)
-receive buffers: 7.3   (117 kiB)
-send buffers:    4.8   (77 kiB)
-hash temp:       0.001 (19 Bytes)
-
-

The receive buffers is proportional to the number of connections we make, and is -limited by the total number of connections in the session (default is 200).

-

The send buffers is proportional to the number of upload slots that are allowed -in the session. The default is auto configured based on the observed upload rate.

-

The read and write cache can be controlled (see section below).

-

The "hash temp" entry size depends on whether or not hashing is optimized for -speed or memory usage. In this test run it was optimized for memory usage.

-
-

disable disk cache

-

The bulk of the memory libtorrent will use is used for the disk cache. To save -the absolute most amount of memory, you can disable the cache by setting -session_settings::cache_size to 0. You might want to consider using the cache -but just disable caching read operations. You do this by settings -session_settings::use_read_cache to false. This is the main factor in how much -memory will be used by the client. Keep in mind that you will degrade performance -by disabling the cache. You should benchmark the disk access in order to make an -informed trade-off.

-
-
-

remove torrents

-

Torrents that have been added to libtorrent will inevitably use up memory, even -when it's paused. A paused torrent will not use any peer connection objects or -any send or receive buffers though. Any added torrent holds the entire .torrent -file in memory, it also remembers the entire list of peers that it's heard about -(which can be fairly long unless it's capped). It also retains information about -which blocks and pieces we have on disk, which can be significant for torrents -with many pieces.

-

If you need to minimize the memory footprint, consider removing torrents from -the session rather than pausing them. This will likely only make a difference -when you have a very large number of torrents in a session.

-

The downside of removing them is that they will no longer be auto-managed. Paused -auto managed torrents are scraped periodically, to determine which torrents are -in the greatest need of seeding, and libtorrent will prioritize to seed those.

-
-
-

socket buffer sizes

-

You can make libtorrent explicitly set the kernel buffer sizes of all its peer -sockets. If you set this to a low number, you may see reduced throughput, especially -for high latency connections. It is however an opportunity to save memory per -connection, and might be worth considering if you have a very large number of -peer connections. This memory will not be visible in your process, this sets -the amount of kernel memory is used for your sockets.

-

Change this by setting session_settings::recv_socket_buffer_size and -session_settings::send_socket_buffer_size.

-
-
-

peer list size

-

The default maximum for the peer list is 4000 peers. For IPv4 peers, each peer -entry uses 32 bytes, which ends up using 128 kB per torrent. If seeding 4 popular -torrents, the peer lists alone uses about half a megabyte.

-

The default limit is the same for paused torrents as well, so if you have a -large number of paused torrents (that are popular) it will be even more -significant.

-

If you're short of memory, you should consider lowering the limit. 500 is probably -enough. You can do this by setting session_settings::max_peerlist_size to -the max number of peers you want in a torrent's peer list. This limit applies per -torrent. For 5 torrents, the total number of peers in peerlists will be 5 times -the setting.

-

You should also lower the same limit but for paused torrents. It might even make sense -to set that even lower, since you only need a few peers to start up while waiting -for the tracker and DHT to give you fresh ones. The max peer list size for paused -torrents is set by session_settings::max_paused_peerlist_size.

-

The drawback of lowering this number is that if you end up in a position where -the tracker is down for an extended period of time, your only hope of finding live -peers is to go through your list of all peers you've ever seen. Having a large -peer list will also help increase performance when starting up, since the torrent -can start connecting to peers in parallel with connecting to the tracker.

-
-
-

send buffer watermark

-

The send buffer watermark controls when libtorrent will ask the disk I/O thread -to read blocks from disk, and append it to a peer's send buffer.

-

When the send buffer has fewer than or equal number of bytes as -session_settings::send_buffer_watermark, the peer will ask the disk I/O thread -for more data to send. The trade-off here is between wasting memory by having too -much data in the send buffer, and hurting send rate by starving out the socket, -waiting for the disk read operation to complete.

-

If your main objective is memory usage and you're not concerned about being able -to achieve high send rates, you can set the watermark to 9 bytes. This will guarantee -that no more than a single (16 kiB) block will be on the send buffer at a time, for -all peers. This is the least amount of memory possible for the send buffer.

-

You should benchmark your max send rate when adjusting this setting. If you have -a very fast disk, you are less likely see a performance hit.

-
-
-

optimize hashing for memory usage

-

When libtorrent is doing hash checks of a file, or when it re-reads a piece that -was just completed to verify its hash, there are two options. The default one -is optimized for speed, which allocates buffers for the entire piece, reads in -the whole piece in one read call, then hashes it.

-

The second option is to optimize for memory usage instead, where a single buffer -is allocated, and the piece is read one block at a time, hashing it as each -block is read from the file. For low memory environments, this latter approach -is recommended. Change this by settings session_settings::optimize_hashing_for_speed -to false. This will significantly reduce peak memory usage, especially for -torrents with very large pieces.

-
-
-

reduce executable size

-

Compilers generally add a significant number of bytes to executables that make use -of C++ exceptions. By disabling exceptions (-fno-exceptions on GCC), you can -reduce the executable size with up to 45%. In order to build without exception -support, you need to patch parts of boost.

-

Also make sure to optimize for size when compiling.

-

Another way of reducing the executable size is to disable code that isn't used. -There are a number of TORRENT_* macros that control which features are included -in libtorrent. If these macros are used to strip down libtorrent, make sure the same -macros are defined when building libtorrent as when linking against it. If these -are different the structures will look different from the libtorrent side and from -the client side and memory corruption will follow.

-

One, probably, safe macro to define is TORRENT_NO_DEPRECATE which removes all -deprecated functions and struct members. As long as no deprecated functions are -relied upon, this should be a simple way to eliminate a little bit of code.

-

For all available options, see the building libtorrent secion.

-
-
-
-

play nice with the disk

-

When checking a torrent, libtorrent will try to read as fast as possible from the disk. -The only thing that might hold it back is a CPU that is slow at calculating SHA-1 hashes, -but typically the file checking is limited by disk read speed. Most operating systems -today do not prioritize disk access based on the importance of the operation, this means -that checking a torrent might delay other disk accesses, such as virtual memory swapping -or just loading file by other (interactive) applications.

-

In order to play nicer with the disk, and leave some spare time for it to service other -processes that might be of higher importance to the end-user, you can introduce a sleep -between the disc accesses. This is a direct tradeoff between how fast you can check a -torrent and how soft you will hit the disk.

-

You control this by setting the session_settings::file_checks_delay_per_block to greater -than zero. This number is the number of milliseconds to sleep between each read of 16 kiB.

-

The sleeps are not necessarily in between each 16 kiB block (it might be read in larger chunks), -but the number will be multiplied by the number of blocks that were read, to maintain the -same semantics.

-
-
-

high performance seeding

-

In the case of a high volume seed, there are two main concerns. Performance and scalability. -This translates into high send rates, and low memory and CPU usage per peer connection.

-
-

file pool

-

libtorrent keeps an LRU file cache. Each file that is opened, is stuck in the cache. The main -purpose of this is because of anti-virus software that hooks on file-open and file close to -scan the file. Anti-virus software that does that will significantly increase the cost of -opening and closing files. However, for a high performance seed, the file open/close might -be so frequent that it becomes a significant cost. It might therefore be a good idea to allow -a large file descriptor cache. Adjust this though session_settings::file_pool_size.

-

Don't forget to set a high rlimit for file descriptors in your process as well. This limit -must be high enough to keep all connections and files open.

-
-
-

disk cache

-

You typically want to set the cache size to as high as possible. The -session_settings::cache_size is specified in 16 kiB blocks. Since you're seeding, -the cache would be useless unless you also set session_settings::use_read_cache -to true.

-

In order to increase the possibility of read cache hits, set the -session_settings::cache_expiry to a large number. This won't degrade anything as -long as the client is only seeding, and not downloading any torrents.

-

In order to increase the disk cache hit rate, you can enable suggest messages based on -what's in the read cache. To do this, set session_settings::suggest_mode to -session_settings::suggest_read_cache. This will send suggest messages to peers -for the most recently used pieces in the read cache. This is especially useful if you -also enable explicit read cache, by settings session_settings::explicit_read_cache -to the number of pieces to keep in the cache. The explicit read cache will make the -disk read cache stick, and not be evicted by cache misses. The explicit read cache -will automatically pull in the rarest pieces in the read cache.

-

Assuming that you seed much more data than you can keep in the cache, to a large -numbers of peers (so that the read cache wouldn't be useful anyway), this may be a -good idea.

-

When peers first connect, libtorrent will send them a number of allow-fast messages, -which lets the peers download certain pieces even when they are choked, since peers -are choked by default, this often triggers immediate requests for those pieces. In the -case of using explicit read cache and suggesting those pieces, allowing fast pieces -should be disabled, to not systematically trigger requests for pieces that are not cached -for all peers. You can turn off allow-fast by settings session_settings::allowed_fast_set_size -to 0.

-

As an alternative to the explicit cache and suggest messages, there's a guided cache -mode. This means the size of the read cache line that's stored in the cache is determined -based on the upload rate to the peer that triggered the read operation. The idea being -that slow peers don't use up a disproportional amount of space in the cache. This -is enabled through session_settings::guided_read_cache.

-

In cases where the assumption is that the cache is only used as a read-ahead, and that no -other peer will ever request the same block while it's still in the cache, the read -cache can be set to be volatile. This means that every block that is requested out of -the read cache is removed immediately. This saves a significant amount of cache space -which can be used as read-ahead for other peers. This mode should never be combined -with either explicit_read_cache or suggest_read_cache, since those uses opposite -strategies for the read cache. You don't want to on one hand attract peers to request -the same pieces, and on the other hand assume that they won't request the same pieces -and drop them when the first peer requests it. To enable volatile read cache, set -session_settings::volatile_read_cache to true.

-
-
-

SSD as level 2 cache

-

It is possible to introduce a second level of cache, below the RAM disk cache. This is done -by setting session_settings::mmap_cache to a file path pointing to the SSD drive, and -increasing the session_settings::cache_size to the number of 16 kiB blocks would fit -on the drive (or less).

-

This will allocate disk buffers (for reading and writing) from a memory region that has -been mapped to the specified file. If the drive this file lives on is not significantly -faster than the destination drive, performance will be degraded. The point is to take -advantage primarily of the fast read speed from SSD drives and use it to extend the read -cache, improving seed performance.

-

Which parts of the cache that actually live in RAM is determined by the operating system.

-

Note that when using this feature, any block which ends up being pulled from the mmapped -file will be considered a cache hit.

-
-
-

uTP-TCP mixed mode

-

libtorrent supports uTP, which has a delay based congestion controller. In order to -avoid having a single TCP bittorrent connection completely starve out any uTP connection, -there is a mixed mode algorithm. This attempts to detect congestion on the uTP peers and -throttle TCP to avoid it taking over all bandwidth. This balances the bandwidth resources -between the two protocols. When running on a network where the bandwidth is in such an -abundance that it's virtually infinite, this algorithm is no longer necessary, and might -even be harmful to throughput. It is adviced to experiment with the -session_setting::mixed_mode_algorithm, setting it to session_settings::prefer_tcp. -This setting entirely disables the balancing and unthrottles all connections. On a typical -home connection, this would mean that none of the benefits of uTP would be preserved -(the modem's send buffer would be full at all times) and uTP connections would for the most -part be squashed by the TCP traffic.

-
-
-

send buffer low watermark

-

libtorrent uses a low watermark for send buffers to determine when a new piece should -be requested from the disk I/O subsystem, to be appended to the send buffer. The low -watermark is determined based on the send rate of the socket. It needs to be large -enough to not draining the socket's send buffer before the disk operation completes.

-

The watermark is bound to a max value, to avoid buffer sizes growing out of control. -The default max send buffer size might not be enough to sustain very high upload rates, -and you might have to increase it. It's specified in bytes in -session_settings::send_buffer_watermark.

-
-
-

peers

-

First of all, in order to allow many connections, set the global connection limit -high, session::set_max_connections(). Also set the upload rate limit to -infinite, session::set_upload_rate_limit(), passing 0 means infinite.

-

When dealing with a large number of peers, it might be a good idea to have slightly -stricter timeouts, to get rid of lingering connections as soon as possible.

-

There are a couple of relevant settings: session_settings::request_timeout, -session_settings::peer_timeout and session_settings::inactivity_timeout.

-

For seeds that are critical for a delivery system, you most likely want to allow -multiple connections from the same IP. That way two people from behind the same NAT -can use the service simultaneously. This is controlled by -session_settings::allow_multiple_connections_per_ip.

-

In order to always unchoke peers, turn off automatic unchoke -session_settings::auto_upload_slots and set the number of upload slots to a large -number via session::set_max_uploads(), or use -1 (which means infinite).

-
-
-

torrent limits

-

To seed thousands of torrents, you need to increase the session_settings::active_limit -and session_settings::active_seeds.

-
-
-

SHA-1 hashing

-

When downloading at very high rates, it is possible to have the CPU be the bottleneck -for passing every downloaded byte through SHA-1. In order to enable calculating SHA-1 -hashes in parallel, on multi-core systems, set session_settings::hashing_threads -to the number of threads libtorrent should start to do SHA-1 hashing. This defaults -to 1, and only if that thread is close to saturating one core does it make sense to -increase the number of threads.

-
-
-
-

scalability

-

In order to make more efficient use of the libtorrent interface when running a large -number of torrents simultaneously, one can use the session::get_torrent_status() call -together with session::refresh_torrent_status(). Keep in mind that every call into -libtorrent that return some value have to block your thread while posting a message to -the main network thread and then wait for a response (calls that don't return any data -will simply post the message and then immediately return). The time this takes might -become significant once you reach a few hundred torrents (depending on how many calls -you make to each torrent and how often). get_torrent_status lets you query the -status of all torrents in a single call. This will actually loop through all torrents -and run a provided predicate function to determine whether or not to include it in -the returned vector. If you have a lot of torrents, you might want to update the status -of only certain torrents. For instance, you might only be interested in torrents that -are being downloaded.

-

The intended use of these functions is to start off by calling get_torrent_status -to get a list of all torrents that match your criteria. Then call refresh_torrent_status -on that list. This will only refresh the status for the torrents in your list, and thus -ignore all other torrents you might be running. This may save a significant amount of -time, especially if the number of torrents you're interested in is small. In order to -keep your list of interested torrents up to date, you can either call get_torrent_status -from time to time, to include torrents you might have become interested in since the last -time. In order to stop refreshing a certain torrent, simply remove it from the list.

-

A more efficient way however, would be to subscribe to status alert notifications, and -update your list based on these alerts. There are alerts for when torrents are added, removed, -paused, resumed, completed etc. Doing this ensures that you only query status for the -minimal set of torrents you are actually interested in.

-
-
-

benchmarking

-

There is a bunch of built-in instrumentation of libtorrent that can be used to get an insight -into what it's doing and how well it performs. This instrumentation is enabled by defining -preprocessor symbols when building.

-

There are also a number of scripts that parses the log files and generates graphs (requires -gnuplot and python).

-
-

disk metrics

-

To enable disk I/O instrumentation, define TORRENT_DISK_STATS when building. When built -with this configuration libtorrent will create three log files, measuring various aspects of -the disk I/O. The following table is an overview of these files and what they measure.

- ---- - - - - - - - - - - - - - - - - -
filenamedescription
disk_io_thread.logThis is a log of which operation the disk I/O thread is -engaged in, with timestamps. This tells you what the thread -is spending its time doing.
disk_buffers.logThis log keeps track of what the buffers allocated from the -disk buffer pool are used for. There are 5 categories. -receive buffer, send buffer, write cache, read cache and -temporary hash storage. This is key when optimizing memory -usage.
disk_access.logThis is a low level log of read and write operations, with -timestamps and file offsets. The file offsets are byte -offsets in the torrent (not in any particular file, in the -case of a multi-file torrent). This can be used as an -estimate of the physical drive location. The purpose of -this log is to identify the amount of seeking the drive has -to do.
-
-

disk_io_thread.log

-

The structure of this log is simple. For each line, there are two columns, a timestamp and -the operation that was started. There is a special operation called idle which means -it looped back to the top and started waiting for new jobs. If there are more jobs to -handle immediately, the idle state is still there, but the timestamp is the same as the -next job that is handled.

-

Some operations have a 3:rd column with an optional parameter. read and write tells -you the number of bytes that were requested to be read or written. flushing tells you -the number of bytes that were flushed from the disk cache.

-

This is an example excerpt from a log:

-
-3702 idle
-3706 check_fastresume
-3707 idle
-4708 save_resume_data
-4708 idle
-8230 read 16384
-8255 idle
-8431 read 16384
-
-

The script to parse this log and generate a graph is called parse_disk_log.py. It takes -the log file as the first command line argument, and produces a file: disk_io.png. -The time stamp is in milliseconds since start.

-

You can pass in a second, optional, argument to specify the window size it will average -the time measurements over. The default is 5 seconds. For long test runs, it might be interesting -to increase that number. It is specified as a number of seconds.

-disk_io.png -

This is an example graph generated by the parse script.

-
-
-

disk_buffers.log

-

The disk buffer log tells you where the buffer memory is used. The log format has a time stamp, -the name of the buffer usage which use-count changed, colon, and the new number of blocks that are -in use for this particular key. For example:

-
-23671 write cache: 18
-23671 receive buffer: 3
-24153 receive buffer: 2
-24153 write cache: 19
-24154 receive buffer: 3
-24198 receive buffer: 2
-24198 write cache: 20
-24202 receive buffer: 3
-24305 send buffer: 0
-24305 send buffer: 1
-24909 receive buffer: 2
-24909 write cache: 21
-24910 receive buffer: 3
-
-

The time stamp is in milliseconds since start.

-

To generate a graph, use parse_disk_buffer_log.py. It takes the log file as the first -command line argument. It generates disk_buffer.png.

-disk_buffer_sample.png -

This is an example graph generated by the parse script.

-
-
-

disk_access.log

-

The disk access log is now binary

-

The disc access log has three fields. The timestamp (milliseconds since start), operation -and offset. The offset is the absolute offset within the torrent (not within a file). This -log is only useful when you're downloading a single torrent, otherwise the offsets will not -be unique.

-

In order to easily plot this directly in gnuplot, without parsing it, there are two lines -associated with each read or write operation. The first one is the offset where the operation -started, and the second one is where the operation ended.

-

Example:

-
-15437 read 301187072
-15437 read_end 301203456
-16651 read 213385216
-16680 read_end 213647360
-25879 write 249036800
-25879 write_end 249298944
-26811 read 325582848
-26943 read_end 325844992
-36736 read 367001600
-36766 read_end 367263744
-
-

The disk access log does not have any good visualization tool yet. There is however a gnuplot -file, disk_access.gnuplot which assumes disk_access.log is in the current directory.

-disk_access.png -

The density of the disk seeks tells you how hard the drive has to work.

-
-
-
-
-

understanding the disk threads

-

This section is somewhat outdated, there are potentially more than one disk -thread

-

All disk operations are funneled through a separate thread, referred to as the -disk thread. The main interface to the disk thread is a queue where disk jobs -are posted, and the results of these jobs are then posted back on the main -thread's io_service.

-

A disk job is essentially one of:

-
    -
  1. -
    write this block to disk, i.e. a write job. For the most part this is just a
    -

    matter of sticking the block in the disk cache, but if we've run out of -cache space or completed a whole piece, we'll also flush blocks to disk. -This is typically very fast, since the OS just sticks these buffers in its -write cache which will be flushed at a later time, presumably when the drive -head will pass the place on the platter where the blocks go.

    -
    -
    -
  2. -
  3. -
    read this block from disk. The first thing that happens is we look in the
    -

    cache to see if the block is already in RAM. If it is, we'll return -immediately with this block. If it's a cache miss, we'll have to hit the -disk. Here we decide to defer this job. We find the physical offset on the -drive for this block and insert the job in an ordered queue, sorted by the -physical location. At a later time, once we don't have any more non-read -jobs left in the queue, we pick one read job out of the ordered queue and -service it. The order we pick jobs out of the queue is according to an -elevator cursor moving up and down along the ordered queue of read jobs. If -we have enough space in the cache we'll read read_cache_line_size number of -blocks and stick those in the cache. This defaults to 32 blocks. If the -system supports asynchronous I/O (Windows, Linux, Mac OS X, BSD, Solars for -instance), jobs will be issued immediately to the OS. This especially -increases read throughput, since the OS has a much greater flexibility to -reorder the read jobs.

    -
    -
    -
  4. -
-

Other disk job consist of operations that needs to be synchronized with the -disk I/O, like renaming files, closing files, flushing the cache, updating the -settings etc. These are relatively rare though.

-
-
-

contributions

-

If you have added instrumentation for some part of libtorrent that is not -covered here, or if you have improved any of the parser scrips, please consider -contributing it back to the project.

-

If you have run tests and found that some algorithm or default value in -libtorrent is suboptimal, please contribute that knowledge back as well, to -allow us to improve the library.

-

If you have additional suggestions on how to tune libtorrent for any specific -use case, please let us know and we'll update this document.

-
- -
-
-
- -
- -
- - diff --git a/docs/udp_tracker_protocol.html b/docs/udp_tracker_protocol.html deleted file mode 100644 index c111f5b7d..000000000 --- a/docs/udp_tracker_protocol.html +++ /dev/null @@ -1,605 +0,0 @@ - - - - - - -Bittorrent udp-tracker protocol extension - - - - - - - - -
-
- - - - -
-

Bittorrent udp-tracker protocol extension

- --- - - - -
Author:Arvid Norberg, arvid@libtorrent.org
- -
-

introduction

-

A tracker with the protocol "udp://" in its URI -is supposed to be contacted using this protocol.

-

This protocol is supported by -xbt-tracker.

-

For additional information and descritptions of -the terminology used in this document, see -the protocol specification

-

All values are sent in network byte order (big endian). The sizes -are specified with ANSI-C standard types.

-

If no response to a request is received within 15 seconds, resend -the request. If no reply has been received after 60 seconds, stop -retrying.

-
-
-

connecting

-

Client sends packet:

- ----- - - - - - - - - - - - - - - - - - - - - -
sizenamedescription
int64_tconnection_idMust be initialized to 0x41727101980 -in network byte order. This will -identify the protocol.
int32_taction0 for a connection request
int32_ttransaction_idRandomized by client.
-

Server replies with packet:

- ----- - - - - - - - - - - - - - - - - - - - - -
sizenamedescription
int32_tactionDescribes the type of packet, in this -case it should be 0, for connect. -If 3 (for error) see errors.
int32_ttransaction_idMust match the transaction_id sent -from the client.
int64_tconnection_idA connection id, this is used when -further information is exchanged with -the tracker, to identify you. -This connection id can be reused for -multiple requests, but if it's cached -for too long, it will not be valid -anymore.
-
-
-

announcing

-

Client sends packet:

- ----- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
sizenamedescription
int64_tconnection_idThe connection id acquired from -establishing the connection.
int32_tactionAction. in this case, 1 for announce. -See actions.
int32_ttransaction_idRandomized by client.
int8_t[20]info_hashThe info-hash of the torrent you want -announce yourself in.
int8_t[20]peer_idYour peer id.
int64_tdownloadedThe number of byte you've downloaded -in this session.
int64_tleftThe number of bytes you have left to -download until you're finished.
int64_tuploadedThe number of bytes you have uploaded -in this session.
int32_tevent

The event, one of

-
-
    -
  • none = 0
  • -
  • completed = 1
  • -
  • started = 2
  • -
  • stopped = 3
  • -
-
-
uint32_tipYour ip address. Set to 0 if you want -the tracker to use the sender of -this udp packet.
uint32_tkeyA unique key that is randomized by the -client.
int32_tnum_wantThe maximum number of peers you want -in the reply. Use -1 for default.
uint16_tportThe port you're listening on.
uint16_textensionsSee extensions
-

Server replies with packet:

- ----- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
sizenamedescription
int32_tactionThe action this is a reply to. Should -in this case be 1 for announce. -If 3 (for error) see errors. -See actions.
int32_ttransaction_idMust match the transaction_id sent -in the announce request.
int32_tintervalthe number of seconds you should wait -until reannouncing yourself.
int32_tleechersThe number of peers in the swarm that -has not finished downloading.
int32_tseedersThe number of peers in the swarm that -has finished downloading and are -seeding.
-

The rest of the server reply is a variable number of the following structure:

- ----- - - - - - - - - - - - - - - - - -
sizenamedescription
int32_tipThe ip of a peer in the swarm.
uint16_tportThe peer's listen port.
-
-
-

scraping

-

Client sends packet:

- ----- - - - - - - - - - - - - - - - - - - - - -
sizenamedescription
int64_tconnection_idThe connection id retreived from the -establishing of the connection.
int32_tactionThe action, in this case, 2 for -scrape. See actions.
int32_ttransaction_idRandomized by client.
-

The following structure is repeated for each info-hash to scrape, but limited by -the MTU.

- ----- - - - - - - - - - - - - -
sizenamedescription
int8_t[20]info_hashThe info hash that is to be scraped.
-

Server replies with packet:

- ----- - - - - - - - - - - - - - - - - -
sizenamedescription
int32_tactionThe action, should in this case be -2 for scrape. -If 3 (for error) see errors.
int32_ttransaction_idMust match the sent transaction id.
-

The rest of the packet contains the following structures once for each info-hash -you asked in the scrape request.

- ----- - - - - - - - - - - - - - - - - - - - - -
sizenamedescription
int32_tcompleteThe current number of connected seeds.
int32_tdownloadedThe number of times this torrent has -been downloaded.
int32_tincompleteThe current number of connected -leechers.
-
-
-

errors

-

In case of a tracker error,

-

server replies packet:

- ----- - - - - - - - - - - - - - - - - - - - - -
sizenamedescription
int32_tactionThe action, in this case 3, for error. -See actions.
int32_ttransaction_idMust match the transaction_id sent -from the client.
int8_t[]error_stringThe rest of the packet is a string -describing the error.
-
-
-

actions

-

The action fields has the following encoding:

-
-
    -
  • connect = 0
  • -
  • announce = 1
  • -
  • scrape = 2
  • -
  • error = 3 (only in server replies)
  • -
-
-
-
-

extensions

-

The extensions field is a bitmask. The following -bits are assigned:

-
- -
-

If multiple bits are present in the extension field, the extension -bodies are appended to the packet in the order of least significant -bit first. For instance, if both bit 1 and 2 are set, the extension -represented by bit 1 comes first, followed by the extension represented -by bit 2.

-
-

authentication

-

The packet will have an authentication part -appended to it. It has the following format:

- ----- - - - - - - - - - - - - - - - - - - - - -
sizenamedescription
int8_tusername_lengthThe number of characters in the -username.
int8_t[]usernameThe username, the number of characters -as specified in the previous field.
uint8_t[8]passwd_hashsha1(packet + sha1(password)) -The packet in this case means the -entire packet except these 8 bytes -that are the password hash. These are -the 8 first bytes (most significant) -from the 20 bytes hash calculated.
-
-
-
-

request string

-

The request string extension is meant to allow torrent creators pass along -cookies back to the tracker. This can be useful for authenticating that a -torrent is allowed to be tracked by a tracker for instance. It could also -be used to authenticate users by generating torrents with unique tokens -in the tracker URL for each user. The extension body has the following format:

- ----- - - - - - - - - - - - - - - - - -
sizenamedescription
int8_trequest lengthThe number of bytes in the request -string.
int8_t[]request stringThe string that comes after the host- -name and port in the udp tracker URL. -Typically this starts with "/announce" -The bittorrent client is not expected -to append query string arguments for -stats reporting, like "uploaded" and -"downloaded" since this is already -reported in the udp tracker protocol. -However, the client is free to add -arguments as extensions.
-
-
-

credits

-

Protocol designed by Olaf van der Spek and extended by Arvid Norberg

-
- -
-
-
- -
- -
- - diff --git a/docs/utp.html b/docs/utp.html deleted file mode 100644 index dd29d0588..000000000 --- a/docs/utp.html +++ /dev/null @@ -1,370 +0,0 @@ - - - - - - -libtorrent manual - - - - - - - - -
-
- - - - -
-

libtorrent manual

- --- - - - - - -
Author:Arvid Norberg, arvid@libtorrent.org
Version:1.1.0
- -
-

uTP

-

uTP (uTorrent transport protocol) is a transport protocol which uses one-way -delay measurements for its congestion controller. This article is about uTP -in general and specifically about libtorrent's implementation of it.

-
-

rationale

-

One of the most common problems users are experiencing using bittorrent is -that their internet "stops working". This can be caused by a number of things, -for example:

-
    -
  1. a home router that crashes or slows down when its NAT pin-hole -table overflows, triggered by DHT or simply many TCP connections.
  2. -
  3. a home router that crashes or slows down by UDP traffic (caused by -the DHT)
  4. -
  5. a home DSL or cable modem having its send buffer filled up by outgoing -data, and the buffer fits seconds worth of bytes. This adds seconds -of delay on interactive traffic. For a web site that needs 10 round -trips to load this may mean 10s of seconds of delay to load compared -to without bittorrent. Skype or other delay sensitive applications -would be affected even more.
  6. -
-

This document will cover (3).

-

Typically this is solved by asking the user to enter a number of bytes -that the client is allowed to send per second (i.e. setting an upload -rate limit). The common recommendation is to set this limit to 80% of the -uplink's capacity. This is to leave some headroom for things like TCP -ACKs as well as the user's interactive use of the connection such as -browsing the web or checking email.

-

There are two major drawbacks with this technique:

-
    -
  1. The user needs to actively make this setting (very few protocols -require the user to provide this sort of information). This also -means the user needs to figure out what its up-link capacity is. -This is unfortunately a number that many ISPs are not advertizing -(because it's often much lower than the download capacity) which -might make it hard to find.
  2. -
  3. The 20% headroom is wasted most of the time. Whenever the user -is not using the internet connection for anything, those extra 20% -could have been used by bittorrent to upload, but they're already -allocated for interactive traffic. On top of that, 20% of the up-link -is often not enough to give a good and responsive browsing experience.
  4. -
-

The ideal bandwidth allocation would be to use 100% for bittorrent when -there is no interactive cross traffic, and 100% for interactive traffic -whenever there is any. This would not waste any bandwidth while the user -is idling, and it would make for a much better experience when the user -is using the internet connection for other things.

-

This is what uTP does.

-
-
-

TCP

-

The reason TCP will fill the send buffer, and cause the delay on all traffic, -is because its congestion control is only based on packet loss (and timeout).

-

Since the modem is buffering, packets won't get dropped until the entire queue -is full, and no more packets will fit. The packets will be dropped, TCP will -detect this within an RTT or so. When TCP notices a packet loss, it will slow -down its send rate and the queue will start to drain again. However, TCP will -immediately start to ramp up its send rate again until the buffer is full and -it detects packet loss again.

-

TCP is designed to fully utilize the link capacity, without causing congestion. -Whenever it sense congestion (through packet loss) it backs off. TCP is not -designed to keep delays low. When you get the first packet loss (assuming the -kind of queue described above, tail-queue) it is already too late. Your queue -is full and you have the maximum amount of delay your modem can provide.

-

TCP controls its send rate by limiting the number of bytes in-flight at any -given time. This limit is called congestion window (cwnd for short). During -steady state, the congestion window is constantly increasing linearly. Each -packet that is successfully transferred will increase cwnd.

-
-            cwnd
-send_rate = ----
-            RTT
-
-

Send rate is proportional to cwnd divided by RTT. A smaller cwnd will cause -the send rate to be lower and a larger cwnd will cause the send rate to be -higher.

-

Using a congestion window instead of controlling the rate directly is simple -because it also introduces an upper bound for memory usage for packets that -haven't been ACKed yet and needs to be kept around.

-

The behavior of TCP, where it bumps up against the ceiling, backs off and then -starts increasing again until it hits the ceiling again, forms a saw tooth shape. -If the modem wouldn't have any send buffer at all, a single TCP stream would -not be able to fully utilize the link because of this behavior, since it would -only fully utilize the link right before the packet loss and the back-off.

-
-
-

LEDBAT congestion controller

-

The congestion controller in uTP is called LEDBAT, which also is an IETF working -group attempting to standardize it. The congestion controller, on top of reacting -to packet loss the same way TCP does, also reacts to changes in delays.

-

For any uTP (or LEDBAT) implementation, there is a target delay. This is the -amount of delay that is acceptable, and is in fact targeted for the connection. -The target delay is defined to 25 ms in LEDBAT, uTorrent uses 100 ms and -libtorrent uses 75 ms. Whenever a delay measurement is lower than the target, -cwnd is increased proportional to (target_delay - delay). Whenever the measurement -is higher than the target, cwnd is decreased proportional to (delay - target_delay).

-

It can simply be expressed as:

-
-cwnd += gain * (target_delay - delay)
-
-cwnd_thumb.png -

Similarly to TCP, this is scaled so that the increase is evened out over one RTT.

-

The linear controller will adjust the cwnd more for delays that are far off the -target, and less for delays that are close to the target. This makes it converge -at the target delay. Although, due to noise there is almost always some amount of -oscillation. This oscillation is typically smaller than the saw tooth TCP forms.

-

The figure to the right shows how (TCP) cross traffic causese uTP to essentially -entirely stop sending anything. Its delay measurements are mostly well above the target -during this time. The cross traffic is only a single TCP stream in this test.

-

As soon as the cross traffic ceases, uTP will pick up its original send rate within -a second.

-

Since uTP constantly measures the delay, with every single packet, the reaction time -to cross traffic causing delays is a single RTT (typically a fraction of a second).

-
-
-

one way delays

-

uTP measures the delay imposed on packets being sent to the other end -of the connection. This measurement only includes buffering delay along -the link, not propagation delay (the speed of light times distance) nor -the routing delay (the time routers spend figuring out where to forward -the packet). It does this by always comparing all measurements to a -baseline measurement, to cancel out any fixed delay. By focusing on the -variable delay along a link, it will specifically detect points where -there might be congestion, since those points will have buffers.

-delays_thumb.png -

Delay on the return link is explicitly not included in the delay measurement. -This is because in a peer-to-peer application, the other end is likely to also -be connected via a modem, with the same send buffer restrictions as we assume -for the sending side. The other end having its send queue full is not an indication -of congestion on the path going the other way.

-

In order to measure one way delays for packets, we cannot rely on clocks being -synchronized, especially not at the microsecond level. Instead, the actual time -it takes for a packet to arrive at the destination is not measured, only the changes -in the transit time is measured.

-

Each packet that is sent includes a time stamp of the current time, in microseconds, -of the sending machine. The receiving machine calculates the difference between its -own timestamp and the one in the packet and sends this back in the ACK. This difference, -since it is in microseconds, will essentially be a random 32 bit number. However, -the difference will stay somewhat similar over time. Any changes in this difference -indicates that packets are either going through faster or slower.

-

In order to measure the one-way buffering delay, a base delay is established. The -base delay is the lowest ever seen value of the time stamp difference. Each delay -sample we receive back, is compared against the base delay and the delay is the -difference.

-

This is the delay that's fed into the congestion controller.

-

A histogram of typical delay measurements is shown to the right. This is from -a transfer between a cable modem connection and a DSL connection.

-

The details of the delay measurements are slightly more complicated since the -values needs to be able to wrap (cross the 2^32 boundry and start over at 0).

-
-
-

Path MTU discovery

-

MTU is short for Maximum Transfer Unit and describes the largest packet size that -can be sent over a link. Any datagrams which size exceeds this limit will either -be fragmented or dropped. A fragmented datagram means that the payload is split up -in multiple packets, each with its own individual packet header.

-

There are several reasons to avoid sending datagrams that get fragmented:

-
    -
  1. A fragmented datagram is more likely to be lost. If any fragment is lost, -the whole datagram is dropped.
  2. -
  3. Bandwidth is likely to be wasted. If the datagram size is not divisible -by the MTU the last packet will not contain as much payload as it could, and the -payload over protocol header ratio decreases.
  4. -
  5. It's expensive to fragment datagrams. Few routers are optimized to handle large -numbers of fragmented packets. Datagrams that have to fragment are likely to -be delayed significantly, and contribute to more CPU being used on routers. -Typically fragmentation (and other advanced IP features) are implemented in -software (slow) and not hardware (fast).
  6. -
-

The path MTU is the lowest MTU of any link along a path from two endpoints on the -internet. The MTU bottleneck isn't necessarily at one of the endpoints, but can -be anywhere in between.

-

The most common MTU is 1500 bytes, which is the largest packet size for ethernet -networks. Many home DSL connections, however, tunnel IP through PPPoE (Point to -Point Protocol over Ethernet. Yes, that is the old dial-up modem protocol). This -protocol uses up 8 bytes per packet for its own header.

-

If the user happens to be on an internet connection over a VPN, it will add another -layer, with its own packet headers.

-

In short; if you would pick the largest possible packet size on an ethernet network, -1472, and stick with it, you would be quite likely to generate fragments for a lot -of connections. The fragments that will be created will be very small and especially -inflate the overhead waste.

-

The other approach of picking a very conservative packet size, that would be very -unlikely to get fragmented has the following drawbacks:

-
    -
  1. People on good, normal, networks will be penalized with a small packet size. -Both in terms of router load but also bandwidth waste.
  2. -
  3. Software routers are typically not limited by the number of bytes they can route, -but the number of packets. Small packets means more of them, and more load on -software routers.
  4. -
-

The solution to the problem of finding the optimal packet size, is to dynamically -adjust the packet size and search for the largest size that can make it through -without being fragmented along the path.

-

To help do this, you can set the DF bit (Don't Fragment) in your Datagrams. This -asks routers that otherwise would fragment packets to instead drop them, and send -back an ICMP message reporting the MTU of the link the packet couldn't fit. With -this message, it's very simple to discover the path MTU. You simply mark your packets -not to be fragmented, and change your packet size whenever you receive the ICMP -packet-too-big message.

-

Unfortunately it's not quite that simple. There are a significant number of firewalls -in the wild blocking all ICMP messages. This means we can't rely on them, we also have -to guess that a packet was dropped because of its size. This is done by only marking -certain packets with DF, and if all other packets go through, except for the MTU probes, -we know that we need to lower our packet sizes.

-

If we set up bounds for the path MTU (say the minimum internet MTU, 576 and ethernet's 1500), -we can do a binary search for the MTU. This would let us find it in just a few round-trips.

-

On top of this, libtorrent has an optimization where it figures out which interface a -uTP connection will be sent over, and initialize the MTU ceiling to that interface's MTU. -This means that a VPN tunnel would advertize its MTU as lower, and the uTP connection would -immediately know to send smaller packets, no search required. It also has the side-effect -of being able to use much larger packet sizes for non-ethernet interfaces or ethernet links -with jumbo frames.

-
-
-

clock drift

-our_delay_base_thumb.png -

Clock drift is clocks progressing at different rates. It's different from clock -skew which means clocks set to different values (but which may progress at the same -rate).

-

Any clock drift between the two machines involved in a uTP transfer will result -in systematically inflated or deflated delay measurements.

-

This can be solved by letting the base delay be the lowest seen sample in the last -n minutes. This is a trade-off between seeing a single packet go straight through -the queue, with no delay, and the amount of clock drift one can assume on normal computers.

-

It turns out that it's fairly safe to assume that one of your packets will in fact go -straight through without any significant delay, once every 20 minutes or so. However, -the clock drift between normal computers can be as much as 17 ms in 10 minutes. 17 ms -is quite significant, especially if your target delay is 25 ms (as in the LEDBAT spec).

-

Clocks progresses at different rates depending on temperature. This means computers -running hot are likely to have a clock drift compared to computers running cool.

-

So, by updating the delay base periodically based on the lowest seen sample, you'll either -end up changing it upwards (artificaially making the delay samples appear small) without -the congestion or delay actually having changed, or you'll end up with a significant clock -drift and have artificially low samples because of that.

-

The solution to this problem is based on the fact that the clock drift is only a problem -for one of the sides of the connection. Only when your delay measurements keep increasing -is it a problem. If your delay measurements keep decreasing, the samples will simply push -down the delay base along with it. With this in mind, we can simply keep track of the -other end's delay measurements as well, applying the same logic to it. Whenever the -other end's base delay is adjusted downwards, we adjust our base delay upwards by the same -amount.

-

This will accurately keep the base delay updated with the clock drift and improve -the delay measurements. The figure on the right shows the absolute timestamp differences -along with the base delay. The slope of the measurements is caused by clock drift.

-

For more information on the clock drift compensation, see the slides from BitTorrent's -presentation at IPTPS10.

-
-
-

features

-

libtorrent's uTP implementation includes the following features:

-
    -
  • Path MTU discovery, including jumbo frames and detecting restricted -MTU tunnels. Binary search packet sizes to find the largest non-fragmented.
  • -
  • Selective ACK. The ability to acknowledge individual packets in the -event of packet loss
  • -
  • Fast resend. The first time a packet is lost, it's resent immediately. -Triggered by duplicate ACKs.
  • -
  • Nagle's algorithm. Minimize protocol overhead by attempting to lump -full packets of payload together before sending a packet.
  • -
  • Delayed ACKs to minimize protocol overhead.
  • -
  • Microsecond resolution timestamps.
  • -
  • Advertised receive window, to support download rate limiting.
  • -
  • Correct handling of wrapping sequence numbers.
  • -
  • Easy configuration of target-delay, gain-factor, timeouts, delayed-ack -and socket buffers.
  • -
-
-
- -
-
-
- -
- -
- -