Merge pull request #555 from arvidn/remove-html-1.1

remove html and other generated files from the repository
This commit is contained in:
Arvid Norberg 2016-03-19 07:51:07 -04:00
commit b624d3be58
22 changed files with 2 additions and 22174 deletions

View File

@ -47,6 +47,8 @@ before_install:
echo ''import site; site.addsitedir("/usr/local/lib/python2.7/site-packages")'' >> /Users/travis/Library/Python/2.7/lib/python/site-packages/homebrew.pth;
sudo easy_install Pygments;
sudo easy_install -U aafigure;
brew install --quiet graphviz;
brew install --quiet Homebrew/python/pillow;
fi'
# disable simulations on OSX for now. It hangs on travis

View File

@ -1,765 +0,0 @@
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.12: http://docutils.sourceforge.net/" />
<title>libtorrent manual</title>
<meta name="author" content="Arvid Norberg, arvid&#64;libtorrent.org" />
<meta name=viewport content="width=device-width, initial-scale=1">
<script type="text/javascript">
/* <![CDATA[ */
(function() {
var s = document.createElement('script'), t = document.getElementsByTagName('script')[0];
s.type = 'text/javascript';
s.async = true;
s.src = 'http://api.flattr.com/js/0.6/load.js?mode=auto';
t.parentNode.insertBefore(s, t);
})();
/* ]]> */
</script>
<link rel="stylesheet" href="style.css" type="text/css" />
<link rel="stylesheet" type="text/css" href="rst.css" />
<style type="text/css">
/* Hides from IE-mac \*/
* html pre { height: 1%; }
/* End hide from IE-mac */
</style>
</head>
<body>
<div class="document" id="libtorrent-manual">
<div id="container">
<table id="header">
<tr><td id="orange"></td>
<td id="logo">libtorrent</td></tr>
</table>
<div id="main">
<h1 class="title">libtorrent manual</h1>
<table class="docinfo" frame="void" rules="none">
<col class="docinfo-name" />
<col class="docinfo-content" />
<tbody valign="top">
<tr><th class="docinfo-name">Author:</th>
<td>Arvid Norberg, <a class="last reference external" href="mailto:arvid&#64;libtorrent.org">arvid&#64;libtorrent.org</a></td></tr>
<tr><th class="docinfo-name">Version:</th>
<td>1.1.0</td></tr>
</tbody>
</table>
<div class="contents topic" id="table-of-contents">
<p class="topic-title first">Table of contents</p>
<ul class="simple">
<li><a class="reference internal" href="#downloading-and-building" id="id8">downloading and building</a><ul>
<li><a class="reference internal" href="#building-from-svn" id="id9">building from svn</a></li>
<li><a class="reference internal" href="#building-with-bbv2" id="id10">building with BBv2</a></li>
<li><a class="reference internal" href="#building-with-autotools" id="id11">building with autotools</a></li>
<li><a class="reference internal" href="#building-with-other-build-systems" id="id12">building with other build systems</a></li>
<li><a class="reference internal" href="#build-configurations" id="id13">build configurations</a></li>
<li><a class="reference internal" href="#building-openssl-for-windows" id="id14">building openssl for windows</a></li>
</ul>
</li>
</ul>
</div>
<div class="section" id="downloading-and-building">
<h1>downloading and building</h1>
<p>To acquire the latest version of libtorrent, you'll have to grab it from SVN.
You'll find instructions on how to do this <a class="reference external" href="http://sourceforge.net/svn/?group_id=79942">here</a> (see subversion access).</p>
<p>The build systems supported &quot;out of the box&quot; in libtorrent are boost-build v2
(BBv2) and autotools (for unix-like systems). If you still can't build after
following these instructions, you can usually get help in the <tt class="docutils literal">#libtorrent</tt>
IRC channel on <tt class="docutils literal">irc.freenode.net</tt>.</p>
<div class="warning">
<p class="first admonition-title">Warning</p>
<p>A common mistake when building and linking against libtorrent is
to build with one set of configuration options (#defines) and
link against it using a different set of configuration options. Since
libtorrent has some code in header files, that code will not be
compatible with the built library if they see different configurations.</p>
<p>Always make sure that the same TORRENT_* macros are defined when you
link against libtorrent as when you build it.</p>
<p class="last">Boost-build supports propagating configuration options to dependencies.
When building using the makefiles, this is handled by setting the
configuration options in the pkg-config file. Always use pkg-config
when linking against libtorrent.</p>
</div>
<div class="section" id="building-from-svn">
<h2>building from svn</h2>
<p>To build libtorrent from svn you need to check out the libtorrent sources from
sourceforge. If you downloaded a release tarball, you can skip this section.</p>
<p>To check out libtorrent follow these <a class="reference external" href="http://sourceforge.net/svn/?group_id=79942">instructions</a>.</p>
</div>
<div class="section" id="building-with-bbv2">
<h2>building with BBv2</h2>
<p>The primary reason to use boost-build is that it will automatically build the
dependent boost libraries with the correct compiler settings, in order to
ensure that the build targets are link compatible (see <a class="reference external" href="http://boost.org/more/separate_compilation.html">boost guidelines</a>
for some details on this issue).</p>
<p>Since BBv2 will build the boost libraries for you, you need the full boost
source package. Having boost installed via some package system is usually not
enough (and even if it is enough, the necessary environment variables are
usually not set by the package installer).</p>
<p>If you want to build against an installed copy of boost, you can skip directly
to step 3 (assuming you also have boost build installed).</p>
<div class="section" id="step-1-download-boost">
<h3>Step 1: Download boost</h3>
<p>You'll find boost <a class="reference external" href="http://sourceforge.net/project/showfiles.php?group_id=7586&amp;package_id=8041&amp;release_id=619445">here</a>.</p>
<p>Extract the archive to some directory where you want it. For the sake of this
guide, let's assume you extract the package to <tt class="docutils literal"><span class="pre">c:\boost_1_55_0</span></tt> (I'm using
a windows path in this example since if you're on linux/unix you're more likely
to use the autotools). You'll need at least version 1.49 of the boost library
in order to build libtorrent.</p>
</div>
<div class="section" id="step-2-setup-bbv2">
<h3>Step 2: Setup BBv2</h3>
<p>First you need to build <tt class="docutils literal">bjam</tt>. You do this by opening a terminal (In
windows, run <tt class="docutils literal">cmd</tt>). Change directory to
<tt class="docutils literal"><span class="pre">c:\boost_1_55_0\tools\jam\src</span></tt>. Then run the script called
<tt class="docutils literal">build.bat</tt> or <tt class="docutils literal">build.sh</tt> on a unix system. This will build <tt class="docutils literal">bjam</tt> and
place it in a directory starting with <tt class="docutils literal">bin.</tt> and then have the name of your
platform. Copy the <tt class="docutils literal">bjam.exe</tt> (or <tt class="docutils literal">bjam</tt> on a unix system) to a place
that's in you shell's <tt class="docutils literal">PATH</tt>. On linux systems a place commonly used may be
<tt class="docutils literal">/usr/local/bin</tt> or on windows <tt class="docutils literal"><span class="pre">c:\windows</span></tt> (you can also add directories
to the search paths by modifying the environment variable called <tt class="docutils literal">PATH</tt>).</p>
<p>Now you have <tt class="docutils literal">bjam</tt> installed. <tt class="docutils literal">bjam</tt> can be considered an interpreter
that the boost-build system is implemented on. So boost-build uses <tt class="docutils literal">bjam</tt>.
So, to complete the installation you need to make two more things. You need to
set the environment variable <tt class="docutils literal">BOOST_BUILD_PATH</tt>. This is the path that tells
<tt class="docutils literal">bjam</tt> where it can find boost-build, your configuration file and all the
toolsets (descriptions used by boost-build to know how to use different
compilers on different platforms). Assuming the boost install path above, set
it to <tt class="docutils literal"><span class="pre">c:\boost_1_55_0\tools\build\v2</span></tt>.</p>
<p>To set an environment variable in windows, type for example:</p>
<pre class="literal-block">
set BOOST_BUILD_PATH=c:\boost_1_55_0\tools\build\v2
</pre>
<p>In a terminal window.</p>
<p>The last thing to do to complete the setup of BBv2 is to modify your
<tt class="docutils literal"><span class="pre">user-config.jam</span></tt> file. It is located in <tt class="docutils literal"><span class="pre">c:\boost_1_55_0\tools\build\v2</span></tt>.
Depending on your platform and which compiler you're using, you should add a
line for each compiler and compiler version you have installed on your system
that you want to be able to use with BBv2. For example, if you're using
Microsoft Visual Studio 12 (2013), just add a line:</p>
<pre class="literal-block">
using msvc : 12.0 ;
</pre>
<p>If you use GCC, add the line:</p>
<pre class="literal-block">
using gcc ;
</pre>
<p>If you have more than one version of GCC installed, you can add the
commandline used to invoke g++ after the version number, like this:</p>
<pre class="literal-block">
using gcc : 3.3 : g++-3.3 ;
using gcc : 4.0 : g++-4.0 ;
</pre>
<p>Another toolset worth mentioning is the <tt class="docutils literal">darwin</tt> toolset (For MacOS X).
From Tiger (10.4) MacOS X comes with both GCC 3.3 and GCC 4.0. Then you can
use the following toolsets:</p>
<pre class="literal-block">
using darwin : 3.3 : g++-3.3 ;
using darwin : 4.0 : g++-4.0 ;
</pre>
<p>Note that the spaces around the semi-colons and colons are important!</p>
<p>Also see the <a class="reference external" href="http://www.boost.org/doc/html/bbv2/installation.html">official installation instructions</a>.</p>
</div>
<div class="section" id="step-3-building-libtorrent">
<h3>Step 3: Building libtorrent</h3>
<p>When building libtorrent, the <tt class="docutils literal">Jamfile</tt> expects the environment variable
<tt class="docutils literal">BOOST_ROOT</tt> to be set to the boost installation directory. It uses this to
find the boost libraries it depends on, so they can be built and their headers
files found. So, set this to <tt class="docutils literal"><span class="pre">c:\boost_1_55_0</span></tt>. You only need this if you're
building against a source distribution of boost.</p>
<p>Then the only thing left is simply to invoke <tt class="docutils literal">bjam</tt>. If you want to specify
a specific toolset to use (compiler) you can just add that to the commandline.
For example:</p>
<pre class="literal-block">
bjam msvc-7.1
bjam gcc-3.3
bjam darwin-4.0
</pre>
<div class="note">
<p class="first admonition-title">Note</p>
<p class="last">If the environment variable <tt class="docutils literal">BOOST_ROOT</tt> is not set, the jamfile will
attempt to link against &quot;installed&quot; boost libraries. i.e. assume the headers
and libraries are available in default search paths.</p>
</div>
<p>To build different versions you can also just add the name of the build
variant. Some default build variants in BBv2 are <tt class="docutils literal">release</tt>, <tt class="docutils literal">debug</tt>,
<tt class="docutils literal">profile</tt>.</p>
<p>You can build libtorrent as a dll too, by typing <tt class="docutils literal">link=shared</tt>, or
<tt class="docutils literal">link=static</tt> to build a static library.</p>
<p>If you want to explicitly say how to link against the runtime library, you
can set the <tt class="docutils literal"><span class="pre">runtime-link</span></tt> feature on the commandline, either to <tt class="docutils literal">shared</tt>
or <tt class="docutils literal">static</tt>. Most operating systems will only allow linking shared against
the runtime, but on windows you can do both. Example:</p>
<pre class="literal-block">
bjam msvc-7.1 link=static runtime-link=static
</pre>
<div class="note">
<p class="first admonition-title">Note</p>
<p class="last">When building on windows, the path boost-build puts targets in may be too
long. If you get an error message like: &quot;The input line is long&quot;, try to
pass --abbreviate-paths on the bjam command line.</p>
</div>
<div class="warning">
<p class="first admonition-title">Warning</p>
<p class="last">If you link statically to the runtime library, you cannot build libtorrent
as a shared library (DLL), since you will get separate heaps in the library
and in the client application. It will result in crashes and possibly link
errors.</p>
</div>
<div class="note">
<p class="first admonition-title">Note</p>
<p class="last">With boost-build V2 (Milestone 11), the darwin toolset uses the <tt class="docutils literal"><span class="pre">-s</span></tt> linker
option to strip debug symbols. This option is buggy in Apple's GCC, and
will make the executable crash on startup. On Mac OS X, instead build
your release executables with the <tt class="docutils literal"><span class="pre">debug-symbols=on</span></tt> option, and
later strip your executable with <tt class="docutils literal">strip</tt>.</p>
</div>
<div class="note">
<p class="first admonition-title">Note</p>
<p class="last">Some linux systems requires linking against <tt class="docutils literal">librt</tt> in order to access
the POSIX clock functions. If you get an error complaining about a missing
symbol <tt class="docutils literal">clock_gettime</tt>, you have to give <tt class="docutils literal"><span class="pre">need-librt=yes</span></tt> on the
bjam command line. This will make libtorrent link against <tt class="docutils literal">librt</tt>.</p>
</div>
<div class="note">
<p class="first admonition-title">Note</p>
<p class="last">When building on Solaris, you might have to specify <tt class="docutils literal"><span class="pre">stdlib=sun-stlport</span></tt>
on the bjam command line.</p>
</div>
<p>The build targets are put in a directory called bin, and under it they are
sorted in directories depending on the toolset and build variant used.</p>
<p>To build the examples, just change directory to the examples directory and
invoke <tt class="docutils literal">bjam</tt> from there. To build and run the tests, go to the test
directory and run <tt class="docutils literal">bjam</tt>.</p>
<p>Note that if you're building on windows using the <tt class="docutils literal">msvc</tt> toolset, you cannot run it
from a cygwin terminal, you'll have to run it from a <tt class="docutils literal">cmd</tt> terminal. The same goes for
cygwin, if you're building with gcc in cygwin you'll have to run it from a cygwin terminal.
Also, make sure the paths are correct in the different environments. In cygwin, the paths
(<tt class="docutils literal">BOOST_BUILD_PATH</tt> and <tt class="docutils literal">BOOST_ROOT</tt>) should be in the typical unix-format (e.g.
<tt class="docutils literal">/cygdrive/c/boost_1_55_0</tt>). In the windows environment, they should have the typical
windows format (<tt class="docutils literal"><span class="pre">c:/boost_1_55_0</span></tt>).</p>
<div class="note">
<p class="first admonition-title">Note</p>
<p class="last">In Jamfiles, spaces are separators. It's typically easiest to avoid spaces
in path names. If you want spaces in your paths, make sure to quote them
with double quotes (&quot;).</p>
</div>
<p>The <tt class="docutils literal">Jamfile</tt> will define <tt class="docutils literal">NDEBUG</tt> when it's building a release build.
For more build configuration flags see <a class="reference internal" href="#build-configurations">Build configurations</a>.</p>
<p>Build features:</p>
<table border="1" class="docutils">
<colgroup>
<col width="33%" />
<col width="67%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">boost build feature</th>
<th class="head">values</th>
</tr>
</thead>
<tbody valign="top">
<tr><td><tt class="docutils literal"><span class="pre">boost-link</span></tt></td>
<td><ul class="first last simple">
<li><tt class="docutils literal">static</tt> - links statically against the boost
libraries.</li>
<li><tt class="docutils literal">shared</tt> - links dynamically against the boost
libraries.</li>
</ul>
</td>
</tr>
<tr><td><tt class="docutils literal">logging</tt></td>
<td><ul class="first last simple">
<li><tt class="docutils literal">off</tt> - logging alerts disabled.</li>
<li><tt class="docutils literal">on</tt> - default. logging alerts available,
still need to be enabled by the alert mask. The
reason to disable logging is to keep the binary
size down.</li>
</ul>
</td>
</tr>
<tr><td><tt class="docutils literal">dht</tt></td>
<td><ul class="first last simple">
<li><tt class="docutils literal">on</tt> - build with support for tracker less
torrents and DHT support.</li>
<li><tt class="docutils literal">logging</tt> - build with DHT support and verbose
logging of the DHT protocol traffic.</li>
<li><tt class="docutils literal">off</tt> - build without DHT support.</li>
</ul>
</td>
</tr>
<tr><td><tt class="docutils literal">asserts</tt></td>
<td><ul class="first last simple">
<li><tt class="docutils literal">auto</tt> - asserts are on if in debug mode</li>
<li><tt class="docutils literal">on</tt> - asserts are on, even in release mode</li>
<li><tt class="docutils literal">off</tt> - asserts are disabled</li>
<li><tt class="docutils literal">production</tt> - assertion failures are logged
to <tt class="docutils literal">asserts.log</tt> in the current working
directory, but won't abort the process.
The file they are logged to can be customized
by setting the global pointer <tt class="docutils literal">extern char
const* libtorrent_assert_log</tt> to a different
filename.</li>
<li><tt class="docutils literal">system</tt> use the libc assert macro</li>
</ul>
</td>
</tr>
<tr><td><tt class="docutils literal">encryption</tt></td>
<td><ul class="first last simple">
<li><tt class="docutils literal">on</tt> - encrypted bittorrent connections
enabled. (Message Stream encryption).</li>
<li><tt class="docutils literal">off</tt> - turns off support for encrypted
connections. The shipped public domain SHA-1
implementation is used.</li>
</ul>
</td>
</tr>
<tr><td><tt class="docutils literal"><span class="pre">mutable-torrents</span></tt></td>
<td><ul class="first last simple">
<li><tt class="docutils literal">on</tt> - mutable torrents are supported
(<a class="reference external" href="http://www.bittorrent.org/beps/bep_0038.html">BEP 38</a>) (default).</li>
<li><tt class="docutils literal">off</tt> - mutable torrents are not supported.</li>
</ul>
</td>
</tr>
<tr><td><tt class="docutils literal">crypto</tt></td>
<td><ul class="first last simple">
<li><tt class="docutils literal"><span class="pre">built-in</span></tt> - (default) uses built-in SHA-1
implementation.</li>
<li><tt class="docutils literal">openssl</tt> - links against openssl and
libcrypto to use for SHA-1 hashing.</li>
<li><tt class="docutils literal">gcrypt</tt> - links against libgcrypt to use for
SHA-1 hashing.</li>
</ul>
</td>
</tr>
<tr><td><tt class="docutils literal">allocator</tt></td>
<td><ul class="first last simple">
<li><tt class="docutils literal">pool</tt> - default, uses pool allocators for
send buffers.</li>
<li><tt class="docutils literal">system</tt> - uses <tt class="docutils literal">malloc()</tt> and <tt class="docutils literal">free()</tt>
instead. Might be useful to debug buffer issues
with tools like electric fence or libgmalloc.</li>
<li><tt class="docutils literal">debug</tt> - instruments buffer usage to catch
bugs in libtorrent.</li>
</ul>
</td>
</tr>
<tr><td><tt class="docutils literal">link</tt></td>
<td><ul class="first last simple">
<li><tt class="docutils literal">static</tt> - builds libtorrent as a static
library (.a / .lib)</li>
<li><tt class="docutils literal">shared</tt> - builds libtorrent as a shared
library (.so / .dll).</li>
</ul>
</td>
</tr>
<tr><td><tt class="docutils literal"><span class="pre">runtime-link</span></tt></td>
<td><ul class="first last simple">
<li><tt class="docutils literal">static</tt> - links statically against the
run-time library (if available on your
platform).</li>
<li><tt class="docutils literal">shared</tt> - link dynamically against the
run-time library (default).</li>
</ul>
</td>
</tr>
<tr><td><tt class="docutils literal">variant</tt></td>
<td><ul class="first last simple">
<li><tt class="docutils literal">debug</tt> - builds libtorrent with debug
information and invariant checks.</li>
<li><tt class="docutils literal">release</tt> - builds libtorrent in release mode
without invariant checks and with optimization.</li>
<li><tt class="docutils literal">profile</tt> - builds libtorrent with profile
information.</li>
</ul>
</td>
</tr>
<tr><td><tt class="docutils literal"><span class="pre">character-set</span></tt></td>
<td><p class="first">This setting will only have an affect on windows.
Other platforms are expected to support UTF-8.</p>
<ul class="last simple">
<li><tt class="docutils literal">unicode</tt> - The unicode version of the win32
API is used. This is default.</li>
<li><tt class="docutils literal">ansi</tt> - The ansi version of the win32 API is
used.</li>
</ul>
</td>
</tr>
<tr><td><tt class="docutils literal"><span class="pre">invariant-checks</span></tt></td>
<td><p class="first">This setting only affects debug builds (where
<tt class="docutils literal">NDEBUG</tt> is not defined). It defaults to <tt class="docutils literal">on</tt>.</p>
<ul class="last simple">
<li><tt class="docutils literal">on</tt> - internal invariant checks are enabled.</li>
<li><tt class="docutils literal">off</tt> - internal invariant checks are
disabled. The resulting executable will run
faster than a regular debug build.</li>
<li><tt class="docutils literal">full</tt> - turns on extra expensive invariant
checks.</li>
</ul>
</td>
</tr>
<tr><td><tt class="docutils literal"><span class="pre">debug-symbols</span></tt></td>
<td><ul class="first last simple">
<li><tt class="docutils literal">on</tt> - default for debug builds. This setting
is useful for building release builds with
symbols.</li>
<li><tt class="docutils literal">off</tt> - default for release builds.</li>
</ul>
</td>
</tr>
<tr><td><tt class="docutils literal"><span class="pre">deprecated-functions</span></tt></td>
<td><ul class="first last simple">
<li><tt class="docutils literal">on</tt> - default. Includes deprecated functions
of the API (might produce warnings during build
when deprecated functions are used).</li>
<li><tt class="docutils literal">off</tt> - excludes deprecated functions from the
API. Generates build errors when deprecated
functions are used.</li>
</ul>
</td>
</tr>
<tr><td><tt class="docutils literal">iconv</tt></td>
<td><ul class="first last simple">
<li><tt class="docutils literal">auto</tt> - use iconv for string conversions for
linux and mingw and other posix platforms.</li>
<li><tt class="docutils literal">on</tt> - force use of iconv</li>
<li><tt class="docutils literal">off</tt> - force not using iconv (disables locale
awareness except on windows).</li>
</ul>
</td>
</tr>
<tr><td><tt class="docutils literal">i2p</tt></td>
<td><ul class="first last simple">
<li><tt class="docutils literal">on</tt> - build with I2P support</li>
<li><tt class="docutils literal">off</tt> - build without I2P support</li>
</ul>
</td>
</tr>
<tr><td><tt class="docutils literal"><span class="pre">profile-calls</span></tt></td>
<td><ul class="first last simple">
<li><tt class="docutils literal">off</tt> - default. No additional call profiling.</li>
<li><tt class="docutils literal">on</tt> - Enable logging of stack traces of
calls into libtorrent that are blocking. On
session shutdown, a file <tt class="docutils literal">blocking_calls.txt</tt>
is written with stack traces of blocking calls
ordered by the number of them.</li>
</ul>
</td>
</tr>
</tbody>
</table>
<p>The <tt class="docutils literal">variant</tt> feature is <em>implicit</em>, which means you don't need to specify
the name of the feature, just the value.</p>
<p>The logs created when building vlog or log mode are put in a directory called
<tt class="docutils literal">libtorrent_logs</tt> in the current working directory.</p>
<p>When building the example client on windows, you need to build with
<tt class="docutils literal">link=static</tt> otherwise you may get unresolved external symbols for some
boost.program-options symbols.</p>
<p>For more information, see the <a class="reference external" href="http://www.boost.org/tools/build/v2/index.html">Boost build v2 documentation</a>, or more
specifically <a class="reference external" href="http://www.boost.org/doc/html/bbv2/reference.html#bbv2.advanced.builtins.features">the section on builtin features</a>.</p>
</div>
</div>
<div class="section" id="building-with-autotools">
<h2>building with autotools</h2>
<p>First of all, you need to install <tt class="docutils literal">automake</tt> and <tt class="docutils literal">autoconf</tt>. Many
unix/linux systems comes with these preinstalled.</p>
<p>The prerequisites for building libtorrent are boost.thread, boost.date_time
and boost.filesystem. Those are the <em>compiled</em> boost libraries needed. The
headers-only libraries needed include (but is not necessarily limited to)
boost.bind, boost.ref, boost.multi_index, boost.optional, boost.lexical_cast,
boost.integer, boost.iterator, boost.tuple, boost.array, boost.function,
boost.smart_ptr, boost.preprocessor, boost.static_assert.</p>
<p>If you want to build the <tt class="docutils literal">client_test</tt> example, you'll also need boost.regex
and boost.program_options.</p>
<div class="section" id="step-1-generating-the-build-system">
<h3>Step 1: Generating the build system</h3>
<p>No build system is present if libtorrent is checked out from CVS - it
needs to be generated first. If you're building from a released tarball,
you may skip directly to <a class="reference internal" href="#step-2-running-configure">Step 2: Running configure</a>.</p>
<p>Execute the following command to generate the build system:</p>
<pre class="literal-block">
./autotool.sh
</pre>
</div>
<div class="section" id="step-2-running-configure">
<h3>Step 2: Running configure</h3>
<p>In your shell, change directory to the libtorrent directory and run
<tt class="docutils literal">./configure</tt>. This will look for libraries and C++ features that libtorrent
is dependent on. If something is missing or can't be found it will print an
error telling you what failed.</p>
<p>The most likely problem you may encounter is that the configure script won't
find the boost libraries. Make sure you have boost installed on your system.
The easiest way to install boost is usually to use the preferred package
system on your platform. Usually libraries and headers are installed in
standard directories where the compiler will find them, but sometimes that
may not be the case. For example when installing boost on darwin using
darwinports (the package system based on BSD ports) all libraries are
installed to <tt class="docutils literal">/opt/local/lib</tt> and headers are installed to
<tt class="docutils literal">/opt/local/include</tt>. By default the compiler will not look in these
directories. You have to set the enviornment variables <tt class="docutils literal">LDFLAGS</tt> and
<tt class="docutils literal">CXXFLAGS</tt> in order to make the compiler find those libs. In this example
you'd set them like this:</p>
<pre class="literal-block">
export LDFLAGS=-L/opt/local/lib
export CXXFLAGS=-I/opt/local/include
</pre>
<p>It was observed on FreeBSD (release 6.0) that one needs to add '-lpthread' to
LDFLAGS, as Boost::Thread detection will fail without it, even if
Boost::Thread is installed.</p>
<p>If you need to set these variables, it may be a good idea to add those lines
to your <tt class="docutils literal"><span class="pre">~/.profile</span></tt> or <tt class="docutils literal"><span class="pre">~/.tcshrc</span></tt> depending on your shell.</p>
<p>If the boost libraries are named with a suffix on your platform, you may use
the <tt class="docutils literal"><span class="pre">--with-boost-thread=</span></tt> option to specify the suffix used for the thread
library in this case. For more information about these options, run:</p>
<pre class="literal-block">
./configure --help
</pre>
<p>On gentoo the boost libraries that are built with multi-threading support have
the suffix <tt class="docutils literal">mt</tt>.</p>
<p>You know that the boost libraries were found if you see the following output
from the configure script:</p>
<pre class="literal-block">
checking whether the Boost::DateTime library is available... yes
checking for main in -lboost_date_time... yes
checking whether the Boost::Filesystem library is available... yes
checking for main in -lboost_filesystem... yes
checking whether the Boost::Thread library is available... yes
checking for main in -lboost_thread... yes
</pre>
<p>Another possible source of problems may be if the path to your libtorrent
directory contains spaces. Make sure you either rename the directories with
spaces in their names to remove the spaces or move the libtorrent directory.</p>
</div>
<div class="section" id="creating-a-debug-build">
<h3>Creating a debug build</h3>
<p>To tell configure to build a debug version (with debug info, asserts
and invariant checks enabled), you have to run the configure script
with the following option:</p>
<pre class="literal-block">
./configure --enable-debug=yes
</pre>
</div>
<div class="section" id="creating-a-release-build">
<h3>Creating a release build</h3>
<p>To tell the configure to build a release version (without debug info,
asserts and invariant checks), you have to run the configure script
with the following option:</p>
<pre class="literal-block">
./configure --enable-debug=no
</pre>
<p>The above option make use of -DNDEBUG, which is used throughout libtorrent.</p>
</div>
<div class="section" id="id7">
<h3>Step 3: Building libtorrent</h3>
<p>Once the configure script is run successfully, you just type <tt class="docutils literal">make</tt> and
libtorrent, the examples and the tests will be built.</p>
<p>When libtorrent is built it may be a good idea to run the tests, you do this
by running <tt class="docutils literal">make check</tt>.</p>
<p>If you want to build a release version (without debug info, asserts and
invariant checks), you have to rerun the configure script and rebuild, like this:</p>
<pre class="literal-block">
./configure --disable-debug
make clean
make
</pre>
</div>
</div>
<div class="section" id="building-with-other-build-systems">
<h2>building with other build systems</h2>
<p>If you're building in MS Visual Studio, you may have to set the compiler
options &quot;force conformance in for loop scope&quot;, &quot;treat wchar_t as built-in
type&quot; and &quot;Enable Run-Time Type Info&quot; to Yes.</p>
</div>
<div class="section" id="build-configurations">
<h2>build configurations</h2>
<p>By default libtorrent is built In debug mode, and will have pretty expensive
invariant checks and asserts built into it. If you want to disable such checks
(you want to do that in a release build) you can see the table below for which
defines you can use to control the build.</p>
<table border="1" class="docutils">
<colgroup>
<col width="45%" />
<col width="55%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">macro</th>
<th class="head">description</th>
</tr>
</thead>
<tbody valign="top">
<tr><td><tt class="docutils literal">NDEBUG</tt></td>
<td>If you define this macro, all asserts,
invariant checks and general debug code will be
removed. Since there is quite a lot of code in
in header files in libtorrent, it may be
important to define the symbol consistently
across compilation units, including the clients
files. Potential problems is different
compilation units having different views of
structs and class layouts and sizes.</td>
</tr>
<tr><td><tt class="docutils literal">TORRENT_DISABLE_LOGGING</tt></td>
<td>This macro will disable support for logging
alerts, like log_alert, torrent_log_alert and
peer_log_alert. With this build flag, you
cannot enable those alerts.</td>
</tr>
<tr><td><tt class="docutils literal">TORRENT_STORAGE_DEBUG</tt></td>
<td>This will enable extra expensive invariant
checks in the storage, including logging of
piece sorting.</td>
</tr>
<tr><td><tt class="docutils literal">TORRENT_DISK_STATS</tt></td>
<td>This will create a log of all disk activity
which later can parsed and graphed using
<tt class="docutils literal">parse_disk_log.py</tt>.</td>
</tr>
<tr><td><tt class="docutils literal">UNICODE</tt></td>
<td>If building on windows this will make sure the
UTF-8 strings in pathnames are converted into
UTF-16 before they are passed to the file
operations.</td>
</tr>
<tr><td><tt class="docutils literal">TORRENT_DISABLE_POOL_ALLOCATOR</tt></td>
<td>Disables use of <tt class="docutils literal"><span class="pre">boost::pool&lt;&gt;</span></tt>.</td>
</tr>
<tr><td><tt class="docutils literal">TORRENT_DISABLE_MUTABLE_TORRENTS</tt></td>
<td>Disables mutable torrent support (<a class="reference external" href="http://www.bittorrent.org/beps/bep_0038.html">BEP 38</a>)</td>
</tr>
<tr><td><tt class="docutils literal">TORRENT_LINKING_SHARED</tt></td>
<td>If this is defined when including the
libtorrent headers, the classes and functions
will be tagged with <tt class="docutils literal">__declspec(dllimport)</tt>
on msvc and default visibility on GCC 4 and
later. Set this in your project if you're
linking against libtorrent as a shared library.
(This is set by the Jamfile when
<tt class="docutils literal">link=shared</tt> is set).</td>
</tr>
<tr><td><tt class="docutils literal">TORRENT_BUILDING_SHARED</tt></td>
<td>If this is defined, the functions and classes
in libtorrent are marked with
<tt class="docutils literal">__declspec(dllexport)</tt> on msvc, or with
default visibility on GCC 4 and later. This
should be defined when building libtorrent as
a shared library. (This is set by the Jamfile
when <tt class="docutils literal">link=shared</tt> is set).</td>
</tr>
<tr><td><tt class="docutils literal">TORRENT_DISABLE_DHT</tt></td>
<td>If this is defined, the support for trackerless
torrents will be disabled.</td>
</tr>
<tr><td><tt class="docutils literal">TORRENT_DISABLE_ENCRYPTION</tt></td>
<td>This will disable any encryption support and
the dependencies of a crypto library.
Encryption support is the peer connection
encrypted supported by clients such as
uTorrent, Azureus and KTorrent.
If this is not defined, either
<tt class="docutils literal">TORRENT_USE_OPENSSL</tt> or
<tt class="docutils literal">TORRENT_USE_GCRYPT</tt> must be defined.</td>
</tr>
<tr><td><tt class="docutils literal">TORRENT_DISABLE_EXTENSIONS</tt></td>
<td>When defined, libtorrent plugin support is
disabled along with support for the extension
handskake (BEP 10).</td>
</tr>
<tr><td><tt class="docutils literal">_UNICODE</tt></td>
<td>On windows, this will cause the file IO
use wide character API, to properly support
non-ansi characters.</td>
</tr>
<tr><td><tt class="docutils literal">TORRENT_DISABLE_RESOLVE_COUNTRIES</tt></td>
<td>Defining this will disable the ability to
resolve countries of origin for peer IPs.</td>
</tr>
<tr><td><tt class="docutils literal">TORRENT_DISABLE_INVARIANT_CHECKS</tt></td>
<td>This will disable internal invariant checks in
libtorrent. The invariant checks can sometime
be quite expensive, they typically don't scale
very well. This option can be used to still
build in debug mode, with asserts enabled, but
make the resulting executable faster.</td>
</tr>
<tr><td><tt class="docutils literal">TORRENT_EXPENSIVE_INVARIANT_CHECKS</tt></td>
<td>This will enable extra expensive invariant
checks. Useful for finding particular bugs
or for running before releases.</td>
</tr>
<tr><td><tt class="docutils literal">TORRENT_NO_DEPRECATE</tt></td>
<td>This will exclude all deprecated functions from
the header files and cpp files.</td>
</tr>
<tr><td><tt class="docutils literal">TORRENT_PRODUCTION_ASSERTS</tt></td>
<td>Define to either 0 or 1. Enables assert logging
in release builds.</td>
</tr>
<tr><td><tt class="docutils literal">TORRENT_USE_ASSERTS</tt></td>
<td>Define as 0 to disable asserts unconditionally.</td>
</tr>
<tr><td><tt class="docutils literal">TORRENT_USE_SYSTEM_ASSERTS</tt></td>
<td>Uses the libc assert macro rather then the
custom one.</td>
</tr>
</tbody>
</table>
<p>If you experience that libtorrent uses unreasonable amounts of cpu, it will
definitely help to define <tt class="docutils literal">NDEBUG</tt>, since it will remove the invariant checks
within the library.</p>
</div>
<div class="section" id="building-openssl-for-windows">
<h2>building openssl for windows</h2>
<p>To build openssl for windows with Visual Studio 7.1 (2003) execute the following commands
in a command shell:</p>
<pre class="literal-block">
perl Configure VC-WIN32 --prefix=&quot;c:/openssl
call ms\do_nasm
call &quot;C:\Program Files\Microsoft Visual Studio .NET 2003\vc7\bin\vcvars32.bat&quot;
nmake -f ms\nt.mak
copy inc32\openssl &quot;C:\Program Files\Microsoft Visual Studio .NET 2003\vc7\include\&quot;
copy out32\libeay32.lib &quot;C:\Program Files\Microsoft Visual Studio .NET 2003\vc7\lib&quot;
copy out32\ssleay32.lib &quot;C:\Program Files\Microsoft Visual Studio .NET 2003\vc7\lib&quot;
</pre>
<p>This will also install the headers and library files in the visual studio directories to
be picked up by libtorrent.</p>
</div>
</div>
</div>
</div>
<div id="gradient"></div>
<div id="footer">
<table>
<tr>
<td><a href="index.html">home</a></td>
<td><a href="http://blog.libtorrent.org">blog</a></td>
<td><a href="utp.html">uTP</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/libtorrent/">download</a></td>
<td><a href="reference.html">documentation</a></td>
<td><a href="dht_store.html">DHT put extension</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/py-libtorrent/">python bindings</a></td>
<td><a href="features.html">features</a></td>
<td><a href="dht_sec.html">DHT security extension</a></td>
</tr>
<tr>
<td><a href="http://dir.gmane.org/gmane.network.bit-torrent.libtorrent">mailing list archive</a></td>
<td><a href="contributing.html">contributing</a></td>
<td><a href="streaming.html">streaming</a></td>
</tr>
<tr>
<td><a href="http://code.google.com/p/libtorrent/issues/entry">report a bug</a></td>
<td><a href="building.html">building</a></td>
<td><a href="bittorrent.pdf">bittorrent slides</a></td>
</tr>
</table>
</div>
<div id="filler"></div>
</div>
</body>
</html>

View File

@ -1,112 +0,0 @@
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.12: http://docutils.sourceforge.net/" />
<title>client_test example program</title>
<meta name=viewport content="width=device-width, initial-scale=1">
<script type="text/javascript">
/* <![CDATA[ */
(function() {
var s = document.createElement('script'), t = document.getElementsByTagName('script')[0];
s.type = 'text/javascript';
s.async = true;
s.src = 'http://api.flattr.com/js/0.6/load.js?mode=auto';
t.parentNode.insertBefore(s, t);
})();
/* ]]> */
</script>
<link rel="stylesheet" href="style.css" type="text/css" />
<link rel="stylesheet" type="text/css" href="rst.css" />
<style type="text/css">
/* Hides from IE-mac \*/
* html pre { height: 1%; }
/* End hide from IE-mac */
</style>
</head>
<body>
<div class="document" id="client-test-example-program">
<div id="container">
<table id="header">
<tr><td id="orange"></td>
<td id="logo">libtorrent</td></tr>
</table>
<div id="main">
<h1 class="title">client_test example program</h1>
<p>Client test is a, more or less, complete bittorrent client. It lacks most
settings and you can't start or stop torrents once you've started it. All
the settings are hardcoded. The commandline arguments are:</p>
<pre class="literal-block">
client_test &lt;filename1.torrent&gt; &lt;filename2.torrent&gt; ...
</pre>
<p>You can start any number of torrent downloads/seeds via the commandline.
If one argument starts with <tt class="docutils literal"><span class="pre">http://</span></tt> it is interpreted as a tracker
announce url, and it expects an info-hash as the next argument. The info-hash
has to be hex-encoded. For example: <tt class="docutils literal">2410d4554d5ed856d69f426c38791673c59f4418</tt>.
If you pass an announce url and info-hash, a torrent-less download is started.
It relies on that at least one peer on the tracker is running a libtorrent based
client and has the metadata (.torrent file). The metadata extension in
libtorrent will then download it from that peer (or from those peers if more
than one).</p>
<p>While running, the <tt class="docutils literal">client_test</tt> sample will look something like this:</p>
<img alt="client_test.png" src="client_test.png" />
<p>The commands available in the client are:</p>
<ul class="simple">
<li><tt class="docutils literal">q</tt> quits the client (there will be a delay while the client waits
for tracker responses)</li>
<li><tt class="docutils literal">l</tt> toggle log. Will display the log at the bottom, informing about
tracker and peer events.</li>
<li><tt class="docutils literal">i</tt> toggles torrent info. Will show the peer list for each torrent.</li>
<li><tt class="docutils literal">d</tt> toggle download info. Will show the block list for each torrent,
showing downloaded and requested blocks.</li>
<li><tt class="docutils literal">p</tt> pause all torrents.</li>
<li><tt class="docutils literal">u</tt> unpause all torrents.</li>
<li><tt class="docutils literal">r</tt> force tracker reannounce for all torrents.</li>
<li><tt class="docutils literal">f</tt> toggle show file progress. Displays a list of all files and the
download progress for each file.</li>
</ul>
<p>The list at the bottom (shown if you press <tt class="docutils literal">d</tt>) shows which blocks has
been requested from which peer. The green background means that it has been
downloaded. It shows that fast peers will prefer to request whole pieces
instead of dowloading parts of pieces. It may make it easier to determine
which peer that sent the corrupt data if a piece fails the hash test.</p>
</div>
</div>
<div id="gradient"></div>
<div id="footer">
<table>
<tr>
<td><a href="index.html">home</a></td>
<td><a href="http://blog.libtorrent.org">blog</a></td>
<td><a href="utp.html">uTP</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/libtorrent/">download</a></td>
<td><a href="reference.html">documentation</a></td>
<td><a href="dht_store.html">DHT put extension</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/py-libtorrent/">python bindings</a></td>
<td><a href="features.html">features</a></td>
<td><a href="dht_sec.html">DHT security extension</a></td>
</tr>
<tr>
<td><a href="http://dir.gmane.org/gmane.network.bit-torrent.libtorrent">mailing list archive</a></td>
<td><a href="contributing.html">contributing</a></td>
<td><a href="streaming.html">streaming</a></td>
</tr>
<tr>
<td><a href="http://code.google.com/p/libtorrent/issues/entry">report a bug</a></td>
<td><a href="building.html">building</a></td>
<td><a href="bittorrent.pdf">bittorrent slides</a></td>
</tr>
</table>
</div>
<div id="filler"></div>
</div>
</body>
</html>

View File

@ -1,136 +0,0 @@
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.12: http://docutils.sourceforge.net/" />
<title>libtorrent manual</title>
<meta name="author" content="Arvid Norberg, arvid&#64;libtorrent.org" />
<meta name=viewport content="width=device-width, initial-scale=1">
<script type="text/javascript">
/* <![CDATA[ */
(function() {
var s = document.createElement('script'), t = document.getElementsByTagName('script')[0];
s.type = 'text/javascript';
s.async = true;
s.src = 'http://api.flattr.com/js/0.6/load.js?mode=auto';
t.parentNode.insertBefore(s, t);
})();
/* ]]> */
</script>
<link rel="stylesheet" href="style.css" type="text/css" />
<link rel="stylesheet" type="text/css" href="rst.css" />
<style type="text/css">
/* Hides from IE-mac \*/
* html pre { height: 1%; }
/* End hide from IE-mac */
</style>
</head>
<body>
<div class="document" id="libtorrent-manual">
<div id="container">
<table id="header">
<tr><td id="orange"></td>
<td id="logo">libtorrent</td></tr>
</table>
<div id="main">
<h1 class="title">libtorrent manual</h1>
<table class="docinfo" frame="void" rules="none">
<col class="docinfo-name" />
<col class="docinfo-content" />
<tbody valign="top">
<tr><th class="docinfo-name">Author:</th>
<td>Arvid Norberg, <a class="last reference external" href="mailto:arvid&#64;libtorrent.org">arvid&#64;libtorrent.org</a></td></tr>
<tr><th class="docinfo-name">Version:</th>
<td>1.1.0</td></tr>
</tbody>
</table>
<div class="contents topic" id="table-of-contents">
<p class="topic-title first">Table of contents</p>
<ul class="simple">
<li><a class="reference internal" href="#contributing-to-libtorrent" id="id1">contributing to libtorrent</a></li>
</ul>
</div>
<div class="section" id="contributing-to-libtorrent">
<h1>contributing to libtorrent</h1>
<p>There are several ways to contribute to libtorrent at various levels. Any help is
much appreciated. If you're interested in something libtorrent related that's not
enumerated on this page, please contact <a class="reference external" href="mailto:arvid&#64;libtorrent.org">arvid&#64;libtorrent.org</a> or the <a class="reference external" href="http://lists.sourceforge.net/lists/listinfo/libtorrent-discuss">mailing list</a>.</p>
<ol class="arabic">
<li><dl class="first docutils">
<dt>Testing</dt>
<dd><p class="first">This is not just limited to finding bugs and ways to reproduce crashes, but also
sub-optimal behavior is certain scenarios and finding ways to reproduce those. Please
report any issue to the bug tracker at <a class="reference external" href="https://github.com/arvidn/libtorrent/issues">github</a>.</p>
<p class="last">New features that need testing are streaming (<tt class="docutils literal">set_piece_deadline()</tt>), the different
choking algorithms (especially the new BitTyrant choker), the disk cache options (such
as <tt class="docutils literal">explicit_cache</tt>).</p>
</dd>
</dl>
</li>
</ol>
<ol class="arabic" start="2">
<li><dl class="first docutils">
<dt>Documentation</dt>
<dd><p class="first last">Finding typos or outdated sections in the documentation. Contributing documentation
based on your own experience and experimentation with the library or with BitTorrent
in general. Non-reference documentation is very much welcome as well, higher level
descriptions on how to configure libtorrent for various situations for instance.
The reference documentation for libtorrent is generated from the header files.
For updates, please submit a <a class="reference external" href="https://github.com/arvidn/libtorrent">pull request</a>.</p>
</dd>
</dl>
</li>
<li><dl class="first docutils">
<dt>Code</dt>
<dd><p class="first">Contributing code for new features or bug-fixes is highly welcome. If you're interested
in adding a feature but not sure where to start, please contact the <a class="reference external" href="http://lists.sourceforge.net/lists/listinfo/libtorrent-discuss">mailing list</a> or
<tt class="docutils literal">#libtorrent</tt> &#64; <tt class="docutils literal">irc.freenode.net</tt>. For proposed fixes or udpates, please
submit a <a class="reference external" href="https://github.com/arvidn/libtorrent">pull request</a>.</p>
<p class="last">New features might be better support for integrating with other services, new choking
algorithms, seeding policies, ports to new platforms etc.</p>
</dd>
</dl>
</li>
</ol>
<p>For an overview of the internals of libtorrent, see the <a class="reference external" href="hacking.html">hacking</a> page.</p>
<p>For outstanding things to do, see the <a class="reference external" href="todo.html">todo list</a>.</p>
</div>
</div>
</div>
<div id="gradient"></div>
<div id="footer">
<table>
<tr>
<td><a href="index.html">home</a></td>
<td><a href="http://blog.libtorrent.org">blog</a></td>
<td><a href="utp.html">uTP</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/libtorrent/">download</a></td>
<td><a href="reference.html">documentation</a></td>
<td><a href="dht_store.html">DHT put extension</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/py-libtorrent/">python bindings</a></td>
<td><a href="features.html">features</a></td>
<td><a href="dht_sec.html">DHT security extension</a></td>
</tr>
<tr>
<td><a href="http://dir.gmane.org/gmane.network.bit-torrent.libtorrent">mailing list archive</a></td>
<td><a href="contributing.html">contributing</a></td>
<td><a href="streaming.html">streaming</a></td>
</tr>
<tr>
<td><a href="http://code.google.com/p/libtorrent/issues/entry">report a bug</a></td>
<td><a href="building.html">building</a></td>
<td><a href="bittorrent.pdf">bittorrent slides</a></td>
</tr>
</table>
</div>
<div id="filler"></div>
</div>
</body>
</html>

View File

@ -1,148 +0,0 @@
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.12: http://docutils.sourceforge.net/" />
<title></title>
<meta name="author" content="Arvid Norberg, arvid&#64;libtorrent.org" />
<meta name=viewport content="width=device-width, initial-scale=1">
<script type="text/javascript">
/* <![CDATA[ */
(function() {
var s = document.createElement('script'), t = document.getElementsByTagName('script')[0];
s.type = 'text/javascript';
s.async = true;
s.src = 'http://api.flattr.com/js/0.6/load.js?mode=auto';
t.parentNode.insertBefore(s, t);
})();
/* ]]> */
</script>
<link rel="stylesheet" href="style.css" type="text/css" />
<link rel="stylesheet" type="text/css" href="rst.css" />
<style type="text/css">
/* Hides from IE-mac \*/
* html pre { height: 1%; }
/* End hide from IE-mac */
</style>
</head>
<body>
<div class="document">
<div id="container">
<table id="header">
<tr><td id="orange"></td>
<td id="logo">libtorrent</td></tr>
</table>
<div id="main">
<table class="docinfo" frame="void" rules="none">
<col class="docinfo-name" />
<col class="docinfo-content" />
<tbody valign="top">
<tr><th class="docinfo-name">Author:</th>
<td>Arvid Norberg, <a class="last reference external" href="mailto:arvid&#64;libtorrent.org">arvid&#64;libtorrent.org</a></td></tr>
</tbody>
</table>
<div class="section" id="mainline-dht-extensions">
<h1>Mainline DHT extensions</h1>
<p>libtorrent implements a few extensions to the Mainline DHT protocol.</p>
<div class="section" id="get-peers-response">
<h2>get_peers response</h2>
<p>libtorrent always responds with <tt class="docutils literal">nodes</tt> to a get_peers request. If it has
peers for the specified info-hash, it will return <tt class="docutils literal">values</tt> as well. This is
because just because some peer announced to us, doesn't mean that we are
among the 8 closest nodes of the info hash. libtorrent also keeps traversing
nodes using get_peers until it has found the 8 closest ones, and then announces
to those nodes.</p>
</div>
<div class="section" id="forward-compatibility">
<h2>forward compatibility</h2>
<p>In order to support future DHT messages, any message which is not recognized
but has either an <tt class="docutils literal">info_hash</tt> or <tt class="docutils literal">target</tt> argument is interpreted as
find node for that target. i.e. it returns nodes. This allows future messages
to be properly forwarded by clients that don't understand them instead of
being blocked.</p>
</div>
<div class="section" id="client-identification">
<h2>client identification</h2>
<p>In each DHT packet, an extra key is inserted named &quot;v&quot;. This is a string
describing the client and version used. This can help alot when debugging
and finding errors in client implementations. The string is encoded as four
characters, two characters describing the client and two characters interpreted
as a binary number describing the client version.</p>
<p>Currently known clients:</p>
<table border="1" class="docutils">
<colgroup>
<col width="65%" />
<col width="35%" />
</colgroup>
<tbody valign="top">
<tr><td>uTorrent</td>
<td><tt class="docutils literal">UT</tt></td>
</tr>
<tr><td>libtorrent</td>
<td><tt class="docutils literal">LT</tt></td>
</tr>
<tr><td>MooPolice</td>
<td><tt class="docutils literal">MP</tt></td>
</tr>
<tr><td>GetRight</td>
<td><tt class="docutils literal">GR</tt></td>
</tr>
</tbody>
</table>
</div>
<div class="section" id="ipv6-support">
<h2>IPv6 support</h2>
<p><strong>This extension is superseeded by</strong> <a class="reference external" href="http://bittorrent.org/beps/bep_0032.html">BEP 32</a>.</p>
<p>The DHT messages that don't support IPv6 are the <tt class="docutils literal">nodes</tt> replies.
They encode all the contacts as 6 bytes packed together in sequence in a
string. The problem is that IPv6 endpoints cannot be encoded as 6 bytes, but
needs 18 bytes. The extension libtorrent applies is to add another key, called
<tt class="docutils literal">nodes2</tt>.</p>
<p><tt class="docutils literal">nodes2</tt> may be present in replies that contains a <tt class="docutils literal">nodes</tt> key. It is encoded
as a list of strings. Each string represents one contact and is encoded as 20
bytes node-id and then a variable length encoded IP address (6 bytes in IPv4 case
and 18 bytes in IPv6 case).</p>
<p>As an optimization, libtorrent does not include the extra key in case there are
only IPv4 nodes present.</p>
</div>
</div>
</div>
</div>
<div id="gradient"></div>
<div id="footer">
<table>
<tr>
<td><a href="index.html">home</a></td>
<td><a href="http://blog.libtorrent.org">blog</a></td>
<td><a href="utp.html">uTP</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/libtorrent/">download</a></td>
<td><a href="reference.html">documentation</a></td>
<td><a href="dht_store.html">DHT put extension</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/py-libtorrent/">python bindings</a></td>
<td><a href="features.html">features</a></td>
<td><a href="dht_sec.html">DHT security extension</a></td>
</tr>
<tr>
<td><a href="http://dir.gmane.org/gmane.network.bit-torrent.libtorrent">mailing list archive</a></td>
<td><a href="contributing.html">contributing</a></td>
<td><a href="streaming.html">streaming</a></td>
</tr>
<tr>
<td><a href="http://code.google.com/p/libtorrent/issues/entry">report a bug</a></td>
<td><a href="building.html">building</a></td>
<td><a href="bittorrent.pdf">bittorrent slides</a></td>
</tr>
</table>
</div>
<div id="filler"></div>
</div>
</body>
</html>

View File

@ -1,435 +0,0 @@
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.12: http://docutils.sourceforge.net/" />
<title>BitTorrent extension for DHT RSS feeds</title>
<meta name="author" content="Arvid Norberg, arvid&#64;libtorrent.org" />
<meta name=viewport content="width=device-width, initial-scale=1">
<script type="text/javascript">
/* <![CDATA[ */
(function() {
var s = document.createElement('script'), t = document.getElementsByTagName('script')[0];
s.type = 'text/javascript';
s.async = true;
s.src = 'http://api.flattr.com/js/0.6/load.js?mode=auto';
t.parentNode.insertBefore(s, t);
})();
/* ]]> */
</script>
<link rel="stylesheet" href="style.css" type="text/css" />
<link rel="stylesheet" type="text/css" href="rst.css" />
<style type="text/css">
/* Hides from IE-mac \*/
* html pre { height: 1%; }
/* End hide from IE-mac */
</style>
</head>
<body>
<div class="document" id="bittorrent-extension-for-dht-rss-feeds">
<div id="container">
<table id="header">
<tr><td id="orange"></td>
<td id="logo">libtorrent</td></tr>
</table>
<div id="main">
<h1 class="title">BitTorrent extension for DHT RSS feeds</h1>
<table class="docinfo" frame="void" rules="none">
<col class="docinfo-name" />
<col class="docinfo-content" />
<tbody valign="top">
<tr><th class="docinfo-name">Author:</th>
<td>Arvid Norberg, <a class="last reference external" href="mailto:arvid&#64;libtorrent.org">arvid&#64;libtorrent.org</a></td></tr>
<tr><th class="docinfo-name">Version:</th>
<td>1.1.0</td></tr>
</tbody>
</table>
<div class="contents topic" id="table-of-contents">
<p class="topic-title first">Table of contents</p>
<ul class="simple">
<li><a class="reference internal" href="#terminology" id="id1">terminology</a></li>
<li><a class="reference internal" href="#linked-lists" id="id2">linked lists</a></li>
<li><a class="reference internal" href="#skip-lists" id="id3">skip lists</a></li>
<li><a class="reference internal" href="#list-head" id="id4">list-head</a></li>
<li><a class="reference internal" href="#messages" id="id5">messages</a><ul>
<li><a class="reference internal" href="#requesting-items" id="id6">requesting items</a></li>
<li><a class="reference internal" href="#request-item-response" id="id7">request item response</a></li>
<li><a class="reference internal" href="#announcing-items" id="id8">announcing items</a></li>
</ul>
</li>
<li><a class="reference internal" href="#re-announcing" id="id9">re-announcing</a></li>
<li><a class="reference internal" href="#timeouts" id="id10">timeouts</a></li>
<li><a class="reference internal" href="#rss-feeds" id="id11">RSS feeds</a><ul>
<li><a class="reference internal" href="#example" id="id12">example</a></li>
</ul>
</li>
<li><a class="reference internal" href="#rss-feed-uri-scheme" id="id13">RSS feed URI scheme</a></li>
<li><a class="reference internal" href="#rationale" id="id14">rationale</a></li>
</ul>
</div>
<p>This proposal has been superseded by the <a class="reference external" href="dht_store.html">dht_put</a> feature. This may
still be implemented on top of that.</p>
<p>This is a proposal for an extension to the BitTorrent DHT to allow
for decentralized RSS feed like functionality.</p>
<p>The intention is to allow the creation of repositories of torrents
where only a single identity has the authority to add new content. For
this repository to be robust against network failures and resilient
to attacks at the source.</p>
<p>The target ID under which the repository is stored in the DHT, is the
SHA-1 hash of a feed name and the 512 bit public key. This private key
in this pair MUST be used to sign every item stored in the repository.
Every message that contain signed items MUST also include this key, to
allow the receiver to verify the key itself against the target ID as well
as the validity of the signatures of the items. Every recipient of a
message with feed items in it MUST verify both the validity of the public
key against the target ID it is stored under, as well as the validity of
the signatures of each individual item.</p>
<p>As with normal DHT announces, the write-token mechanism is used to
prevent IP spoof attacks.</p>
<div class="section" id="terminology">
<h1>terminology</h1>
<p>In this document, a <em>storage node</em> refers to the node in the DHT to which
an item is being announce. A <em>subscribing node</em> refers to a node which
makes look ups in the DHT to find the storage nodes, to request items
from them.</p>
</div>
<div class="section" id="linked-lists">
<h1>linked lists</h1>
<p>Items are chained together in a geneal singly linked list. A linked
list does not necessarily contain RSS items, and no RSS related items
are mandatory. However, RSS items will be used as examples in this BEP:</p>
<pre class="literal-block">
key = SHA1(name + key)
+---------+
| head | key = SHA1(bencode(item))
| +---------+ +---------+
| | next |--------&gt;| item | key = SHA1(bencode(item))
| | key | | +---------+ +---------+
| | name | | | next |-------&gt;| item |
| | seq | | | key | | +---------+
| | ... | | | ... | | | next |---&gt;0
| +---------+ | +---------+ | | key |
| sig | | sig | | | ... |
+---------+ +---------+ | +---------+
| sig |
+---------+
</pre>
<p>The <tt class="docutils literal">next</tt> pointer is at least 20 byte ID in the DHT key space pointing to where the next
item in the list is announced. The list is terminated with an ID of all zeroes.</p>
<p>The ID an items is announced to is determined by the SHA1 hash of the bencoded representation
of the item iteself. This contains all fields in the item, except the signature.
The only mandatory fields in an item are <tt class="docutils literal">next</tt>, <tt class="docutils literal">key</tt> and <tt class="docutils literal">sig</tt>.</p>
<p>The <tt class="docutils literal">key</tt> field MUST match the public key of the list head node. The <tt class="docutils literal">sig</tt> field
MUST be the signature of the bencoded representation of <tt class="docutils literal">item</tt> or <tt class="docutils literal">head</tt> (whichever
is included in the message).</p>
<p>All subscribers MUST verify that the item is announced under the correct DHT key
and MUST verify the signature is valid and MUST verify the public key is the same
as the list-head. If a node fails any of these checks, it must be ignored and the
chain of items considered terminated.</p>
<p>Each item holds a bencoded dictionary with arbitrary keys, except two mandatory keys:
<tt class="docutils literal">next</tt> and <tt class="docutils literal">key</tt>. The signature <tt class="docutils literal">sig</tt> is transferred outside of this dictionary
and is the signature of all of it. An implementation should stora any arbitrary keys that
are announced to an item, within reasonable restriction such as nesting, size and numeric
range of integers.</p>
</div>
<div class="section" id="skip-lists">
<h1>skip lists</h1>
<p>The <tt class="docutils literal">next</tt> key stored in the list head and the items is a string of at least length
20 bytes, it may be any length divisible by 20. Each 20 bytes are the ID of the next
item in the list, the item 2 hops away, 4 hops away, 8 hops away, and so on. For
simplicity, only the first ID (1 hop) in the <tt class="docutils literal">next</tt> field is illustrated above.</p>
<p>A publisher of an item SHOULD include as many IDs in the <tt class="docutils literal">next</tt> field as the remaining
size of the list warrants, within reason.</p>
<p>These skip lists allow for parallelized lookups of items and also makes it more efficient
to search for specific items. It also mitigates breaking lists missing some items.</p>
<p>Figure of the skip list in the first list item:</p>
<pre class="literal-block">
n Item0 Item1 Item2 Item3 Item4 Item5 Item6 Item7 Item8 Item9 Item10
0 O-----&gt;
20 O------------&gt;
40 O--------------------------&gt;
60 O------------------------------------------------------&gt;
</pre>
<p><em>n</em> refers to the byte offset into the <tt class="docutils literal">next</tt> field.</p>
</div>
<div class="section" id="list-head">
<h1>list-head</h1>
<p>The list head item is special in that it can be updated, without changing its
DHT key. This is required to prepend new items to the linked list. To authenticate
that only the original publisher can update the head, the whole linked list head
is signed. In order to avoid a malicious node to overwrite the list head with an old
version, the sequence number <tt class="docutils literal">seq</tt> must be monotonically increasing for each update,
and a node hosting the list node MUST not downgrade a list head from a higher sequence
number to a lower one, only upgrade.</p>
<p>The list head's DHT key (which it is announced to) MUST be the SHA1 hash of the name
(<tt class="docutils literal">n</tt>) and <tt class="docutils literal">key</tt> fields concatenated.</p>
<p>Any node MUST reject any list head which is announced under any other ID.</p>
</div>
<div class="section" id="messages">
<h1>messages</h1>
<p>These are the messages to deal with linked lists.</p>
<p>The <tt class="docutils literal">id</tt> field in these messages has the same semantics as the standard DHT messages,
i.e. the node ID of the node sending the message, to maintain the structure of the DHT
network.</p>
<p>The <tt class="docutils literal">token</tt> field also has the same semantics as the standard DHT message <tt class="docutils literal">get_peers</tt>
and <tt class="docutils literal">announce_peer</tt>, when requesting an item and to write an item respectively.</p>
<p><tt class="docutils literal">nodes</tt> and <tt class="docutils literal">nodes6</tt> has the same semantics as in its <tt class="docutils literal">get_peers</tt> response.</p>
<div class="section" id="requesting-items">
<h2>requesting items</h2>
<p>This message can be used to request both a list head and a list item. When requesting
a list head, the <tt class="docutils literal">n</tt> (name) field MUST be specified. When requesting a list item the
<tt class="docutils literal">n</tt> field is not required.</p>
<pre class="literal-block">
{
&quot;a&quot;:
{
&quot;id&quot;: <em>&lt;20 byte ID of sending node&gt;</em>,
&quot;key&quot;: <em>&lt;64 byte public curve25519 key for this list&gt;</em>,
&quot;n&quot;: <em>&lt;list name&gt;</em>
&quot;target&quot;: <em>&lt;target-id for 'head' or 'item'&gt;</em>
},
&quot;q&quot;: &quot;get_item&quot;,
&quot;t&quot;: <em>&lt;transaction-id&gt;</em>,
&quot;y&quot;: &quot;q&quot;,
}
</pre>
<p>When requesting a list-head the <tt class="docutils literal">target</tt> MUST always be SHA-1(<em>feed_name</em> + <em>public_key</em>).
<tt class="docutils literal">target</tt> is the target node ID the item was written to.</p>
<p>The <tt class="docutils literal">n</tt> field is the name of the list. If specified, It MUST be UTF-8 encoded string
and it MUST match the name of the feed in the receiving node.</p>
</div>
<div class="section" id="request-item-response">
<h2>request item response</h2>
<p>This is the format of a response of a list head:</p>
<pre class="literal-block">
{
&quot;r&quot;:
{
&quot;head&quot;:
{
&quot;key&quot;: <em>&lt;64 byte public curve25519 key for this list&gt;</em>,
&quot;next&quot;: <em>&lt;20 bytes item ID&gt;</em>,
&quot;n&quot;: <em>&lt;name of the linked list&gt;</em>,
&quot;seq&quot;: <em>&lt;monotonically increasing sequence number&gt;</em>
},
&quot;sig&quot;: <em>&lt;curve25519 signature of 'head' entry (in bencoded form)&gt;</em>,
&quot;id&quot;: <em>&lt;20 byte id of sending node&gt;</em>,
&quot;token&quot;: <em>&lt;write-token&gt;</em>,
&quot;nodes&quot;: <em>&lt;n * compact IPv4-port pair&gt;</em>,
&quot;nodes6&quot;: <em>&lt;n * compact IPv6-port pair&gt;</em>
},
&quot;t&quot;: <em>&lt;transaction-id&gt;</em>,
&quot;y&quot;: &quot;r&quot;,
}
</pre>
<p>This is the format of a response of a list item:</p>
<pre class="literal-block">
{
&quot;r&quot;:
{
&quot;item&quot;:
{
&quot;key&quot;: <em>&lt;64 byte public curve25519 key for this list&gt;</em>,
&quot;next&quot;: <em>&lt;20 bytes item ID&gt;</em>,
...
},
&quot;sig&quot;: <em>&lt;curve25519 signature of 'item' entry (in bencoded form)&gt;</em>,
&quot;id&quot;: <em>&lt;20 byte id of sending node&gt;</em>,
&quot;token&quot;: <em>&lt;write-token&gt;</em>,
&quot;nodes&quot;: <em>&lt;n * compact IPv4-port pair&gt;</em>,
&quot;nodes6&quot;: <em>&lt;n * compact IPv6-port pair&gt;</em>
},
&quot;t&quot;: <em>&lt;transaction-id&gt;</em>,
&quot;y&quot;: &quot;r&quot;,
}
</pre>
<p>A client receiving a <tt class="docutils literal">get_item</tt> response MUST verify the signature in the <tt class="docutils literal">sig</tt>
field against the bencoded representation of the <tt class="docutils literal">item</tt> field, using the <tt class="docutils literal">key</tt> as
the public key. The <tt class="docutils literal">key</tt> MUST match the public key of the feed.</p>
<p>The <tt class="docutils literal">item</tt> dictionary MAY contain arbitrary keys, and all keys MUST be stored for
items.</p>
</div>
<div class="section" id="announcing-items">
<h2>announcing items</h2>
<p>The message format for announcing a list head:</p>
<pre class="literal-block">
{
&quot;a&quot;:
{
&quot;head&quot;:
{
&quot;key&quot;: <em>&lt;64 byte public curve25519 key for this list&gt;</em>,
&quot;next&quot;: <em>&lt;20 bytes item ID&gt;</em>,
&quot;n&quot;: <em>&lt;name of the linked list&gt;</em>,
&quot;seq&quot;: <em>&lt;monotonically increasing sequence number&gt;</em>
},
&quot;sig&quot;: <em>&lt;curve25519 signature of 'head' entry (in bencoded form)&gt;</em>,
&quot;id&quot;: <em>&lt;20 byte node-id of origin node&gt;</em>,
&quot;target&quot;: <em>&lt;target-id as derived from public key and name&gt;</em>,
&quot;token&quot;: <em>&lt;write-token as obtained by previous request&gt;</em>
},
&quot;y&quot;: &quot;q&quot;,
&quot;q&quot;: &quot;announce_item&quot;,
&quot;t&quot;: <em>&lt;transaction-id&gt;</em>
}
</pre>
<p>The message format for announcing a list item:</p>
<pre class="literal-block">
{
&quot;a&quot;:
{
&quot;item&quot;:
{
&quot;key&quot;: <em>&lt;64 byte public curve25519 key for this list&gt;</em>,
&quot;next&quot;: <em>&lt;20 bytes item ID&gt;</em>,
...
},
&quot;sig&quot;: <em>&lt;curve25519 signature of 'item' entry (in bencoded form)&gt;</em>,
&quot;id&quot;: <em>&lt;20 byte node-id of origin node&gt;</em>,
&quot;target&quot;: <em>&lt;target-id as derived from item dict&gt;</em>,
&quot;token&quot;: <em>&lt;write-token as obtained by previous request&gt;</em>
},
&quot;y&quot;: &quot;q&quot;,
&quot;q&quot;: &quot;announce_item&quot;,
&quot;t&quot;: <em>&lt;transaction-id&gt;</em>
}
</pre>
<p>A storage node MAY reject items and heads whose bencoded representation is
greater than 1024 bytes.</p>
</div>
</div>
<div class="section" id="re-announcing">
<h1>re-announcing</h1>
<p>In order to keep feeds alive, subscriber nodes SHOULD help out in announcing
items they have downloaded to the DHT.</p>
<p>Every subscriber node SHOULD store items in long term storage, across sessions,
in order to keep items alive for as long as possible, with as few sources as possible.</p>
<p>Subscribers to a feed SHOULD also announce items that they know of, to the feed.
Since a feed may have many subscribers and many items, subscribers should re-announce
items according to the following algorithm.</p>
<pre class="literal-block">
1. pick one random item (<em>i</em>) from the local repository (except
items already announced this round)
2. If all items in the local repository have been announced
2.1 terminate
3. look up item <em>i</em> in the DHT
4. If fewer than 8 nodes returned the item
4.1 announce <em>i</em> to the DHT
4.2 goto 1
</pre>
<p>This ensures a balanced load on the DHT while still keeping items alive</p>
</div>
<div class="section" id="timeouts">
<h1>timeouts</h1>
<p>Items SHOULD be announced to the DHT every 30 minutes. A storage node MAY time
out an item after 60 minutes of no one announcing it.</p>
<p>A storing node MAY extend the timeout when it receives a request for it. Since
items are immutable, the data doesn't go stale. Therefore it doesn't matter if
the storing node no longer is in the set of the 8 closest nodes.</p>
</div>
<div class="section" id="rss-feeds">
<h1>RSS feeds</h1>
<p>For RSS feeds, following keys are mandatory in the list item's <tt class="docutils literal">item</tt> dictionary.</p>
<dl class="docutils">
<dt>ih</dt>
<dd>The torrent's info hash</dd>
<dt>size</dt>
<dd>The size (in bytes) of all files the torrent</dd>
<dt>n</dt>
<dd>name of the torrent</dd>
</dl>
<div class="section" id="example">
<h2>example</h2>
<p>This is an example of an <tt class="docutils literal">announce_item</tt> message:</p>
<pre class="literal-block">
{
&quot;a&quot;:
{
&quot;item&quot;:
{
&quot;key&quot;: &quot;6bc1de5443d1a7c536cdf69433ac4a7163d3c63e2f9c92d
78f6011cf63dbcd5b638bbc2119cdad0c57e4c61bc69ba5e2c08
b918c2db8d1848cf514bd9958d307&quot;,
&quot;info-hash&quot;: &quot;7ea94c240691311dc0916a2a91eb7c3db2c6f3e4&quot;,
&quot;size&quot;: 24315329,
&quot;n&quot;: &quot;my stuff&quot;,
&quot;next&quot;: &quot;c68f29156404e8e0aas8761ef5236bcagf7f8f2e&quot;
}
&quot;sig&quot;: <em>&lt;signature&gt;</em>
&quot;id&quot;: &quot;b46989156404e8e0acdb751ef553b210ef77822e&quot;,
&quot;target&quot;: &quot;b4692ef0005639e86d7165bf378474107bf3a762&quot;
&quot;token&quot;: &quot;23ba&quot;
},
&quot;y&quot;: &quot;q&quot;,
&quot;q&quot;: &quot;announce_item&quot;,
&quot;t&quot;: &quot;a421&quot;
}
</pre>
<p>Strings are printed in hex for printability, but actual encoding is binary.</p>
<p>Note that <tt class="docutils literal">target</tt> is in fact SHA1 hash of the same data the signature <tt class="docutils literal">sig</tt>
is the signature of, i.e.:</p>
<pre class="literal-block">
d9:info-hash20:7ea94c240691311dc0916a2a91eb7c3db2c6f3e43:key64:6bc1de5443d1
a7c536cdf69433ac4a7163d3c63e2f9c92d78f6011cf63dbcd5b638bbc2119cdad0c57e4c61
bc69ba5e2c08b918c2db8d1848cf514bd9958d3071:n8:my stuff4:next20:c68f29156404
e8e0aas8761ef5236bcagf7f8f2e4:sizei24315329ee
</pre>
<p>(note that binary data is printed as hex)</p>
</div>
</div>
<div class="section" id="rss-feed-uri-scheme">
<h1>RSS feed URI scheme</h1>
<p>The proposed URI scheme for DHT feeds is:</p>
<pre class="literal-block">
magnet:?xt=btfd:<em>&lt;base16-curve25519-public-key&gt;</em> &amp;dn= <em>&lt;feed name&gt;</em>
</pre>
<p>Note that a difference from regular torrent magnet links is the <strong>btfd</strong>
versus <strong>btih</strong> used in regular magnet links to torrents.</p>
<p>The <em>feed name</em> is mandatory since it is used in the request and when
calculating the target ID.</p>
</div>
<div class="section" id="rationale">
<h1>rationale</h1>
<p>The reason to use <a class="reference external" href="http://cr.yp.to/ecdh.html">curve25519</a> instead of, for instance, RSA is compactness. According to
<a class="reference external" href="http://cr.yp.to/">http://cr.yp.to/</a>, curve25519 is free from patent claims and there are open implementations
in both C and Java.</p>
</div>
</div>
</div>
<div id="gradient"></div>
<div id="footer">
<table>
<tr>
<td><a href="index.html">home</a></td>
<td><a href="http://blog.libtorrent.org">blog</a></td>
<td><a href="utp.html">uTP</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/libtorrent/">download</a></td>
<td><a href="reference.html">documentation</a></td>
<td><a href="dht_store.html">DHT put extension</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/py-libtorrent/">python bindings</a></td>
<td><a href="features.html">features</a></td>
<td><a href="dht_sec.html">DHT security extension</a></td>
</tr>
<tr>
<td><a href="http://dir.gmane.org/gmane.network.bit-torrent.libtorrent">mailing list archive</a></td>
<td><a href="contributing.html">contributing</a></td>
<td><a href="streaming.html">streaming</a></td>
</tr>
<tr>
<td><a href="http://code.google.com/p/libtorrent/issues/entry">report a bug</a></td>
<td><a href="building.html">building</a></td>
<td><a href="bittorrent.pdf">bittorrent slides</a></td>
</tr>
</table>
</div>
<div id="filler"></div>
</div>
</body>
</html>

View File

@ -1,305 +0,0 @@
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.12: http://docutils.sourceforge.net/" />
<title>BitTorrent DHT security extension</title>
<meta name="author" content="Arvid Norberg, arvid&#64;libtorrent.org" />
<meta name=viewport content="width=device-width, initial-scale=1">
<script type="text/javascript">
/* <![CDATA[ */
(function() {
var s = document.createElement('script'), t = document.getElementsByTagName('script')[0];
s.type = 'text/javascript';
s.async = true;
s.src = 'http://api.flattr.com/js/0.6/load.js?mode=auto';
t.parentNode.insertBefore(s, t);
})();
/* ]]> */
</script>
<link rel="stylesheet" href="style.css" type="text/css" />
<link rel="stylesheet" type="text/css" href="rst.css" />
<style type="text/css">
/* Hides from IE-mac \*/
* html pre { height: 1%; }
/* End hide from IE-mac */
</style>
</head>
<body>
<div class="document" id="bittorrent-dht-security-extension">
<div id="container">
<table id="header">
<tr><td id="orange"></td>
<td id="logo">libtorrent</td></tr>
</table>
<div id="main">
<h1 class="title">BitTorrent DHT security extension</h1>
<table class="docinfo" frame="void" rules="none">
<col class="docinfo-name" />
<col class="docinfo-content" />
<tbody valign="top">
<tr><th class="docinfo-name">Author:</th>
<td>Arvid Norberg, <a class="last reference external" href="mailto:arvid&#64;libtorrent.org">arvid&#64;libtorrent.org</a></td></tr>
<tr><th class="docinfo-name">Version:</th>
<td>1.1.0</td></tr>
</tbody>
</table>
<div class="contents topic" id="table-of-contents">
<p class="topic-title first">Table of contents</p>
<ul class="simple">
<li><a class="reference internal" href="#id1" id="id2">BitTorrent DHT security extension</a></li>
<li><a class="reference internal" href="#considerations" id="id3">considerations</a></li>
<li><a class="reference internal" href="#node-id-restriction" id="id4">Node ID restriction</a></li>
<li><a class="reference internal" href="#bootstrapping" id="id5">bootstrapping</a></li>
<li><a class="reference internal" href="#rationale" id="id6">rationale</a></li>
<li><a class="reference internal" href="#enforcement" id="id7">enforcement</a></li>
<li><a class="reference internal" href="#backwards-compatibility-and-transition" id="id8">backwards compatibility and transition</a></li>
<li><a class="reference internal" href="#forward-compatibility" id="id9">forward compatibility</a></li>
</ul>
</div>
<div class="section" id="id1">
<h1>BitTorrent DHT security extension</h1>
<p>The purpose of this extension is to make it harder to launch a few
specific attacks against the BitTorrent DHT and also to make it harder
to snoop the network.</p>
<p>Specifically the attack this extension intends to make harder is launching
8 or more DHT nodes which node-IDs selected close to a specific target
info-hash, in order to become the main nodes hosting peers for it. Currently
this is very easy to do and lets the attacker not only see all the traffic
related to this specific info-hash but also block access to it by other
peers.</p>
<p>The proposed guard against this is to enforce restrictions on which node-ID
a node can choose, based on its external IP address.</p>
</div>
<div class="section" id="considerations">
<h1>considerations</h1>
<p>One straight forward scheme to tie the node ID to an IP would be to hash
the IP and force the node ID to share the prefix of that hash. One main
draw back of this approach is that an entities control over the DHT key
space grows linearly with its control over the IP address space.</p>
<p>In order to successfully launch an attack, you just need to find 8 IPs
whose hash will be <em>closest</em> to the target info-hash. Given the current
size of the DHT, that is quite likely to be possible by anyone in control
of a /8 IP block.</p>
<p>The size of the DHT is approximately 8.4 million nodes. This is estmiated
by observing that a typical routing table typically has about 20 of its
top routing table buckets full. That means the key space is dense enough
to contain 8 nodes for every combination of the 20 top bits of node IDs.</p>
<blockquote>
<tt class="docutils literal">2^20 * 8 = 8388608</tt></blockquote>
<p>By controlling that many IP addresses, an attacker could snoop any info-hash.
By controlling 8 times that many IP addresses, an attacker could actually
take over any info-hash.</p>
<p>With IPv4, snooping would require a /8 IP block, giving access to 16.7 million
Ips.</p>
<p>Another problem with hashing the IP is that multiple users behind a NAT are
forced to run their DHT nodes on the same node ID.</p>
</div>
<div class="section" id="node-id-restriction">
<h1>Node ID restriction</h1>
<p>In order to avoid the number node IDs controlled to grow linearly by the number
of IPs, as well as allowing more than one node ID per external IP, the node
ID can be restricted at each class level of the IP.</p>
<p>Another important property of the restriction put on node IDs is that the
distribution of the IDs remoain uniform. This is why CRC32C (Castagnoli) was
chosen as the hash function.</p>
<p>The expression to calculate a valid ID prefix (from an IPv4 address) is:</p>
<pre class="literal-block">
crc32c((ip &amp; 0x030f3fff) | (r &lt;&lt; 29))
</pre>
<p>And for an IPv6 address (<tt class="docutils literal">ip</tt> is the high 64 bits of the address):</p>
<pre class="literal-block">
crc32c((ip &amp; 0x0103070f1f3f7fff) | (r &lt;&lt; 61))
</pre>
<p><tt class="docutils literal">r</tt> is a random number in the range [0, 7]. The resulting integer,
representing the masked IP address is supposed to be big-endian before
hashed. The &quot;|&quot; operator means bit-wise OR.</p>
<p>The details of implementing this is to evaluate the expression, store the
result in a big endian 64 bit integer and hash those 8 bytes with CRC32C.</p>
<p>The first (most significant) 21 bits of the node ID used in the DHT MUST
match the first 21 bits of the resulting hash. The last byte of the hash MUST
match the random number (<tt class="docutils literal">r</tt>) used to generate the hash.</p>
<img alt="ip_id_v4.png" src="ip_id_v4.png" />
<img alt="ip_id_v6.png" src="ip_id_v6.png" />
<p>Example code code for calculating a valid node ID:</p>
<pre class="literal-block">
uint8_t* ip; // our external IPv4 or IPv6 address (network byte order)
int num_octets; // the number of octets to consider in ip (4 or 8)
uint8_t node_id[20]; // resulting node ID
uint8_t v4_mask[] = { 0x03, 0x0f, 0x3f, 0xff };
uint8_t v6_mask[] = { 0x01, 0x03, 0x07, 0x0f, 0x1f, 0x3f, 0x7f, 0xff };
uint8_t* mask = num_octets == 4 ? v4_mask : v6_mask;
for (int i = 0; i &lt; num_octets; ++i)
ip[i] &amp;= mask[i];
uint32_t rand = std::rand() &amp; 0xff;
uint8_t r = rand &amp; 0x7;
ip[0] |= r &lt;&lt; 5;
uint32_t crc = 0;
crc = crc32c(crc, ip, num_octets);
// only take the top 21 bits from crc
node_id[0] = (crc &gt;&gt; 24) &amp; 0xff;
node_id[1] = (crc &gt;&gt; 16) &amp; 0xff;
node_id[2] = ((crc &gt;&gt; 8) &amp; 0xf8) | (std::rand() &amp; 0x7);
for (int i = 3; i &lt; 19; ++i) node_id[i] = std::rand();
node_id[19] = rand;
</pre>
<p>test vectors:</p>
<pre class="literal-block">
IP rand example node ID
============ ===== ==========================================
124.31.75.21 1 <strong>5fbfbf</strong> f10c5d6a4ec8a88e4c6ab4c28b95eee4 <strong>01</strong>
21.75.31.124 86 <strong>5a3ce9</strong> c14e7a08645677bbd1cfe7d8f956d532 <strong>56</strong>
65.23.51.170 22 <strong>a5d432</strong> 20bc8f112a3d426c84764f8c2a1150e6 <strong>16</strong>
84.124.73.14 65 <strong>1b0321</strong> dd1bb1fe518101ceef99462b947a01ff <strong>41</strong>
43.213.53.83 90 <strong>e56f6c</strong> bf5b7c4be0237986d5243b87aa6d5130 <strong>5a</strong>
</pre>
<p>The bold parts of the node ID are the important parts. The rest are
random numbers. The last bold number of each row has only its most significant
bit pulled from the CRC32C function. The lower 3 bits are random.</p>
</div>
<div class="section" id="bootstrapping">
<h1>bootstrapping</h1>
<p>In order to set ones initial node ID, the external IP needs to be known. This
is not a trivial problem. With this extension, <em>all</em> DHT responses SHOULD include
a <em>top-level</em> field called <tt class="docutils literal">ip</tt>, containing a compact binary representation of
the requestor's IP and port. That is big endian IP followed by 2 bytes of big endian
port.</p>
<p>The IP portion is the same byte sequence used to verify the node ID.</p>
<p>It is important that the <tt class="docutils literal">ip</tt> field is in the top level dictionary. Nodes that
enforce the node-ID will respond with an error message (&quot;y&quot;: &quot;e&quot;, &quot;e&quot;: { ... }),
whereas a node that supports this extension but without enforcing it will respond
with a normal reply (&quot;y&quot;: &quot;r&quot;, &quot;r&quot;: { ... }).</p>
<p>A DHT node which receives an <tt class="docutils literal">ip</tt> result in a request SHOULD consider restarting
its DHT node with a new node ID, taking this IP into account. Since a single node
can not be trusted, there should be some mechanism to determine whether or
not the node has a correct understanding of its external IP or not. This could
be done by voting, or only restart the DHT once at least a certain number of
nodes, from separate searches, tells you your node ID is incorrect.</p>
</div>
<div class="section" id="rationale">
<h1>rationale</h1>
<p>The choice of using CRC32C instead of a more traditional cryptographic hash
function is justified primarily of these reasons:</p>
<ol class="arabic simple">
<li>it is a fast function</li>
<li>produces well distributed results</li>
<li>there is no need for the hash function to be one-way (the input set is
so small that any hash function could be reversed).</li>
<li>CRC32C (Castagnoli) is supported in hardware by SSE 4.2, which can
significantly speed up computation</li>
</ol>
<p>There are primarily two tests run on SHA-1 and CRC32C to establish the
distribution of results. The first one is the number of bits in the output
set that contain every possible combination of bits. The CRC32C function
has a longer such prefix in its output than SHA-1. This means nodes will still
have well uniformly distributed IDs, even when IP addresses in use are not
uniformly distributed.</p>
<p>The following graph illustrate a few different hash functions with regard
to this property.</p>
<img alt="complete_bit_prefixes.png" src="complete_bit_prefixes.png" />
<p>This test takes into account IP addresses that are not globally routable, i.e.
reserved for local networks, multicast and other things. It also takes into
account that some /8 blocks are not in use by end-users and exremely unlikely
to ever run a DHT node. This makes the results likely to be very similar to
what we would see in the wild.</p>
<p>These results indicate that CRC32C provides the best uniformity in the results
in terms of bit prefixes where all possibilities are represented, and that
no more than 21 bits should be used from the result. If more than 21 bits
were to be used, there would be certain node IDs that would be impossible to
have, which would make routing sub-optimal.</p>
<p>The second test is more of a sanity test for the uniform distribution property.
The target space (32 bit interger) is divided up into 1000 buckets. Every valid
IP and <tt class="docutils literal">r</tt> input is run through the algorithm and the result is put in the
bucket it falls in. The expectation is that each bucket has roughly an equal
number of results falling into it. The following graph shows the resulting
histogram, comparing SHA-1 and CRC32C.</p>
<img alt="hash_distribution.png" src="hash_distribution.png" />
<p>The source code for these tests can be found <a class="reference external" href="https://github.com/arvidn/hash_complete_prefix">here</a>.</p>
<p>The reason to use CRC32C instead of the CRC32 implemented by zlib is that
Intel CPUs have hardware support for the CRC32C calculations. The input
being exactly 4 bytes is also deliberate, to make it fit in a single
instruction.</p>
</div>
<div class="section" id="enforcement">
<h1>enforcement</h1>
<p>Once enforced, write tokens from peers whose node ID does not match its external
IP should be considered dropped. In other words, a peer that uses a non-matching
ID MUST never be used to store information on, regardless of which request. In the
original DHT specification only <tt class="docutils literal">announce_peer</tt> stores data in the network,
but any future extension which stores data in the network SHOULD use the same
restriction.</p>
<p>Any peer on a local network address is exempt from this node ID verification.
This includes the following IP blocks:</p>
<dl class="docutils">
<dt>10.0.0.0/8</dt>
<dd>reserved for local networks</dd>
<dt>172.16.0.0/12</dt>
<dd>reserved for local networks</dd>
<dt>192.168.0.0/16</dt>
<dd>reserved for local networks</dd>
<dt>169.254.0.0/16</dt>
<dd>reserved for self-assigned IPs</dd>
<dt>127.0.0.0/8</dt>
<dd>reserved for loopback</dd>
</dl>
</div>
<div class="section" id="backwards-compatibility-and-transition">
<h1>backwards compatibility and transition</h1>
<p>During some transition period, this restriction should not be enforced, and
peers whose node ID does not match this formula relative to their external IP
should not be blocked.</p>
<p>Requests from peers whose node ID does not match their external IP should
always be serviced, even after the transition period. The attack this protects
from is storing data on an attacker's node, not servicing an attackers request.</p>
</div>
<div class="section" id="forward-compatibility">
<h1>forward compatibility</h1>
<p>If the total size of the DHT grows to the point where the inherent size limit
in this proposal is too small, the modulus constants can be updated in a new
proposal, and another transition period where both sets of modulus constants
are accepted.</p>
</div>
</div>
</div>
<div id="gradient"></div>
<div id="footer">
<table>
<tr>
<td><a href="index.html">home</a></td>
<td><a href="http://blog.libtorrent.org">blog</a></td>
<td><a href="utp.html">uTP</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/libtorrent/">download</a></td>
<td><a href="reference.html">documentation</a></td>
<td><a href="dht_store.html">DHT put extension</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/py-libtorrent/">python bindings</a></td>
<td><a href="features.html">features</a></td>
<td><a href="dht_sec.html">DHT security extension</a></td>
</tr>
<tr>
<td><a href="http://dir.gmane.org/gmane.network.bit-torrent.libtorrent">mailing list archive</a></td>
<td><a href="contributing.html">contributing</a></td>
<td><a href="streaming.html">streaming</a></td>
</tr>
<tr>
<td><a href="http://code.google.com/p/libtorrent/issues/entry">report a bug</a></td>
<td><a href="building.html">building</a></td>
<td><a href="bittorrent.pdf">bittorrent slides</a></td>
</tr>
</table>
</div>
<div id="filler"></div>
</div>
</body>
</html>

View File

@ -1,538 +0,0 @@
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.12: http://docutils.sourceforge.net/" />
<title>BitTorrent extension for arbitrary DHT store</title>
<meta name="author" content="Arvid Norberg, arvid&#64;libtorrent.org" />
<meta name=viewport content="width=device-width, initial-scale=1">
<script type="text/javascript">
/* <![CDATA[ */
(function() {
var s = document.createElement('script'), t = document.getElementsByTagName('script')[0];
s.type = 'text/javascript';
s.async = true;
s.src = 'http://api.flattr.com/js/0.6/load.js?mode=auto';
t.parentNode.insertBefore(s, t);
})();
/* ]]> */
</script>
<link rel="stylesheet" href="style.css" type="text/css" />
<link rel="stylesheet" type="text/css" href="rst.css" />
<style type="text/css">
/* Hides from IE-mac \*/
* html pre { height: 1%; }
/* End hide from IE-mac */
</style>
</head>
<body>
<div class="document" id="bittorrent-extension-for-arbitrary-dht-store">
<div id="container">
<table id="header">
<tr><td id="orange"></td>
<td id="logo">libtorrent</td></tr>
</table>
<div id="main">
<h1 class="title">BitTorrent extension for arbitrary DHT store</h1>
<table class="docinfo" frame="void" rules="none">
<col class="docinfo-name" />
<col class="docinfo-content" />
<tbody valign="top">
<tr><th class="docinfo-name">Author:</th>
<td>Arvid Norberg, <a class="last reference external" href="mailto:arvid&#64;libtorrent.org">arvid&#64;libtorrent.org</a></td></tr>
<tr><th class="docinfo-name">Version:</th>
<td>1.1.0</td></tr>
</tbody>
</table>
<div class="contents topic" id="table-of-contents">
<p class="topic-title first">Table of contents</p>
<ul class="simple">
<li><a class="reference internal" href="#terminology" id="id3">terminology</a></li>
<li><a class="reference internal" href="#messages" id="id4">messages</a></li>
<li><a class="reference internal" href="#immutable-items" id="id5">immutable items</a><ul>
<li><a class="reference internal" href="#put-message" id="id6">put message</a></li>
<li><a class="reference internal" href="#get-message" id="id7">get message</a></li>
</ul>
</li>
<li><a class="reference internal" href="#mutable-items" id="id8">mutable items</a><ul>
<li><a class="reference internal" href="#id1" id="id9">put message</a></li>
<li><a class="reference internal" href="#cas" id="id10">CAS</a></li>
<li><a class="reference internal" href="#response" id="id11">response</a></li>
<li><a class="reference internal" href="#errors" id="id12">errors</a></li>
<li><a class="reference internal" href="#id2" id="id13">get message</a></li>
</ul>
</li>
<li><a class="reference internal" href="#signature-verification" id="id14">signature verification</a></li>
<li><a class="reference internal" href="#expiration" id="id15">expiration</a></li>
<li><a class="reference internal" href="#test-vectors" id="id16">test vectors</a><ul>
<li><a class="reference internal" href="#test-1-mutable" id="id17">test 1 (mutable)</a></li>
<li><a class="reference internal" href="#test-2-mutable-with-salt" id="id18">test 2 (mutable with salt)</a></li>
<li><a class="reference internal" href="#test-3-immutable" id="id19">test 3 (immutable)</a></li>
</ul>
</li>
<li><a class="reference internal" href="#resources" id="id20">resources</a></li>
</ul>
</div>
<p>This is a proposal for an extension to the BitTorrent DHT to allow
storing and retrieving of arbitrary data.</p>
<p>It supports both storing <em>immutable</em> items, where the key is
the SHA-1 hash of the data itself, and <em>mutable</em> items, where
the key is the public key of the key pair used to sign the data.</p>
<p>There are two new proposed messages, <tt class="docutils literal">put</tt> and <tt class="docutils literal">get</tt>.</p>
<div class="section" id="terminology">
<h1>terminology</h1>
<p>In this document, a <em>storage node</em> refers to the node in the DHT to which
an item is being announced and stored on. A <em>requesting node</em> refers to
a node which makes look-ups in the DHT to find the storage nodes, to
request items from them, and possibly re-announce those items to keep them
alive.</p>
</div>
<div class="section" id="messages">
<h1>messages</h1>
<p>The proposed new messages <tt class="docutils literal">get</tt> and <tt class="docutils literal">put</tt> are similar to the existing
<tt class="docutils literal">get_peers</tt> and <tt class="docutils literal">announce_peer</tt>.</p>
<p>Responses to <tt class="docutils literal">get</tt> should always include <tt class="docutils literal">nodes</tt> and <tt class="docutils literal">nodes6</tt>. Those
fields have the same semantics as in its <tt class="docutils literal">get_peers</tt> response. It should also
include a write token, <tt class="docutils literal">token</tt>, with the same semantics as int <tt class="docutils literal">get_peers</tt>.
The write token MAY be tied specifically to the key which <tt class="docutils literal">get</tt> requested.
i.e. the <tt class="docutils literal">token</tt> can only be used to store values under that one key. It may
also be tied to the node ID and IP address of the requesting node.</p>
<p>The <tt class="docutils literal">id</tt> field in these messages has the same semantics as the standard DHT
messages, i.e. the node ID of the node sending the message, to maintain the
structure of the DHT network.</p>
<p>The <tt class="docutils literal">token</tt> field also has the same semantics as the standard DHT message
<tt class="docutils literal">get_peers</tt> and <tt class="docutils literal">announce_peer</tt>, when requesting an item and to write an
item respectively.</p>
<p>The <tt class="docutils literal">k</tt> field is the 32 byte ed25519 public key, which the signature can be
authenticated with. When looking up a mutable item, the <tt class="docutils literal">target</tt> field MUST be
the SHA-1 hash of this key concatenated with the <tt class="docutils literal">salt</tt>, if present.</p>
<p>The distinction between storing mutable and immutable items is the inclusion of
a public key, a sequence number, signature and an optional salt (<tt class="docutils literal">k</tt>, <tt class="docutils literal">seq</tt>,
<tt class="docutils literal">sig</tt> and <tt class="docutils literal">salt</tt>).</p>
<p><tt class="docutils literal">get</tt> requests for mutable items and immutable items cannot be distinguished
from eachother. An implementation can either store mutable and immutable items
in the same hash table internally, or in separate ones and potentially do two
lookups for <tt class="docutils literal">get</tt> requests.</p>
<p>The <tt class="docutils literal">v</tt> field is the <em>value</em> to be stored. It is allowed to be any bencoded
type (list, dict, string or integer). When it's being hashed (for verifying its
signature or to calculate its key), its flattened, bencoded, form is used. It is
important to use the verbatim bencoded representation as it appeared in the
message. decoding and then re-encoding bencoded structures is not necessarily an
identity operation.</p>
<p>Storing nodes MAY reject <tt class="docutils literal">put</tt> requests where the bencoded form of <tt class="docutils literal">v</tt> is
longer than 1000 bytes. In other words, it's not safe to assume storing more
than 1000 bytes will succeed.</p>
</div>
<div class="section" id="immutable-items">
<h1>immutable items</h1>
<p>Immutable items are stored under their SHA-1 hash, and since they cannot be
modified, there is no need to authenticate the origin of them. This makes
immutable items simple.</p>
<p>A node making a lookup SHOULD verify the data it receives from the network, to
verify that its hash matches the target that was looked up.</p>
<div class="section" id="put-message">
<h2>put message</h2>
<p>Request:</p>
<pre class="literal-block">
{
&quot;a&quot;:
{
&quot;id&quot;: <em>&lt;20 byte id of sending node (string)&gt;</em>,
&quot;v&quot;: <em>&lt;any bencoded type, whose encoded size &lt;= 1000&gt;</em>
},
&quot;t&quot;: <em>&lt;transaction-id (string)&gt;</em>,
&quot;y&quot;: &quot;q&quot;,
&quot;q&quot;: &quot;put&quot;
}
</pre>
<p>Response:</p>
<pre class="literal-block">
{
&quot;r&quot;: { &quot;id&quot;: <em>&lt;20 byte id of sending node (string)&gt;</em> },
&quot;t&quot;: <em>&lt;transaction-id (string)&gt;</em>,
&quot;y&quot;: &quot;r&quot;,
}
</pre>
</div>
<div class="section" id="get-message">
<h2>get message</h2>
<p>Request:</p>
<pre class="literal-block">
{
&quot;a&quot;:
{
&quot;id&quot;: <em>&lt;20 byte id of sending node (string)&gt;</em>,
&quot;target&quot;: <em>&lt;SHA-1 hash of item (string)&gt;</em>,
},
&quot;t&quot;: <em>&lt;transaction-id (string)&gt;</em>,
&quot;y&quot;: &quot;q&quot;,
&quot;q&quot;: &quot;get&quot;
}
</pre>
<p>Response:</p>
<pre class="literal-block">
{
&quot;r&quot;:
{
&quot;id&quot;: <em>&lt;20 byte id of sending node (string)&gt;</em>,
&quot;token&quot;: <em>&lt;write token (string)&gt;</em>,
&quot;v&quot;: <em>&lt;any bencoded type whose SHA-1 hash matches 'target'&gt;</em>,
&quot;nodes&quot;: <em>&lt;IPv4 nodes close to 'target'&gt;</em>,
&quot;nodes6&quot;: <em>&lt;IPv6 nodes close to 'target'&gt;</em>
},
&quot;t&quot;: <em>&lt;transaction-id&gt;</em>,
&quot;y&quot;: &quot;r&quot;,
}
</pre>
</div>
</div>
<div class="section" id="mutable-items">
<h1>mutable items</h1>
<p>Mutable items can be updated, without changing their DHT keys. To authenticate
that only the original publisher can update an item, it is signed by a private
key generated by the original publisher. The target ID mutable items are stored
under is the SHA-1 hash of the public key (as it appears in the <tt class="docutils literal">put</tt>
message).</p>
<p>In order to avoid a malicious node to overwrite the list head with an old
version, the sequence number <tt class="docutils literal">seq</tt> must be monotonically increasing for each
update, and a node hosting the list node MUST not downgrade a list head from a
higher sequence number to a lower one, only upgrade. The sequence number SHOULD
not exceed <tt class="docutils literal">MAX_INT64</tt>, (i.e. <tt class="docutils literal">0x7fffffffffffffff</tt>. A client MAY reject any
message with a sequence number exceeding this. A client MAY also reject any
message with a negative sequence number.</p>
<p>The signature is a 64 byte ed25519 signature of the bencoded sequence number
concatenated with the <tt class="docutils literal">v</tt> key. e.g. something like this:</p>
<pre class="literal-block">
3:seqi4e1:v12:Hello world!
</pre>
<p>If the <tt class="docutils literal">salt</tt> key is present and non-empty, the salt string must be included
in what's signed. Note that if <tt class="docutils literal">salt</tt> is specified and an empty string, it is
as if it was not specified and nothing in addition to the sequence number and
the data is signed. The salt string may not be longer than 64 bytes.</p>
<p>When a salt is included in what is signed, the key <tt class="docutils literal">salt</tt> with the value of
the key is prepended in its bencoded form. For example, if <tt class="docutils literal">salt</tt> is &quot;foobar&quot;,
the buffer to be signed is:</p>
<pre class="literal-block">
4:salt6:foobar3:seqi4e1:v12:Hello world!
</pre>
<div class="section" id="id1">
<h2>put message</h2>
<p>Request:</p>
<pre class="literal-block">
{
&quot;a&quot;:
{
&quot;cas&quot;: <em>&lt;optional expected seq-nr (int)&gt;</em>,
&quot;id&quot;: <em>&lt;20 byte id of sending node (string)&gt;</em>,
&quot;k&quot;: <em>&lt;ed25519 public key (32 bytes string)&gt;</em>,
&quot;salt&quot;: <em>&lt;optional salt to be appended to &quot;k&quot; when hashing (string)&gt;</em>
&quot;seq&quot;: <em>&lt;monotonically increasing sequence number (integer)&gt;</em>,
&quot;sig&quot;: <em>&lt;ed25519 signature (64 bytes string)&gt;</em>,
&quot;token&quot;: <em>&lt;write-token (string)&gt;</em>,
&quot;v&quot;: <em>&lt;any bencoded type, whose encoded size &lt; 1000&gt;</em>
},
&quot;t&quot;: <em>&lt;transaction-id (string)&gt;</em>,
&quot;y&quot;: &quot;q&quot;,
&quot;q&quot;: &quot;put&quot;
}
</pre>
<p>Storing nodes receiving a <tt class="docutils literal">put</tt> request where <tt class="docutils literal">seq</tt> is lower than or equal
to what's already stored on the node, MUST reject the request. If the sequence
number is equal, and the value is also the same, the node SHOULD reset its
timeout counter.</p>
<p>If the sequence number in the <tt class="docutils literal">put</tt> message is lower than the sequence number
associated with the currently stored value, the storing node MAY return an error
message with code 302 (see error codes below).</p>
<p>Note that this request does not contain a target hash. The target hash under
which this blob is stored is implied by the <tt class="docutils literal">k</tt> argument. The key is the SHA-1
hash of the key (<tt class="docutils literal">k</tt>).</p>
<p>In order to support a single key being used to store separate items in the DHT,
an optional <tt class="docutils literal">salt</tt> can be specified in the <tt class="docutils literal">put</tt> request of mutable items.
If the salt entry is not present, it can be assumed to be an empty string, and
its semantics should be identical as specifying a salt key with an empty string.
The salt can be any binary string (but probably most conveniently a hash of
something). This string is appended to the key, as specified in the <tt class="docutils literal">k</tt> field,
when calculating the key to store the blob under (i.e. the key <tt class="docutils literal">get</tt> requests
specify to retrieve this data).</p>
<p>This lets a single entity, with a single key, publish any number of unrelated
items, with a single key that readers can verify. This is useful if the
publisher doesn't know ahead of time how many different items are to be
published. It can distribute a single public key for users to authenticate the
published blobs.</p>
<p>Note that the salt is not returned in the response to a <tt class="docutils literal">get</tt> request. This
is intentional. When issuing a <tt class="docutils literal">get</tt> request for an item is expected to
know what the salt is (because it is part of what the target ID that is being
looked up is derived from). There is no need to repeat it back for bystanders
to see.</p>
</div>
<div class="section" id="cas">
<h2>CAS</h2>
<p>CAS is short for <em>compare and swap</em>, it has similar semantics as CAS CPU
instructions. It is used to avoid race conditions when multiple nodes are
writing to the same slot in the DHT.</p>
<p>The <tt class="docutils literal">cas</tt> field is optional. If present it specifies the sequence number of
the data blob being overwritten by the put. When present, the storing node
MUST compare this number to the current sequence number it has stored under
this key. Only if the <tt class="docutils literal">cas</tt> matches the stored sequence number is the put
performed. If it mismatches, the store fails and an error is returned.
See <a class="reference internal" href="#errors">errors</a> below.</p>
<p>The <tt class="docutils literal">cas</tt> field only applies to mutable puts. If there is no current
value, the <tt class="docutils literal">cas</tt> field SHOULD be ignored.</p>
<p>When sending a <tt class="docutils literal">put</tt> request to a node that did not return any data for the
<tt class="docutils literal">get</tt>, the <tt class="docutils literal">cas</tt> field SHOULD NOT be included.</p>
</div>
<div class="section" id="response">
<h2>response</h2>
<p>Response:</p>
<pre class="literal-block">
{
&quot;r&quot;: { &quot;id&quot;: <em>&lt;20 byte id of sending node (string)&gt;</em> },
&quot;t&quot;: <em>&lt;transaction-id (string)&gt;</em>,
&quot;y&quot;: &quot;r&quot;,
}
</pre>
</div>
<div class="section" id="errors">
<h2>errors</h2>
<p>If the store fails for any reason an error message is returned instead of the
message template above, i.e. one where &quot;y&quot; is &quot;e&quot; and &quot;e&quot; is a tuple of
[error-code, message]). Failures include <tt class="docutils literal">cas</tt> mismatches and the sequence
number is outdated.</p>
<p>The error message (as specified by <a class="reference external" href="http://www.bittorrent.org/beps/bep_0005.html">BEP5</a>) looks like this:</p>
<pre class="literal-block">
{
&quot;e&quot;: [ <em>&lt;error-code (integer)&gt;</em>, <em>&lt;error-string (string)&gt;</em> ],
&quot;t&quot;: <em>&lt;transaction-id (string)&gt;</em>,
&quot;y&quot;: &quot;e&quot;,
}
</pre>
<p>In addition to the error codes defined in <a class="reference external" href="http://www.bittorrent.org/beps/bep_0005.html">BEP5</a>, this specification defines
some additional error codes.</p>
<table border="1" class="docutils">
<colgroup>
<col width="29%" />
<col width="71%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">error-code</th>
<th class="head">description</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>205</td>
<td>message (<tt class="docutils literal">v</tt> field)
too big.</td>
</tr>
<tr><td>206</td>
<td>invalid signature</td>
</tr>
<tr><td>207</td>
<td>salt (<tt class="docutils literal">salt</tt> field)
too big.</td>
</tr>
<tr><td>301</td>
<td>the CAS hash mismatched,
re-read value and try
again.</td>
</tr>
<tr><td>302</td>
<td>sequence number less than
current.</td>
</tr>
</tbody>
</table>
<p>An implementation MUST emit 301 errors if the cas mismatches. This is a
critical feature in synchronization of multiple agents sharing an immutable
item.</p>
</div>
<div class="section" id="id2">
<h2>get message</h2>
<p>Request:</p>
<pre class="literal-block">
{
&quot;a&quot;:
{
&quot;id&quot;: <em>&lt;20 byte id of sending node (string)&gt;</em>,
&quot;target:&quot; <em>&lt;20 byte SHA-1 hash of public key and salt (string)&gt;</em>
},
&quot;t&quot;: <em>&lt;transaction-id (string)&gt;</em>,
&quot;y&quot;: &quot;q&quot;,
&quot;q&quot;: &quot;get&quot;
}
</pre>
<p>Response:</p>
<pre class="literal-block">
{
&quot;r&quot;:
{
&quot;id&quot;: <em>&lt;20 byte id of sending node (string)&gt;</em>,
&quot;k&quot;: <em>&lt;ed25519 public key (32 bytes string)&gt;</em>,
&quot;nodes&quot;: <em>&lt;IPv4 nodes close to 'target'&gt;</em>,
&quot;nodes6&quot;: <em>&lt;IPv6 nodes close to 'target'&gt;</em>,
&quot;seq&quot;: <em>&lt;monotonically increasing sequence number (integer)&gt;</em>,
&quot;sig&quot;: <em>&lt;ed25519 signature (64 bytes string)&gt;</em>,
&quot;token&quot;: <em>&lt;write-token (string)&gt;</em>,
&quot;v&quot;: <em>&lt;any bencoded type, whose encoded size &lt;= 1000&gt;</em>
},
&quot;t&quot;: <em>&lt;transaction-id (string)&gt;</em>,
&quot;y&quot;: &quot;r&quot;,
}
</pre>
</div>
</div>
<div class="section" id="signature-verification">
<h1>signature verification</h1>
<p>In order to make it maximally difficult to attack the bencoding parser, signing
and verification of the value and sequence number should be done as follows:</p>
<ol class="arabic simple">
<li>encode value and sequence number separately</li>
<li>concatenate (&quot;4:salt&quot; <em>length-of-salt</em> &quot;:&quot; <em>salt</em>) &quot;3:seqi&quot; <em>seq</em>
&quot;e1:v&quot; <em>len</em> &quot;:&quot; and the encoded value.
sequence number 1 of value &quot;Hello World!&quot; would be converted to:
&quot;3:seqi1e1:v12:Hello World!&quot;. In this way it is not possible to convince a
node that part of the length is actually part of the sequence number even if
the parser contains certain bugs. Furthermore it is not possible to have a
verification failure if a bencoding serializer alters the order of entries in
the dictionary. The salt is in parenthesis because it is optional. It is only
prepended if a non-empty salt is specified in the <tt class="docutils literal">put</tt> request.</li>
<li>sign or verify the concatenated string</li>
</ol>
<p>On the storage node, the signature MUST be verified before accepting the store
command. The data MUST be stored under the SHA-1 hash of the public key (as it
appears in the bencoded dict) and the salt (if present).</p>
<p>On the requesting nodes, the key they get back from a <tt class="docutils literal">get</tt> request MUST be
verified to hash to the target ID the lookup was made for, as well as verifying
the signature. If any of these fail, the response SHOULD be considered invalid.</p>
</div>
<div class="section" id="expiration">
<h1>expiration</h1>
<p>Without re-announcement, these items MAY expire in 2 hours. In order
to keep items alive, they SHOULD be re-announced once an hour.</p>
<p>Any node that's interested in keeping a blob in the DHT alive may announce it.
It would simply repeat the signature for a mutable put without having the
private key.</p>
</div>
<div class="section" id="test-vectors">
<h1>test vectors</h1>
<div class="section" id="test-1-mutable">
<h2>test 1 (mutable)</h2>
<p>value:</p>
<pre class="literal-block">
12:Hello World!
</pre>
<p>buffer being signed:</p>
<pre class="literal-block">
3:seqi1e1:v12:Hello World!
</pre>
<p>public key:</p>
<pre class="literal-block">
77ff84905a91936367c01360803104f92432fcd904a43511876df5cdf3e7e548
</pre>
<p>private key:</p>
<pre class="literal-block">
e06d3183d14159228433ed599221b80bd0a5ce8352e4bdf0262f76786ef1c74d
b7e7a9fea2c0eb269d61e3b38e450a22e754941ac78479d6c54e1faf6037881d
</pre>
<p><strong>target ID</strong>:</p>
<pre class="literal-block">
4a533d47ec9c7d95b1ad75f576cffc641853b750
</pre>
<p><strong>signature</strong>:</p>
<pre class="literal-block">
305ac8aeb6c9c151fa120f120ea2cfb923564e11552d06a5d856091e5e853cff
1260d3f39e4999684aa92eb73ffd136e6f4f3ecbfda0ce53a1608ecd7ae21f01
</pre>
</div>
<div class="section" id="test-2-mutable-with-salt">
<h2>test 2 (mutable with salt)</h2>
<p>value:</p>
<pre class="literal-block">
12:Hello World!
</pre>
<p>salt:</p>
<pre class="literal-block">
foobar
</pre>
<p>buffer being signed:</p>
<pre class="literal-block">
4:salt6:foobar3:seqi1e1:v12:Hello World!
</pre>
<p>public key:</p>
<pre class="literal-block">
77ff84905a91936367c01360803104f92432fcd904a43511876df5cdf3e7e548
</pre>
<p>private key:</p>
<pre class="literal-block">
e06d3183d14159228433ed599221b80bd0a5ce8352e4bdf0262f76786ef1c74d
b7e7a9fea2c0eb269d61e3b38e450a22e754941ac78479d6c54e1faf6037881d
</pre>
<p><strong>target ID</strong>:</p>
<pre class="literal-block">
411eba73b6f087ca51a3795d9c8c938d365e32c1
</pre>
<p><strong>signature</strong>:</p>
<pre class="literal-block">
6834284b6b24c3204eb2fea824d82f88883a3d95e8b4a21b8c0ded553d17d17d
df9a8a7104b1258f30bed3787e6cb896fca78c58f8e03b5f18f14951a87d9a08
</pre>
</div>
<div class="section" id="test-3-immutable">
<h2>test 3 (immutable)</h2>
<p>value:</p>
<pre class="literal-block">
12:Hello World!
</pre>
<p><strong>target ID</strong>:</p>
<pre class="literal-block">
e5f96f6f38320f0f33959cb4d3d656452117aadb
</pre>
</div>
</div>
<div class="section" id="resources">
<h1>resources</h1>
<p>Libraries that implement ed25519 DSA:</p>
<ul class="simple">
<li><a class="reference external" href="http://nacl.cr.yp.to/">NaCl</a></li>
<li><a class="reference external" href="https://github.com/jedisct1/libsodium">libsodium</a></li>
<li><a class="reference external" href="https://github.com/nightcracker/ed25519">nightcracker's ed25519</a></li>
</ul>
</div>
</div>
</div>
<div id="gradient"></div>
<div id="footer">
<table>
<tr>
<td><a href="index.html">home</a></td>
<td><a href="http://blog.libtorrent.org">blog</a></td>
<td><a href="utp.html">uTP</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/libtorrent/">download</a></td>
<td><a href="reference.html">documentation</a></td>
<td><a href="dht_store.html">DHT put extension</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/py-libtorrent/">python bindings</a></td>
<td><a href="features.html">features</a></td>
<td><a href="dht_sec.html">DHT security extension</a></td>
</tr>
<tr>
<td><a href="http://dir.gmane.org/gmane.network.bit-torrent.libtorrent">mailing list archive</a></td>
<td><a href="contributing.html">contributing</a></td>
<td><a href="streaming.html">streaming</a></td>
</tr>
<tr>
<td><a href="http://code.google.com/p/libtorrent/issues/entry">report a bug</a></td>
<td><a href="building.html">building</a></td>
<td><a href="bittorrent.pdf">bittorrent slides</a></td>
</tr>
</table>
</div>
<div id="filler"></div>
</div>
</body>
</html>

View File

@ -1,763 +0,0 @@
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.12: http://docutils.sourceforge.net/" />
<title>libtorrent Examples</title>
<meta name="author" content="Arvid Norberg, arvid&#64;libtorrent.org" />
<meta name=viewport content="width=device-width, initial-scale=1">
<script type="text/javascript">
/* <![CDATA[ */
(function() {
var s = document.createElement('script'), t = document.getElementsByTagName('script')[0];
s.type = 'text/javascript';
s.async = true;
s.src = 'http://api.flattr.com/js/0.6/load.js?mode=auto';
t.parentNode.insertBefore(s, t);
})();
/* ]]> */
</script>
<link rel="stylesheet" href="style.css" type="text/css" />
<link rel="stylesheet" type="text/css" href="rst.css" />
<style type="text/css">
/* Hides from IE-mac \*/
* html pre { height: 1%; }
/* End hide from IE-mac */
</style>
</head>
<body>
<div class="document" id="libtorrent-examples">
<div id="container">
<table id="header">
<tr><td id="orange"></td>
<td id="logo">libtorrent</td></tr>
</table>
<div id="main">
<h1 class="title">libtorrent Examples</h1>
<table class="docinfo" frame="void" rules="none">
<col class="docinfo-name" />
<col class="docinfo-content" />
<tbody valign="top">
<tr><th class="docinfo-name">Author:</th>
<td>Arvid Norberg, <a class="last reference external" href="mailto:arvid&#64;libtorrent.org">arvid&#64;libtorrent.org</a></td></tr>
<tr><th class="docinfo-name">Version:</th>
<td>1.1.0</td></tr>
</tbody>
</table>
<div class="contents topic" id="table-of-contents">
<p class="topic-title first">Table of contents</p>
<ul class="simple">
<li><a class="reference internal" href="#examples" id="id2">examples</a><ul>
<li><a class="reference internal" href="#simple-client" id="id3">simple client</a></li>
<li><a class="reference internal" href="#make-torrent" id="id4">make_torrent</a></li>
<li><a class="reference internal" href="#dump-torrent" id="id5">dump_torrent</a></li>
</ul>
</li>
</ul>
</div>
<div class="section" id="examples">
<h1>examples</h1>
<p>Except for the example programs in this manual, there's also a bigger example
of a (little bit) more complete client, <tt class="docutils literal">client_test</tt>. There are separate
instructions for how to use it <a class="reference external" href="client_test.html">here</a> if you'd like to try it. Note that building
<tt class="docutils literal">client_test</tt> also requires boost.regex and boost.program_options library.</p>
<div class="section" id="simple-client">
<h2>simple client</h2>
<p>This is a simple client. It doesn't have much output to keep it simple:</p>
<pre class="code c++ literal-block">
<span class="comment preproc">#include &lt;stdlib.h&gt;
#include &lt;boost/make_shared.hpp&gt;
#include &quot;libtorrent/entry.hpp&quot;
#include &quot;libtorrent/bencode.hpp&quot;
#include &quot;libtorrent/session.hpp&quot;
#include &quot;libtorrent/torrent_info.hpp&quot;
</span>
<span class="keyword type">int</span> <span class="name function">main</span><span class="punctuation">(</span><span class="keyword type">int</span> <span class="name">argc</span><span class="punctuation">,</span> <span class="keyword type">char</span><span class="operator">*</span> <span class="name">argv</span><span class="punctuation">[])</span>
<span class="punctuation">{</span>
<span class="keyword">using</span> <span class="keyword">namespace</span> <span class="name">libtorrent</span><span class="punctuation">;</span>
<span class="keyword">namespace</span> <span class="name">lt</span> <span class="operator">=</span> <span class="name">libtorrent</span><span class="punctuation">;</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">argc</span> <span class="operator">!=</span> <span class="literal number integer">2</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">fputs</span><span class="punctuation">(</span><span class="literal string">&quot;usage: ./simple_client torrent-file</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;to stop the client, press return.</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span><span class="punctuation">,</span> <span class="name">stderr</span><span class="punctuation">);</span>
<span class="keyword">return</span> <span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="name">settings_pack</span> <span class="name">sett</span><span class="punctuation">;</span>
<span class="name">sett</span><span class="punctuation">.</span><span class="name">set_str</span><span class="punctuation">(</span><span class="name">settings_pack</span><span class="operator">::</span><span class="name">listen_interfaces</span><span class="punctuation">,</span> <span class="literal string">&quot;0.0.0.0:6881&quot;</span><span class="punctuation">);</span>
<span class="name">lt</span><span class="operator">::</span><span class="name">session</span> <span class="name">s</span><span class="punctuation">(</span><span class="name">sett</span><span class="punctuation">);</span>
<span class="name">error_code</span> <span class="name">ec</span><span class="punctuation">;</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">ec</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">fprintf</span><span class="punctuation">(</span><span class="name">stderr</span><span class="punctuation">,</span> <span class="literal string">&quot;failed to open listen socket: %s</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span><span class="punctuation">,</span> <span class="name">ec</span><span class="punctuation">.</span><span class="name">message</span><span class="punctuation">().</span><span class="name">c_str</span><span class="punctuation">());</span>
<span class="keyword">return</span> <span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="name">add_torrent_params</span> <span class="name">p</span><span class="punctuation">;</span>
<span class="name">p</span><span class="punctuation">.</span><span class="name">save_path</span> <span class="operator">=</span> <span class="literal string">&quot;./&quot;</span><span class="punctuation">;</span>
<span class="name">p</span><span class="punctuation">.</span><span class="name">ti</span> <span class="operator">=</span> <span class="name">boost</span><span class="operator">::</span><span class="name">make_shared</span><span class="operator">&lt;</span><span class="name">torrent_info</span><span class="operator">&gt;</span><span class="punctuation">(</span><span class="name">std</span><span class="operator">::</span><span class="name">string</span><span class="punctuation">(</span><span class="name">argv</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">]),</span> <span class="name">boost</span><span class="operator">::</span><span class="name">ref</span><span class="punctuation">(</span><span class="name">ec</span><span class="punctuation">),</span> <span class="literal number integer">0</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">ec</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">fprintf</span><span class="punctuation">(</span><span class="name">stderr</span><span class="punctuation">,</span> <span class="literal string">&quot;%s</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span><span class="punctuation">,</span> <span class="name">ec</span><span class="punctuation">.</span><span class="name">message</span><span class="punctuation">().</span><span class="name">c_str</span><span class="punctuation">());</span>
<span class="keyword">return</span> <span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="name">s</span><span class="punctuation">.</span><span class="name">add_torrent</span><span class="punctuation">(</span><span class="name">p</span><span class="punctuation">,</span> <span class="name">ec</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">ec</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">fprintf</span><span class="punctuation">(</span><span class="name">stderr</span><span class="punctuation">,</span> <span class="literal string">&quot;%s</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span><span class="punctuation">,</span> <span class="name">ec</span><span class="punctuation">.</span><span class="name">message</span><span class="punctuation">().</span><span class="name">c_str</span><span class="punctuation">());</span>
<span class="keyword">return</span> <span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="comment single">// wait for the user to end
</span> <span class="keyword type">char</span> <span class="name">a</span><span class="punctuation">;</span>
<span class="name">scanf</span><span class="punctuation">(</span><span class="literal string">&quot;%c</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span><span class="punctuation">,</span> <span class="operator">&amp;</span><span class="name">a</span><span class="punctuation">);</span>
<span class="keyword">return</span> <span class="literal number integer">0</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
</pre>
</div>
<div class="section" id="make-torrent">
<h2>make_torrent</h2>
<p>Shows how to create a torrent from a directory tree:</p>
<pre class="code c++ literal-block">
<span class="comment preproc">#include &quot;libtorrent/entry.hpp&quot;
#include &quot;libtorrent/bencode.hpp&quot;
#include &quot;libtorrent/torrent_info.hpp&quot;
#include &quot;libtorrent/file.hpp&quot;
#include &quot;libtorrent/storage.hpp&quot;
#include &quot;libtorrent/hasher.hpp&quot;
#include &quot;libtorrent/create_torrent.hpp&quot;
#include &quot;libtorrent/file.hpp&quot;
#include &quot;libtorrent/file_pool.hpp&quot;
#include &quot;libtorrent/hex.hpp&quot; </span><span class="comment single">// for from_hex
</span>
<span class="comment preproc">#include &lt;boost/bind.hpp&gt;
</span>
<span class="comment preproc">#ifdef TORRENT_WINDOWS
#include &lt;direct.h&gt; </span><span class="comment single">// for _getcwd
</span><span class="comment preproc">#endif
</span>
<span class="keyword">using</span> <span class="keyword">namespace</span> <span class="name">libtorrent</span><span class="punctuation">;</span>
<span class="keyword type">int</span> <span class="name function">load_file</span><span class="punctuation">(</span><span class="name">std</span><span class="operator">::</span><span class="name">string</span> <span class="keyword">const</span><span class="operator">&amp;</span> <span class="name">filename</span><span class="punctuation">,</span> <span class="name">std</span><span class="operator">::</span><span class="name">vector</span><span class="operator">&lt;</span><span class="keyword type">char</span><span class="operator">&gt;&amp;</span> <span class="name">v</span><span class="punctuation">,</span> <span class="name">libtorrent</span><span class="operator">::</span><span class="name">error_code</span><span class="operator">&amp;</span> <span class="name">ec</span><span class="punctuation">,</span> <span class="keyword type">int</span> <span class="name">limit</span> <span class="operator">=</span> <span class="literal number integer">8000000</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">ec</span><span class="punctuation">.</span><span class="name">clear</span><span class="punctuation">();</span>
<span class="keyword type">FILE</span><span class="operator">*</span> <span class="name">f</span> <span class="operator">=</span> <span class="name">fopen</span><span class="punctuation">(</span><span class="name">filename</span><span class="punctuation">.</span><span class="name">c_str</span><span class="punctuation">(),</span> <span class="literal string">&quot;rb&quot;</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">f</span> <span class="operator">==</span> <span class="name builtin">NULL</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">ec</span><span class="punctuation">.</span><span class="name">assign</span><span class="punctuation">(</span><span class="name">errno</span><span class="punctuation">,</span> <span class="name">boost</span><span class="operator">::</span><span class="name">system</span><span class="operator">::</span><span class="name">system_category</span><span class="punctuation">());</span>
<span class="keyword">return</span> <span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="keyword type">int</span> <span class="name">r</span> <span class="operator">=</span> <span class="name">fseek</span><span class="punctuation">(</span><span class="name">f</span><span class="punctuation">,</span> <span class="literal number integer">0</span><span class="punctuation">,</span> <span class="name">SEEK_END</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">r</span> <span class="operator">!=</span> <span class="literal number integer">0</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">ec</span><span class="punctuation">.</span><span class="name">assign</span><span class="punctuation">(</span><span class="name">errno</span><span class="punctuation">,</span> <span class="name">boost</span><span class="operator">::</span><span class="name">system</span><span class="operator">::</span><span class="name">system_category</span><span class="punctuation">());</span>
<span class="name">fclose</span><span class="punctuation">(</span><span class="name">f</span><span class="punctuation">);</span>
<span class="keyword">return</span> <span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="keyword type">long</span> <span class="name">s</span> <span class="operator">=</span> <span class="name">ftell</span><span class="punctuation">(</span><span class="name">f</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">s</span> <span class="operator">&lt;</span> <span class="literal number integer">0</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">ec</span><span class="punctuation">.</span><span class="name">assign</span><span class="punctuation">(</span><span class="name">errno</span><span class="punctuation">,</span> <span class="name">boost</span><span class="operator">::</span><span class="name">system</span><span class="operator">::</span><span class="name">system_category</span><span class="punctuation">());</span>
<span class="name">fclose</span><span class="punctuation">(</span><span class="name">f</span><span class="punctuation">);</span>
<span class="keyword">return</span> <span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">s</span> <span class="operator">&gt;</span> <span class="name">limit</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">fclose</span><span class="punctuation">(</span><span class="name">f</span><span class="punctuation">);</span>
<span class="keyword">return</span> <span class="operator">-</span><span class="literal number integer">2</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="name">r</span> <span class="operator">=</span> <span class="name">fseek</span><span class="punctuation">(</span><span class="name">f</span><span class="punctuation">,</span> <span class="literal number integer">0</span><span class="punctuation">,</span> <span class="name">SEEK_SET</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">r</span> <span class="operator">!=</span> <span class="literal number integer">0</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">ec</span><span class="punctuation">.</span><span class="name">assign</span><span class="punctuation">(</span><span class="name">errno</span><span class="punctuation">,</span> <span class="name">boost</span><span class="operator">::</span><span class="name">system</span><span class="operator">::</span><span class="name">system_category</span><span class="punctuation">());</span>
<span class="name">fclose</span><span class="punctuation">(</span><span class="name">f</span><span class="punctuation">);</span>
<span class="keyword">return</span> <span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="name">v</span><span class="punctuation">.</span><span class="name">resize</span><span class="punctuation">(</span><span class="name">s</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">s</span> <span class="operator">==</span> <span class="literal number integer">0</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">fclose</span><span class="punctuation">(</span><span class="name">f</span><span class="punctuation">);</span>
<span class="keyword">return</span> <span class="literal number integer">0</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="name">r</span> <span class="operator">=</span> <span class="name">fread</span><span class="punctuation">(</span><span class="operator">&amp;</span><span class="name">v</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">],</span> <span class="literal number integer">1</span><span class="punctuation">,</span> <span class="name">v</span><span class="punctuation">.</span><span class="name">size</span><span class="punctuation">(),</span> <span class="name">f</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">r</span> <span class="operator">&lt;</span> <span class="literal number integer">0</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">ec</span><span class="punctuation">.</span><span class="name">assign</span><span class="punctuation">(</span><span class="name">errno</span><span class="punctuation">,</span> <span class="name">boost</span><span class="operator">::</span><span class="name">system</span><span class="operator">::</span><span class="name">system_category</span><span class="punctuation">());</span>
<span class="name">fclose</span><span class="punctuation">(</span><span class="name">f</span><span class="punctuation">);</span>
<span class="keyword">return</span> <span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="name">fclose</span><span class="punctuation">(</span><span class="name">f</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">r</span> <span class="operator">!=</span> <span class="name">s</span><span class="punctuation">)</span> <span class="keyword">return</span> <span class="operator">-</span><span class="literal number integer">3</span><span class="punctuation">;</span>
<span class="keyword">return</span> <span class="literal number integer">0</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="name">std</span><span class="operator">::</span><span class="name">string</span> <span class="name">branch_path</span><span class="punctuation">(</span><span class="name">std</span><span class="operator">::</span><span class="name">string</span> <span class="keyword">const</span><span class="operator">&amp;</span> <span class="name">f</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">f</span><span class="punctuation">.</span><span class="name">empty</span><span class="punctuation">())</span> <span class="keyword">return</span> <span class="name">f</span><span class="punctuation">;</span>
<span class="comment preproc">#ifdef TORRENT_WINDOWS
</span> <span class="keyword">if</span> <span class="punctuation">(</span><span class="name">f</span> <span class="operator">==</span> <span class="literal string">&quot;</span><span class="literal string escape">\\\\</span><span class="literal string">&quot;</span><span class="punctuation">)</span> <span class="keyword">return</span> <span class="literal string">&quot;&quot;</span><span class="punctuation">;</span>
<span class="comment preproc">#endif
</span> <span class="keyword">if</span> <span class="punctuation">(</span><span class="name">f</span> <span class="operator">==</span> <span class="literal string">&quot;/&quot;</span><span class="punctuation">)</span> <span class="keyword">return</span> <span class="literal string">&quot;&quot;</span><span class="punctuation">;</span>
<span class="keyword type">int</span> <span class="name">len</span> <span class="operator">=</span> <span class="name">f</span><span class="punctuation">.</span><span class="name">size</span><span class="punctuation">();</span>
<span class="comment single">// if the last character is / or \ ignore it
</span> <span class="keyword">if</span> <span class="punctuation">(</span><span class="name">f</span><span class="punctuation">[</span><span class="name">len</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span> <span class="operator">==</span> <span class="literal string char">'/'</span> <span class="operator">||</span> <span class="name">f</span><span class="punctuation">[</span><span class="name">len</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span> <span class="operator">==</span> <span class="literal string char">'\\'</span><span class="punctuation">)</span> <span class="operator">--</span><span class="name">len</span><span class="punctuation">;</span>
<span class="keyword">while</span> <span class="punctuation">(</span><span class="name">len</span> <span class="operator">&gt;</span> <span class="literal number integer">0</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="operator">--</span><span class="name">len</span><span class="punctuation">;</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">f</span><span class="punctuation">[</span><span class="name">len</span><span class="punctuation">]</span> <span class="operator">==</span> <span class="literal string char">'/'</span> <span class="operator">||</span> <span class="name">f</span><span class="punctuation">[</span><span class="name">len</span><span class="punctuation">]</span> <span class="operator">==</span> <span class="literal string char">'\\'</span><span class="punctuation">)</span>
<span class="keyword">break</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">f</span><span class="punctuation">[</span><span class="name">len</span><span class="punctuation">]</span> <span class="operator">==</span> <span class="literal string char">'/'</span> <span class="operator">||</span> <span class="name">f</span><span class="punctuation">[</span><span class="name">len</span><span class="punctuation">]</span> <span class="operator">==</span> <span class="literal string char">'\\'</span><span class="punctuation">)</span> <span class="operator">++</span><span class="name">len</span><span class="punctuation">;</span>
<span class="keyword">return</span> <span class="name">std</span><span class="operator">::</span><span class="name">string</span><span class="punctuation">(</span><span class="name">f</span><span class="punctuation">.</span><span class="name">c_str</span><span class="punctuation">(),</span> <span class="name">len</span><span class="punctuation">);</span>
<span class="punctuation">}</span>
<span class="comment single">// do not include files and folders whose
// name starts with a .
</span><span class="keyword type">bool</span> <span class="name">file_filter</span><span class="punctuation">(</span><span class="name">std</span><span class="operator">::</span><span class="name">string</span> <span class="keyword">const</span><span class="operator">&amp;</span> <span class="name">f</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">f</span><span class="punctuation">.</span><span class="name">empty</span><span class="punctuation">())</span> <span class="keyword">return</span> <span class="name builtin">false</span><span class="punctuation">;</span>
<span class="keyword type">char</span> <span class="keyword">const</span><span class="operator">*</span> <span class="name">first</span> <span class="operator">=</span> <span class="name">f</span><span class="punctuation">.</span><span class="name">c_str</span><span class="punctuation">();</span>
<span class="keyword type">char</span> <span class="keyword">const</span><span class="operator">*</span> <span class="name">sep</span> <span class="operator">=</span> <span class="name">strrchr</span><span class="punctuation">(</span><span class="name">first</span><span class="punctuation">,</span> <span class="literal string char">'/'</span><span class="punctuation">);</span>
<span class="comment preproc">#if defined(TORRENT_WINDOWS) || defined(TORRENT_OS2)
</span> <span class="keyword type">char</span> <span class="keyword">const</span><span class="operator">*</span> <span class="name">altsep</span> <span class="operator">=</span> <span class="name">strrchr</span><span class="punctuation">(</span><span class="name">first</span><span class="punctuation">,</span> <span class="literal string char">'\\'</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">sep</span> <span class="operator">==</span> <span class="name builtin">NULL</span> <span class="operator">||</span> <span class="name">altsep</span> <span class="operator">&gt;</span> <span class="name">sep</span><span class="punctuation">)</span> <span class="name">sep</span> <span class="operator">=</span> <span class="name">altsep</span><span class="punctuation">;</span>
<span class="comment preproc">#endif
</span> <span class="comment single">// if there is no parent path, just set 'sep'
</span> <span class="comment single">// to point to the filename.
</span> <span class="comment single">// if there is a parent path, skip the '/' character
</span> <span class="keyword">if</span> <span class="punctuation">(</span><span class="name">sep</span> <span class="operator">==</span> <span class="name builtin">NULL</span><span class="punctuation">)</span> <span class="name">sep</span> <span class="operator">=</span> <span class="name">first</span><span class="punctuation">;</span>
<span class="keyword">else</span> <span class="operator">++</span><span class="name">sep</span><span class="punctuation">;</span>
<span class="comment single">// return false if the first character of the filename is a .
</span> <span class="keyword">if</span> <span class="punctuation">(</span><span class="name">sep</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">]</span> <span class="operator">==</span> <span class="literal string char">'.'</span><span class="punctuation">)</span> <span class="keyword">return</span> <span class="name builtin">false</span><span class="punctuation">;</span>
<span class="name">fprintf</span><span class="punctuation">(</span><span class="name">stderr</span><span class="punctuation">,</span> <span class="literal string">&quot;%s</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span><span class="punctuation">,</span> <span class="name">f</span><span class="punctuation">.</span><span class="name">c_str</span><span class="punctuation">());</span>
<span class="keyword">return</span> <span class="name builtin">true</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="keyword type">void</span> <span class="name">print_progress</span><span class="punctuation">(</span><span class="keyword type">int</span> <span class="name">i</span><span class="punctuation">,</span> <span class="keyword type">int</span> <span class="name">num</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">fprintf</span><span class="punctuation">(</span><span class="name">stderr</span><span class="punctuation">,</span> <span class="literal string">&quot;</span><span class="literal string escape">\r</span><span class="literal string">%d/%d&quot;</span><span class="punctuation">,</span> <span class="name">i</span><span class="operator">+</span><span class="literal number integer">1</span><span class="punctuation">,</span> <span class="name">num</span><span class="punctuation">);</span>
<span class="punctuation">}</span>
<span class="keyword type">void</span> <span class="name">print_usage</span><span class="punctuation">()</span>
<span class="punctuation">{</span>
<span class="name">fputs</span><span class="punctuation">(</span><span class="literal string">&quot;usage: make_torrent FILE [OPTIONS]</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;Generates a torrent file from the specified file</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;or directory and writes it to standard out</span><span class="literal string escape">\n\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;OPTIONS:</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;-m file generate a merkle hash tree torrent.</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot; merkle torrents require client support</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot; the resulting full merkle tree is written to</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot; the specified file</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;-w url adds a web seed to the torrent with</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot; the specified url</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;-t url adds the specified tracker to the</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot; torrent. For multiple trackers, specify more</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot; -t options</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;-c comment sets the comment to the specified string</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;-C creator sets the created-by field to the specified string</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;-p bytes enables padding files. Files larger</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot; than bytes will be piece-aligned</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;-s bytes specifies a piece size for the torrent</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot; This has to be a multiple of 16 kiB</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;-l Don't follow symlinks, instead encode them as</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot; links in the torrent file</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;-o file specifies the output filename of the torrent file</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot; If this is not specified, the torrent file is</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot; printed to the standard out, except on windows</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot; where the filename defaults to a.torrent</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;-r file add root certificate to the torrent, to verify</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot; the HTTPS tracker</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;-S info-hash add a similar torrent by info-hash. The similar</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot; torrent is expected to share some files with this one</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;-L collection add a collection name to this torrent. Other torrents</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot; in the same collection is expected to share files</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot; with this one.</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;-M make the torrent compatible with mutable torrents</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot; this means aligning large files and pad them in order</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot; for piece hashes to uniquely indentify a file without</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot; overlap</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="punctuation">,</span> <span class="name">stderr</span><span class="punctuation">);</span>
<span class="punctuation">}</span>
<span class="keyword type">int</span> <span class="name">main</span><span class="punctuation">(</span><span class="keyword type">int</span> <span class="name">argc</span><span class="punctuation">,</span> <span class="keyword type">char</span><span class="operator">*</span> <span class="name">argv</span><span class="punctuation">[])</span>
<span class="punctuation">{</span>
<span class="keyword">using</span> <span class="keyword">namespace</span> <span class="name">libtorrent</span><span class="punctuation">;</span>
<span class="name">std</span><span class="operator">::</span><span class="name">string</span> <span class="name">creator_str</span> <span class="operator">=</span> <span class="literal string">&quot;libtorrent&quot;</span><span class="punctuation">;</span>
<span class="name">std</span><span class="operator">::</span><span class="name">string</span> <span class="name">comment_str</span><span class="punctuation">;</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">argc</span> <span class="operator">&lt;</span> <span class="literal number integer">2</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">print_usage</span><span class="punctuation">();</span>
<span class="keyword">return</span> <span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="comment preproc">#ifndef BOOST_NO_EXCEPTIONS
</span> <span class="name">try</span>
<span class="punctuation">{</span>
<span class="comment preproc">#endif
</span> <span class="name">std</span><span class="operator">::</span><span class="name">vector</span><span class="operator">&lt;</span><span class="name">std</span><span class="operator">::</span><span class="name">string</span><span class="operator">&gt;</span> <span class="name">web_seeds</span><span class="punctuation">;</span>
<span class="name">std</span><span class="operator">::</span><span class="name">vector</span><span class="operator">&lt;</span><span class="name">std</span><span class="operator">::</span><span class="name">string</span><span class="operator">&gt;</span> <span class="name">trackers</span><span class="punctuation">;</span>
<span class="name">std</span><span class="operator">::</span><span class="name">vector</span><span class="operator">&lt;</span><span class="name">std</span><span class="operator">::</span><span class="name">string</span><span class="operator">&gt;</span> <span class="name">collections</span><span class="punctuation">;</span>
<span class="name">std</span><span class="operator">::</span><span class="name">vector</span><span class="operator">&lt;</span><span class="name">sha1_hash</span><span class="operator">&gt;</span> <span class="name">similar</span><span class="punctuation">;</span>
<span class="keyword type">int</span> <span class="name">pad_file_limit</span> <span class="operator">=</span> <span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="keyword type">int</span> <span class="name">piece_size</span> <span class="operator">=</span> <span class="literal number integer">0</span><span class="punctuation">;</span>
<span class="keyword type">int</span> <span class="name">flags</span> <span class="operator">=</span> <span class="literal number integer">0</span><span class="punctuation">;</span>
<span class="name">std</span><span class="operator">::</span><span class="name">string</span> <span class="name">root_cert</span><span class="punctuation">;</span>
<span class="name">std</span><span class="operator">::</span><span class="name">string</span> <span class="name">outfile</span><span class="punctuation">;</span>
<span class="name">std</span><span class="operator">::</span><span class="name">string</span> <span class="name">merklefile</span><span class="punctuation">;</span>
<span class="comment preproc">#ifdef TORRENT_WINDOWS
</span> <span class="comment single">// don't ever write binary data to the console on windows
</span> <span class="comment single">// it will just be interpreted as text and corrupted
</span> <span class="name">outfile</span> <span class="operator">=</span> <span class="literal string">&quot;a.torrent&quot;</span><span class="punctuation">;</span>
<span class="comment preproc">#endif
</span>
<span class="keyword">for</span> <span class="punctuation">(</span><span class="keyword type">int</span> <span class="name">i</span> <span class="operator">=</span> <span class="literal number integer">2</span><span class="punctuation">;</span> <span class="name">i</span> <span class="operator">&lt;</span> <span class="name">argc</span><span class="punctuation">;</span> <span class="operator">++</span><span class="name">i</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">argv</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">][</span><span class="literal number integer">0</span><span class="punctuation">]</span> <span class="operator">!=</span> <span class="literal string char">'-'</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">print_usage</span><span class="punctuation">();</span>
<span class="keyword">return</span> <span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="keyword">switch</span> <span class="punctuation">(</span><span class="name">argv</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">][</span><span class="literal number integer">1</span><span class="punctuation">])</span>
<span class="punctuation">{</span>
<span class="keyword">case</span> <span class="literal string char">'w'</span><span class="operator">:</span>
<span class="operator">++</span><span class="name">i</span><span class="punctuation">;</span>
<span class="name">web_seeds</span><span class="punctuation">.</span><span class="name">push_back</span><span class="punctuation">(</span><span class="name">argv</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">]);</span>
<span class="keyword">break</span><span class="punctuation">;</span>
<span class="keyword">case</span> <span class="literal string char">'t'</span><span class="operator">:</span>
<span class="operator">++</span><span class="name">i</span><span class="punctuation">;</span>
<span class="name">trackers</span><span class="punctuation">.</span><span class="name">push_back</span><span class="punctuation">(</span><span class="name">argv</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">]);</span>
<span class="keyword">break</span><span class="punctuation">;</span>
<span class="keyword">case</span> <span class="literal string char">'M'</span><span class="operator">:</span>
<span class="name">flags</span> <span class="operator">|=</span> <span class="name">create_torrent</span><span class="operator">::</span><span class="name">mutable_torrent_support</span><span class="punctuation">;</span>
<span class="name">pad_file_limit</span> <span class="operator">=</span> <span class="literal number hex">0x4000</span><span class="punctuation">;</span>
<span class="keyword">break</span><span class="punctuation">;</span>
<span class="keyword">case</span> <span class="literal string char">'p'</span><span class="operator">:</span>
<span class="operator">++</span><span class="name">i</span><span class="punctuation">;</span>
<span class="name">pad_file_limit</span> <span class="operator">=</span> <span class="name">atoi</span><span class="punctuation">(</span><span class="name">argv</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">]);</span>
<span class="name">flags</span> <span class="operator">|=</span> <span class="name">create_torrent</span><span class="operator">::</span><span class="name">optimize</span><span class="punctuation">;</span>
<span class="keyword">break</span><span class="punctuation">;</span>
<span class="keyword">case</span> <span class="literal string char">'s'</span><span class="operator">:</span>
<span class="operator">++</span><span class="name">i</span><span class="punctuation">;</span>
<span class="name">piece_size</span> <span class="operator">=</span> <span class="name">atoi</span><span class="punctuation">(</span><span class="name">argv</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">]);</span>
<span class="keyword">break</span><span class="punctuation">;</span>
<span class="keyword">case</span> <span class="literal string char">'m'</span><span class="operator">:</span>
<span class="operator">++</span><span class="name">i</span><span class="punctuation">;</span>
<span class="name">merklefile</span> <span class="operator">=</span> <span class="name">argv</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">];</span>
<span class="name">flags</span> <span class="operator">|=</span> <span class="name">create_torrent</span><span class="operator">::</span><span class="name">merkle</span><span class="punctuation">;</span>
<span class="keyword">break</span><span class="punctuation">;</span>
<span class="keyword">case</span> <span class="literal string char">'o'</span><span class="operator">:</span>
<span class="operator">++</span><span class="name">i</span><span class="punctuation">;</span>
<span class="name">outfile</span> <span class="operator">=</span> <span class="name">argv</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">];</span>
<span class="keyword">break</span><span class="punctuation">;</span>
<span class="keyword">case</span> <span class="literal string char">'l'</span><span class="operator">:</span>
<span class="name">flags</span> <span class="operator">|=</span> <span class="name">create_torrent</span><span class="operator">::</span><span class="name">symlinks</span><span class="punctuation">;</span>
<span class="keyword">break</span><span class="punctuation">;</span>
<span class="keyword">case</span> <span class="literal string char">'C'</span><span class="operator">:</span>
<span class="operator">++</span><span class="name">i</span><span class="punctuation">;</span>
<span class="name">creator_str</span> <span class="operator">=</span> <span class="name">argv</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">];</span>
<span class="keyword">break</span><span class="punctuation">;</span>
<span class="keyword">case</span> <span class="literal string char">'c'</span><span class="operator">:</span>
<span class="operator">++</span><span class="name">i</span><span class="punctuation">;</span>
<span class="name">comment_str</span> <span class="operator">=</span> <span class="name">argv</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">];</span>
<span class="keyword">break</span><span class="punctuation">;</span>
<span class="keyword">case</span> <span class="literal string char">'r'</span><span class="operator">:</span>
<span class="operator">++</span><span class="name">i</span><span class="punctuation">;</span>
<span class="name">root_cert</span> <span class="operator">=</span> <span class="name">argv</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">];</span>
<span class="keyword">break</span><span class="punctuation">;</span>
<span class="keyword">case</span> <span class="literal string char">'S'</span><span class="operator">:</span>
<span class="punctuation">{</span>
<span class="operator">++</span><span class="name">i</span><span class="punctuation">;</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">strlen</span><span class="punctuation">(</span><span class="name">argv</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">])</span> <span class="operator">!=</span> <span class="literal number integer">40</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">fprintf</span><span class="punctuation">(</span><span class="name">stderr</span><span class="punctuation">,</span> <span class="literal string">&quot;invalid info-hash for -S. &quot;</span>
<span class="literal string">&quot;Expected 40 hex characters</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span><span class="punctuation">);</span>
<span class="name">print_usage</span><span class="punctuation">();</span>
<span class="keyword">return</span> <span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="name">sha1_hash</span> <span class="name">ih</span><span class="punctuation">;</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="operator">!</span><span class="name">from_hex</span><span class="punctuation">(</span><span class="name">argv</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">],</span> <span class="literal number integer">40</span><span class="punctuation">,</span> <span class="punctuation">(</span><span class="keyword type">char</span><span class="operator">*</span><span class="punctuation">)</span><span class="operator">&amp;</span><span class="name">ih</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">]))</span>
<span class="punctuation">{</span>
<span class="name">fprintf</span><span class="punctuation">(</span><span class="name">stderr</span><span class="punctuation">,</span> <span class="literal string">&quot;invalid info-hash for -S</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span><span class="punctuation">);</span>
<span class="name">print_usage</span><span class="punctuation">();</span>
<span class="keyword">return</span> <span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="name">similar</span><span class="punctuation">.</span><span class="name">push_back</span><span class="punctuation">(</span><span class="name">ih</span><span class="punctuation">);</span>
<span class="punctuation">}</span>
<span class="keyword">break</span><span class="punctuation">;</span>
<span class="keyword">case</span> <span class="literal string char">'L'</span><span class="operator">:</span>
<span class="operator">++</span><span class="name">i</span><span class="punctuation">;</span>
<span class="name">collections</span><span class="punctuation">.</span><span class="name">push_back</span><span class="punctuation">(</span><span class="name">argv</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">]);</span>
<span class="keyword">break</span><span class="punctuation">;</span>
<span class="keyword">default</span><span class="operator">:</span>
<span class="name">print_usage</span><span class="punctuation">();</span>
<span class="keyword">return</span> <span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="punctuation">}</span>
<span class="name">file_storage</span> <span class="name">fs</span><span class="punctuation">;</span>
<span class="name">std</span><span class="operator">::</span><span class="name">string</span> <span class="name">full_path</span> <span class="operator">=</span> <span class="name">argv</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">];</span>
<span class="comment preproc">#ifdef TORRENT_WINDOWS
</span> <span class="keyword">if</span> <span class="punctuation">(</span><span class="name">full_path</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">]</span> <span class="operator">!=</span> <span class="literal string char">':'</span><span class="punctuation">)</span>
<span class="comment preproc">#else
</span> <span class="keyword">if</span> <span class="punctuation">(</span><span class="name">full_path</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">]</span> <span class="operator">!=</span> <span class="literal string char">'/'</span><span class="punctuation">)</span>
<span class="comment preproc">#endif
</span> <span class="punctuation">{</span>
<span class="keyword type">char</span> <span class="name">cwd</span><span class="punctuation">[</span><span class="name">TORRENT_MAX_PATH</span><span class="punctuation">];</span>
<span class="comment preproc">#ifdef TORRENT_WINDOWS
</span> <span class="name">_getcwd</span><span class="punctuation">(</span><span class="name">cwd</span><span class="punctuation">,</span> <span class="keyword">sizeof</span><span class="punctuation">(</span><span class="name">cwd</span><span class="punctuation">));</span>
<span class="name">full_path</span> <span class="operator">=</span> <span class="name">cwd</span> <span class="operator">+</span> <span class="punctuation">(</span><span class="literal string">&quot;</span><span class="literal string escape">\\</span><span class="literal string">&quot;</span> <span class="operator">+</span> <span class="name">full_path</span><span class="punctuation">);</span>
<span class="comment preproc">#else
</span> <span class="name">getcwd</span><span class="punctuation">(</span><span class="name">cwd</span><span class="punctuation">,</span> <span class="keyword">sizeof</span><span class="punctuation">(</span><span class="name">cwd</span><span class="punctuation">));</span>
<span class="name">full_path</span> <span class="operator">=</span> <span class="name">cwd</span> <span class="operator">+</span> <span class="punctuation">(</span><span class="literal string">&quot;/&quot;</span> <span class="operator">+</span> <span class="name">full_path</span><span class="punctuation">);</span>
<span class="comment preproc">#endif
</span> <span class="punctuation">}</span>
<span class="name">add_files</span><span class="punctuation">(</span><span class="name">fs</span><span class="punctuation">,</span> <span class="name">full_path</span><span class="punctuation">,</span> <span class="name">file_filter</span><span class="punctuation">,</span> <span class="name">flags</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">fs</span><span class="punctuation">.</span><span class="name">num_files</span><span class="punctuation">()</span> <span class="operator">==</span> <span class="literal number integer">0</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">fputs</span><span class="punctuation">(</span><span class="literal string">&quot;no files specified.</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span><span class="punctuation">,</span> <span class="name">stderr</span><span class="punctuation">);</span>
<span class="keyword">return</span> <span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="name">create_torrent</span> <span class="name">t</span><span class="punctuation">(</span><span class="name">fs</span><span class="punctuation">,</span> <span class="name">piece_size</span><span class="punctuation">,</span> <span class="name">pad_file_limit</span><span class="punctuation">,</span> <span class="name">flags</span><span class="punctuation">);</span>
<span class="keyword type">int</span> <span class="name">tier</span> <span class="operator">=</span> <span class="literal number integer">0</span><span class="punctuation">;</span>
<span class="keyword">for</span> <span class="punctuation">(</span><span class="name">std</span><span class="operator">::</span><span class="name">vector</span><span class="operator">&lt;</span><span class="name">std</span><span class="operator">::</span><span class="name">string</span><span class="operator">&gt;::</span><span class="name">iterator</span> <span class="name">i</span> <span class="operator">=</span> <span class="name">trackers</span><span class="punctuation">.</span><span class="name">begin</span><span class="punctuation">()</span>
<span class="punctuation">,</span> <span class="name">end</span><span class="punctuation">(</span><span class="name">trackers</span><span class="punctuation">.</span><span class="name">end</span><span class="punctuation">());</span> <span class="name">i</span> <span class="operator">!=</span> <span class="name">end</span><span class="punctuation">;</span> <span class="operator">++</span><span class="name">i</span><span class="punctuation">,</span> <span class="operator">++</span><span class="name">tier</span><span class="punctuation">)</span>
<span class="name">t</span><span class="punctuation">.</span><span class="name">add_tracker</span><span class="punctuation">(</span><span class="operator">*</span><span class="name">i</span><span class="punctuation">,</span> <span class="name">tier</span><span class="punctuation">);</span>
<span class="keyword">for</span> <span class="punctuation">(</span><span class="name">std</span><span class="operator">::</span><span class="name">vector</span><span class="operator">&lt;</span><span class="name">std</span><span class="operator">::</span><span class="name">string</span><span class="operator">&gt;::</span><span class="name">iterator</span> <span class="name">i</span> <span class="operator">=</span> <span class="name">web_seeds</span><span class="punctuation">.</span><span class="name">begin</span><span class="punctuation">()</span>
<span class="punctuation">,</span> <span class="name">end</span><span class="punctuation">(</span><span class="name">web_seeds</span><span class="punctuation">.</span><span class="name">end</span><span class="punctuation">());</span> <span class="name">i</span> <span class="operator">!=</span> <span class="name">end</span><span class="punctuation">;</span> <span class="operator">++</span><span class="name">i</span><span class="punctuation">)</span>
<span class="name">t</span><span class="punctuation">.</span><span class="name">add_url_seed</span><span class="punctuation">(</span><span class="operator">*</span><span class="name">i</span><span class="punctuation">);</span>
<span class="keyword">for</span> <span class="punctuation">(</span><span class="name">std</span><span class="operator">::</span><span class="name">vector</span><span class="operator">&lt;</span><span class="name">std</span><span class="operator">::</span><span class="name">string</span><span class="operator">&gt;::</span><span class="name">iterator</span> <span class="name">i</span> <span class="operator">=</span> <span class="name">collections</span><span class="punctuation">.</span><span class="name">begin</span><span class="punctuation">()</span>
<span class="punctuation">,</span> <span class="name">end</span><span class="punctuation">(</span><span class="name">collections</span><span class="punctuation">.</span><span class="name">end</span><span class="punctuation">());</span> <span class="name">i</span> <span class="operator">!=</span> <span class="name">end</span><span class="punctuation">;</span> <span class="operator">++</span><span class="name">i</span><span class="punctuation">)</span>
<span class="name">t</span><span class="punctuation">.</span><span class="name">add_collection</span><span class="punctuation">(</span><span class="operator">*</span><span class="name">i</span><span class="punctuation">);</span>
<span class="keyword">for</span> <span class="punctuation">(</span><span class="name">std</span><span class="operator">::</span><span class="name">vector</span><span class="operator">&lt;</span><span class="name">sha1_hash</span><span class="operator">&gt;::</span><span class="name">iterator</span> <span class="name">i</span> <span class="operator">=</span> <span class="name">similar</span><span class="punctuation">.</span><span class="name">begin</span><span class="punctuation">()</span>
<span class="punctuation">,</span> <span class="name">end</span><span class="punctuation">(</span><span class="name">similar</span><span class="punctuation">.</span><span class="name">end</span><span class="punctuation">());</span> <span class="name">i</span> <span class="operator">!=</span> <span class="name">end</span><span class="punctuation">;</span> <span class="operator">++</span><span class="name">i</span><span class="punctuation">)</span>
<span class="name">t</span><span class="punctuation">.</span><span class="name">add_similar_torrent</span><span class="punctuation">(</span><span class="operator">*</span><span class="name">i</span><span class="punctuation">);</span>
<span class="name">error_code</span> <span class="name">ec</span><span class="punctuation">;</span>
<span class="name">set_piece_hashes</span><span class="punctuation">(</span><span class="name">t</span><span class="punctuation">,</span> <span class="name">branch_path</span><span class="punctuation">(</span><span class="name">full_path</span><span class="punctuation">)</span>
<span class="punctuation">,</span> <span class="name">boost</span><span class="operator">::</span><span class="name">bind</span><span class="punctuation">(</span><span class="operator">&amp;</span><span class="name">print_progress</span><span class="punctuation">,</span> <span class="name">_1</span><span class="punctuation">,</span> <span class="name">t</span><span class="punctuation">.</span><span class="name">num_pieces</span><span class="punctuation">()),</span> <span class="name">ec</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">ec</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">fprintf</span><span class="punctuation">(</span><span class="name">stderr</span><span class="punctuation">,</span> <span class="literal string">&quot;%s</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span><span class="punctuation">,</span> <span class="name">ec</span><span class="punctuation">.</span><span class="name">message</span><span class="punctuation">().</span><span class="name">c_str</span><span class="punctuation">());</span>
<span class="keyword">return</span> <span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="name">fprintf</span><span class="punctuation">(</span><span class="name">stderr</span><span class="punctuation">,</span> <span class="literal string">&quot;</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span><span class="punctuation">);</span>
<span class="name">t</span><span class="punctuation">.</span><span class="name">set_creator</span><span class="punctuation">(</span><span class="name">creator_str</span><span class="punctuation">.</span><span class="name">c_str</span><span class="punctuation">());</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="operator">!</span><span class="name">comment_str</span><span class="punctuation">.</span><span class="name">empty</span><span class="punctuation">())</span>
<span class="name">t</span><span class="punctuation">.</span><span class="name">set_comment</span><span class="punctuation">(</span><span class="name">comment_str</span><span class="punctuation">.</span><span class="name">c_str</span><span class="punctuation">());</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="operator">!</span><span class="name">root_cert</span><span class="punctuation">.</span><span class="name">empty</span><span class="punctuation">())</span>
<span class="punctuation">{</span>
<span class="name">std</span><span class="operator">::</span><span class="name">vector</span><span class="operator">&lt;</span><span class="keyword type">char</span><span class="operator">&gt;</span> <span class="name">pem</span><span class="punctuation">;</span>
<span class="name">load_file</span><span class="punctuation">(</span><span class="name">root_cert</span><span class="punctuation">,</span> <span class="name">pem</span><span class="punctuation">,</span> <span class="name">ec</span><span class="punctuation">,</span> <span class="literal number integer">10000</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">ec</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">fprintf</span><span class="punctuation">(</span><span class="name">stderr</span><span class="punctuation">,</span> <span class="literal string">&quot;failed to load root certificate for tracker: %s</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span><span class="punctuation">,</span> <span class="name">ec</span><span class="punctuation">.</span><span class="name">message</span><span class="punctuation">().</span><span class="name">c_str</span><span class="punctuation">());</span>
<span class="punctuation">}</span>
<span class="keyword">else</span>
<span class="punctuation">{</span>
<span class="name">t</span><span class="punctuation">.</span><span class="name">set_root_cert</span><span class="punctuation">(</span><span class="name">std</span><span class="operator">::</span><span class="name">string</span><span class="punctuation">(</span><span class="operator">&amp;</span><span class="name">pem</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">],</span> <span class="name">pem</span><span class="punctuation">.</span><span class="name">size</span><span class="punctuation">()));</span>
<span class="punctuation">}</span>
<span class="punctuation">}</span>
<span class="comment single">// create the torrent and print it to stdout
</span> <span class="name">std</span><span class="operator">::</span><span class="name">vector</span><span class="operator">&lt;</span><span class="keyword type">char</span><span class="operator">&gt;</span> <span class="name">torrent</span><span class="punctuation">;</span>
<span class="name">bencode</span><span class="punctuation">(</span><span class="name">back_inserter</span><span class="punctuation">(</span><span class="name">torrent</span><span class="punctuation">),</span> <span class="name">t</span><span class="punctuation">.</span><span class="name">generate</span><span class="punctuation">());</span>
<span class="keyword type">FILE</span><span class="operator">*</span> <span class="name">output</span> <span class="operator">=</span> <span class="name">stdout</span><span class="punctuation">;</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="operator">!</span><span class="name">outfile</span><span class="punctuation">.</span><span class="name">empty</span><span class="punctuation">())</span>
<span class="name">output</span> <span class="operator">=</span> <span class="name">fopen</span><span class="punctuation">(</span><span class="name">outfile</span><span class="punctuation">.</span><span class="name">c_str</span><span class="punctuation">(),</span> <span class="literal string">&quot;wb+&quot;</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">output</span> <span class="operator">==</span> <span class="name builtin">NULL</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">fprintf</span><span class="punctuation">(</span><span class="name">stderr</span><span class="punctuation">,</span> <span class="literal string">&quot;failed to open file </span><span class="literal string escape">\&quot;</span><span class="literal string">%s</span><span class="literal string escape">\&quot;</span><span class="literal string">: (%d) %s</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="punctuation">,</span> <span class="name">outfile</span><span class="punctuation">.</span><span class="name">c_str</span><span class="punctuation">(),</span> <span class="name">errno</span><span class="punctuation">,</span> <span class="name">strerror</span><span class="punctuation">(</span><span class="name">errno</span><span class="punctuation">));</span>
<span class="keyword">return</span> <span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="name">fwrite</span><span class="punctuation">(</span><span class="operator">&amp;</span><span class="name">torrent</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">],</span> <span class="literal number integer">1</span><span class="punctuation">,</span> <span class="name">torrent</span><span class="punctuation">.</span><span class="name">size</span><span class="punctuation">(),</span> <span class="name">output</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">output</span> <span class="operator">!=</span> <span class="name">stdout</span><span class="punctuation">)</span>
<span class="name">fclose</span><span class="punctuation">(</span><span class="name">output</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="operator">!</span><span class="name">merklefile</span><span class="punctuation">.</span><span class="name">empty</span><span class="punctuation">())</span>
<span class="punctuation">{</span>
<span class="name">output</span> <span class="operator">=</span> <span class="name">fopen</span><span class="punctuation">(</span><span class="name">merklefile</span><span class="punctuation">.</span><span class="name">c_str</span><span class="punctuation">(),</span> <span class="literal string">&quot;wb+&quot;</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">output</span> <span class="operator">==</span> <span class="name builtin">NULL</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">fprintf</span><span class="punctuation">(</span><span class="name">stderr</span><span class="punctuation">,</span> <span class="literal string">&quot;failed to open file </span><span class="literal string escape">\&quot;</span><span class="literal string">%s</span><span class="literal string escape">\&quot;</span><span class="literal string">: (%d) %s</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="punctuation">,</span> <span class="name">merklefile</span><span class="punctuation">.</span><span class="name">c_str</span><span class="punctuation">(),</span> <span class="name">errno</span><span class="punctuation">,</span> <span class="name">strerror</span><span class="punctuation">(</span><span class="name">errno</span><span class="punctuation">));</span>
<span class="keyword">return</span> <span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="keyword type">int</span> <span class="name">ret</span> <span class="operator">=</span> <span class="name">fwrite</span><span class="punctuation">(</span><span class="operator">&amp;</span><span class="name">t</span><span class="punctuation">.</span><span class="name">merkle_tree</span><span class="punctuation">()[</span><span class="literal number integer">0</span><span class="punctuation">],</span> <span class="literal number integer">20</span><span class="punctuation">,</span> <span class="name">t</span><span class="punctuation">.</span><span class="name">merkle_tree</span><span class="punctuation">().</span><span class="name">size</span><span class="punctuation">(),</span> <span class="name">output</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">ret</span> <span class="operator">!=</span> <span class="keyword type">int</span><span class="punctuation">(</span><span class="name">t</span><span class="punctuation">.</span><span class="name">merkle_tree</span><span class="punctuation">().</span><span class="name">size</span><span class="punctuation">()))</span>
<span class="punctuation">{</span>
<span class="name">fprintf</span><span class="punctuation">(</span><span class="name">stderr</span><span class="punctuation">,</span> <span class="literal string">&quot;failed to write %s: (%d) %s</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="punctuation">,</span> <span class="name">merklefile</span><span class="punctuation">.</span><span class="name">c_str</span><span class="punctuation">(),</span> <span class="name">errno</span><span class="punctuation">,</span> <span class="name">strerror</span><span class="punctuation">(</span><span class="name">errno</span><span class="punctuation">));</span>
<span class="punctuation">}</span>
<span class="name">fclose</span><span class="punctuation">(</span><span class="name">output</span><span class="punctuation">);</span>
<span class="punctuation">}</span>
<span class="comment preproc">#ifndef BOOST_NO_EXCEPTIONS
</span> <span class="punctuation">}</span>
<span class="keyword">catch</span> <span class="punctuation">(</span><span class="name">std</span><span class="operator">::</span><span class="name">exception</span><span class="operator">&amp;</span> <span class="name">e</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">fprintf</span><span class="punctuation">(</span><span class="name">stderr</span><span class="punctuation">,</span> <span class="literal string">&quot;%s</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span><span class="punctuation">,</span> <span class="name">e</span><span class="punctuation">.</span><span class="name">what</span><span class="punctuation">());</span>
<span class="punctuation">}</span>
<span class="comment preproc">#endif
</span>
<span class="keyword">return</span> <span class="literal number integer">0</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
</pre>
</div>
<div class="section" id="dump-torrent">
<h2>dump_torrent</h2>
<p>This is an example of a program that will take a torrent-file as a parameter and
print information about it to std out:</p>
<pre class="code c++ literal-block">
<span class="comment preproc">#include &quot;libtorrent/entry.hpp&quot;
#include &quot;libtorrent/bencode.hpp&quot;
#include &quot;libtorrent/torrent_info.hpp&quot;
#include &quot;libtorrent/announce_entry.hpp&quot;
#include &quot;libtorrent/bdecode.hpp&quot;
#include &quot;libtorrent/magnet_uri.hpp&quot;
</span>
<span class="keyword type">int</span> <span class="name function">load_file</span><span class="punctuation">(</span><span class="name">std</span><span class="operator">::</span><span class="name">string</span> <span class="keyword">const</span><span class="operator">&amp;</span> <span class="name">filename</span><span class="punctuation">,</span> <span class="name">std</span><span class="operator">::</span><span class="name">vector</span><span class="operator">&lt;</span><span class="keyword type">char</span><span class="operator">&gt;&amp;</span> <span class="name">v</span>
<span class="punctuation">,</span> <span class="name">libtorrent</span><span class="operator">::</span><span class="name">error_code</span><span class="operator">&amp;</span> <span class="name">ec</span><span class="punctuation">,</span> <span class="keyword type">int</span> <span class="name">limit</span> <span class="operator">=</span> <span class="literal number integer">8000000</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">ec</span><span class="punctuation">.</span><span class="name">clear</span><span class="punctuation">();</span>
<span class="keyword type">FILE</span><span class="operator">*</span> <span class="name">f</span> <span class="operator">=</span> <span class="name">fopen</span><span class="punctuation">(</span><span class="name">filename</span><span class="punctuation">.</span><span class="name">c_str</span><span class="punctuation">(),</span> <span class="literal string">&quot;rb&quot;</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">f</span> <span class="operator">==</span> <span class="name builtin">NULL</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">ec</span><span class="punctuation">.</span><span class="name">assign</span><span class="punctuation">(</span><span class="name">errno</span><span class="punctuation">,</span> <span class="name">boost</span><span class="operator">::</span><span class="name">system</span><span class="operator">::</span><span class="name">system_category</span><span class="punctuation">());</span>
<span class="keyword">return</span> <span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="keyword type">int</span> <span class="name">r</span> <span class="operator">=</span> <span class="name">fseek</span><span class="punctuation">(</span><span class="name">f</span><span class="punctuation">,</span> <span class="literal number integer">0</span><span class="punctuation">,</span> <span class="name">SEEK_END</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">r</span> <span class="operator">!=</span> <span class="literal number integer">0</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">ec</span><span class="punctuation">.</span><span class="name">assign</span><span class="punctuation">(</span><span class="name">errno</span><span class="punctuation">,</span> <span class="name">boost</span><span class="operator">::</span><span class="name">system</span><span class="operator">::</span><span class="name">system_category</span><span class="punctuation">());</span>
<span class="name">fclose</span><span class="punctuation">(</span><span class="name">f</span><span class="punctuation">);</span>
<span class="keyword">return</span> <span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="keyword type">long</span> <span class="name">s</span> <span class="operator">=</span> <span class="name">ftell</span><span class="punctuation">(</span><span class="name">f</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">s</span> <span class="operator">&lt;</span> <span class="literal number integer">0</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">ec</span><span class="punctuation">.</span><span class="name">assign</span><span class="punctuation">(</span><span class="name">errno</span><span class="punctuation">,</span> <span class="name">boost</span><span class="operator">::</span><span class="name">system</span><span class="operator">::</span><span class="name">system_category</span><span class="punctuation">());</span>
<span class="name">fclose</span><span class="punctuation">(</span><span class="name">f</span><span class="punctuation">);</span>
<span class="keyword">return</span> <span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">s</span> <span class="operator">&gt;</span> <span class="name">limit</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">fclose</span><span class="punctuation">(</span><span class="name">f</span><span class="punctuation">);</span>
<span class="keyword">return</span> <span class="operator">-</span><span class="literal number integer">2</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="name">r</span> <span class="operator">=</span> <span class="name">fseek</span><span class="punctuation">(</span><span class="name">f</span><span class="punctuation">,</span> <span class="literal number integer">0</span><span class="punctuation">,</span> <span class="name">SEEK_SET</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">r</span> <span class="operator">!=</span> <span class="literal number integer">0</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">ec</span><span class="punctuation">.</span><span class="name">assign</span><span class="punctuation">(</span><span class="name">errno</span><span class="punctuation">,</span> <span class="name">boost</span><span class="operator">::</span><span class="name">system</span><span class="operator">::</span><span class="name">system_category</span><span class="punctuation">());</span>
<span class="name">fclose</span><span class="punctuation">(</span><span class="name">f</span><span class="punctuation">);</span>
<span class="keyword">return</span> <span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="name">v</span><span class="punctuation">.</span><span class="name">resize</span><span class="punctuation">(</span><span class="name">s</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">s</span> <span class="operator">==</span> <span class="literal number integer">0</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">fclose</span><span class="punctuation">(</span><span class="name">f</span><span class="punctuation">);</span>
<span class="keyword">return</span> <span class="literal number integer">0</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="name">r</span> <span class="operator">=</span> <span class="name">fread</span><span class="punctuation">(</span><span class="operator">&amp;</span><span class="name">v</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">],</span> <span class="literal number integer">1</span><span class="punctuation">,</span> <span class="name">v</span><span class="punctuation">.</span><span class="name">size</span><span class="punctuation">(),</span> <span class="name">f</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">r</span> <span class="operator">&lt;</span> <span class="literal number integer">0</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">ec</span><span class="punctuation">.</span><span class="name">assign</span><span class="punctuation">(</span><span class="name">errno</span><span class="punctuation">,</span> <span class="name">boost</span><span class="operator">::</span><span class="name">system</span><span class="operator">::</span><span class="name">system_category</span><span class="punctuation">());</span>
<span class="name">fclose</span><span class="punctuation">(</span><span class="name">f</span><span class="punctuation">);</span>
<span class="keyword">return</span> <span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="name">fclose</span><span class="punctuation">(</span><span class="name">f</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">r</span> <span class="operator">!=</span> <span class="name">s</span><span class="punctuation">)</span> <span class="keyword">return</span> <span class="operator">-</span><span class="literal number integer">3</span><span class="punctuation">;</span>
<span class="keyword">return</span> <span class="literal number integer">0</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="keyword type">int</span> <span class="name function">main</span><span class="punctuation">(</span><span class="keyword type">int</span> <span class="name">argc</span><span class="punctuation">,</span> <span class="keyword type">char</span><span class="operator">*</span> <span class="name">argv</span><span class="punctuation">[])</span>
<span class="punctuation">{</span>
<span class="keyword">using</span> <span class="keyword">namespace</span> <span class="name">libtorrent</span><span class="punctuation">;</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">argc</span> <span class="operator">&lt;</span> <span class="literal number integer">2</span> <span class="operator">||</span> <span class="name">argc</span> <span class="operator">&gt;</span> <span class="literal number integer">4</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">fputs</span><span class="punctuation">(</span><span class="literal string">&quot;usage: dump_torrent torrent-file [total-items-limit] [recursion-limit]</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span><span class="punctuation">,</span> <span class="name">stderr</span><span class="punctuation">);</span>
<span class="keyword">return</span> <span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="keyword type">int</span> <span class="name">item_limit</span> <span class="operator">=</span> <span class="literal number integer">1000000</span><span class="punctuation">;</span>
<span class="keyword type">int</span> <span class="name">depth_limit</span> <span class="operator">=</span> <span class="literal number integer">1000</span><span class="punctuation">;</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">argc</span> <span class="operator">&gt;</span> <span class="literal number integer">2</span><span class="punctuation">)</span> <span class="name">item_limit</span> <span class="operator">=</span> <span class="name">atoi</span><span class="punctuation">(</span><span class="name">argv</span><span class="punctuation">[</span><span class="literal number integer">2</span><span class="punctuation">]);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">argc</span> <span class="operator">&gt;</span> <span class="literal number integer">3</span><span class="punctuation">)</span> <span class="name">depth_limit</span> <span class="operator">=</span> <span class="name">atoi</span><span class="punctuation">(</span><span class="name">argv</span><span class="punctuation">[</span><span class="literal number integer">3</span><span class="punctuation">]);</span>
<span class="name">std</span><span class="operator">::</span><span class="name">vector</span><span class="operator">&lt;</span><span class="keyword type">char</span><span class="operator">&gt;</span> <span class="name">buf</span><span class="punctuation">;</span>
<span class="name">error_code</span> <span class="name">ec</span><span class="punctuation">;</span>
<span class="keyword type">int</span> <span class="name">ret</span> <span class="operator">=</span> <span class="name">load_file</span><span class="punctuation">(</span><span class="name">argv</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">],</span> <span class="name">buf</span><span class="punctuation">,</span> <span class="name">ec</span><span class="punctuation">,</span> <span class="literal number integer">40</span> <span class="operator">*</span> <span class="literal number integer">1000000</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">ret</span> <span class="operator">==</span> <span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">fprintf</span><span class="punctuation">(</span><span class="name">stderr</span><span class="punctuation">,</span> <span class="literal string">&quot;file too big, aborting</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span><span class="punctuation">);</span>
<span class="keyword">return</span> <span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">ret</span> <span class="operator">!=</span> <span class="literal number integer">0</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">fprintf</span><span class="punctuation">(</span><span class="name">stderr</span><span class="punctuation">,</span> <span class="literal string">&quot;failed to load file: %s</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span><span class="punctuation">,</span> <span class="name">ec</span><span class="punctuation">.</span><span class="name">message</span><span class="punctuation">().</span><span class="name">c_str</span><span class="punctuation">());</span>
<span class="keyword">return</span> <span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="name">bdecode_node</span> <span class="name">e</span><span class="punctuation">;</span>
<span class="keyword type">int</span> <span class="name">pos</span> <span class="operator">=</span> <span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="name">printf</span><span class="punctuation">(</span><span class="literal string">&quot;decoding. recursion limit: %d total item count limit: %d</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="punctuation">,</span> <span class="name">depth_limit</span><span class="punctuation">,</span> <span class="name">item_limit</span><span class="punctuation">);</span>
<span class="name">ret</span> <span class="operator">=</span> <span class="name">bdecode</span><span class="punctuation">(</span><span class="operator">&amp;</span><span class="name">buf</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">],</span> <span class="operator">&amp;</span><span class="name">buf</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">]</span> <span class="operator">+</span> <span class="name">buf</span><span class="punctuation">.</span><span class="name">size</span><span class="punctuation">(),</span> <span class="name">e</span><span class="punctuation">,</span> <span class="name">ec</span><span class="punctuation">,</span> <span class="operator">&amp;</span><span class="name">pos</span>
<span class="punctuation">,</span> <span class="name">depth_limit</span><span class="punctuation">,</span> <span class="name">item_limit</span><span class="punctuation">);</span>
<span class="name">printf</span><span class="punctuation">(</span><span class="literal string">&quot;</span><span class="literal string escape">\n\n</span><span class="literal string">----- raw info -----</span><span class="literal string escape">\n\n</span><span class="literal string">%s</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span><span class="punctuation">,</span> <span class="name">print_entry</span><span class="punctuation">(</span><span class="name">e</span><span class="punctuation">).</span><span class="name">c_str</span><span class="punctuation">());</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">ret</span> <span class="operator">!=</span> <span class="literal number integer">0</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">fprintf</span><span class="punctuation">(</span><span class="name">stderr</span><span class="punctuation">,</span> <span class="literal string">&quot;failed to decode: '%s' at character: %d</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span><span class="punctuation">,</span> <span class="name">ec</span><span class="punctuation">.</span><span class="name">message</span><span class="punctuation">().</span><span class="name">c_str</span><span class="punctuation">(),</span> <span class="name">pos</span><span class="punctuation">);</span>
<span class="keyword">return</span> <span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="name">torrent_info</span> <span class="name">t</span><span class="punctuation">(</span><span class="name">e</span><span class="punctuation">,</span> <span class="name">ec</span><span class="punctuation">);</span>
<span class="keyword">if</span> <span class="punctuation">(</span><span class="name">ec</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">fprintf</span><span class="punctuation">(</span><span class="name">stderr</span><span class="punctuation">,</span> <span class="literal string">&quot;%s</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span><span class="punctuation">,</span> <span class="name">ec</span><span class="punctuation">.</span><span class="name">message</span><span class="punctuation">().</span><span class="name">c_str</span><span class="punctuation">());</span>
<span class="keyword">return</span> <span class="literal number integer">1</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
<span class="name">e</span><span class="punctuation">.</span><span class="name">clear</span><span class="punctuation">();</span>
<span class="name">std</span><span class="operator">::</span><span class="name">vector</span><span class="operator">&lt;</span><span class="keyword type">char</span><span class="operator">&gt;</span><span class="punctuation">().</span><span class="name">swap</span><span class="punctuation">(</span><span class="name">buf</span><span class="punctuation">);</span>
<span class="comment single">// print info about torrent
</span> <span class="name">printf</span><span class="punctuation">(</span><span class="literal string">&quot;</span><span class="literal string escape">\n\n</span><span class="literal string">----- torrent file info -----</span><span class="literal string escape">\n\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;nodes:</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span><span class="punctuation">);</span>
<span class="keyword">typedef</span> <span class="name">std</span><span class="operator">::</span><span class="name">vector</span><span class="operator">&lt;</span><span class="name">std</span><span class="operator">::</span><span class="name">pair</span><span class="operator">&lt;</span><span class="name">std</span><span class="operator">::</span><span class="name">string</span><span class="punctuation">,</span> <span class="keyword type">int</span><span class="operator">&gt;</span> <span class="operator">&gt;</span> <span class="name">node_vec</span><span class="punctuation">;</span>
<span class="name">node_vec</span> <span class="keyword">const</span><span class="operator">&amp;</span> <span class="name">nodes</span> <span class="operator">=</span> <span class="name">t</span><span class="punctuation">.</span><span class="name">nodes</span><span class="punctuation">();</span>
<span class="keyword">for</span> <span class="punctuation">(</span><span class="name">node_vec</span><span class="operator">::</span><span class="name">const_iterator</span> <span class="name">i</span> <span class="operator">=</span> <span class="name">nodes</span><span class="punctuation">.</span><span class="name">begin</span><span class="punctuation">(),</span> <span class="name">end</span><span class="punctuation">(</span><span class="name">nodes</span><span class="punctuation">.</span><span class="name">end</span><span class="punctuation">());</span>
<span class="name">i</span> <span class="operator">!=</span> <span class="name">end</span><span class="punctuation">;</span> <span class="operator">++</span><span class="name">i</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">printf</span><span class="punctuation">(</span><span class="literal string">&quot;%s: %d</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span><span class="punctuation">,</span> <span class="name">i</span><span class="operator">-&gt;</span><span class="name">first</span><span class="punctuation">.</span><span class="name">c_str</span><span class="punctuation">(),</span> <span class="name">i</span><span class="operator">-&gt;</span><span class="name">second</span><span class="punctuation">);</span>
<span class="punctuation">}</span>
<span class="name">puts</span><span class="punctuation">(</span><span class="literal string">&quot;trackers:</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span><span class="punctuation">);</span>
<span class="keyword">for</span> <span class="punctuation">(</span><span class="name">std</span><span class="operator">::</span><span class="name">vector</span><span class="operator">&lt;</span><span class="name">announce_entry</span><span class="operator">&gt;::</span><span class="name">const_iterator</span> <span class="name">i</span> <span class="operator">=</span> <span class="name">t</span><span class="punctuation">.</span><span class="name">trackers</span><span class="punctuation">().</span><span class="name">begin</span><span class="punctuation">();</span>
<span class="name">i</span> <span class="operator">!=</span> <span class="name">t</span><span class="punctuation">.</span><span class="name">trackers</span><span class="punctuation">().</span><span class="name">end</span><span class="punctuation">();</span> <span class="operator">++</span><span class="name">i</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="name">printf</span><span class="punctuation">(</span><span class="literal string">&quot;%2d: %s</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span><span class="punctuation">,</span> <span class="name">i</span><span class="operator">-&gt;</span><span class="name">tier</span><span class="punctuation">,</span> <span class="name">i</span><span class="operator">-&gt;</span><span class="name">url</span><span class="punctuation">.</span><span class="name">c_str</span><span class="punctuation">());</span>
<span class="punctuation">}</span>
<span class="keyword type">char</span> <span class="name">ih</span><span class="punctuation">[</span><span class="literal number integer">41</span><span class="punctuation">];</span>
<span class="name">to_hex</span><span class="punctuation">((</span><span class="keyword type">char</span> <span class="keyword">const</span><span class="operator">*</span><span class="punctuation">)</span><span class="operator">&amp;</span><span class="name">t</span><span class="punctuation">.</span><span class="name">info_hash</span><span class="punctuation">()[</span><span class="literal number integer">0</span><span class="punctuation">],</span> <span class="literal number integer">20</span><span class="punctuation">,</span> <span class="name">ih</span><span class="punctuation">);</span>
<span class="name">printf</span><span class="punctuation">(</span><span class="literal string">&quot;number of pieces: %d</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;piece length: %d</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;info hash: %s</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;comment: %s</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;created by: %s</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;magnet link: %s</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;name: %s</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;number of files: %d</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="literal string">&quot;files:</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="punctuation">,</span> <span class="name">t</span><span class="punctuation">.</span><span class="name">num_pieces</span><span class="punctuation">()</span>
<span class="punctuation">,</span> <span class="name">t</span><span class="punctuation">.</span><span class="name">piece_length</span><span class="punctuation">()</span>
<span class="punctuation">,</span> <span class="name">ih</span>
<span class="punctuation">,</span> <span class="name">t</span><span class="punctuation">.</span><span class="name">comment</span><span class="punctuation">().</span><span class="name">c_str</span><span class="punctuation">()</span>
<span class="punctuation">,</span> <span class="name">t</span><span class="punctuation">.</span><span class="name">creator</span><span class="punctuation">().</span><span class="name">c_str</span><span class="punctuation">()</span>
<span class="punctuation">,</span> <span class="name">make_magnet_uri</span><span class="punctuation">(</span><span class="name">t</span><span class="punctuation">).</span><span class="name">c_str</span><span class="punctuation">()</span>
<span class="punctuation">,</span> <span class="name">t</span><span class="punctuation">.</span><span class="name">name</span><span class="punctuation">().</span><span class="name">c_str</span><span class="punctuation">()</span>
<span class="punctuation">,</span> <span class="name">t</span><span class="punctuation">.</span><span class="name">num_files</span><span class="punctuation">());</span>
<span class="name">file_storage</span> <span class="keyword">const</span><span class="operator">&amp;</span> <span class="name">st</span> <span class="operator">=</span> <span class="name">t</span><span class="punctuation">.</span><span class="name">files</span><span class="punctuation">();</span>
<span class="keyword">for</span> <span class="punctuation">(</span><span class="keyword type">int</span> <span class="name">i</span> <span class="operator">=</span> <span class="literal number integer">0</span><span class="punctuation">;</span> <span class="name">i</span> <span class="operator">&lt;</span> <span class="name">st</span><span class="punctuation">.</span><span class="name">num_files</span><span class="punctuation">();</span> <span class="operator">++</span><span class="name">i</span><span class="punctuation">)</span>
<span class="punctuation">{</span>
<span class="keyword type">int</span> <span class="name">first</span> <span class="operator">=</span> <span class="name">st</span><span class="punctuation">.</span><span class="name">map_file</span><span class="punctuation">(</span><span class="name">i</span><span class="punctuation">,</span> <span class="literal number integer">0</span><span class="punctuation">,</span> <span class="literal number integer">0</span><span class="punctuation">).</span><span class="name">piece</span><span class="punctuation">;</span>
<span class="keyword type">int</span> <span class="name">last</span> <span class="operator">=</span> <span class="name">st</span><span class="punctuation">.</span><span class="name">map_file</span><span class="punctuation">(</span><span class="name">i</span><span class="punctuation">,</span> <span class="punctuation">(</span><span class="name">std</span><span class="operator">::</span><span class="name">max</span><span class="punctuation">)(</span><span class="name">boost</span><span class="operator">::</span><span class="keyword type">int64_t</span><span class="punctuation">(</span><span class="name">st</span><span class="punctuation">.</span><span class="name">file_size</span><span class="punctuation">(</span><span class="name">i</span><span class="punctuation">))</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">,</span> <span class="name">boost</span><span class="operator">::</span><span class="keyword type">int64_t</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">)),</span> <span class="literal number integer">0</span><span class="punctuation">).</span><span class="name">piece</span><span class="punctuation">;</span>
<span class="keyword type">int</span> <span class="name">flags</span> <span class="operator">=</span> <span class="name">st</span><span class="punctuation">.</span><span class="name">file_flags</span><span class="punctuation">(</span><span class="name">i</span><span class="punctuation">);</span>
<span class="name">printf</span><span class="punctuation">(</span><span class="literal string">&quot; %8&quot;</span> <span class="name">PRIx64</span> <span class="literal string">&quot; %11&quot;</span> <span class="name">PRId64</span> <span class="literal string">&quot; %c%c%c%c [ %5d, %5d ] %7u %s %s %s%s</span><span class="literal string escape">\n</span><span class="literal string">&quot;</span>
<span class="punctuation">,</span> <span class="name">st</span><span class="punctuation">.</span><span class="name">file_offset</span><span class="punctuation">(</span><span class="name">i</span><span class="punctuation">)</span>
<span class="punctuation">,</span> <span class="name">st</span><span class="punctuation">.</span><span class="name">file_size</span><span class="punctuation">(</span><span class="name">i</span><span class="punctuation">)</span>
<span class="punctuation">,</span> <span class="punctuation">((</span><span class="name">flags</span> <span class="operator">&amp;</span> <span class="name">file_storage</span><span class="operator">::</span><span class="name">flag_pad_file</span><span class="punctuation">)</span><span class="operator">?</span><span class="literal string char">'p'</span><span class="operator">:</span><span class="literal string char">'-'</span><span class="punctuation">)</span>
<span class="punctuation">,</span> <span class="punctuation">((</span><span class="name">flags</span> <span class="operator">&amp;</span> <span class="name">file_storage</span><span class="operator">::</span><span class="name">flag_executable</span><span class="punctuation">)</span><span class="operator">?</span><span class="literal string char">'x'</span><span class="operator">:</span><span class="literal string char">'-'</span><span class="punctuation">)</span>
<span class="punctuation">,</span> <span class="punctuation">((</span><span class="name">flags</span> <span class="operator">&amp;</span> <span class="name">file_storage</span><span class="operator">::</span><span class="name">flag_hidden</span><span class="punctuation">)</span><span class="operator">?</span><span class="literal string char">'h'</span><span class="operator">:</span><span class="literal string char">'-'</span><span class="punctuation">)</span>
<span class="punctuation">,</span> <span class="punctuation">((</span><span class="name">flags</span> <span class="operator">&amp;</span> <span class="name">file_storage</span><span class="operator">::</span><span class="name">flag_symlink</span><span class="punctuation">)</span><span class="operator">?</span><span class="literal string char">'l'</span><span class="operator">:</span><span class="literal string char">'-'</span><span class="punctuation">)</span>
<span class="punctuation">,</span> <span class="name">first</span><span class="punctuation">,</span> <span class="name">last</span>
<span class="punctuation">,</span> <span class="name">boost</span><span class="operator">::</span><span class="keyword type">uint32_t</span><span class="punctuation">(</span><span class="name">st</span><span class="punctuation">.</span><span class="name">mtime</span><span class="punctuation">(</span><span class="name">i</span><span class="punctuation">))</span>
<span class="punctuation">,</span> <span class="name">st</span><span class="punctuation">.</span><span class="name">hash</span><span class="punctuation">(</span><span class="name">i</span><span class="punctuation">)</span> <span class="operator">!=</span> <span class="name">sha1_hash</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">)</span> <span class="operator">?</span> <span class="name">to_hex</span><span class="punctuation">(</span><span class="name">st</span><span class="punctuation">.</span><span class="name">hash</span><span class="punctuation">(</span><span class="name">i</span><span class="punctuation">).</span><span class="name">to_string</span><span class="punctuation">()).</span><span class="name">c_str</span><span class="punctuation">()</span> <span class="operator">:</span> <span class="literal string">&quot;&quot;</span>
<span class="punctuation">,</span> <span class="name">st</span><span class="punctuation">.</span><span class="name">file_path</span><span class="punctuation">(</span><span class="name">i</span><span class="punctuation">).</span><span class="name">c_str</span><span class="punctuation">()</span>
<span class="punctuation">,</span> <span class="punctuation">(</span><span class="name">flags</span> <span class="operator">&amp;</span> <span class="name">file_storage</span><span class="operator">::</span><span class="name">flag_symlink</span><span class="punctuation">)</span> <span class="operator">?</span> <span class="literal string">&quot;-&gt; &quot;</span> <span class="operator">:</span> <span class="literal string">&quot;&quot;</span>
<span class="punctuation">,</span> <span class="punctuation">(</span><span class="name">flags</span> <span class="operator">&amp;</span> <span class="name">file_storage</span><span class="operator">::</span><span class="name">flag_symlink</span><span class="punctuation">)</span> <span class="operator">?</span> <span class="name">st</span><span class="punctuation">.</span><span class="name">symlink</span><span class="punctuation">(</span><span class="name">i</span><span class="punctuation">).</span><span class="name">c_str</span><span class="punctuation">()</span> <span class="operator">:</span> <span class="literal string">&quot;&quot;</span><span class="punctuation">);</span>
<span class="punctuation">}</span>
<span class="keyword">return</span> <span class="literal number integer">0</span><span class="punctuation">;</span>
<span class="punctuation">}</span>
</pre>
</div>
</div>
</div>
</div>
<div id="gradient"></div>
<div id="footer">
<table>
<tr>
<td><a href="index.html">home</a></td>
<td><a href="http://blog.libtorrent.org">blog</a></td>
<td><a href="utp.html">uTP</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/libtorrent/">download</a></td>
<td><a href="reference.html">documentation</a></td>
<td><a href="dht_store.html">DHT put extension</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/py-libtorrent/">python bindings</a></td>
<td><a href="features.html">features</a></td>
<td><a href="dht_sec.html">DHT security extension</a></td>
</tr>
<tr>
<td><a href="http://dir.gmane.org/gmane.network.bit-torrent.libtorrent">mailing list archive</a></td>
<td><a href="contributing.html">contributing</a></td>
<td><a href="streaming.html">streaming</a></td>
</tr>
<tr>
<td><a href="http://code.google.com/p/libtorrent/issues/entry">report a bug</a></td>
<td><a href="building.html">building</a></td>
<td><a href="bittorrent.pdf">bittorrent slides</a></td>
</tr>
</table>
</div>
<div id="filler"></div>
</div>
</body>
</html>

View File

@ -1,498 +0,0 @@
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.12: http://docutils.sourceforge.net/" />
<title></title>
<meta name="author" content="Arvid Norberg, arvid&#64;libtorrent.org Ludvig Strigeus, ludde&#64;utorrent.com" />
<meta name=viewport content="width=device-width, initial-scale=1">
<script type="text/javascript">
/* <![CDATA[ */
(function() {
var s = document.createElement('script'), t = document.getElementsByTagName('script')[0];
s.type = 'text/javascript';
s.async = true;
s.src = 'http://api.flattr.com/js/0.6/load.js?mode=auto';
t.parentNode.insertBefore(s, t);
})();
/* ]]> */
</script>
<link rel="stylesheet" href="style.css" type="text/css" />
<link rel="stylesheet" type="text/css" href="rst.css" />
<style type="text/css">
/* Hides from IE-mac \*/
* html pre { height: 1%; }
/* End hide from IE-mac */
</style>
</head>
<body>
<div class="document">
<div id="container">
<table id="header">
<tr><td id="orange"></td>
<td id="logo">libtorrent</td></tr>
</table>
<div id="main">
<table class="docinfo" frame="void" rules="none">
<col class="docinfo-name" />
<col class="docinfo-content" />
<tbody valign="top">
<tr><th class="docinfo-name">Author:</th>
<td>Arvid Norberg, <a class="reference external" href="mailto:arvid&#64;libtorrent.org">arvid&#64;libtorrent.org</a>
Ludvig Strigeus, <a class="last reference external" href="mailto:ludde&#64;utorrent.com">ludde&#64;utorrent.com</a></td></tr>
</tbody>
</table>
<div class="section" id="extension-protocol-for-bittorrent">
<h1>extension protocol for bittorrent</h1>
<p>The intention of this protocol is to provide a simple and thin transport
for extensions to the bittorrent protocol. Supporting this protocol makes
it easy to add new extensions without interfering with the standard
bittorrent protocol or clients that don't support this extension or the
one you want to add.</p>
<p>To advertise to other clients that you support, one bit from the reserved
bytes is used.</p>
<p>The bit selected for the extension protocol is bit 20 from the right (counting
starts at 0). So (reserved_byte[5] &amp; 0x10) is the expression to use for checking
if the client supports extended messaging.</p>
<p>Once support for the protocol is established, the client is supposed to
support 1 new message:</p>
<table border="1" class="docutils">
<colgroup>
<col width="86%" />
<col width="14%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">name</th>
<th class="head">id</th>
</tr>
</thead>
<tbody valign="top">
<tr><td><tt class="docutils literal">extended</tt></td>
<td>20</td>
</tr>
</tbody>
</table>
<p>This message is sent as any other bittorrent message, with a 4 byte length
prefix and a single byte identifying the message (the single byte being 20
in this case). At the start of the payload of the message, is a single byte
message identifier. This identifier can refer to different extension messages
and only one ID is specified, 0. If the ID is 0, the message is a handshake
message which is described below. The layout of a general <tt class="docutils literal">extended</tt> message
follows (including the message headers used by the bittorrent protocol):</p>
<table border="1" class="docutils">
<colgroup>
<col width="15%" />
<col width="85%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">size</th>
<th class="head">description</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>uint32_t</td>
<td>length prefix. Specifies the number of bytes for the
entire message. (Big endian)</td>
</tr>
<tr><td>uint8_t</td>
<td>bittorrent message ID, = 20</td>
</tr>
<tr><td>uint8_t</td>
<td>extended message ID. 0 = handshake, &gt;0 = extended
message as specified by the handshake.</td>
</tr>
</tbody>
</table>
<div class="section" id="handshake-message">
<h2>handshake message</h2>
<p>The payload of the handshake message is a bencoded dictionary. All items
in the dictionary are optional. Any unknown names should be ignored
by the client. All parts of the dictionary are case sensitive.
This is the defined item in the dictionary:</p>
<table border="1" class="docutils">
<colgroup>
<col width="11%" />
<col width="89%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">name</th>
<th class="head">description</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>m</td>
<td><p class="first">Dictionary of supported extension messages which maps
names of extensions to an extended message ID for each
extension message. The only requirement on these IDs
is that no extension message share the same one. Setting
an extension number to zero means that the extension is
not supported/disabled. The client should ignore any
extension names it doesn't recognize.</p>
<p class="last">The extension message IDs are the IDs used to send the
extension messages to the peer sending this handshake.
i.e. The IDs are local to this particular peer.</p>
</td>
</tr>
</tbody>
</table>
<p>Here are some other items that an implementation may choose to support:</p>
<table border="1" class="docutils">
<colgroup>
<col width="12%" />
<col width="88%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">name</th>
<th class="head">description</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>p</td>
<td>Local TCP listen port. Allows each side to learn about
the TCP port number of the other side. Note that there is
no need for the receiving side of the connection to send
this extension message, since its port number is already
known.</td>
</tr>
<tr><td>v</td>
<td>Client name and version (as a utf-8 string).
This is a much more reliable way of identifying the
client than relying on the peer id encoding.</td>
</tr>
<tr><td>yourip</td>
<td>A string containing the compact representation of the ip
address this peer sees you as. i.e. this is the
receiver's external ip address (no port is included).
This may be either an IPv4 (4 bytes) or an IPv6
(16 bytes) address.</td>
</tr>
<tr><td>ipv6</td>
<td>If this peer has an IPv6 interface, this is the compact
representation of that address (16 bytes). The client may
prefer to connect back via the IPv6 address.</td>
</tr>
<tr><td>ipv4</td>
<td>If this peer has an IPv4 interface, this is the compact
representation of that address (4 bytes). The client may
prefer to connect back via this interface.</td>
</tr>
<tr><td>reqq</td>
<td>An integer, the number of outstanding request messages
this client supports without dropping any. The default in
in libtorrent is 250.</td>
</tr>
</tbody>
</table>
<p>The handshake dictionary could also include extended handshake
information, such as support for encrypted headers or anything
imaginable.</p>
<p>An example of what the payload of a handshake message could look like:</p>
<table border="1" class="docutils">
<colgroup>
<col width="36%" />
<col width="64%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head" colspan="2">Dictionary</th>
</tr>
</thead>
<tbody valign="top">
<tr><td><tt class="docutils literal">m</tt></td>
<td><table border="1" class="first last docutils">
<colgroup>
<col width="88%" />
<col width="12%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head" colspan="2">Dictionary</th>
</tr>
</thead>
<tbody valign="top">
<tr><td><tt class="docutils literal">LT_metadata</tt></td>
<td>1</td>
</tr>
<tr><td><tt class="docutils literal">ut_pex</tt></td>
<td>2</td>
</tr>
</tbody>
</table>
</td>
</tr>
<tr><td><tt class="docutils literal">p</tt></td>
<td>6881</td>
</tr>
<tr><td><tt class="docutils literal">v</tt></td>
<td>&quot;uTorrent 1.2&quot;</td>
</tr>
</tbody>
</table>
<p>and in the encoded form:</p>
<p><tt class="docutils literal">d1:md11:LT_metadatai1e6:ut_pexi2ee1:pi6881e1:v12:uTorrent 1.2e</tt></p>
<p>To make sure the extension names do not collide by mistake, they should be
prefixed with the two (or one) character code that is used to identify the
client that introduced the extension. This applies for both the names of
extension messages, and for any additional information put inside the
top-level dictionary. All one and two byte identifiers are invalid to use
unless defined by this specification.</p>
<p>This message should be sent immediately after the standard bittorrent handshake
to any peer that supports this extension protocol. It is valid to send the
handshake message more than once during the lifetime of a connection,
the sending client should not be disconnected. An implementation may choose
to ignore the subsequent handshake messages (or parts of them).</p>
<p>Subsequent handshake messages can be used to enable/disable extensions
without restarting the connection. If a peer supports changing extensions
at run time, it should note that the <tt class="docutils literal">m</tt> dictionary is additive.
It's enough that it contains the actual <em>CHANGES</em> to the extension list.
To disable the support for <tt class="docutils literal">LT_metadata</tt> at run-time, without affecting
any other extensions, this message should be sent:
<tt class="docutils literal">d11:LT_metadatai0ee</tt>.
As specified above, the value 0 is used to turn off an extension.</p>
<p>The extension IDs must be stored for every peer, becuase every peer may have
different IDs for the same extension.</p>
<p>This specification, deliberately, does not specify any extensions such as
peer-exchange or metadata exchange. This protocol is merely a transport
for the actual extensions to the bittorrent protocol and the extensions
named in the example above (such as <tt class="docutils literal">p</tt>) are just examples of possible
extensions.</p>
</div>
<div class="section" id="rationale">
<h2>rationale</h2>
<p>The reason why the extension messages' IDs would be defined in the handshake
is to avoid having a global registry of message IDs. Instead the names of the
extension messages requires unique names, which is much easier to do without
a global registry. The convention is to use a two letter prefix on the
extension message names, the prefix would identify the client first
implementing the extension message. e.g. <tt class="docutils literal">LT_metadata</tt> is implemented by
libtorrent, and hence it has the <tt class="docutils literal">LT</tt> prefix.</p>
<p>If the client supporting the extensions can decide which numbers the messages
it receives will have, it means they are constants within that client. i.e.
they can be used in <tt class="docutils literal">switch</tt> statements. It's easy for the other end to
store an array with the ID's we expect for each message and use that for
lookups each time it sends an extension message.</p>
<p>The reason for having a dictionary instead of having an array (using
implicitly assigned index numbers to the extensions) is that if a client
want to disable some extensions, the ID numbers would change, and it wouldn't
be able to use constants (and hence, not use them in a <tt class="docutils literal">switch</tt>). If the
messages IDs would map directly to bittorrent message IDs, It would also make
it possible to map extensions in the handshake to existing extensions with
fixed message IDs.</p>
<p>The reasoning behind having a single byte as extended message identifier is
to follow the the bittorrent spec. with its single byte message identifiers.
It is also considered to be enough. It won't limit the total number of
extensions, only the number of extensions used simultaneously.</p>
<p>The reason for using single byte identifiers for the standardized handshake
identifiers is 1) The mainline DHT uses single byte identifiers. 2) Saves
bandwidth. The only advantage of longer messages is that it makes the
protocol more readable for a human, but the BT protocol wasn't designed to
be a human readable protocol, so why bother.</p>
</div>
</div>
<div class="section" id="extensions">
<h1>extensions</h1>
<p>These extensions all operates within the <a class="reference external" href="extension_protocol.html">extension protocol</a>. The name of the
extension is the name used in the extension-list packets, and the payload is
the data in the extended message (not counting the length-prefix, message-id
nor extension-id).</p>
<p>Note that since this protocol relies on one of the reserved bits in the
handshake, it may be incompatible with future versions of the mainline
bittorrent client.</p>
<p>These are the extensions that are currently implemented.</p>
<div class="section" id="metadata-from-peers">
<h2>metadata from peers</h2>
<p>Extension name: &quot;LT_metadata&quot;</p>
<div class="note">
<p class="first admonition-title">Note</p>
<p class="last">This extension is deprecated in favor of the more widely supported
<tt class="docutils literal">ut_metadata</tt> extension, see <a class="reference external" href="http://bittorrent.org/beps/bep_0009.html">BEP 9</a>.</p>
</div>
<p>The point with this extension is that you don't have to distribute the
metadata (.torrent-file) separately. The metadata can be distributed
through the bittorrent swarm. The only thing you need to download such
a torrent is the tracker url and the info-hash of the torrent.</p>
<p>It works by assuming that the initial seeder has the metadata and that the
metadata will propagate through the network as more peers join.</p>
<p>There are three kinds of messages in the metadata extension. These packets are
put as payload to the extension message. The three packets are:</p>
<blockquote>
<ul class="simple">
<li>request metadata</li>
<li>metadata</li>
<li>don't have metadata</li>
</ul>
</blockquote>
<p>request metadata:</p>
<table border="1" class="docutils">
<colgroup>
<col width="17%" />
<col width="23%" />
<col width="61%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">size</th>
<th class="head">name</th>
<th class="head">description</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>uint8_t</td>
<td>msg_type</td>
<td>Determines the kind of message this is
0 means 'request metadata'</td>
</tr>
<tr><td>uint8_t</td>
<td>start</td>
<td>The start of the metadata block that
is requested. It is given in 256:ths
of the total size of the metadata,
since the requesting client don't know
the size of the metadata.</td>
</tr>
<tr><td>uint8_t</td>
<td>size</td>
<td>The size of the metadata block that is
requested. This is also given in
256:ths of the total size of the
metadata. The size is given as size-1.
That means that if this field is set
0, the request wants one 256:th of the
metadata.</td>
</tr>
</tbody>
</table>
<p>metadata:</p>
<table border="1" class="docutils">
<colgroup>
<col width="17%" />
<col width="23%" />
<col width="61%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">size</th>
<th class="head">name</th>
<th class="head">description</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>uint8_t</td>
<td>msg_type</td>
<td>1 means 'metadata'</td>
</tr>
<tr><td>int32_t</td>
<td>total_size</td>
<td>The total size of the metadata, given
in number of bytes.</td>
</tr>
<tr><td>int32_t</td>
<td>offset</td>
<td>The offset of where the metadata block
in this message belongs in the final
metadata. This is given in bytes.</td>
</tr>
<tr><td>uint8_t[]</td>
<td>metadata</td>
<td>The actual metadata block. The size of
this part is given implicit by the
length prefix in the bittorrent
protocol packet.</td>
</tr>
</tbody>
</table>
<p>Don't have metadata:</p>
<table border="1" class="docutils">
<colgroup>
<col width="17%" />
<col width="23%" />
<col width="61%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">size</th>
<th class="head">name</th>
<th class="head">description</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>uint8_t</td>
<td>msg_type</td>
<td>2 means 'I don't have metadata'.
This message is sent as a reply to a
metadata request if the the client
doesn't have any metadata.</td>
</tr>
</tbody>
</table>
</div>
<div class="section" id="dont-have">
<h2>dont_have</h2>
<p>Extension name: &quot;lt_donthave&quot;</p>
<p>The <tt class="docutils literal">dont_have</tt> extension message is used to tell peers that the client no
longer has a specific piece. The extension message should be advertised in the
<tt class="docutils literal">m</tt> dictionary as <tt class="docutils literal">lt_donthave</tt>. The message format mimics the regular
<tt class="docutils literal">HAVE</tt> bittorrent message.</p>
<p>Just like all extension messages, the first 2 bytes in the mssage itself are 20
(the bittorrent extension message) and the message ID assigned to this
extension in the <tt class="docutils literal">m</tt> dictionary in the handshake.</p>
<table border="1" class="docutils">
<colgroup>
<col width="17%" />
<col width="23%" />
<col width="61%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">size</th>
<th class="head">name</th>
<th class="head">description</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>uint32_t</td>
<td>piece</td>
<td>index of the piece the peer no longer
has.</td>
</tr>
</tbody>
</table>
<p>The length of this message (including the extension message prefix) is 6 bytes,
i.e. one byte longer than the normal <tt class="docutils literal">HAVE</tt> message, because of the extension
message wrapping.</p>
</div>
</div>
</div>
</div>
<div id="gradient"></div>
<div id="footer">
<table>
<tr>
<td><a href="index.html">home</a></td>
<td><a href="http://blog.libtorrent.org">blog</a></td>
<td><a href="utp.html">uTP</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/libtorrent/">download</a></td>
<td><a href="reference.html">documentation</a></td>
<td><a href="dht_store.html">DHT put extension</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/py-libtorrent/">python bindings</a></td>
<td><a href="features.html">features</a></td>
<td><a href="dht_sec.html">DHT security extension</a></td>
</tr>
<tr>
<td><a href="http://dir.gmane.org/gmane.network.bit-torrent.libtorrent">mailing list archive</a></td>
<td><a href="contributing.html">contributing</a></td>
<td><a href="streaming.html">streaming</a></td>
</tr>
<tr>
<td><a href="http://code.google.com/p/libtorrent/issues/entry">report a bug</a></td>
<td><a href="building.html">building</a></td>
<td><a href="bittorrent.pdf">bittorrent slides</a></td>
</tr>
</table>
</div>
<div id="filler"></div>
</div>
</body>
</html>

View File

@ -1,415 +0,0 @@
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.12: http://docutils.sourceforge.net/" />
<title>libtorrent manual</title>
<meta name="author" content="Arvid Norberg, arvid&#64;libtorrent.org" />
<meta name=viewport content="width=device-width, initial-scale=1">
<script type="text/javascript">
/* <![CDATA[ */
(function() {
var s = document.createElement('script'), t = document.getElementsByTagName('script')[0];
s.type = 'text/javascript';
s.async = true;
s.src = 'http://api.flattr.com/js/0.6/load.js?mode=auto';
t.parentNode.insertBefore(s, t);
})();
/* ]]> */
</script>
<link rel="stylesheet" href="style.css" type="text/css" />
<link rel="stylesheet" type="text/css" href="rst.css" />
<style type="text/css">
/* Hides from IE-mac \*/
* html pre { height: 1%; }
/* End hide from IE-mac */
</style>
</head>
<body>
<div class="document" id="libtorrent-manual">
<div id="container">
<table id="header">
<tr><td id="orange"></td>
<td id="logo">libtorrent</td></tr>
</table>
<div id="main">
<h1 class="title">libtorrent manual</h1>
<table class="docinfo" frame="void" rules="none">
<col class="docinfo-name" />
<col class="docinfo-content" />
<tbody valign="top">
<tr><th class="docinfo-name">Author:</th>
<td>Arvid Norberg, <a class="last reference external" href="mailto:arvid&#64;libtorrent.org">arvid&#64;libtorrent.org</a></td></tr>
<tr><th class="docinfo-name">Version:</th>
<td>1.1.0</td></tr>
</tbody>
</table>
<div class="contents topic" id="table-of-contents">
<p class="topic-title first">Table of contents</p>
<ul class="simple">
<li><a class="reference internal" href="#introduction" id="id2">introduction</a></li>
<li><a class="reference internal" href="#features" id="id3">features</a><ul>
<li><a class="reference internal" href="#extensions" id="id4">extensions</a></li>
<li><a class="reference internal" href="#disk-management" id="id5">disk management</a></li>
<li><a class="reference internal" href="#network" id="id6">network</a></li>
</ul>
</li>
<li><a class="reference internal" href="#highlighted-features" id="id7">highlighted features</a><ul>
<li><a class="reference internal" href="#disk-caching" id="id8">disk caching</a></li>
<li><a class="reference internal" href="#high-performance-disk-subsystem" id="id9">high performance disk subsystem</a></li>
<li><a class="reference internal" href="#network-buffers" id="id10">network buffers</a></li>
<li><a class="reference internal" href="#piece-picker" id="id11">piece picker</a></li>
<li><a class="reference internal" href="#share-mode" id="id12">share mode</a></li>
<li><a class="reference internal" href="#merkle-hash-tree-torrents" id="id13">merkle hash tree torrents</a></li>
<li><a class="reference internal" href="#customizable-file-storage" id="id14">customizable file storage</a></li>
<li><a class="reference internal" href="#easy-to-use-api" id="id15">easy to use API</a></li>
</ul>
</li>
<li><a class="reference internal" href="#portability" id="id16">portability</a></li>
</ul>
</div>
<div class="section" id="introduction">
<h1>introduction</h1>
<p>libtorrent is a feature complete C++ bittorrent implementation focusing
on efficiency and scalability. It runs on embedded devices as well as
desktops. It boasts a well documented library interface that is easy to
use. It comes with a simple bittorrent client demonstrating the use of
the library.</p>
</div>
<div class="section" id="features">
<h1>features</h1>
<p>libtorrent is an ongoing project under active development. Its
current state supports and includes the following features:</p>
<div class="section" id="extensions">
<h2>extensions</h2>
<ul class="simple">
<li>plugin interface for implementing custom bittorrent extensions
without having to modify libtorrent</li>
<li>supports trackerless torrents (using the Mainline kademlia DHT protocol) with
some <a class="reference external" href="dht_extensions.html">DHT extensions</a>. <a class="reference external" href="http://bittorrent.org/beps/bep_0005.html">BEP 5</a>.</li>
<li>supports the bittorrent <a class="reference external" href="extension_protocol.html">extension protocol</a>. See <a class="reference external" href="manual-ref.html#extensions">extensions</a>. <a class="reference external" href="http://bittorrent.org/beps/bep_0010.html">BEP 10</a>.</li>
<li>supports the uTorrent metadata transfer protocol <a class="reference external" href="http://bittorrent.org/beps/bep_0009.html">BEP 9</a> (i.e. magnet links).</li>
<li>supports the uTorrent peer exchange protocol (PEX).</li>
<li>supports local peer discovery (multicasts for peers on the same local network)</li>
<li>multitracker extension support (supports both strict <a class="reference external" href="http://bittorrent.org/beps/bep_0012.html">BEP 12</a> and the
uTorrent interpretation).</li>
<li>tracker scrapes</li>
<li>supports lt_trackers extension, to exchange trackers between peers</li>
<li><a class="reference external" href="manual-ref.html#http-seeding">HTTP seeding</a>, as specified in <a class="reference external" href="http://bittorrent.org/beps/bep_0017.html">BEP 17</a> and <a class="reference external" href="http://bittorrent.org/beps/bep_0019.html">BEP 19</a>.</li>
<li>supports the udp-tracker protocol. (<a class="reference external" href="http://bittorrent.org/beps/bep_0015.html">BEP 15</a>).</li>
<li>supports the <tt class="docutils literal">no_peer_id=1</tt> extension that will ease the load off trackers.</li>
<li>supports the <tt class="docutils literal">compact=1</tt> tracker parameter.</li>
<li>super seeding/initial seeding (<a class="reference external" href="http://bittorrent.org/beps/bep_0016.html">BEP 16</a>).</li>
<li>private torrents (<a class="reference external" href="http://bittorrent.org/beps/bep_0027.html">BEP 27</a>).</li>
<li>upload-only extension (<a class="reference external" href="http://bittorrent.org/beps/bep_0021.html">BEP 21</a>).</li>
<li>support for IPv6, including <a class="reference external" href="http://bittorrent.org/beps/bep_0007.html">BEP 7</a> and <a class="reference external" href="http://bittorrent.org/beps/bep_0024.html">BEP 24</a>.</li>
<li>support for merkle hash tree torrents. This makes the size of torrent files
scale well with the size of the content.</li>
<li>share-mode. This is a special mode torrents can be put in to optimize share
ratio rather than downloading the torrent.</li>
</ul>
</div>
<div class="section" id="disk-management">
<h2>disk management</h2>
<ul class="simple">
<li>can use multipled disk I/O threads to not have the disk block network or
client interaction.</li>
<li>supports verifying the SHA-1 hash of pieces in multiple threads, to take
advantage of multi core machines.</li>
<li>supports files &gt; 2 gigabytes.</li>
<li>fast resume support, a way to avoid the costly piece check at the
start of a resumed torrent. Saves the storage state, piece_picker state
as well as all local peers in a fast-resume file.</li>
<li>has an adjustable read and write disk cache for improved disk throughput.</li>
<li>queues torrents for file check, instead of checking all of them in parallel.</li>
<li>does not have any requirements on the piece order in a torrent that it
resumes. This means it can resume a torrent downloaded by any client.</li>
<li>seed mode, where the files on disk are assumed to be complete, and each
piece's hash is verified the first time it is requested.</li>
<li>implements an ARC disk cache, tuned for performing well under bittorrent work
loads</li>
</ul>
</div>
<div class="section" id="network">
<h2>network</h2>
<ul class="simple">
<li>a high quality uTP implementation (<a class="reference external" href="http://bittorrent.org/beps/bep_0029.html">BEP 29</a>). A transport protocol with
delay based congestion control. See separate <a class="reference external" href="utp.html">article</a>.</li>
<li>adjusts the length of the request queue depending on download rate.</li>
<li>serves multiple torrents on a single port and in a single thread</li>
<li>piece picking on block-level (as opposed to piece-level).
This means it can download parts of the same piece from different peers.
It will also prefer to download whole pieces from single peers if the
download speed is high enough from that particular peer.</li>
<li>supports http proxies and basic proxy authentication</li>
<li>supports gzipped tracker-responses</li>
<li>can limit the upload and download bandwidth usage and the maximum number of
unchoked peers</li>
<li>possibility to limit the number of connections.</li>
<li>delays have messages if there's no other outgoing traffic to the peer, and
doesn't send have messages to peers that already has the piece. This saves
bandwidth.</li>
<li>selective downloading. The ability to select which parts of a torrent you
want to download.</li>
<li>ip filter to disallow ip addresses and ip ranges from connecting and
being connected.</li>
<li>NAT-PMP and UPnP support (automatic port mapping on routers that supports it)</li>
<li>implements automatic upload slots, to optimize download rate without spreading
upload capacity too thin. The number of upload slots is adjusted based on the
peers' download capacity to work even for connections that are orders of
magnitude faster than others.</li>
</ul>
</div>
</div>
<div class="section" id="highlighted-features">
<h1>highlighted features</h1>
<div class="section" id="disk-caching">
<h2>disk caching</h2>
<p>All disk I/O in libtorrent is done asynchronously to the network thread, by the
disk io thread. When a block is read, the disk io thread reads all subsequent
blocks from that piece into the read cache, assuming that the peer requesting
the block will also request more blocks from the same piece. This decreases the
number of syscalls for reading data. It also decreases delay from seeking.</p>
<p>Similarly, for write requests, blocks are cached and flushed to disk once one full
piece is complete or the piece is the least recently updated one when more cache
space is needed. The cache dynamically allocates space between the write and read
cache. The write cache is strictly prioritized over the read cache.</p>
<p>The cache blocks that are in used, are locked into physical memory to avoid it
being paged out to disk. Allowing the disk cache to be paged out to disk means
that it would become extremely inefficient to flush it, since it would have to be
read back into physical memory only to be flushed back out to disk again.</p>
<p>In order to conserve memory, and system calls, iovec file operations are
used to flush multiple cache blocks in a single call.</p>
<p>On low-memory systems, the disk cache can be disabled altogether or set to smaller
limit, to save memory.</p>
<p>The disk caching algorithm is configurable between 'LRU' and 'largest contiguous'.
The largest contiguous algorithm is the default and flushes the largest contiguous
block of buffers, instead of flushing all blocks belonging to the piece which was
written to least recently.</p>
<p>For version 0.15 a lot of work went into optimizing the cache algorithm, trying
to increase the cache hit rate and utilization. The graph to the left shows the
memory utilization in 0.14. This cache is a straight forward, fairly naive, implementation.
Every block read will also read all subsequent blocks in that piece into the cache.
Whenever we need more space, the entire oldest piece is evicted from the cache. Caching
writes always takes presedence over the read cache. Whenever a piece is fully downloaded,
it is flushed to disk.</p>
<img alt="disk_buffer_before_optimization.png" src="disk_buffer_before_optimization.png" style="width: 49%;" />
<img alt="disk_buffer.png" src="disk_buffer.png" style="width: 49%;" />
<p>The left graph shows the problem of evicting entire pieces at a time, and waiting until
an entire piece is downloaded until flushing it. These graphs were generated for a torrent
with fairly large pieces. This means that granularity was poor in 0.14, since it only
dealt with entire pieces. In 0.15, the granularity problem has been fixed by evicting one
block at a time from the read cache. This maximizes the read cache utilization. The write
cache is also flushed when a sufficient number of contiguous blocks have been downloaded
for a piece, which is not tied to the piece size anymore. This way the cache scales a lot
better with piece sizes.</p>
<p>The graph to the right shows the same download but with the new optimized disk cache
algorithm. It clearly shows an increased utilization, which means higher read hit rates
or smaller caches with maintained hit rate.</p>
</div>
<div class="section" id="high-performance-disk-subsystem">
<h2>high performance disk subsystem</h2>
<p>In some circumstances, the disk cache may not suffice to provide maximum performance.
One such example is high performance seeding, to a large number of peers, over a fast
up-link. In such a case, the amount of RAM may simply not be enough to cache disk
reads. When there's not enough RAM to cache disk reads, the disk throughput would
typically degrade to perform as poorly as with no cache at all, with the majority
of the time spent waiting for the disk head to seek.</p>
<p>To solve this problem, libtorrent sorts read requests by their physical offset on the
disk. They are processed by having the disk read head sweep back and forth over the drive.</p>
<p>This makes libtorrent very suitable for large scale, high-throughput seeding.</p>
<img alt="disk_access_no_elevator.png" src="disk_access_no_elevator.png" style="width: 49%;" />
<img alt="disk_access_elevator.png" src="disk_access_elevator.png" style="width: 49%;" />
<p>These plots illustrates the physical disk offset for reads over time. The left plot
is of a run where disk operation re-ordering is turned off and the righ is when it's
turned on. The right one has a relatively smooth sine wave shape whereas the left
one is more random and involves much longer seeks back and forth over the disk.</p>
<p>True physical disk offset queries are only supported on newer linux kernels, Mac OS X and
Windows 2000 and up.</p>
</div>
<div class="section" id="network-buffers">
<h2>network buffers</h2>
<p>On CPUs with small L2 caches, copying memory can be expensive operations. It is important
to keep copying to a minimum on such machines. This mostly applies to embedded systems.</p>
<p>In order to minimize the number of times received data is copied, the receive buffer
for payload data is received directly into a page aligned disk buffer. If the connection
is encrypted, the buffer is decrypted in-place. The buffer is then moved into the disk
cache without being copied. Once all the blocks for a piece have been received, or the
cache needs to be flushed, all the blocks are passed directly to <tt class="docutils literal">writev()</tt> to flush
them in a single syscall. This means a single copy into user space memory, and a single
copy back into kernel memory, as illustrated by this figure:</p>
<img alt="write_disk_buffers.png" src="write_disk_buffers.png" style="width: 100%;" />
<p>When seeding and uploading in general, unnecessary copying is avoided by caching blocks
in aligned buffers, that are copied once into the peer's send buffer. The peer's send buffer
is not guaranteed to be aligned, even though it is most of the time. The send buffer is
then encrypted with the peer specific key and chained onto the <tt class="docutils literal">iovec</tt> for sending.
This means there is one user space copy in order to allow unaligned peer requests and
peer-specific encryption. This is illustrated by the following figure:</p>
<img alt="read_disk_buffers.png" src="read_disk_buffers.png" style="width: 100%;" />
</div>
<div class="section" id="piece-picker">
<h2>piece picker</h2>
<p>The piece picker is a central component in a bittorrent implementation. The piece picker
in libtorrent is optimized for quickly finding the rarest pieces. It keeps a list of all
available pieces sorted by rarity, and pieces with the same rarity, shuffled. The rarest
first mode is the dominant piece picker mode. Other modes are supported as well, and
used by peers in specific situations.</p>
<p>The piece picker allows to combine the availability of a piece with a priority. Together
they determine the sort order of the piece list. Pieces with priority 0 will never be
picked, which is used for the selective download feature.</p>
<p>In order to have as few partially finished pieces as possible, peers have an affinity
towards picking blocks from the same pieces as other peers in the same speed category.
The speed category is a coarse categorization of peers based on their download rate. This
makes slow peers pick blocks from the same piece, and fast peers pick from the same piece,
and hence decreasing the likelihood of slow peers blocking the completion of pieces.</p>
<p>The piece picker can also be set to download pieces in sequential order.</p>
</div>
<div class="section" id="share-mode">
<h2>share mode</h2>
<p>The share mode feature in libtorrent is intended for users who are only interested in
helping out swarms, not downloading the torrents.</p>
<p>It works by predicting the demand for pieces, and only download pieces if there is enough
demand. New pieces will only be downloaded once the share ratio has hit a certain target.</p>
<p>This feature is especially useful when combined with RSS, so that a client can be set up
to provide additional bandwidth to an entire feed.</p>
</div>
<div class="section" id="merkle-hash-tree-torrents">
<h2>merkle hash tree torrents</h2>
<img alt="merkle_tree.png" class="align-right" src="merkle_tree.png" />
<p>Merkle hash tree torrents is an extension that lets a torrent file only contain the
root hash of the hash tree forming the piece hashes. The main benefit of this feature
is that regardless of how many pieces there is in a torrent, the .torrent file will
always be the same size. It will only grow with the number of files (since it still
has to contain the file names).</p>
<p>With regular torrents, clients have to request multiple blocks for pieces, typically
from different peers, before the data can be verified against the piece hash. The
larger the pieces are, the longer it will take to download a complete piece and verify
it. Before the piece is verified, it cannot be shared with the swarm, which means the
larger piece sizes, the slower turnaround data has when it is downloaded by peers.
Since on average the data has to sit around, waiting, in client buffers before it has
been verified and can be uploaded again.</p>
<p>Another problem with large piece sizes is that it is harder for a client to pinpoint
the malicious or buggy peer when a piece fails, and it will take longer to re-download
it and take more tries before the piece succeeds the larger the pieces are.</p>
<p>The piece size in regular torrents is a tradeoff between the size of the .torrent file
itself and the piece size. Often, for files that are 4 GB, the piece size is 2 or 4 MB,
just to avoid making the .torrent file too big.</p>
<p>Merkle torrents solves these problems by removing the tradeoff between .torrent size and
piece size. With merkle torrents, the piece size can be the minimum block size (16 kB),
which lets peers verify every block of data received from peers, immediately. This
gives a minimum turnaround time and completely removes the problem of identifying malicious
peers.</p>
<p>The root hash is built by hashing all the piece hashes pair-wise, until they all collapse
down to the root.</p>
</div>
<div class="section" id="customizable-file-storage">
<h2>customizable file storage</h2>
<img alt="storage.png" class="align-right" src="storage.png" />
<p>libtorrent's storage implementation is customizable. That means a special purpose bittorrent
client can replace the default way to store files on disk.</p>
<p>When implementing a bittorrent cache, it doesn't matter how the data is stored on disk, as
long as it can be retrieved and seeded. In that case a new storage class can be implemented
(inheriting from the <tt class="docutils literal">storage_interface</tt> class) that avoids the unnecessary step of mapping
slots to files and offsets. The storage can ignore the file boundaries and just store the
entire torrent in a single file (which will end up being all the files concatenated). The main
advantage of this, other than a slight cpu performance gain, is that all file operations would
be page (and sector) aligned. This enables efficient unbuffered I/O, and can potentially
lead to more efficient read caching (using the built in disk cache rather than relying on the
operating system's disk cache).</p>
<p>The storage interface supports operating systems where you can ask for sparse regions
(such as Windows and Solaris). The advantage of this is that when checking files, the regions
that are known to be sparse can be skipped, which can reduce the time to check a torrent
significantly.</p>
</div>
<div class="section" id="easy-to-use-api">
<h2>easy to use API</h2>
<p>One of the design goals of the libtorrent API is to make common operations simple, but still
have it possible to do complicated and advanced operations. This is best illustrated by example
code to implement a simple bittorrent client:</p>
<pre class="literal-block">
#include &lt;iostream&gt;
#include &quot;libtorrent/session.hpp&quot;
// usage a.out [torrent-file]
int main(int argc, char* argv[]) try
{
using namespace libtorrent;
session s;
s.listen_on(std::make_pair(6881, 6889));
add_torrent_params p;
p.save_path = &quot;./&quot;;
p.ti = new torrent_info(argv[1]);
s.add_torrent(p);
// wait for the user to end
char a;
std::cin.unsetf(std::ios_base::skipws);
std::cin &gt;&gt; a;
return 0;
}
catch (std::exception&amp; e)
{
std::cerr &lt;&lt; ec.what() &lt;&lt; std::endl;
return 1;
}
</pre>
<p>This client doesn't give the user any status information or progress about the torrent, but
it is fully functional.</p>
<p>libtorrent also comes with python bindings for easy access for python developers.</p>
</div>
</div>
<div class="section" id="portability">
<h1>portability</h1>
<p>libtorrent runs on most major operating systems, including Windows,
MacOS X, Linux, BSD and Solaris.
It uses Boost.Thread, Boost.Filesystem, Boost.Date_time and various other
boost libraries. At least version 1.46.1 of boost is required.</p>
<p>libtorrent uses asio, hence it will take full advantage of high performance
network APIs on the most popular platforms. I/O completion ports on windows,
epoll on linux and kqueue on MacOS X and BSD.</p>
<p>libtorrent does not build with the following compilers:</p>
<ul class="simple">
<li>GCC 2.95.4</li>
<li>Visual Studio 6, 7.0, 7.1</li>
</ul>
</div>
</div>
</div>
<div id="gradient"></div>
<div id="footer">
<table>
<tr>
<td><a href="index.html">home</a></td>
<td><a href="http://blog.libtorrent.org">blog</a></td>
<td><a href="utp.html">uTP</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/libtorrent/">download</a></td>
<td><a href="reference.html">documentation</a></td>
<td><a href="dht_store.html">DHT put extension</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/py-libtorrent/">python bindings</a></td>
<td><a href="features.html">features</a></td>
<td><a href="dht_sec.html">DHT security extension</a></td>
</tr>
<tr>
<td><a href="http://dir.gmane.org/gmane.network.bit-torrent.libtorrent">mailing list archive</a></td>
<td><a href="contributing.html">contributing</a></td>
<td><a href="streaming.html">streaming</a></td>
</tr>
<tr>
<td><a href="http://code.google.com/p/libtorrent/issues/entry">report a bug</a></td>
<td><a href="building.html">building</a></td>
<td><a href="bittorrent.pdf">bittorrent slides</a></td>
</tr>
</table>
</div>
<div id="filler"></div>
</div>
</body>
</html>

View File

@ -1,171 +0,0 @@
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.12: http://docutils.sourceforge.net/" />
<title></title>
<meta name="author" content="Arvid Norberg, arvid&#64;libtorrent.org" />
<meta name=viewport content="width=device-width, initial-scale=1">
<script type="text/javascript">
/* <![CDATA[ */
(function() {
var s = document.createElement('script'), t = document.getElementsByTagName('script')[0];
s.type = 'text/javascript';
s.async = true;
s.src = 'http://api.flattr.com/js/0.6/load.js?mode=auto';
t.parentNode.insertBefore(s, t);
})();
/* ]]> */
</script>
<link rel="stylesheet" href="style.css" type="text/css" />
<link rel="stylesheet" type="text/css" href="rst.css" />
<style type="text/css">
/* Hides from IE-mac \*/
* html pre { height: 1%; }
/* End hide from IE-mac */
</style>
</head>
<body>
<div class="document">
<div id="container">
<table id="header">
<tr><td id="orange"></td>
<td id="logo">libtorrent</td></tr>
</table>
<div id="main">
<table class="docinfo" frame="void" rules="none">
<col class="docinfo-name" />
<col class="docinfo-content" />
<tbody valign="top">
<tr><th class="docinfo-name">Author:</th>
<td>Arvid Norberg, <a class="last reference external" href="mailto:arvid&#64;libtorrent.org">arvid&#64;libtorrent.org</a></td></tr>
<tr><th class="docinfo-name">Version:</th>
<td>1.1.0</td></tr>
</tbody>
</table>
<div id="librarySidebar"><ul class="simple">
<li><a class="reference external" href="https://github.com/arvidn/libtorrent/releases">download</a></li>
<li><a class="reference external" href="features.html">features</a></li>
<li><a class="reference external" href="examples.html">examples</a></li>
<li><a class="reference external" href="manual-ref.html">overview</a></li>
<li><a class="reference external" href="reference.html">reference documentation</a></li>
<li><a class="reference external" href="contributing.html">contributing</a></li>
<li><a class="reference external" href="building.html">building</a></li>
<li><a class="reference external" href="troubleshooting.html">troubleshooting</a></li>
<li><a class="reference external" href="tuning.html">tuning</a></li>
<li><a class="reference external" href="client_test.png">screenshot</a></li>
<li><a class="reference external" href="http://lists.sourceforge.net/lists/listinfo/libtorrent-discuss">mailing list</a> (<a class="reference external" href="http://dir.gmane.org/gmane.network.bit-torrent.libtorrent">archive</a>)</li>
<li><a class="reference external" href="projects.html">who's using libtorrent?</a></li>
<li><a class="reference external" href="https://github.com/arvidn/libtorrent/issues">report bugs</a></li>
<li><a class="reference external" href="https://github.com/arvidn/libtorrent">github page</a></li>
<li><a class="reference external" href="http://blog.libtorrent.org">blog</a></li>
</ul>
<hr class="docutils" />
<p>Extensions</p>
<ul class="simple">
<li><a class="reference external" href="utp.html">uTP</a></li>
<li><a class="reference external" href="extension_protocol.html">extensions protocol</a></li>
<li><a class="reference external" href="reference-Plugins.html">plugin interface</a></li>
<li><a class="reference external" href="streaming.html">streaming</a></li>
<li><a class="reference external" href="dht_extensions.html">DHT extensions</a></li>
<li><a class="reference external" href="dht_sec.html">DHT security extension</a></li>
<li><a class="reference external" href="dht_store.html">DHT store extension</a></li>
<li><a class="reference external" href="udp_tracker_protocol.html">UDP tracker protocol</a></li>
<li><a class="reference external" href="http://www.getright.com/seedtorrent.html">HTTP seed</a></li>
<li><a class="reference external" href="http://bittorrent.org/beps/bep_0012.html">multitracker</a></li>
</ul>
<hr class="docutils" />
<p>Bindings</p>
<ul class="simple">
<li><a class="reference external" href="http://libtorrent-ruby.rubyforge.org/">ruby bindings</a></li>
<li><a class="reference external" href="python_binding.html">python bindings</a></li>
</ul>
<hr class="docutils" />
<ul class="simple">
<li><a class="reference external" href="bittorrent.pdf">Introduction, slides</a></li>
</ul>
</div>
<div id="libraryBody"><div class="section" id="libtorrent">
<h1>libtorrent</h1>
<p>libtorrent is a feature complete C++ bittorrent implementation focusing
on efficiency and scalability. It runs on embedded devices as well as
desktops. It boasts a well documented library interface that is easy to
use. It comes with a <a class="reference external" href="client_test.html">simple bittorrent client</a> demonstrating the use of
the library.</p>
<p>The main goals of libtorrent are:</p>
<ul class="simple">
<li>to be cpu efficient</li>
<li>to be memory efficient</li>
<li>to be very easy to use</li>
</ul>
<div class="section" id="donate">
<h2>Donate</h2>
<p>Support the development of libtorrent</p>
<a class="FlattrButton" style="display:none;" href="http://libtorrent.org"></a>
<noscript><a href="https://flattr.com/thing/95662/libtorrent" target="_blank">
<img src="http://api.flattr.com/button/flattr-badge-large.png" alt="Flattr this" title="Flattr this" border="0" /></a></noscript></div>
<div class="section" id="feedback">
<h2>Feedback</h2>
<p>There's a <a class="reference external" href="http://lists.sourceforge.net/lists/listinfo/libtorrent-discuss">mailing list</a>, general libtorrent discussion.</p>
<p>You can usually find me as hydri in <tt class="docutils literal">#libtorrent</tt> on <tt class="docutils literal">irc.freenode.net</tt>.</p>
</div>
<div class="section" id="license">
<h2>license</h2>
<p>libtorrent is released under the <a class="reference external" href="http://opensource.org/licenses/bsd-license.php">BSD-license</a>.</p>
<p>This means that you can use the library in your project without having to
release its source code. The only requirement is that you give credit
to the author of the library by including the libtorrent license in your
software or documentation.</p>
<p>It is however greatly appreciated if additional features are contributed
back to the open source project. Patches can be emailed to the mailing
list or posted to the <a class="reference external" href="https://github.com/arvidn/libtorrent/issues">bug tracker</a>.</p>
</div>
<div class="section" id="acknowledgements">
<h2>Acknowledgements</h2>
<p>Written by Arvid Norberg. Copyright © 2003-2016</p>
<p>Contributions by Steven Siloti, Magnus Jonsson, Daniel Wallin and Cory Nelson</p>
<p>Thanks to Reimond Retz for bugfixes, suggestions and testing</p>
<p>Thanks to <a class="reference external" href="http://www.cs.umu.se">Umeå University</a> for providing development and test hardware.</p>
<p>Project is hosted by <a class="reference external" href="https://www.github.com/arvidn/libtorrent">github</a>.</p>
</div></div>
</div>
</div>
</div>
<div id="gradient"></div>
<div id="footer">
<table>
<tr>
<td><a href="index.html">home</a></td>
<td><a href="http://blog.libtorrent.org">blog</a></td>
<td><a href="utp.html">uTP</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/libtorrent/">download</a></td>
<td><a href="reference.html">documentation</a></td>
<td><a href="dht_store.html">DHT put extension</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/py-libtorrent/">python bindings</a></td>
<td><a href="features.html">features</a></td>
<td><a href="dht_sec.html">DHT security extension</a></td>
</tr>
<tr>
<td><a href="http://dir.gmane.org/gmane.network.bit-torrent.libtorrent">mailing list archive</a></td>
<td><a href="contributing.html">contributing</a></td>
<td><a href="streaming.html">streaming</a></td>
</tr>
<tr>
<td><a href="http://code.google.com/p/libtorrent/issues/entry">report a bug</a></td>
<td><a href="building.html">building</a></td>
<td><a href="bittorrent.pdf">bittorrent slides</a></td>
</tr>
</table>
</div>
<div id="filler"></div>
</div>
</body>
</html>

View File

@ -1,222 +0,0 @@
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.12: http://docutils.sourceforge.net/" />
<title>projects using libtorrent</title>
<meta name=viewport content="width=device-width, initial-scale=1">
<script type="text/javascript">
/* <![CDATA[ */
(function() {
var s = document.createElement('script'), t = document.getElementsByTagName('script')[0];
s.type = 'text/javascript';
s.async = true;
s.src = 'http://api.flattr.com/js/0.6/load.js?mode=auto';
t.parentNode.insertBefore(s, t);
})();
/* ]]> */
</script>
<link rel="stylesheet" href="style.css" type="text/css" />
<link rel="stylesheet" type="text/css" href="rst.css" />
<style type="text/css">
/* Hides from IE-mac \*/
* html pre { height: 1%; }
/* End hide from IE-mac */
</style>
</head>
<body>
<div class="document" id="projects-using-libtorrent">
<div id="container">
<table id="header">
<tr><td id="orange"></td>
<td id="logo">libtorrent</td></tr>
</table>
<div id="main">
<h1 class="title">projects using libtorrent</h1>
<p>These are some of the public projects that uses libtorrent. If you want your
project listed here, let <a class="reference external" href="mailto:arvid&#64;libtorrent.org">me</a> know.</p>
<div class="section" id="wyzo">
<h1>Wyzo</h1>
<p><a class="reference external" href="http://www.wyzo.com/">wyzo</a> is a media browser with built-in bittorrent support.</p>
</div>
<div class="section" id="deluge">
<h1>deluge</h1>
<p><a class="reference external" href="http://deluge-torrent.org/">deluge Torrent</a> is a more full-featured yet still lightweight bittorrent
client. It has the ability to automatically resume partial downloads and
background to the system tray.</p>
</div>
<div class="section" id="qbittorrent">
<h1>qBittorrent</h1>
<p><a class="reference external" href="http://www.qbittorrent.org/">qBittorrent</a> is a QT bittorrent client available for linux (likely portable to
most other desktops as well). Written by Christophe Dumez.</p>
</div>
<div class="section" id="tonidoplug">
<h1>tonidoplug</h1>
<p><a class="reference external" href="http://www.tonidoplug.com/">Tonidoplug</a> is a tiny, low-power, low-cost home server and
NAS device powered by Tonido software that allows you to access
your apps, files, music and media from anywhere.</p>
</div>
<div class="section" id="folx">
<h1>Folx</h1>
<p><a class="reference external" href="http://www.mac-downloader.com/">Folx</a> is a torrent client and download manager for Mac OS X.
The Free version of Folx has all the basic functionality of the torrent
client, which allows users to download and create torrent files.
Folx PRO (available for a small fee) features the possibility to search
for torrent files just from Folx interface. So there is no need to
browse through multiple torrent trackers searching for particular file.</p>
</div>
<div class="section" id="miro">
<h1>Miro</h1>
<p><a class="reference external" href="http://getmiro.com">Miro</a> is a free application for channels of internet video (also known as
video podcasts and video rss). Miro is designed to be easy to use and to give
you an elegant fullscreen viewing experience.</p>
</div>
<div class="section" id="moopolice">
<h1>MooPolice</h1>
<p><a class="reference external" href="http://www.moopolice.de">MooPolice</a> is a windows bittorrent client with a unique look.</p>
</div>
<div class="section" id="leechcraft">
<h1>LeechCraft</h1>
<p><a class="reference external" href="http://leechcraft.org/">LeechCraft</a> LeechCraft is a free open source cross-platform extensible
software, which primary goal is support of file sharing networks and protocols
like HTTP and FTP</p>
</div>
<div class="section" id="free-download-manager">
<h1>Free download manager</h1>
<p><a class="reference external" href="http://www.freedownloadmanager.org/">FDM</a> is a powerful, easy-to-use and absolutely free download accelerator and
manager. Moreover, FDM is 100% safe, open-source software distributed under
GPL License.</p>
</div>
<div class="section" id="btg">
<h1>btg</h1>
<p><a class="reference external" href="http://btg.berlios.de//">btg</a> is a unix bittorrent client which is run as a daemon. It has multiple user
interfaces which connects to the daemon. One GUI (Gtkmm), one terminal
interface (ncurses) and one web interface (accessable through a web browser).
Written by Michael Wojciechowski and Johan Strom.</p>
</div>
<div class="section" id="electric-sheep">
<h1>electric sheep</h1>
<p><a class="reference external" href="http://electricsheep.org">electric sheep</a> is a screensaver which collectively generates animations and
lets the users vote which one to live on.</p>
</div>
<div class="section" id="tvitty">
<h1>Tvitty</h1>
<p><a class="reference external" href="http://tvitty.com">tvitty</a> is a bittorrent client for Vista Media Center, which allows
searching and downloading of torrents directly on your TV.</p>
</div>
<div class="section" id="hrktorrent">
<h1>hrktorrent</h1>
<p><a class="reference external" href="http://50hz.ws/hrktorrent/">hrktorrent</a> hrktorrent is a light console torrent client written in C++.</p>
</div>
<div class="section" id="fatrat">
<h1>FatRat</h1>
<p><a class="reference external" href="http://fatrat.dolezel.info">FatRat</a> is an open source download manager for Linux/Unix systems written in
C++ with the help of the Trolltech Qt 4 library. It's simple to use and
install.</p>
</div>
<div class="section" id="halite-bittorrent">
<h1>halite BitTorrent</h1>
<p><a class="reference external" href="http://www.binarynotions.com/halite-bittorrent-client">Halite</a> is a windows bittorrent client controllabel via an xml-rpc
interface.</p>
</div>
<div class="section" id="arctic-torrent">
<h1>Arctic Torrent</h1>
<p><a class="reference external" href="http://www.int64.org/arctic.html">Arctic Torrent</a> is a light-weight
bittorrent client for windows.
Written by Cory Nelson.</p>
</div>
<div class="section" id="bubba">
<h1>bubba</h1>
<p><a class="reference external" href="http://excito.com/bubba/about-bubba.html">Bubba</a> is a mini-sized server, designed to fit your home better than
an always running PC. Boasting Torrent downloader, DAAP streaming,
Web, E-mail, printer and FTP server etc.</p>
</div>
<div class="section" id="tvblob">
<h1>tvblob</h1>
<p>The <a class="reference external" href="http://www.tvblob.com">BLOBbox</a> represents the ability to harness all of the content available
on the web, without any filtering or pre-selection by a third party just
like surfing the web.</p>
<p>This means that anyone will have the ability to reach viewers via the Internet
directly on TV, without them having to connect a PC.</p>
</div>
<div class="section" id="flush">
<h1>Flush</h1>
<p><a class="reference external" href="https://sourceforge.net/projects/flush/">Flush</a> is a GTK-based BitTorrent client.</p>
</div>
<div class="section" id="lince">
<h1>Lince</h1>
<p><a class="reference external" href="http://lincetorrent.sourceforge.net/">Lince</a> is a bittorrent client using libtorrent to handle bittorrent protocol
and gtkmm for the interface, it has been designed to be a light and full
featured client.</p>
</div>
<div class="section" id="linkage">
<h1>Linkage</h1>
<p><a class="reference external" href="http://code.google.com/p/linkage/">Linkage</a> is a gtkmm client that aims to be middle weight.</p>
</div>
<div class="section" id="bitfox">
<h1>Bitfox</h1>
<p><a class="reference external" href="http://code.google.com/p/bitfox/">Bitfox</a> is a firefox plugin integrating bittorrent downloads in firefox.</p>
</div>
<div class="section" id="bitslug">
<h1>BitSlug</h1>
<p><a class="reference external" href="http://bitslug.sourceforge.net/">BitSlug</a> is a MacOSX cocoa client.</p>
</div>
<div class="section" id="delco">
<h1>DelCo</h1>
<p><a class="reference external" href="http://delco.cs.tut.fi/">DelCo</a> is a research project at Tampere university of technology, finland.</p>
</div>
<div class="section" id="torrent2exe">
<h1>Torrent2Exe</h1>
<p><a class="reference external" href="http://torrent2exe.com">Torrent2Exe</a> Torrent2exe is a small BitTorrent client. Its basic idea is to
let users download a custom-built EXE program with the torrent file
integrated into it.</p>
</div>
<div class="section" id="zyxel-nsa-220">
<h1>ZyXEL NSA-220</h1>
<p><a class="reference external" href="http://us.zyxel.com/Products/details.aspx?PC1IndexFlag=20050125090459&amp;CategoryGroupNo=758BFE64-3A95-463C-9E1E-3D30E3B58D9C">ZyXEL</a> NSA220 makes it easy to store, protect and share files between users
on your home network. The built-in DLNA server works with many set top boxes
to allow you to play back music, watch video files, or view photos on your
home theater system, while the built in download manager can automatically
download video and audio podcasts as well as allow you to download bittorrent
files without needing to leave your computer on.</p>
</div>
</div>
</div>
<div id="gradient"></div>
<div id="footer">
<table>
<tr>
<td><a href="index.html">home</a></td>
<td><a href="http://blog.libtorrent.org">blog</a></td>
<td><a href="utp.html">uTP</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/libtorrent/">download</a></td>
<td><a href="reference.html">documentation</a></td>
<td><a href="dht_store.html">DHT put extension</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/py-libtorrent/">python bindings</a></td>
<td><a href="features.html">features</a></td>
<td><a href="dht_sec.html">DHT security extension</a></td>
</tr>
<tr>
<td><a href="http://dir.gmane.org/gmane.network.bit-torrent.libtorrent">mailing list archive</a></td>
<td><a href="contributing.html">contributing</a></td>
<td><a href="streaming.html">streaming</a></td>
</tr>
<tr>
<td><a href="http://code.google.com/p/libtorrent/issues/entry">report a bug</a></td>
<td><a href="building.html">building</a></td>
<td><a href="bittorrent.pdf">bittorrent slides</a></td>
</tr>
</table>
</div>
<div id="filler"></div>
</div>
</body>
</html>

View File

@ -1,209 +0,0 @@
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.12: http://docutils.sourceforge.net/" />
<title>libtorrent python binding</title>
<meta name="author" content="Arvid Norberg, arvid&#64;libtorrent.org" />
<meta name=viewport content="width=device-width, initial-scale=1">
<script type="text/javascript">
/* <![CDATA[ */
(function() {
var s = document.createElement('script'), t = document.getElementsByTagName('script')[0];
s.type = 'text/javascript';
s.async = true;
s.src = 'http://api.flattr.com/js/0.6/load.js?mode=auto';
t.parentNode.insertBefore(s, t);
})();
/* ]]> */
</script>
<link rel="stylesheet" href="style.css" type="text/css" />
<link rel="stylesheet" type="text/css" href="rst.css" />
<style type="text/css">
/* Hides from IE-mac \*/
* html pre { height: 1%; }
/* End hide from IE-mac */
</style>
</head>
<body>
<div class="document" id="libtorrent-python-binding">
<div id="container">
<table id="header">
<tr><td id="orange"></td>
<td id="logo">libtorrent</td></tr>
</table>
<div id="main">
<h1 class="title">libtorrent python binding</h1>
<table class="docinfo" frame="void" rules="none">
<col class="docinfo-name" />
<col class="docinfo-content" />
<tbody valign="top">
<tr><th class="docinfo-name">Author:</th>
<td>Arvid Norberg, <a class="last reference external" href="mailto:arvid&#64;libtorrent.org">arvid&#64;libtorrent.org</a></td></tr>
</tbody>
</table>
<div class="contents topic" id="table-of-contents">
<p class="topic-title first">Table of contents</p>
<ul class="simple">
<li><a class="reference internal" href="#building" id="id2">building</a><ul>
<li><a class="reference internal" href="#building-using-setup-py" id="id3">building using setup.py</a></li>
<li><a class="reference internal" href="#building-using-boost-build" id="id4">building using boost build</a></li>
</ul>
</li>
<li><a class="reference internal" href="#using-libtorrent-in-python" id="id5">using libtorrent in python</a></li>
</ul>
</div>
<div class="section" id="building">
<h1>building</h1>
<p>Building the libtorrent python bindings will produce a shared library (DLL)
which is a python module that can be imported in a python program.</p>
<div class="section" id="building-using-setup-py">
<h2>building using setup.py</h2>
<p>There is a <tt class="docutils literal">setup.py</tt> shipped with libtorrent that can be used on windows.
On windows the setup.py will invoke <tt class="docutils literal">bjam</tt> and assume that you have boost
sources at <tt class="docutils literal">$BOOST_PATH</tt>. The resulting executable is self-contained, it does
not depend any boost or libtorrent dlls.</p>
<p>On other systems, the setup.py is generated by running
<tt class="docutils literal">./configure <span class="pre">--enable-python-binding</span></tt>.</p>
<p>To build the Python bindings do:</p>
<ol class="arabic">
<li><p class="first">Run:</p>
<pre class="literal-block">
python setup.py build
</pre>
</li>
<li><p class="first">As root, run:</p>
<pre class="literal-block">
python setup.py install
</pre>
</li>
</ol>
</div>
<div class="section" id="building-using-boost-build">
<h2>building using boost build</h2>
<p>To set up your build environment, you need to add some settings to your
<tt class="docutils literal"><span class="pre">$BOOST_BUILD_PATH/user-config.jam</span></tt>.</p>
<p>Make sure your user config contains the following line:</p>
<pre class="literal-block">
using python : 2.3 ;
</pre>
<p>Set the version to the version of python you have installed or want to use. If
you've installed python in a non-standard location, you have to add the prefix
path used when you installed python as a second option. Like this:</p>
<pre class="literal-block">
using python : 2.6 : /usr/bin/python2.6 : /usr/include/python2.6 : /usr/lib/python2.6 ;
</pre>
<p>The bindings require <em>at least</em> python version 2.2.</p>
<p>For more information on how to install and set up boost-build, see the
<a class="reference external" href="building.html#step-2-setup-bbv2">building libtorrent</a> section.</p>
<p>Once you have boost-build set up, you cd to the <tt class="docutils literal">bindings/python</tt>
directory and invoke <tt class="docutils literal">bjam</tt> with the apropriate settings. For the available
build variants, see <a class="reference external" href="building.html#step-3-building-libtorrent">libtorrent build options</a>.</p>
<p>For example:</p>
<pre class="literal-block">
$ bjam dht-support=on link=static
</pre>
<p>On Mac OS X, this will produce the following python module:</p>
<pre class="literal-block">
bin/darwin-4.0/release/dht-support-on/link-static/logging-none/threading-multi/libtorrent.so
</pre>
</div>
</div>
<div class="section" id="using-libtorrent-in-python">
<h1>using libtorrent in python</h1>
<p>The python interface is nearly identical to the C++ interface. Please refer to
the <a class="reference external" href="reference.html">library reference</a>. The main differences are:</p>
<dl class="docutils">
<dt>asio::tcp::endpoint</dt>
<dd>The endpoint type is represented as a tuple of a string (as the address) and an int for
the port number. E.g. <tt class="docutils literal">('127.0.0.1', 6881)</tt> represents the localhost port 6881.</dd>
<dt>libtorrent::time_duration</dt>
<dd>The time duration is represented as a number of seconds in a regular integer.</dd>
</dl>
<p>The following functions takes a reference to a container that is filled with
entries by the function. The python equivalent of these functions instead returns
a list of entries.</p>
<ul class="simple">
<li>torrent_handle::get_peer_info</li>
<li>torrent_handle::file_progress</li>
<li>torrent_handle::get_download_queue</li>
<li>torrent_handle::piece_availability</li>
</ul>
<p><tt class="docutils literal"><span class="pre">create_torrent::add_node()</span></tt> takes two arguments, one string and one integer,
instead of a pair. The string is the address and the integer is the port.</p>
<p><tt class="docutils literal"><span class="pre">session::set_settings()</span></tt> not only accepts a <tt class="docutils literal">session_settings</tt> object, but also
a dictionary with keys matching the names of the members of the <tt class="docutils literal">session_settings</tt> struct.
When calling <tt class="docutils literal">set_settings</tt>, the dictionary does not need to have every settings set,
keys that are not present, are set to their default value.</p>
<p>For backwards compatibility, <tt class="docutils literal"><span class="pre">session::settings()</span></tt> still returns a <tt class="docutils literal">session_settings</tt>
struct. To get a python dictionary of the settings, call <tt class="docutils literal"><span class="pre">session::get_settings</span></tt>.</p>
<p>For an example python program, see <tt class="docutils literal">client.py</tt> in the <tt class="docutils literal">bindings/python</tt>
directory.</p>
<p>A very simple example usage of the module would be something like this:</p>
<pre class="literal-block">
import libtorrent as lt
import time
ses = lt.session()
ses.listen_on(6881, 6891)
e = lt.bdecode(open(&quot;test.torrent&quot;, 'rb').read())
info = lt.torrent_info(e)
params = { 'save_path': '.', \
'storage_mode': lt.storage_mode_t.storage_mode_sparse, \
'ti': info }
h = ses.add_torrent(params)
s = h.status()
while (not s.is_seeding):
s = h.status()
state_str = ['queued', 'checking', 'downloading metadata', \
'downloading', 'finished', 'seeding', 'allocating']
print '%.2f%% complete (down: %.1f kb/s up: %.1f kB/s peers: %d) %s' % \
(s.progress * 100, s.download_rate / 1000, s.upload_rate / 1000, \
s.num_peers, state_str[s.state])
time.sleep(1)
</pre>
</div>
</div>
</div>
<div id="gradient"></div>
<div id="footer">
<table>
<tr>
<td><a href="index.html">home</a></td>
<td><a href="http://blog.libtorrent.org">blog</a></td>
<td><a href="utp.html">uTP</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/libtorrent/">download</a></td>
<td><a href="reference.html">documentation</a></td>
<td><a href="dht_store.html">DHT put extension</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/py-libtorrent/">python bindings</a></td>
<td><a href="features.html">features</a></td>
<td><a href="dht_sec.html">DHT security extension</a></td>
</tr>
<tr>
<td><a href="http://dir.gmane.org/gmane.network.bit-torrent.libtorrent">mailing list archive</a></td>
<td><a href="contributing.html">contributing</a></td>
<td><a href="streaming.html">streaming</a></td>
</tr>
<tr>
<td><a href="http://code.google.com/p/libtorrent/issues/entry">report a bug</a></td>
<td><a href="building.html">building</a></td>
<td><a href="bittorrent.pdf">bittorrent slides</a></td>
</tr>
</table>
</div>
<div id="filler"></div>
</div>
</body>
</html>

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because one or more lines are too long

View File

@ -1,90 +0,0 @@
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.12: http://docutils.sourceforge.net/" />
<title>libtorrent manual</title>
<meta name="author" content="Arvid Norberg, arvid&#64;libtorrent.org" />
<meta name=viewport content="width=device-width, initial-scale=1">
<script type="text/javascript">
/* <![CDATA[ */
(function() {
var s = document.createElement('script'), t = document.getElementsByTagName('script')[0];
s.type = 'text/javascript';
s.async = true;
s.src = 'http://api.flattr.com/js/0.6/load.js?mode=auto';
t.parentNode.insertBefore(s, t);
})();
/* ]]> */
</script>
<link rel="stylesheet" href="style.css" type="text/css" />
<link rel="stylesheet" type="text/css" href="rst.css" />
<style type="text/css">
/* Hides from IE-mac \*/
* html pre { height: 1%; }
/* End hide from IE-mac */
</style>
</head>
<body>
<div class="document" id="libtorrent-manual">
<div id="container">
<table id="header">
<tr><td id="orange"></td>
<td id="logo">libtorrent</td></tr>
</table>
<div id="main">
<h1 class="title">libtorrent manual</h1>
<table class="docinfo" frame="void" rules="none">
<col class="docinfo-name" />
<col class="docinfo-content" />
<tbody valign="top">
<tr><th class="docinfo-name">Author:</th>
<td>Arvid Norberg, <a class="last reference external" href="mailto:arvid&#64;libtorrent.org">arvid&#64;libtorrent.org</a></td></tr>
<tr><th class="docinfo-name">Version:</th>
<td>1.1.0</td></tr>
</tbody>
</table>
<p>The following troubleshooting chart may help in finding out why torrents fail
to download. It is not complete, please submit suggestions via mail to
<a class="reference external" href="mailto:arvid&#64;libtorrent.org">arvid&#64;libtorrent.org</a> or to the <a class="reference external" href="http://lists.sourceforge.net/lists/listinfo/libtorrent-discuss">mailing list</a>. Ideally in the form of patches
against <tt class="docutils literal">docs/troubleshooting.dot</tt>.</p>
<p><a class="reference external" href="troubleshooting.png"><img alt="thumb" src="troubleshooting_thumb.png" /></a></p>
</div>
</div>
<div id="gradient"></div>
<div id="footer">
<table>
<tr>
<td><a href="index.html">home</a></td>
<td><a href="http://blog.libtorrent.org">blog</a></td>
<td><a href="utp.html">uTP</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/libtorrent/">download</a></td>
<td><a href="reference.html">documentation</a></td>
<td><a href="dht_store.html">DHT put extension</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/py-libtorrent/">python bindings</a></td>
<td><a href="features.html">features</a></td>
<td><a href="dht_sec.html">DHT security extension</a></td>
</tr>
<tr>
<td><a href="http://dir.gmane.org/gmane.network.bit-torrent.libtorrent">mailing list archive</a></td>
<td><a href="contributing.html">contributing</a></td>
<td><a href="streaming.html">streaming</a></td>
</tr>
<tr>
<td><a href="http://code.google.com/p/libtorrent/issues/entry">report a bug</a></td>
<td><a href="building.html">building</a></td>
<td><a href="bittorrent.pdf">bittorrent slides</a></td>
</tr>
</table>
</div>
<div id="filler"></div>
</div>
</body>
</html>

Binary file not shown.

Before

Width:  |  Height:  |  Size: 365 KiB

View File

@ -1,628 +0,0 @@
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.12: http://docutils.sourceforge.net/" />
<title>libtorrent manual</title>
<meta name="author" content="Arvid Norberg, arvid&#64;libtorrent.org" />
<meta name=viewport content="width=device-width, initial-scale=1">
<script type="text/javascript">
/* <![CDATA[ */
(function() {
var s = document.createElement('script'), t = document.getElementsByTagName('script')[0];
s.type = 'text/javascript';
s.async = true;
s.src = 'http://api.flattr.com/js/0.6/load.js?mode=auto';
t.parentNode.insertBefore(s, t);
})();
/* ]]> */
</script>
<link rel="stylesheet" href="style.css" type="text/css" />
<link rel="stylesheet" type="text/css" href="rst.css" />
<style type="text/css">
/* Hides from IE-mac \*/
* html pre { height: 1%; }
/* End hide from IE-mac */
</style>
</head>
<body>
<div class="document" id="libtorrent-manual">
<div id="container">
<table id="header">
<tr><td id="orange"></td>
<td id="logo">libtorrent</td></tr>
</table>
<div id="main">
<h1 class="title">libtorrent manual</h1>
<table class="docinfo" frame="void" rules="none">
<col class="docinfo-name" />
<col class="docinfo-content" />
<tbody valign="top">
<tr><th class="docinfo-name">Author:</th>
<td>Arvid Norberg, <a class="last reference external" href="mailto:arvid&#64;libtorrent.org">arvid&#64;libtorrent.org</a></td></tr>
<tr><th class="docinfo-name">Version:</th>
<td>1.1.0</td></tr>
</tbody>
</table>
<div class="contents topic" id="table-of-contents">
<p class="topic-title first">Table of contents</p>
<ul class="simple">
<li><a class="reference internal" href="#tuning-libtorrent" id="id1">tuning libtorrent</a></li>
<li><a class="reference internal" href="#reducing-memory-footprint" id="id2">reducing memory footprint</a><ul>
<li><a class="reference internal" href="#disable-disk-cache" id="id3">disable disk cache</a></li>
<li><a class="reference internal" href="#remove-torrents" id="id4">remove torrents</a></li>
<li><a class="reference internal" href="#socket-buffer-sizes" id="id5">socket buffer sizes</a></li>
<li><a class="reference internal" href="#peer-list-size" id="id6">peer list size</a></li>
<li><a class="reference internal" href="#send-buffer-watermark" id="id7">send buffer watermark</a></li>
<li><a class="reference internal" href="#optimize-hashing-for-memory-usage" id="id8">optimize hashing for memory usage</a></li>
<li><a class="reference internal" href="#reduce-executable-size" id="id9">reduce executable size</a></li>
</ul>
</li>
<li><a class="reference internal" href="#play-nice-with-the-disk" id="id10">play nice with the disk</a></li>
<li><a class="reference internal" href="#high-performance-seeding" id="id11">high performance seeding</a><ul>
<li><a class="reference internal" href="#file-pool" id="id12">file pool</a></li>
<li><a class="reference internal" href="#disk-cache" id="id13">disk cache</a></li>
<li><a class="reference internal" href="#ssd-as-level-2-cache" id="id14">SSD as level 2 cache</a></li>
<li><a class="reference internal" href="#utp-tcp-mixed-mode" id="id15">uTP-TCP mixed mode</a></li>
<li><a class="reference internal" href="#send-buffer-low-watermark" id="id16">send buffer low watermark</a></li>
<li><a class="reference internal" href="#peers" id="id17">peers</a></li>
<li><a class="reference internal" href="#torrent-limits" id="id18">torrent limits</a></li>
<li><a class="reference internal" href="#sha-1-hashing" id="id19">SHA-1 hashing</a></li>
</ul>
</li>
<li><a class="reference internal" href="#scalability" id="id20">scalability</a></li>
<li><a class="reference internal" href="#benchmarking" id="id21">benchmarking</a><ul>
<li><a class="reference internal" href="#disk-metrics" id="id22">disk metrics</a></li>
</ul>
</li>
<li><a class="reference internal" href="#understanding-the-disk-threads" id="id23">understanding the disk threads</a></li>
<li><a class="reference internal" href="#contributions" id="id24">contributions</a></li>
</ul>
</div>
<div class="section" id="tuning-libtorrent">
<h1>tuning libtorrent</h1>
<p>libtorrent expose most constants used in the bittorrent engine for
customization through the <tt class="docutils literal">session_settings</tt>. This makes it possible to
test and tweak the parameters for certain algorithms to make a client
that fits a wide range of needs. From low memory embedded devices to
servers seeding thousands of torrents. The default settings in libtorrent
are tuned for an end-user bittorrent client running on a normal desktop
computer.</p>
<p>This document describes techniques to benchmark libtorrent performance
and how parameters are likely to affect it.</p>
</div>
<div class="section" id="reducing-memory-footprint">
<h1>reducing memory footprint</h1>
<p>These are things you can do to reduce the memory footprint of libtorrent. You get
some of this by basing your default <tt class="docutils literal">session_settings</tt> on the <tt class="docutils literal">min_memory_usage()</tt>
setting preset function.</p>
<p>Keep in mind that lowering memory usage will affect performance, always profile
and benchmark your settings to determine if it's worth the trade-off.</p>
<p>The typical buffer usage of libtorrent, for a single download, with the cache
size set to 256 blocks (256 * 16 kiB = 4 MiB) is:</p>
<pre class="literal-block">
read cache: 128.6 (2058 kiB)
write cache: 103.5 (1656 kiB)
receive buffers: 7.3 (117 kiB)
send buffers: 4.8 (77 kiB)
hash temp: 0.001 (19 Bytes)
</pre>
<p>The receive buffers is proportional to the number of connections we make, and is
limited by the total number of connections in the session (default is 200).</p>
<p>The send buffers is proportional to the number of upload slots that are allowed
in the session. The default is auto configured based on the observed upload rate.</p>
<p>The read and write cache can be controlled (see section below).</p>
<p>The &quot;hash temp&quot; entry size depends on whether or not hashing is optimized for
speed or memory usage. In this test run it was optimized for memory usage.</p>
<div class="section" id="disable-disk-cache">
<h2>disable disk cache</h2>
<p>The bulk of the memory libtorrent will use is used for the disk cache. To save
the absolute most amount of memory, you can disable the cache by setting
<tt class="docutils literal"><span class="pre">session_settings::cache_size</span></tt> to 0. You might want to consider using the cache
but just disable caching read operations. You do this by settings
<tt class="docutils literal"><span class="pre">session_settings::use_read_cache</span></tt> to false. This is the main factor in how much
memory will be used by the client. Keep in mind that you will degrade performance
by disabling the cache. You should benchmark the disk access in order to make an
informed trade-off.</p>
</div>
<div class="section" id="remove-torrents">
<h2>remove torrents</h2>
<p>Torrents that have been added to libtorrent will inevitably use up memory, even
when it's paused. A paused torrent will not use any peer connection objects or
any send or receive buffers though. Any added torrent holds the entire .torrent
file in memory, it also remembers the entire list of peers that it's heard about
(which can be fairly long unless it's capped). It also retains information about
which blocks and pieces we have on disk, which can be significant for torrents
with many pieces.</p>
<p>If you need to minimize the memory footprint, consider removing torrents from
the session rather than pausing them. This will likely only make a difference
when you have a very large number of torrents in a session.</p>
<p>The downside of removing them is that they will no longer be auto-managed. Paused
auto managed torrents are scraped periodically, to determine which torrents are
in the greatest need of seeding, and libtorrent will prioritize to seed those.</p>
</div>
<div class="section" id="socket-buffer-sizes">
<h2>socket buffer sizes</h2>
<p>You can make libtorrent explicitly set the kernel buffer sizes of all its peer
sockets. If you set this to a low number, you may see reduced throughput, especially
for high latency connections. It is however an opportunity to save memory per
connection, and might be worth considering if you have a very large number of
peer connections. This memory will not be visible in your process, this sets
the amount of kernel memory is used for your sockets.</p>
<p>Change this by setting <tt class="docutils literal"><span class="pre">session_settings::recv_socket_buffer_size</span></tt> and
<tt class="docutils literal"><span class="pre">session_settings::send_socket_buffer_size</span></tt>.</p>
</div>
<div class="section" id="peer-list-size">
<h2>peer list size</h2>
<p>The default maximum for the peer list is 4000 peers. For IPv4 peers, each peer
entry uses 32 bytes, which ends up using 128 kB per torrent. If seeding 4 popular
torrents, the peer lists alone uses about half a megabyte.</p>
<p>The default limit is the same for paused torrents as well, so if you have a
large number of paused torrents (that are popular) it will be even more
significant.</p>
<p>If you're short of memory, you should consider lowering the limit. 500 is probably
enough. You can do this by setting <tt class="docutils literal"><span class="pre">session_settings::max_peerlist_size</span></tt> to
the max number of peers you want in a torrent's peer list. This limit applies per
torrent. For 5 torrents, the total number of peers in peerlists will be 5 times
the setting.</p>
<p>You should also lower the same limit but for paused torrents. It might even make sense
to set that even lower, since you only need a few peers to start up while waiting
for the tracker and DHT to give you fresh ones. The max peer list size for paused
torrents is set by <tt class="docutils literal"><span class="pre">session_settings::max_paused_peerlist_size</span></tt>.</p>
<p>The drawback of lowering this number is that if you end up in a position where
the tracker is down for an extended period of time, your only hope of finding live
peers is to go through your list of all peers you've ever seen. Having a large
peer list will also help increase performance when starting up, since the torrent
can start connecting to peers in parallel with connecting to the tracker.</p>
</div>
<div class="section" id="send-buffer-watermark">
<h2>send buffer watermark</h2>
<p>The send buffer watermark controls when libtorrent will ask the disk I/O thread
to read blocks from disk, and append it to a peer's send buffer.</p>
<p>When the send buffer has fewer than or equal number of bytes as
<tt class="docutils literal"><span class="pre">session_settings::send_buffer_watermark</span></tt>, the peer will ask the disk I/O thread
for more data to send. The trade-off here is between wasting memory by having too
much data in the send buffer, and hurting send rate by starving out the socket,
waiting for the disk read operation to complete.</p>
<p>If your main objective is memory usage and you're not concerned about being able
to achieve high send rates, you can set the watermark to 9 bytes. This will guarantee
that no more than a single (16 kiB) block will be on the send buffer at a time, for
all peers. This is the least amount of memory possible for the send buffer.</p>
<p>You should benchmark your max send rate when adjusting this setting. If you have
a very fast disk, you are less likely see a performance hit.</p>
</div>
<div class="section" id="optimize-hashing-for-memory-usage">
<h2>optimize hashing for memory usage</h2>
<p>When libtorrent is doing hash checks of a file, or when it re-reads a piece that
was just completed to verify its hash, there are two options. The default one
is optimized for speed, which allocates buffers for the entire piece, reads in
the whole piece in one read call, then hashes it.</p>
<p>The second option is to optimize for memory usage instead, where a single buffer
is allocated, and the piece is read one block at a time, hashing it as each
block is read from the file. For low memory environments, this latter approach
is recommended. Change this by settings <tt class="docutils literal"><span class="pre">session_settings::optimize_hashing_for_speed</span></tt>
to false. This will significantly reduce peak memory usage, especially for
torrents with very large pieces.</p>
</div>
<div class="section" id="reduce-executable-size">
<h2>reduce executable size</h2>
<p>Compilers generally add a significant number of bytes to executables that make use
of C++ exceptions. By disabling exceptions (-fno-exceptions on GCC), you can
reduce the executable size with up to 45%. In order to build without exception
support, you need to patch parts of boost.</p>
<p>Also make sure to optimize for size when compiling.</p>
<p>Another way of reducing the executable size is to disable code that isn't used.
There are a number of <tt class="docutils literal">TORRENT_*</tt> macros that control which features are included
in libtorrent. If these macros are used to strip down libtorrent, make sure the same
macros are defined when building libtorrent as when linking against it. If these
are different the structures will look different from the libtorrent side and from
the client side and memory corruption will follow.</p>
<p>One, probably, safe macro to define is <tt class="docutils literal">TORRENT_NO_DEPRECATE</tt> which removes all
deprecated functions and struct members. As long as no deprecated functions are
relied upon, this should be a simple way to eliminate a little bit of code.</p>
<p>For all available options, see the <a class="reference external" href="building.html">building libtorrent</a> secion.</p>
</div>
</div>
<div class="section" id="play-nice-with-the-disk">
<h1>play nice with the disk</h1>
<p>When checking a torrent, libtorrent will try to read as fast as possible from the disk.
The only thing that might hold it back is a CPU that is slow at calculating SHA-1 hashes,
but typically the file checking is limited by disk read speed. Most operating systems
today do not prioritize disk access based on the importance of the operation, this means
that checking a torrent might delay other disk accesses, such as virtual memory swapping
or just loading file by other (interactive) applications.</p>
<p>In order to play nicer with the disk, and leave some spare time for it to service other
processes that might be of higher importance to the end-user, you can introduce a sleep
between the disc accesses. This is a direct tradeoff between how fast you can check a
torrent and how soft you will hit the disk.</p>
<p>You control this by setting the <tt class="docutils literal"><span class="pre">session_settings::file_checks_delay_per_block</span></tt> to greater
than zero. This number is the number of milliseconds to sleep between each read of 16 kiB.</p>
<p>The sleeps are not necessarily in between each 16 kiB block (it might be read in larger chunks),
but the number will be multiplied by the number of blocks that were read, to maintain the
same semantics.</p>
</div>
<div class="section" id="high-performance-seeding">
<h1>high performance seeding</h1>
<p>In the case of a high volume seed, there are two main concerns. Performance and scalability.
This translates into high send rates, and low memory and CPU usage per peer connection.</p>
<div class="section" id="file-pool">
<h2>file pool</h2>
<p>libtorrent keeps an LRU file cache. Each file that is opened, is stuck in the cache. The main
purpose of this is because of anti-virus software that hooks on file-open and file close to
scan the file. Anti-virus software that does that will significantly increase the cost of
opening and closing files. However, for a high performance seed, the file open/close might
be so frequent that it becomes a significant cost. It might therefore be a good idea to allow
a large file descriptor cache. Adjust this though <tt class="docutils literal"><span class="pre">session_settings::file_pool_size</span></tt>.</p>
<p>Don't forget to set a high rlimit for file descriptors in your process as well. This limit
must be high enough to keep all connections and files open.</p>
</div>
<div class="section" id="disk-cache">
<h2>disk cache</h2>
<p>You typically want to set the cache size to as high as possible. The
<tt class="docutils literal"><span class="pre">session_settings::cache_size</span></tt> is specified in 16 kiB blocks. Since you're seeding,
the cache would be useless unless you also set <tt class="docutils literal"><span class="pre">session_settings::use_read_cache</span></tt>
to true.</p>
<p>In order to increase the possibility of read cache hits, set the
<tt class="docutils literal"><span class="pre">session_settings::cache_expiry</span></tt> to a large number. This won't degrade anything as
long as the client is only seeding, and not downloading any torrents.</p>
<p>In order to increase the disk cache hit rate, you can enable suggest messages based on
what's in the read cache. To do this, set <tt class="docutils literal"><span class="pre">session_settings::suggest_mode</span></tt> to
<tt class="docutils literal"><span class="pre">session_settings::suggest_read_cache</span></tt>. This will send suggest messages to peers
for the most recently used pieces in the read cache. This is especially useful if you
also enable explicit read cache, by settings <tt class="docutils literal"><span class="pre">session_settings::explicit_read_cache</span></tt>
to the number of pieces to keep in the cache. The explicit read cache will make the
disk read cache stick, and not be evicted by cache misses. The explicit read cache
will automatically pull in the rarest pieces in the read cache.</p>
<p>Assuming that you seed much more data than you can keep in the cache, to a large
numbers of peers (so that the read cache wouldn't be useful anyway), this may be a
good idea.</p>
<p>When peers first connect, libtorrent will send them a number of allow-fast messages,
which lets the peers download certain pieces even when they are choked, since peers
are choked by default, this often triggers immediate requests for those pieces. In the
case of using explicit read cache and suggesting those pieces, allowing fast pieces
should be disabled, to not systematically trigger requests for pieces that are not cached
for all peers. You can turn off allow-fast by settings <tt class="docutils literal"><span class="pre">session_settings::allowed_fast_set_size</span></tt>
to 0.</p>
<p>As an alternative to the explicit cache and suggest messages, there's a <em>guided cache</em>
mode. This means the size of the read cache line that's stored in the cache is determined
based on the upload rate to the peer that triggered the read operation. The idea being
that slow peers don't use up a disproportional amount of space in the cache. This
is enabled through <tt class="docutils literal"><span class="pre">session_settings::guided_read_cache</span></tt>.</p>
<p>In cases where the assumption is that the cache is only used as a read-ahead, and that no
other peer will ever request the same block while it's still in the cache, the read
cache can be set to be <em>volatile</em>. This means that every block that is requested out of
the read cache is removed immediately. This saves a significant amount of cache space
which can be used as read-ahead for other peers. This mode should <strong>never</strong> be combined
with either <tt class="docutils literal">explicit_read_cache</tt> or <tt class="docutils literal">suggest_read_cache</tt>, since those uses opposite
strategies for the read cache. You don't want to on one hand attract peers to request
the same pieces, and on the other hand assume that they won't request the same pieces
and drop them when the first peer requests it. To enable volatile read cache, set
<tt class="docutils literal"><span class="pre">session_settings::volatile_read_cache</span></tt> to true.</p>
</div>
<div class="section" id="ssd-as-level-2-cache">
<h2>SSD as level 2 cache</h2>
<p>It is possible to introduce a second level of cache, below the RAM disk cache. This is done
by setting <tt class="docutils literal"><span class="pre">session_settings::mmap_cache</span></tt> to a file path pointing to the SSD drive, and
increasing the <tt class="docutils literal"><span class="pre">session_settings::cache_size</span></tt> to the number of 16 kiB blocks would fit
on the drive (or less).</p>
<p>This will allocate disk buffers (for reading and writing) from a memory region that has
been mapped to the specified file. If the drive this file lives on is not significantly
faster than the destination drive, performance will be degraded. The point is to take
advantage primarily of the fast read speed from SSD drives and use it to extend the read
cache, improving seed performance.</p>
<p>Which parts of the cache that actually live in RAM is determined by the operating system.</p>
<p>Note that when using this feature, any block which ends up being pulled from the mmapped
file will be considered a cache hit.</p>
</div>
<div class="section" id="utp-tcp-mixed-mode">
<h2>uTP-TCP mixed mode</h2>
<p>libtorrent supports <a class="reference external" href="utp.html">uTP</a>, which has a delay based congestion controller. In order to
avoid having a single TCP bittorrent connection completely starve out any uTP connection,
there is a mixed mode algorithm. This attempts to detect congestion on the uTP peers and
throttle TCP to avoid it taking over all bandwidth. This balances the bandwidth resources
between the two protocols. When running on a network where the bandwidth is in such an
abundance that it's virtually infinite, this algorithm is no longer necessary, and might
even be harmful to throughput. It is adviced to experiment with the
<tt class="docutils literal"><span class="pre">session_setting::mixed_mode_algorithm</span></tt>, setting it to <tt class="docutils literal"><span class="pre">session_settings::prefer_tcp</span></tt>.
This setting entirely disables the balancing and unthrottles all connections. On a typical
home connection, this would mean that none of the benefits of uTP would be preserved
(the modem's send buffer would be full at all times) and uTP connections would for the most
part be squashed by the TCP traffic.</p>
</div>
<div class="section" id="send-buffer-low-watermark">
<h2>send buffer low watermark</h2>
<p>libtorrent uses a low watermark for send buffers to determine when a new piece should
be requested from the disk I/O subsystem, to be appended to the send buffer. The low
watermark is determined based on the send rate of the socket. It needs to be large
enough to not draining the socket's send buffer before the disk operation completes.</p>
<p>The watermark is bound to a max value, to avoid buffer sizes growing out of control.
The default max send buffer size might not be enough to sustain very high upload rates,
and you might have to increase it. It's specified in bytes in
<tt class="docutils literal"><span class="pre">session_settings::send_buffer_watermark</span></tt>.</p>
</div>
<div class="section" id="peers">
<h2>peers</h2>
<p>First of all, in order to allow many connections, set the global connection limit
high, <tt class="docutils literal"><span class="pre">session::set_max_connections()</span></tt>. Also set the upload rate limit to
infinite, <tt class="docutils literal"><span class="pre">session::set_upload_rate_limit()</span></tt>, passing 0 means infinite.</p>
<p>When dealing with a large number of peers, it might be a good idea to have slightly
stricter timeouts, to get rid of lingering connections as soon as possible.</p>
<p>There are a couple of relevant settings: <tt class="docutils literal"><span class="pre">session_settings::request_timeout</span></tt>,
<tt class="docutils literal"><span class="pre">session_settings::peer_timeout</span></tt> and <tt class="docutils literal"><span class="pre">session_settings::inactivity_timeout</span></tt>.</p>
<p>For seeds that are critical for a delivery system, you most likely want to allow
multiple connections from the same IP. That way two people from behind the same NAT
can use the service simultaneously. This is controlled by
<tt class="docutils literal"><span class="pre">session_settings::allow_multiple_connections_per_ip</span></tt>.</p>
<p>In order to always unchoke peers, turn off automatic unchoke
<tt class="docutils literal"><span class="pre">session_settings::auto_upload_slots</span></tt> and set the number of upload slots to a large
number via <tt class="docutils literal"><span class="pre">session::set_max_uploads()</span></tt>, or use -1 (which means infinite).</p>
</div>
<div class="section" id="torrent-limits">
<h2>torrent limits</h2>
<p>To seed thousands of torrents, you need to increase the <tt class="docutils literal"><span class="pre">session_settings::active_limit</span></tt>
and <tt class="docutils literal"><span class="pre">session_settings::active_seeds</span></tt>.</p>
</div>
<div class="section" id="sha-1-hashing">
<h2>SHA-1 hashing</h2>
<p>When downloading at very high rates, it is possible to have the CPU be the bottleneck
for passing every downloaded byte through SHA-1. In order to enable calculating SHA-1
hashes in parallel, on multi-core systems, set <tt class="docutils literal"><span class="pre">session_settings::hashing_threads</span></tt>
to the number of threads libtorrent should start to do SHA-1 hashing. This defaults
to 1, and only if that thread is close to saturating one core does it make sense to
increase the number of threads.</p>
</div>
</div>
<div class="section" id="scalability">
<h1>scalability</h1>
<p>In order to make more efficient use of the libtorrent interface when running a large
number of torrents simultaneously, one can use the <tt class="docutils literal"><span class="pre">session::get_torrent_status()</span></tt> call
together with <tt class="docutils literal"><span class="pre">session::refresh_torrent_status()</span></tt>. Keep in mind that every call into
libtorrent that return some value have to block your thread while posting a message to
the main network thread and then wait for a response (calls that don't return any data
will simply post the message and then immediately return). The time this takes might
become significant once you reach a few hundred torrents (depending on how many calls
you make to each torrent and how often). <tt class="docutils literal">get_torrent_status</tt> lets you query the
status of all torrents in a single call. This will actually loop through all torrents
and run a provided predicate function to determine whether or not to include it in
the returned vector. If you have a lot of torrents, you might want to update the status
of only certain torrents. For instance, you might only be interested in torrents that
are being downloaded.</p>
<p>The intended use of these functions is to start off by calling <tt class="docutils literal">get_torrent_status</tt>
to get a list of all torrents that match your criteria. Then call <tt class="docutils literal">refresh_torrent_status</tt>
on that list. This will only refresh the status for the torrents in your list, and thus
ignore all other torrents you might be running. This may save a significant amount of
time, especially if the number of torrents you're interested in is small. In order to
keep your list of interested torrents up to date, you can either call <tt class="docutils literal">get_torrent_status</tt>
from time to time, to include torrents you might have become interested in since the last
time. In order to stop refreshing a certain torrent, simply remove it from the list.</p>
<p>A more efficient way however, would be to subscribe to status alert notifications, and
update your list based on these alerts. There are alerts for when torrents are added, removed,
paused, resumed, completed etc. Doing this ensures that you only query status for the
minimal set of torrents you are actually interested in.</p>
</div>
<div class="section" id="benchmarking">
<h1>benchmarking</h1>
<p>There is a bunch of built-in instrumentation of libtorrent that can be used to get an insight
into what it's doing and how well it performs. This instrumentation is enabled by defining
preprocessor symbols when building.</p>
<p>There are also a number of scripts that parses the log files and generates graphs (requires
gnuplot and python).</p>
<div class="section" id="disk-metrics">
<h2>disk metrics</h2>
<p>To enable disk I/O instrumentation, define <tt class="docutils literal">TORRENT_DISK_STATS</tt> when building. When built
with this configuration libtorrent will create three log files, measuring various aspects of
the disk I/O. The following table is an overview of these files and what they measure.</p>
<table border="1" class="docutils">
<colgroup>
<col width="30%" />
<col width="70%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">filename</th>
<th class="head">description</th>
</tr>
</thead>
<tbody valign="top">
<tr><td><tt class="docutils literal">disk_io_thread.log</tt></td>
<td>This is a log of which operation the disk I/O thread is
engaged in, with timestamps. This tells you what the thread
is spending its time doing.</td>
</tr>
<tr><td><tt class="docutils literal">disk_buffers.log</tt></td>
<td>This log keeps track of what the buffers allocated from the
disk buffer pool are used for. There are 5 categories.
receive buffer, send buffer, write cache, read cache and
temporary hash storage. This is key when optimizing memory
usage.</td>
</tr>
<tr><td><tt class="docutils literal">disk_access.log</tt></td>
<td>This is a low level log of read and write operations, with
timestamps and file offsets. The file offsets are byte
offsets in the torrent (not in any particular file, in the
case of a multi-file torrent). This can be used as an
estimate of the physical drive location. The purpose of
this log is to identify the amount of seeking the drive has
to do.</td>
</tr>
</tbody>
</table>
<div class="section" id="disk-io-thread-log">
<h3>disk_io_thread.log</h3>
<p>The structure of this log is simple. For each line, there are two columns, a timestamp and
the operation that was started. There is a special operation called <tt class="docutils literal">idle</tt> which means
it looped back to the top and started waiting for new jobs. If there are more jobs to
handle immediately, the <tt class="docutils literal">idle</tt> state is still there, but the timestamp is the same as the
next job that is handled.</p>
<p>Some operations have a 3:rd column with an optional parameter. <tt class="docutils literal">read</tt> and <tt class="docutils literal">write</tt> tells
you the number of bytes that were requested to be read or written. <tt class="docutils literal">flushing</tt> tells you
the number of bytes that were flushed from the disk cache.</p>
<p>This is an example excerpt from a log:</p>
<pre class="literal-block">
3702 idle
3706 check_fastresume
3707 idle
4708 save_resume_data
4708 idle
8230 read 16384
8255 idle
8431 read 16384
</pre>
<p>The script to parse this log and generate a graph is called <tt class="docutils literal">parse_disk_log.py</tt>. It takes
the log file as the first command line argument, and produces a file: <tt class="docutils literal">disk_io.png</tt>.
The time stamp is in milliseconds since start.</p>
<p>You can pass in a second, optional, argument to specify the window size it will average
the time measurements over. The default is 5 seconds. For long test runs, it might be interesting
to increase that number. It is specified as a number of seconds.</p>
<img alt="disk_io.png" src="disk_io.png" />
<p>This is an example graph generated by the parse script.</p>
</div>
<div class="section" id="disk-buffers-log">
<h3>disk_buffers.log</h3>
<p>The disk buffer log tells you where the buffer memory is used. The log format has a time stamp,
the name of the buffer usage which use-count changed, colon, and the new number of blocks that are
in use for this particular key. For example:</p>
<pre class="literal-block">
23671 write cache: 18
23671 receive buffer: 3
24153 receive buffer: 2
24153 write cache: 19
24154 receive buffer: 3
24198 receive buffer: 2
24198 write cache: 20
24202 receive buffer: 3
24305 send buffer: 0
24305 send buffer: 1
24909 receive buffer: 2
24909 write cache: 21
24910 receive buffer: 3
</pre>
<p>The time stamp is in milliseconds since start.</p>
<p>To generate a graph, use <tt class="docutils literal">parse_disk_buffer_log.py</tt>. It takes the log file as the first
command line argument. It generates <tt class="docutils literal">disk_buffer.png</tt>.</p>
<img alt="disk_buffer_sample.png" src="disk_buffer_sample.png" />
<p>This is an example graph generated by the parse script.</p>
</div>
<div class="section" id="disk-access-log">
<h3>disk_access.log</h3>
<p><em>The disk access log is now binary</em></p>
<p>The disc access log has three fields. The timestamp (milliseconds since start), operation
and offset. The offset is the absolute offset within the torrent (not within a file). This
log is only useful when you're downloading a single torrent, otherwise the offsets will not
be unique.</p>
<p>In order to easily plot this directly in gnuplot, without parsing it, there are two lines
associated with each read or write operation. The first one is the offset where the operation
started, and the second one is where the operation ended.</p>
<p>Example:</p>
<pre class="literal-block">
15437 read 301187072
15437 read_end 301203456
16651 read 213385216
16680 read_end 213647360
25879 write 249036800
25879 write_end 249298944
26811 read 325582848
26943 read_end 325844992
36736 read 367001600
36766 read_end 367263744
</pre>
<p>The disk access log does not have any good visualization tool yet. There is however a gnuplot
file, <tt class="docutils literal">disk_access.gnuplot</tt> which assumes <tt class="docutils literal">disk_access.log</tt> is in the current directory.</p>
<img alt="disk_access.png" src="disk_access.png" />
<p>The density of the disk seeks tells you how hard the drive has to work.</p>
</div>
</div>
</div>
<div class="section" id="understanding-the-disk-threads">
<h1>understanding the disk threads</h1>
<p><em>This section is somewhat outdated, there are potentially more than one disk
thread</em></p>
<p>All disk operations are funneled through a separate thread, referred to as the
disk thread. The main interface to the disk thread is a queue where disk jobs
are posted, and the results of these jobs are then posted back on the main
thread's io_service.</p>
<p>A disk job is essentially one of:</p>
<ol class="arabic">
<li><dl class="first docutils">
<dt>write this block to disk, i.e. a write job. For the most part this is just a</dt>
<dd><p class="first last">matter of sticking the block in the disk cache, but if we've run out of
cache space or completed a whole piece, we'll also flush blocks to disk.
This is typically very fast, since the OS just sticks these buffers in its
write cache which will be flushed at a later time, presumably when the drive
head will pass the place on the platter where the blocks go.</p>
</dd>
</dl>
</li>
<li><dl class="first docutils">
<dt>read this block from disk. The first thing that happens is we look in the</dt>
<dd><p class="first last">cache to see if the block is already in RAM. If it is, we'll return
immediately with this block. If it's a cache miss, we'll have to hit the
disk. Here we decide to defer this job. We find the physical offset on the
drive for this block and insert the job in an ordered queue, sorted by the
physical location. At a later time, once we don't have any more non-read
jobs left in the queue, we pick one read job out of the ordered queue and
service it. The order we pick jobs out of the queue is according to an
elevator cursor moving up and down along the ordered queue of read jobs. If
we have enough space in the cache we'll read read_cache_line_size number of
blocks and stick those in the cache. This defaults to 32 blocks. If the
system supports asynchronous I/O (Windows, Linux, Mac OS X, BSD, Solars for
instance), jobs will be issued immediately to the OS. This especially
increases read throughput, since the OS has a much greater flexibility to
reorder the read jobs.</p>
</dd>
</dl>
</li>
</ol>
<p>Other disk job consist of operations that needs to be synchronized with the
disk I/O, like renaming files, closing files, flushing the cache, updating the
settings etc. These are relatively rare though.</p>
</div>
<div class="section" id="contributions">
<h1>contributions</h1>
<p>If you have added instrumentation for some part of libtorrent that is not
covered here, or if you have improved any of the parser scrips, please consider
contributing it back to the project.</p>
<p>If you have run tests and found that some algorithm or default value in
libtorrent is suboptimal, please contribute that knowledge back as well, to
allow us to improve the library.</p>
<p>If you have additional suggestions on how to tune libtorrent for any specific
use case, please let us know and we'll update this document.</p>
</div>
</div>
</div>
<div id="gradient"></div>
<div id="footer">
<table>
<tr>
<td><a href="index.html">home</a></td>
<td><a href="http://blog.libtorrent.org">blog</a></td>
<td><a href="utp.html">uTP</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/libtorrent/">download</a></td>
<td><a href="reference.html">documentation</a></td>
<td><a href="dht_store.html">DHT put extension</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/py-libtorrent/">python bindings</a></td>
<td><a href="features.html">features</a></td>
<td><a href="dht_sec.html">DHT security extension</a></td>
</tr>
<tr>
<td><a href="http://dir.gmane.org/gmane.network.bit-torrent.libtorrent">mailing list archive</a></td>
<td><a href="contributing.html">contributing</a></td>
<td><a href="streaming.html">streaming</a></td>
</tr>
<tr>
<td><a href="http://code.google.com/p/libtorrent/issues/entry">report a bug</a></td>
<td><a href="building.html">building</a></td>
<td><a href="bittorrent.pdf">bittorrent slides</a></td>
</tr>
</table>
</div>
<div id="filler"></div>
</div>
</body>
</html>

View File

@ -1,605 +0,0 @@
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.12: http://docutils.sourceforge.net/" />
<title>Bittorrent udp-tracker protocol extension</title>
<meta name="author" content="Arvid Norberg, arvid&#64;libtorrent.org" />
<meta name=viewport content="width=device-width, initial-scale=1">
<script type="text/javascript">
/* <![CDATA[ */
(function() {
var s = document.createElement('script'), t = document.getElementsByTagName('script')[0];
s.type = 'text/javascript';
s.async = true;
s.src = 'http://api.flattr.com/js/0.6/load.js?mode=auto';
t.parentNode.insertBefore(s, t);
})();
/* ]]> */
</script>
<link rel="stylesheet" href="style.css" type="text/css" />
<link rel="stylesheet" type="text/css" href="rst.css" />
<style type="text/css">
/* Hides from IE-mac \*/
* html pre { height: 1%; }
/* End hide from IE-mac */
</style>
</head>
<body>
<div class="document" id="bittorrent-udp-tracker-protocol-extension">
<div id="container">
<table id="header">
<tr><td id="orange"></td>
<td id="logo">libtorrent</td></tr>
</table>
<div id="main">
<h1 class="title">Bittorrent udp-tracker protocol extension</h1>
<table class="docinfo" frame="void" rules="none">
<col class="docinfo-name" />
<col class="docinfo-content" />
<tbody valign="top">
<tr><th class="docinfo-name">Author:</th>
<td>Arvid Norberg, <a class="last reference external" href="mailto:arvid&#64;libtorrent.org">arvid&#64;libtorrent.org</a></td></tr>
</tbody>
</table>
<div class="contents topic" id="table-of-contents">
<p class="topic-title first">Table of contents</p>
<ul class="simple">
<li><a class="reference internal" href="#introduction" id="id2">introduction</a></li>
<li><a class="reference internal" href="#connecting" id="id3">connecting</a></li>
<li><a class="reference internal" href="#announcing" id="id4">announcing</a></li>
<li><a class="reference internal" href="#scraping" id="id5">scraping</a></li>
<li><a class="reference internal" href="#errors" id="id6">errors</a></li>
<li><a class="reference internal" href="#actions" id="id7">actions</a></li>
<li><a class="reference internal" href="#extensions" id="id8">extensions</a><ul>
<li><a class="reference internal" href="#authentication" id="id9">authentication</a></li>
</ul>
</li>
<li><a class="reference internal" href="#request-string" id="id10">request string</a></li>
<li><a class="reference internal" href="#credits" id="id11">credits</a></li>
</ul>
</div>
<div class="section" id="introduction">
<h1>introduction</h1>
<p>A tracker with the protocol &quot;udp://&quot; in its URI
is supposed to be contacted using this protocol.</p>
<p>This protocol is supported by
<a class="reference external" href="http://xbtt.sourceforge.net">xbt-tracker</a>.</p>
<p>For additional information and descritptions of
the terminology used in this document, see
the <a class="reference external" href="http://wiki.theory.org/index.php/BitTorrentSpecification">protocol specification</a></p>
<p>All values are sent in network byte order (big endian). The sizes
are specified with ANSI-C standard types.</p>
<p>If no response to a request is received within 15 seconds, resend
the request. If no reply has been received after 60 seconds, stop
retrying.</p>
</div>
<div class="section" id="connecting">
<h1>connecting</h1>
<p>Client sends packet:</p>
<table border="1" class="docutils">
<colgroup>
<col width="18%" />
<col width="28%" />
<col width="54%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">size</th>
<th class="head">name</th>
<th class="head">description</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>int64_t</td>
<td>connection_id</td>
<td>Must be initialized to 0x41727101980
in network byte order. This will
identify the protocol.</td>
</tr>
<tr><td>int32_t</td>
<td>action</td>
<td>0 for a connection request</td>
</tr>
<tr><td>int32_t</td>
<td>transaction_id</td>
<td>Randomized by client.</td>
</tr>
</tbody>
</table>
<p>Server replies with packet:</p>
<table border="1" class="docutils">
<colgroup>
<col width="18%" />
<col width="28%" />
<col width="54%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">size</th>
<th class="head">name</th>
<th class="head">description</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>int32_t</td>
<td>action</td>
<td>Describes the type of packet, in this
case it should be 0, for connect.
If 3 (for error) see <a class="reference internal" href="#errors">errors</a>.</td>
</tr>
<tr><td>int32_t</td>
<td>transaction_id</td>
<td>Must match the transaction_id sent
from the client.</td>
</tr>
<tr><td>int64_t</td>
<td>connection_id</td>
<td>A connection id, this is used when
further information is exchanged with
the tracker, to identify you.
This connection id can be reused for
multiple requests, but if it's cached
for too long, it will not be valid
anymore.</td>
</tr>
</tbody>
</table>
</div>
<div class="section" id="announcing">
<h1>announcing</h1>
<p>Client sends packet:</p>
<table border="1" class="docutils">
<colgroup>
<col width="18%" />
<col width="28%" />
<col width="54%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">size</th>
<th class="head">name</th>
<th class="head">description</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>int64_t</td>
<td>connection_id</td>
<td>The connection id acquired from
establishing the connection.</td>
</tr>
<tr><td>int32_t</td>
<td>action</td>
<td>Action. in this case, 1 for announce.
See <a class="reference internal" href="#actions">actions</a>.</td>
</tr>
<tr><td>int32_t</td>
<td>transaction_id</td>
<td>Randomized by client.</td>
</tr>
<tr><td>int8_t[20]</td>
<td>info_hash</td>
<td>The info-hash of the torrent you want
announce yourself in.</td>
</tr>
<tr><td>int8_t[20]</td>
<td>peer_id</td>
<td>Your peer id.</td>
</tr>
<tr><td>int64_t</td>
<td>downloaded</td>
<td>The number of byte you've downloaded
in this session.</td>
</tr>
<tr><td>int64_t</td>
<td>left</td>
<td>The number of bytes you have left to
download until you're finished.</td>
</tr>
<tr><td>int64_t</td>
<td>uploaded</td>
<td>The number of bytes you have uploaded
in this session.</td>
</tr>
<tr><td>int32_t</td>
<td>event</td>
<td><p class="first">The event, one of</p>
<blockquote class="last">
<ul class="simple">
<li>none = 0</li>
<li>completed = 1</li>
<li>started = 2</li>
<li>stopped = 3</li>
</ul>
</blockquote>
</td>
</tr>
<tr><td>uint32_t</td>
<td>ip</td>
<td>Your ip address. Set to 0 if you want
the tracker to use the <tt class="docutils literal">sender</tt> of
this udp packet.</td>
</tr>
<tr><td>uint32_t</td>
<td>key</td>
<td>A unique key that is randomized by the
client.</td>
</tr>
<tr><td>int32_t</td>
<td>num_want</td>
<td>The maximum number of peers you want
in the reply. Use -1 for default.</td>
</tr>
<tr><td>uint16_t</td>
<td>port</td>
<td>The port you're listening on.</td>
</tr>
<tr><td>uint16_t</td>
<td>extensions</td>
<td>See <a class="reference internal" href="#extensions">extensions</a></td>
</tr>
</tbody>
</table>
<p>Server replies with packet:</p>
<table border="1" class="docutils">
<colgroup>
<col width="18%" />
<col width="28%" />
<col width="54%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">size</th>
<th class="head">name</th>
<th class="head">description</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>int32_t</td>
<td>action</td>
<td>The action this is a reply to. Should
in this case be 1 for announce.
If 3 (for error) see <a class="reference internal" href="#errors">errors</a>.
See <a class="reference internal" href="#actions">actions</a>.</td>
</tr>
<tr><td>int32_t</td>
<td>transaction_id</td>
<td>Must match the transaction_id sent
in the announce request.</td>
</tr>
<tr><td>int32_t</td>
<td>interval</td>
<td>the number of seconds you should wait
until reannouncing yourself.</td>
</tr>
<tr><td>int32_t</td>
<td>leechers</td>
<td>The number of peers in the swarm that
has not finished downloading.</td>
</tr>
<tr><td>int32_t</td>
<td>seeders</td>
<td>The number of peers in the swarm that
has finished downloading and are
seeding.</td>
</tr>
</tbody>
</table>
<p>The rest of the server reply is a variable number of the following structure:</p>
<table border="1" class="docutils">
<colgroup>
<col width="18%" />
<col width="28%" />
<col width="54%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">size</th>
<th class="head">name</th>
<th class="head">description</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>int32_t</td>
<td>ip</td>
<td>The ip of a peer in the swarm.</td>
</tr>
<tr><td>uint16_t</td>
<td>port</td>
<td>The peer's listen port.</td>
</tr>
</tbody>
</table>
</div>
<div class="section" id="scraping">
<h1>scraping</h1>
<p>Client sends packet:</p>
<table border="1" class="docutils">
<colgroup>
<col width="18%" />
<col width="28%" />
<col width="54%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">size</th>
<th class="head">name</th>
<th class="head">description</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>int64_t</td>
<td>connection_id</td>
<td>The connection id retreived from the
establishing of the connection.</td>
</tr>
<tr><td>int32_t</td>
<td>action</td>
<td>The action, in this case, 2 for
scrape. See <a class="reference internal" href="#actions">actions</a>.</td>
</tr>
<tr><td>int32_t</td>
<td>transaction_id</td>
<td>Randomized by client.</td>
</tr>
</tbody>
</table>
<p>The following structure is repeated for each info-hash to scrape, but limited by
the MTU.</p>
<table border="1" class="docutils">
<colgroup>
<col width="18%" />
<col width="28%" />
<col width="54%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">size</th>
<th class="head">name</th>
<th class="head">description</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>int8_t[20]</td>
<td>info_hash</td>
<td>The info hash that is to be scraped.</td>
</tr>
</tbody>
</table>
<p>Server replies with packet:</p>
<table border="1" class="docutils">
<colgroup>
<col width="18%" />
<col width="28%" />
<col width="54%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">size</th>
<th class="head">name</th>
<th class="head">description</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>int32_t</td>
<td>action</td>
<td>The action, should in this case be
2 for scrape.
If 3 (for error) see <a class="reference internal" href="#errors">errors</a>.</td>
</tr>
<tr><td>int32_t</td>
<td>transaction_id</td>
<td>Must match the sent transaction id.</td>
</tr>
</tbody>
</table>
<p>The rest of the packet contains the following structures once for each info-hash
you asked in the scrape request.</p>
<table border="1" class="docutils">
<colgroup>
<col width="18%" />
<col width="28%" />
<col width="54%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">size</th>
<th class="head">name</th>
<th class="head">description</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>int32_t</td>
<td>complete</td>
<td>The current number of connected seeds.</td>
</tr>
<tr><td>int32_t</td>
<td>downloaded</td>
<td>The number of times this torrent has
been downloaded.</td>
</tr>
<tr><td>int32_t</td>
<td>incomplete</td>
<td>The current number of connected
leechers.</td>
</tr>
</tbody>
</table>
</div>
<div class="section" id="errors">
<h1>errors</h1>
<p>In case of a tracker error,</p>
<p>server replies packet:</p>
<table border="1" class="docutils">
<colgroup>
<col width="18%" />
<col width="28%" />
<col width="54%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">size</th>
<th class="head">name</th>
<th class="head">description</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>int32_t</td>
<td>action</td>
<td>The action, in this case 3, for error.
See <a class="reference internal" href="#actions">actions</a>.</td>
</tr>
<tr><td>int32_t</td>
<td>transaction_id</td>
<td>Must match the transaction_id sent
from the client.</td>
</tr>
<tr><td>int8_t[]</td>
<td>error_string</td>
<td>The rest of the packet is a string
describing the error.</td>
</tr>
</tbody>
</table>
</div>
<div class="section" id="actions">
<h1>actions</h1>
<p>The action fields has the following encoding:</p>
<blockquote>
<ul class="simple">
<li>connect = 0</li>
<li>announce = 1</li>
<li>scrape = 2</li>
<li>error = 3 (only in server replies)</li>
</ul>
</blockquote>
</div>
<div class="section" id="extensions">
<h1>extensions</h1>
<p>The extensions field is a bitmask. The following
bits are assigned:</p>
<blockquote>
<ul class="simple">
<li>1 = <a class="reference internal" href="#authentication">authentication</a>.</li>
<li>2 = <a class="reference internal" href="#request-string">request string</a>.</li>
</ul>
</blockquote>
<p>If multiple bits are present in the extension field, the extension
bodies are appended to the packet in the order of least significant
bit first. For instance, if both bit 1 and 2 are set, the extension
represented by bit 1 comes first, followed by the extension represented
by bit 2.</p>
<div class="section" id="authentication">
<h2>authentication</h2>
<p>The packet will have an authentication part
appended to it. It has the following format:</p>
<table border="1" class="docutils">
<colgroup>
<col width="18%" />
<col width="28%" />
<col width="54%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">size</th>
<th class="head">name</th>
<th class="head">description</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>int8_t</td>
<td>username_length</td>
<td>The number of characters in the
username.</td>
</tr>
<tr><td>int8_t[]</td>
<td>username</td>
<td>The username, the number of characters
as specified in the previous field.</td>
</tr>
<tr><td>uint8_t[8]</td>
<td>passwd_hash</td>
<td>sha1(packet + sha1(password))
The packet in this case means the
entire packet except these 8 bytes
that are the password hash. These are
the 8 first bytes (most significant)
from the 20 bytes hash calculated.</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="request-string">
<h1>request string</h1>
<p>The request string extension is meant to allow torrent creators pass along
cookies back to the tracker. This can be useful for authenticating that a
torrent is allowed to be tracked by a tracker for instance. It could also
be used to authenticate users by generating torrents with unique tokens
in the tracker URL for each user. The extension body has the following format:</p>
<table border="1" class="docutils">
<colgroup>
<col width="18%" />
<col width="28%" />
<col width="54%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">size</th>
<th class="head">name</th>
<th class="head">description</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>int8_t</td>
<td>request length</td>
<td>The number of bytes in the request
string.</td>
</tr>
<tr><td>int8_t[]</td>
<td>request string</td>
<td>The string that comes after the host-
name and port in the udp tracker URL.
Typically this starts with &quot;/announce&quot;
The bittorrent client is not expected
to append query string arguments for
stats reporting, like &quot;uploaded&quot; and
&quot;downloaded&quot; since this is already
reported in the udp tracker protocol.
However, the client is free to add
arguments as extensions.</td>
</tr>
</tbody>
</table>
</div>
<div class="section" id="credits">
<h1>credits</h1>
<p>Protocol designed by Olaf van der Spek and extended by Arvid Norberg</p>
</div>
</div>
</div>
<div id="gradient"></div>
<div id="footer">
<table>
<tr>
<td><a href="index.html">home</a></td>
<td><a href="http://blog.libtorrent.org">blog</a></td>
<td><a href="utp.html">uTP</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/libtorrent/">download</a></td>
<td><a href="reference.html">documentation</a></td>
<td><a href="dht_store.html">DHT put extension</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/py-libtorrent/">python bindings</a></td>
<td><a href="features.html">features</a></td>
<td><a href="dht_sec.html">DHT security extension</a></td>
</tr>
<tr>
<td><a href="http://dir.gmane.org/gmane.network.bit-torrent.libtorrent">mailing list archive</a></td>
<td><a href="contributing.html">contributing</a></td>
<td><a href="streaming.html">streaming</a></td>
</tr>
<tr>
<td><a href="http://code.google.com/p/libtorrent/issues/entry">report a bug</a></td>
<td><a href="building.html">building</a></td>
<td><a href="bittorrent.pdf">bittorrent slides</a></td>
</tr>
</table>
</div>
<div id="filler"></div>
</div>
</body>
</html>

View File

@ -1,370 +0,0 @@
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.12: http://docutils.sourceforge.net/" />
<title>libtorrent manual</title>
<meta name="author" content="Arvid Norberg, arvid&#64;libtorrent.org" />
<meta name=viewport content="width=device-width, initial-scale=1">
<script type="text/javascript">
/* <![CDATA[ */
(function() {
var s = document.createElement('script'), t = document.getElementsByTagName('script')[0];
s.type = 'text/javascript';
s.async = true;
s.src = 'http://api.flattr.com/js/0.6/load.js?mode=auto';
t.parentNode.insertBefore(s, t);
})();
/* ]]> */
</script>
<link rel="stylesheet" href="style.css" type="text/css" />
<link rel="stylesheet" type="text/css" href="rst.css" />
<style type="text/css">
/* Hides from IE-mac \*/
* html pre { height: 1%; }
/* End hide from IE-mac */
</style>
</head>
<body>
<div class="document" id="libtorrent-manual">
<div id="container">
<table id="header">
<tr><td id="orange"></td>
<td id="logo">libtorrent</td></tr>
</table>
<div id="main">
<h1 class="title">libtorrent manual</h1>
<table class="docinfo" frame="void" rules="none">
<col class="docinfo-name" />
<col class="docinfo-content" />
<tbody valign="top">
<tr><th class="docinfo-name">Author:</th>
<td>Arvid Norberg, <a class="last reference external" href="mailto:arvid&#64;libtorrent.org">arvid&#64;libtorrent.org</a></td></tr>
<tr><th class="docinfo-name">Version:</th>
<td>1.1.0</td></tr>
</tbody>
</table>
<div class="contents topic" id="table-of-contents">
<p class="topic-title first">Table of contents</p>
<ul class="simple">
<li><a class="reference internal" href="#utp" id="id1">uTP</a><ul>
<li><a class="reference internal" href="#rationale" id="id2">rationale</a></li>
<li><a class="reference internal" href="#tcp" id="id3">TCP</a></li>
<li><a class="reference internal" href="#ledbat-congestion-controller" id="id4">LEDBAT congestion controller</a></li>
<li><a class="reference internal" href="#one-way-delays" id="id5">one way delays</a></li>
<li><a class="reference internal" href="#path-mtu-discovery" id="id6">Path MTU discovery</a></li>
<li><a class="reference internal" href="#clock-drift" id="id7">clock drift</a></li>
<li><a class="reference internal" href="#features" id="id8">features</a></li>
</ul>
</li>
</ul>
</div>
<div class="section" id="utp">
<h1>uTP</h1>
<p>uTP (uTorrent transport protocol) is a transport protocol which uses one-way
delay measurements for its congestion controller. This article is about uTP
in general and specifically about libtorrent's implementation of it.</p>
<div class="section" id="rationale">
<h2>rationale</h2>
<p>One of the most common problems users are experiencing using bittorrent is
that their internet &quot;stops working&quot;. This can be caused by a number of things,
for example:</p>
<ol class="arabic simple">
<li>a home router that crashes or slows down when its NAT pin-hole
table overflows, triggered by DHT or simply many TCP connections.</li>
<li>a home router that crashes or slows down by UDP traffic (caused by
the DHT)</li>
<li>a home DSL or cable modem having its send buffer filled up by outgoing
data, and the buffer fits seconds worth of bytes. This adds seconds
of delay on interactive traffic. For a web site that needs 10 round
trips to load this may mean 10s of seconds of delay to load compared
to without bittorrent. Skype or other delay sensitive applications
would be affected even more.</li>
</ol>
<p>This document will cover (3).</p>
<p>Typically this is solved by asking the user to enter a number of bytes
that the client is allowed to send per second (i.e. setting an upload
rate limit). The common recommendation is to set this limit to 80% of the
uplink's capacity. This is to leave some headroom for things like TCP
ACKs as well as the user's interactive use of the connection such as
browsing the web or checking email.</p>
<p>There are two major drawbacks with this technique:</p>
<ol class="arabic simple">
<li>The user needs to actively make this setting (very few protocols
require the user to provide this sort of information). This also
means the user needs to figure out what its up-link capacity is.
This is unfortunately a number that many ISPs are not advertizing
(because it's often much lower than the download capacity) which
might make it hard to find.</li>
<li>The 20% headroom is wasted most of the time. Whenever the user
is not using the internet connection for anything, those extra 20%
could have been used by bittorrent to upload, but they're already
allocated for interactive traffic. On top of that, 20% of the up-link
is often not enough to give a good and responsive browsing experience.</li>
</ol>
<p>The ideal bandwidth allocation would be to use 100% for bittorrent when
there is no interactive cross traffic, and 100% for interactive traffic
whenever there is any. This would not waste any bandwidth while the user
is idling, and it would make for a much better experience when the user
is using the internet connection for other things.</p>
<p>This is what uTP does.</p>
</div>
<div class="section" id="tcp">
<h2>TCP</h2>
<p>The reason TCP will fill the send buffer, and cause the delay on all traffic,
is because its congestion control is <em>only</em> based on packet loss (and timeout).</p>
<p>Since the modem is buffering, packets won't get dropped until the entire queue
is full, and no more packets will fit. The packets will be dropped, TCP will
detect this within an RTT or so. When TCP notices a packet loss, it will slow
down its send rate and the queue will start to drain again. However, TCP will
immediately start to ramp up its send rate again until the buffer is full and
it detects packet loss again.</p>
<p>TCP is designed to fully utilize the link capacity, without causing congestion.
Whenever it sense congestion (through packet loss) it backs off. TCP is not
designed to keep delays low. When you get the first packet loss (assuming the
kind of queue described above, tail-queue) it is already too late. Your queue
is full and you have the maximum amount of delay your modem can provide.</p>
<p>TCP controls its send rate by limiting the number of bytes in-flight at any
given time. This limit is called congestion window (<em>cwnd</em> for short). During
steady state, the congestion window is constantly increasing linearly. Each
packet that is successfully transferred will increase cwnd.</p>
<pre class="literal-block">
cwnd
send_rate = ----
RTT
</pre>
<p>Send rate is proportional to cwnd divided by RTT. A smaller cwnd will cause
the send rate to be lower and a larger cwnd will cause the send rate to be
higher.</p>
<p>Using a congestion window instead of controlling the rate directly is simple
because it also introduces an upper bound for memory usage for packets that
haven't been ACKed yet and needs to be kept around.</p>
<p>The behavior of TCP, where it bumps up against the ceiling, backs off and then
starts increasing again until it hits the ceiling again, forms a saw tooth shape.
If the modem wouldn't have any send buffer at all, a single TCP stream would
not be able to fully utilize the link because of this behavior, since it would
only fully utilize the link right before the packet loss and the back-off.</p>
</div>
<div class="section" id="ledbat-congestion-controller">
<h2>LEDBAT congestion controller</h2>
<p>The congestion controller in uTP is called <a class="reference external" href="https://datatracker.ietf.org/doc/draft-ietf-ledbat-congestion/">LEDBAT</a>, which also is an IETF working
group attempting to standardize it. The congestion controller, on top of reacting
to packet loss the same way TCP does, also reacts to changes in delays.</p>
<p>For any uTP (or <a class="reference external" href="https://datatracker.ietf.org/doc/draft-ietf-ledbat-congestion/">LEDBAT</a>) implementation, there is a target delay. This is the
amount of delay that is acceptable, and is in fact targeted for the connection.
The target delay is defined to 25 ms in <a class="reference external" href="https://datatracker.ietf.org/doc/draft-ietf-ledbat-congestion/">LEDBAT</a>, uTorrent uses 100 ms and
libtorrent uses 75 ms. Whenever a delay measurement is lower than the target,
cwnd is increased proportional to (target_delay - delay). Whenever the measurement
is higher than the target, cwnd is decreased proportional to (delay - target_delay).</p>
<p>It can simply be expressed as:</p>
<pre class="literal-block">
cwnd += gain * (target_delay - delay)
</pre>
<a class="reference external image-reference" href="cwnd.png"><img alt="cwnd_thumb.png" class="align-right" src="cwnd_thumb.png" /></a>
<p>Similarly to TCP, this is scaled so that the increase is evened out over one RTT.</p>
<p>The linear controller will adjust the cwnd more for delays that are far off the
target, and less for delays that are close to the target. This makes it converge
at the target delay. Although, due to noise there is almost always some amount of
oscillation. This oscillation is typically smaller than the saw tooth TCP forms.</p>
<p>The figure to the right shows how (TCP) cross traffic causese uTP to essentially
entirely stop sending anything. Its delay measurements are mostly well above the target
during this time. The cross traffic is only a single TCP stream in this test.</p>
<p>As soon as the cross traffic ceases, uTP will pick up its original send rate within
a second.</p>
<p>Since uTP constantly measures the delay, with every single packet, the reaction time
to cross traffic causing delays is a single RTT (typically a fraction of a second).</p>
</div>
<div class="section" id="one-way-delays">
<h2>one way delays</h2>
<p>uTP measures the delay imposed on packets being sent to the other end
of the connection. This measurement only includes buffering delay along
the link, not propagation delay (the speed of light times distance) nor
the routing delay (the time routers spend figuring out where to forward
the packet). It does this by always comparing all measurements to a
baseline measurement, to cancel out any fixed delay. By focusing on the
variable delay along a link, it will specifically detect points where
there might be congestion, since those points will have buffers.</p>
<a class="reference external image-reference" href="delays.png"><img alt="delays_thumb.png" class="align-right" src="delays_thumb.png" /></a>
<p>Delay on the return link is explicitly not included in the delay measurement.
This is because in a peer-to-peer application, the other end is likely to also
be connected via a modem, with the same send buffer restrictions as we assume
for the sending side. The other end having its send queue full is not an indication
of congestion on the path going the other way.</p>
<p>In order to measure one way delays for packets, we cannot rely on clocks being
synchronized, especially not at the microsecond level. Instead, the actual time
it takes for a packet to arrive at the destination is not measured, only the changes
in the transit time is measured.</p>
<p>Each packet that is sent includes a time stamp of the current time, in microseconds,
of the sending machine. The receiving machine calculates the difference between its
own timestamp and the one in the packet and sends this back in the ACK. This difference,
since it is in microseconds, will essentially be a random 32 bit number. However,
the difference will stay somewhat similar over time. Any changes in this difference
indicates that packets are either going through faster or slower.</p>
<p>In order to measure the one-way buffering delay, a base delay is established. The
base delay is the lowest ever seen value of the time stamp difference. Each delay
sample we receive back, is compared against the base delay and the delay is the
difference.</p>
<p>This is the delay that's fed into the congestion controller.</p>
<p>A histogram of typical delay measurements is shown to the right. This is from
a transfer between a cable modem connection and a DSL connection.</p>
<p>The details of the delay measurements are slightly more complicated since the
values needs to be able to wrap (cross the 2^32 boundry and start over at 0).</p>
</div>
<div class="section" id="path-mtu-discovery">
<h2>Path MTU discovery</h2>
<p>MTU is short for <em>Maximum Transfer Unit</em> and describes the largest packet size that
can be sent over a link. Any datagrams which size exceeds this limit will either
be <em>fragmented</em> or dropped. A fragmented datagram means that the payload is split up
in multiple packets, each with its own individual packet header.</p>
<p>There are several reasons to avoid sending datagrams that get fragmented:</p>
<ol class="arabic simple">
<li>A fragmented datagram is more likely to be lost. If any fragment is lost,
the whole datagram is dropped.</li>
<li>Bandwidth is likely to be wasted. If the datagram size is not divisible
by the MTU the last packet will not contain as much payload as it could, and the
payload over protocol header ratio decreases.</li>
<li>It's expensive to fragment datagrams. Few routers are optimized to handle large
numbers of fragmented packets. Datagrams that have to fragment are likely to
be delayed significantly, and contribute to more CPU being used on routers.
Typically fragmentation (and other advanced IP features) are implemented in
software (slow) and not hardware (fast).</li>
</ol>
<p>The path MTU is the lowest MTU of any link along a path from two endpoints on the
internet. The MTU bottleneck isn't necessarily at one of the endpoints, but can
be anywhere in between.</p>
<p>The most common MTU is 1500 bytes, which is the largest packet size for ethernet
networks. Many home DSL connections, however, tunnel IP through PPPoE (Point to
Point Protocol over Ethernet. Yes, that is the old dial-up modem protocol). This
protocol uses up 8 bytes per packet for its own header.</p>
<p>If the user happens to be on an internet connection over a VPN, it will add another
layer, with its own packet headers.</p>
<p>In short; if you would pick the largest possible packet size on an ethernet network,
1472, and stick with it, you would be quite likely to generate fragments for a lot
of connections. The fragments that will be created will be very small and especially
inflate the overhead waste.</p>
<p>The other approach of picking a very conservative packet size, that would be very
unlikely to get fragmented has the following drawbacks:</p>
<ol class="arabic simple">
<li>People on good, normal, networks will be penalized with a small packet size.
Both in terms of router load but also bandwidth waste.</li>
<li>Software routers are typically not limited by the number of bytes they can route,
but the number of packets. Small packets means more of them, and more load on
software routers.</li>
</ol>
<p>The solution to the problem of finding the optimal packet size, is to dynamically
adjust the packet size and search for the largest size that can make it through
without being fragmented along the path.</p>
<p>To help do this, you can set the DF bit (Don't Fragment) in your Datagrams. This
asks routers that otherwise would fragment packets to instead drop them, and send
back an ICMP message reporting the MTU of the link the packet couldn't fit. With
this message, it's very simple to discover the path MTU. You simply mark your packets
not to be fragmented, and change your packet size whenever you receive the ICMP
packet-too-big message.</p>
<p>Unfortunately it's not quite that simple. There are a significant number of firewalls
in the wild blocking all ICMP messages. This means we can't rely on them, we also have
to guess that a packet was dropped because of its size. This is done by only marking
certain packets with DF, and if all other packets go through, except for the MTU probes,
we know that we need to lower our packet sizes.</p>
<p>If we set up bounds for the path MTU (say the minimum internet MTU, 576 and ethernet's 1500),
we can do a binary search for the MTU. This would let us find it in just a few round-trips.</p>
<p>On top of this, libtorrent has an optimization where it figures out which interface a
uTP connection will be sent over, and initialize the MTU ceiling to that interface's MTU.
This means that a VPN tunnel would advertize its MTU as lower, and the uTP connection would
immediately know to send smaller packets, no search required. It also has the side-effect
of being able to use much larger packet sizes for non-ethernet interfaces or ethernet links
with jumbo frames.</p>
</div>
<div class="section" id="clock-drift">
<h2>clock drift</h2>
<a class="reference external image-reference" href="our_delay_base.png"><img alt="our_delay_base_thumb.png" class="align-right" src="our_delay_base_thumb.png" /></a>
<p>Clock drift is clocks progressing at different rates. It's different from clock
skew which means clocks set to different values (but which may progress at the same
rate).</p>
<p>Any clock drift between the two machines involved in a uTP transfer will result
in systematically inflated or deflated delay measurements.</p>
<p>This can be solved by letting the base delay be the lowest seen sample in the last
<em>n</em> minutes. This is a trade-off between seeing a single packet go straight through
the queue, with no delay, and the amount of clock drift one can assume on normal computers.</p>
<p>It turns out that it's fairly safe to assume that one of your packets will in fact go
straight through without any significant delay, once every 20 minutes or so. However,
the clock drift between normal computers can be as much as 17 ms in 10 minutes. 17 ms
is quite significant, especially if your target delay is 25 ms (as in the <a class="reference external" href="https://datatracker.ietf.org/doc/draft-ietf-ledbat-congestion/">LEDBAT</a> spec).</p>
<p>Clocks progresses at different rates depending on temperature. This means computers
running hot are likely to have a clock drift compared to computers running cool.</p>
<p>So, by updating the delay base periodically based on the lowest seen sample, you'll either
end up changing it upwards (artificaially making the delay samples appear small) without
the congestion or delay actually having changed, or you'll end up with a significant clock
drift and have artificially low samples because of that.</p>
<p>The solution to this problem is based on the fact that the clock drift is only a problem
for one of the sides of the connection. Only when your delay measurements keep increasing
is it a problem. If your delay measurements keep decreasing, the samples will simply push
down the delay base along with it. With this in mind, we can simply keep track of the
other end's delay measurements as well, applying the same logic to it. Whenever the
other end's base delay is adjusted downwards, we adjust our base delay upwards by the same
amount.</p>
<p>This will accurately keep the base delay updated with the clock drift and improve
the delay measurements. The figure on the right shows the absolute timestamp differences
along with the base delay. The slope of the measurements is caused by clock drift.</p>
<p>For more information on the clock drift compensation, see the slides from BitTorrent's
presentation at <a class="reference external" href="http://www.usenix.org/event/iptps10/tech/slides/cohen.pdf">IPTPS10</a>.</p>
</div>
<div class="section" id="features">
<h2>features</h2>
<p>libtorrent's uTP implementation includes the following features:</p>
<ul class="simple">
<li>Path MTU discovery, including jumbo frames and detecting restricted
MTU tunnels. Binary search packet sizes to find the largest non-fragmented.</li>
<li>Selective ACK. The ability to acknowledge individual packets in the
event of packet loss</li>
<li>Fast resend. The first time a packet is lost, it's resent immediately.
Triggered by duplicate ACKs.</li>
<li>Nagle's algorithm. Minimize protocol overhead by attempting to lump
full packets of payload together before sending a packet.</li>
<li>Delayed ACKs to minimize protocol overhead.</li>
<li>Microsecond resolution timestamps.</li>
<li>Advertised receive window, to support download rate limiting.</li>
<li>Correct handling of wrapping sequence numbers.</li>
<li>Easy configuration of target-delay, gain-factor, timeouts, delayed-ack
and socket buffers.</li>
</ul>
</div>
</div>
</div>
</div>
<div id="gradient"></div>
<div id="footer">
<table>
<tr>
<td><a href="index.html">home</a></td>
<td><a href="http://blog.libtorrent.org">blog</a></td>
<td><a href="utp.html">uTP</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/libtorrent/">download</a></td>
<td><a href="reference.html">documentation</a></td>
<td><a href="dht_store.html">DHT put extension</a></td>
</tr>
<tr>
<td><a href="https://sourceforge.net/projects/libtorrent/files/py-libtorrent/">python bindings</a></td>
<td><a href="features.html">features</a></td>
<td><a href="dht_sec.html">DHT security extension</a></td>
</tr>
<tr>
<td><a href="http://dir.gmane.org/gmane.network.bit-torrent.libtorrent">mailing list archive</a></td>
<td><a href="contributing.html">contributing</a></td>
<td><a href="streaming.html">streaming</a></td>
</tr>
<tr>
<td><a href="http://code.google.com/p/libtorrent/issues/entry">report a bug</a></td>
<td><a href="building.html">building</a></td>
<td><a href="bittorrent.pdf">bittorrent slides</a></td>
</tr>
</table>
</div>
<div id="filler"></div>
</div>
</body>
</html>