<?xml version="1.0" encoding="utf-8" ?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta name="generator" content="Docutils 0.12: http://docutils.sourceforge.net/" /> <title></title> <meta name="author" content="Arvid Norberg, arvid@libtorrent.org Ludvig Strigeus, ludde@utorrent.com" /> <link rel="stylesheet" type="text/css" href="rst.css" /> <script type="text/javascript"> /* <![CDATA[ */ (function() { var s = document.createElement('script'), t = document.getElementsByTagName('script')[0]; s.type = 'text/javascript'; s.async = true; s.src = 'http://api.flattr.com/js/0.6/load.js?mode=auto'; t.parentNode.insertBefore(s, t); })(); /* ]]> */ </script> <link rel="stylesheet" href="style.css" type="text/css" /> <style type="text/css"> /* Hides from IE-mac \*/ * html pre { height: 1%; } /* End hide from IE-mac */ </style> </head> <body> <div class="document"> <div id="container"> <table id="header"> <tr><td id="orange"></td> <td id="logo">libtorrent</td></tr> </table> <div id="main"> <table class="docinfo" frame="void" rules="none"> <col class="docinfo-name" /> <col class="docinfo-content" /> <tbody valign="top"> <tr><th class="docinfo-name">Author:</th> <td>Arvid Norberg, <a class="reference external" href="mailto:arvid@libtorrent.org">arvid@libtorrent.org</a> Ludvig Strigeus, <a class="last reference external" href="mailto:ludde@utorrent.com">ludde@utorrent.com</a></td></tr> </tbody> </table> <div class="section" id="extension-protocol-for-bittorrent"> <h1>extension protocol for bittorrent</h1> <p>The intention of this protocol is to provide a simple and thin transport for extensions to the bittorrent protocol. Supporting this protocol makes it easy to add new extensions without interfering with the standard bittorrent protocol or clients that don't support this extension or the one you want to add.</p> <p>To advertise to other clients that you support, one bit from the reserved bytes is used.</p> <p>The bit selected for the extension protocol is bit 20 from the right (counting starts at 0). So (reserved_byte[5] & 0x10) is the expression to use for checking if the client supports extended messaging.</p> <p>Once support for the protocol is established, the client is supposed to support 1 new message:</p> <table border="1" class="docutils"> <colgroup> <col width="86%" /> <col width="14%" /> </colgroup> <thead valign="bottom"> <tr><th class="head">name</th> <th class="head">id</th> </tr> </thead> <tbody valign="top"> <tr><td><tt class="docutils literal">extended</tt></td> <td>20</td> </tr> </tbody> </table> <p>This message is sent as any other bittorrent message, with a 4 byte length prefix and a single byte identifying the message (the single byte being 20 in this case). At the start of the payload of the message, is a single byte message identifier. This identifier can refer to different extension messages and only one ID is specified, 0. If the ID is 0, the message is a handshake message which is described below. The layout of a general <tt class="docutils literal">extended</tt> message follows (including the message headers used by the bittorrent protocol):</p> <table border="1" class="docutils"> <colgroup> <col width="15%" /> <col width="85%" /> </colgroup> <thead valign="bottom"> <tr><th class="head">size</th> <th class="head">description</th> </tr> </thead> <tbody valign="top"> <tr><td>uint32_t</td> <td>length prefix. Specifies the number of bytes for the entire message. (Big endian)</td> </tr> <tr><td>uint8_t</td> <td>bittorrent message ID, = 20</td> </tr> <tr><td>uint8_t</td> <td>extended message ID. 0 = handshake, >0 = extended message as specified by the handshake.</td> </tr> </tbody> </table> <div class="section" id="handshake-message"> <h2>handshake message</h2> <p>The payload of the handshake message is a bencoded dictionary. All items in the dictionary are optional. Any unknown names should be ignored by the client. All parts of the dictionary are case sensitive. This is the defined item in the dictionary:</p> <table border="1" class="docutils"> <colgroup> <col width="11%" /> <col width="89%" /> </colgroup> <thead valign="bottom"> <tr><th class="head">name</th> <th class="head">description</th> </tr> </thead> <tbody valign="top"> <tr><td>m</td> <td><p class="first">Dictionary of supported extension messages which maps names of extensions to an extended message ID for each extension message. The only requirement on these IDs is that no extension message share the same one. Setting an extension number to zero means that the extension is not supported/disabled. The client should ignore any extension names it doesn't recognize.</p> <p class="last">The extension message IDs are the IDs used to send the extension messages to the peer sending this handshake. i.e. The IDs are local to this particular peer.</p> </td> </tr> </tbody> </table> <p>Here are some other items that an implementation may choose to support:</p> <table border="1" class="docutils"> <colgroup> <col width="12%" /> <col width="88%" /> </colgroup> <thead valign="bottom"> <tr><th class="head">name</th> <th class="head">description</th> </tr> </thead> <tbody valign="top"> <tr><td>p</td> <td>Local TCP listen port. Allows each side to learn about the TCP port number of the other side. Note that there is no need for the receiving side of the connection to send this extension message, since its port number is already known.</td> </tr> <tr><td>v</td> <td>Client name and version (as a utf-8 string). This is a much more reliable way of identifying the client than relying on the peer id encoding.</td> </tr> <tr><td>yourip</td> <td>A string containing the compact representation of the ip address this peer sees you as. i.e. this is the receiver's external ip address (no port is included). This may be either an IPv4 (4 bytes) or an IPv6 (16 bytes) address.</td> </tr> <tr><td>ipv6</td> <td>If this peer has an IPv6 interface, this is the compact representation of that address (16 bytes). The client may prefer to connect back via the IPv6 address.</td> </tr> <tr><td>ipv4</td> <td>If this peer has an IPv4 interface, this is the compact representation of that address (4 bytes). The client may prefer to connect back via this interface.</td> </tr> <tr><td>reqq</td> <td>An integer, the number of outstanding request messages this client supports without dropping any. The default in in libtorrent is 250.</td> </tr> </tbody> </table> <p>The handshake dictionary could also include extended handshake information, such as support for encrypted headers or anything imaginable.</p> <p>An example of what the payload of a handshake message could look like:</p> <table border="1" class="docutils"> <colgroup> <col width="36%" /> <col width="64%" /> </colgroup> <thead valign="bottom"> <tr><th class="head" colspan="2">Dictionary</th> </tr> </thead> <tbody valign="top"> <tr><td><tt class="docutils literal">m</tt></td> <td><table border="1" class="first last docutils"> <colgroup> <col width="88%" /> <col width="12%" /> </colgroup> <thead valign="bottom"> <tr><th class="head" colspan="2">Dictionary</th> </tr> </thead> <tbody valign="top"> <tr><td><tt class="docutils literal">LT_metadata</tt></td> <td>1</td> </tr> <tr><td><tt class="docutils literal">ut_pex</tt></td> <td>2</td> </tr> </tbody> </table> </td> </tr> <tr><td><tt class="docutils literal">p</tt></td> <td>6881</td> </tr> <tr><td><tt class="docutils literal">v</tt></td> <td>"uTorrent 1.2"</td> </tr> </tbody> </table> <p>and in the encoded form:</p> <p><tt class="docutils literal">d1:md11:LT_metadatai1e6:ut_pexi2ee1:pi6881e1:v12:uTorrent 1.2e</tt></p> <p>To make sure the extension names do not collide by mistake, they should be prefixed with the two (or one) character code that is used to identify the client that introduced the extension. This applies for both the names of extension messages, and for any additional information put inside the top-level dictionary. All one and two byte identifiers are invalid to use unless defined by this specification.</p> <p>This message should be sent immediately after the standard bittorrent handshake to any peer that supports this extension protocol. It is valid to send the handshake message more than once during the lifetime of a connection, the sending client should not be disconnected. An implementation may choose to ignore the subsequent handshake messages (or parts of them).</p> <p>Subsequent handshake messages can be used to enable/disable extensions without restarting the connection. If a peer supports changing extensions at run time, it should note that the <tt class="docutils literal">m</tt> dictionary is additive. It's enough that it contains the actual <em>CHANGES</em> to the extension list. To disable the support for <tt class="docutils literal">LT_metadata</tt> at run-time, without affecting any other extensions, this message should be sent: <tt class="docutils literal">d11:LT_metadatai0ee</tt>. As specified above, the value 0 is used to turn off an extension.</p> <p>The extension IDs must be stored for every peer, becuase every peer may have different IDs for the same extension.</p> <p>This specification, deliberately, does not specify any extensions such as peer-exchange or metadata exchange. This protocol is merely a transport for the actual extensions to the bittorrent protocol and the extensions named in the example above (such as <tt class="docutils literal">p</tt>) are just examples of possible extensions.</p> </div> <div class="section" id="rationale"> <h2>rationale</h2> <p>The reason why the extension messages' IDs would be defined in the handshake is to avoid having a global registry of message IDs. Instead the names of the extension messages requires unique names, which is much easier to do without a global registry. The convention is to use a two letter prefix on the extension message names, the prefix would identify the client first implementing the extension message. e.g. <tt class="docutils literal">LT_metadata</tt> is implemented by libtorrent, and hence it has the <tt class="docutils literal">LT</tt> prefix.</p> <p>If the client supporting the extensions can decide which numbers the messages it receives will have, it means they are constants within that client. i.e. they can be used in <tt class="docutils literal">switch</tt> statements. It's easy for the other end to store an array with the ID's we expect for each message and use that for lookups each time it sends an extension message.</p> <p>The reason for having a dictionary instead of having an array (using implicitly assigned index numbers to the extensions) is that if a client want to disable some extensions, the ID numbers would change, and it wouldn't be able to use constants (and hence, not use them in a <tt class="docutils literal">switch</tt>). If the messages IDs would map directly to bittorrent message IDs, It would also make it possible to map extensions in the handshake to existing extensions with fixed message IDs.</p> <p>The reasoning behind having a single byte as extended message identifier is to follow the the bittorrent spec. with its single byte message identifiers. It is also considered to be enough. It won't limit the total number of extensions, only the number of extensions used simultaneously.</p> <p>The reason for using single byte identifiers for the standardized handshake identifiers is 1) The mainline DHT uses single byte identifiers. 2) Saves bandwidth. The only advantage of longer messages is that it makes the protocol more readable for a human, but the BT protocol wasn't designed to be a human readable protocol, so why bother.</p> </div> </div> <div class="section" id="extensions"> <h1>extensions</h1> <p>These extensions all operates within the <a class="reference external" href="extension_protocol.html">extension protocol</a>. The name of the extension is the name used in the extension-list packets, and the payload is the data in the extended message (not counting the length-prefix, message-id nor extension-id).</p> <p>Note that since this protocol relies on one of the reserved bits in the handshake, it may be incompatible with future versions of the mainline bittorrent client.</p> <p>These are the extensions that are currently implemented.</p> <div class="section" id="metadata-from-peers"> <h2>metadata from peers</h2> <p>Extension name: "LT_metadata"</p> <div class="note"> <p class="first admonition-title">Note</p> <p class="last">This extension is deprecated in favor of the more widely supported <tt class="docutils literal">ut_metadata</tt> extension, see <a class="reference external" href="http://bittorrent.org/beps/bep_0009.html">BEP 9</a>.</p> </div> <p>The point with this extension is that you don't have to distribute the metadata (.torrent-file) separately. The metadata can be distributed through the bittorrent swarm. The only thing you need to download such a torrent is the tracker url and the info-hash of the torrent.</p> <p>It works by assuming that the initial seeder has the metadata and that the metadata will propagate through the network as more peers join.</p> <p>There are three kinds of messages in the metadata extension. These packets are put as payload to the extension message. The three packets are:</p> <blockquote> <ul class="simple"> <li>request metadata</li> <li>metadata</li> <li>don't have metadata</li> </ul> </blockquote> <p>request metadata:</p> <table border="1" class="docutils"> <colgroup> <col width="17%" /> <col width="23%" /> <col width="61%" /> </colgroup> <thead valign="bottom"> <tr><th class="head">size</th> <th class="head">name</th> <th class="head">description</th> </tr> </thead> <tbody valign="top"> <tr><td>uint8_t</td> <td>msg_type</td> <td>Determines the kind of message this is 0 means 'request metadata'</td> </tr> <tr><td>uint8_t</td> <td>start</td> <td>The start of the metadata block that is requested. It is given in 256:ths of the total size of the metadata, since the requesting client don't know the size of the metadata.</td> </tr> <tr><td>uint8_t</td> <td>size</td> <td>The size of the metadata block that is requested. This is also given in 256:ths of the total size of the metadata. The size is given as size-1. That means that if this field is set 0, the request wants one 256:th of the metadata.</td> </tr> </tbody> </table> <p>metadata:</p> <table border="1" class="docutils"> <colgroup> <col width="17%" /> <col width="23%" /> <col width="61%" /> </colgroup> <thead valign="bottom"> <tr><th class="head">size</th> <th class="head">name</th> <th class="head">description</th> </tr> </thead> <tbody valign="top"> <tr><td>uint8_t</td> <td>msg_type</td> <td>1 means 'metadata'</td> </tr> <tr><td>int32_t</td> <td>total_size</td> <td>The total size of the metadata, given in number of bytes.</td> </tr> <tr><td>int32_t</td> <td>offset</td> <td>The offset of where the metadata block in this message belongs in the final metadata. This is given in bytes.</td> </tr> <tr><td>uint8_t[]</td> <td>metadata</td> <td>The actual metadata block. The size of this part is given implicit by the length prefix in the bittorrent protocol packet.</td> </tr> </tbody> </table> <p>Don't have metadata:</p> <table border="1" class="docutils"> <colgroup> <col width="17%" /> <col width="23%" /> <col width="61%" /> </colgroup> <thead valign="bottom"> <tr><th class="head">size</th> <th class="head">name</th> <th class="head">description</th> </tr> </thead> <tbody valign="top"> <tr><td>uint8_t</td> <td>msg_type</td> <td>2 means 'I don't have metadata'. This message is sent as a reply to a metadata request if the the client doesn't have any metadata.</td> </tr> </tbody> </table> </div> <div class="section" id="dont-have"> <h2>dont_have</h2> <p>Extension name: "lt_donthave"</p> <p>The <tt class="docutils literal">dont_have</tt> extension message is used to tell peers that the client no longer has a specific piece. The extension message should be advertised in the <tt class="docutils literal">m</tt> dictionary as <tt class="docutils literal">lt_donthave</tt>. The message format mimics the regular <tt class="docutils literal">HAVE</tt> bittorrent message.</p> <p>Just like all extension messages, the first 2 bytes in the mssage itself are 20 (the bittorrent extension message) and the message ID assigned to this extension in the <tt class="docutils literal">m</tt> dictionary in the handshake.</p> <table border="1" class="docutils"> <colgroup> <col width="17%" /> <col width="23%" /> <col width="61%" /> </colgroup> <thead valign="bottom"> <tr><th class="head">size</th> <th class="head">name</th> <th class="head">description</th> </tr> </thead> <tbody valign="top"> <tr><td>uint32_t</td> <td>piece</td> <td>index of the piece the peer no longer has.</td> </tr> </tbody> </table> <p>The length of this message (including the extension message prefix) is 6 bytes, i.e. one byte longer than the normal <tt class="docutils literal">HAVE</tt> message, because of the extension message wrapping.</p> </div> </div> </div> </div> <div id="gradient"></div> <div id="footer"> <table> <tr> <td><a href="index.html">home</a></td> <td><a href="http://blog.libtorrent.org">blog</a></td> <td><a href="utp.html">uTP</a></td> </tr> <tr> <td><a href="https://sourceforge.net/projects/libtorrent/files/libtorrent/">download</a></td> <td><a href="reference.html">documentation</a></td> <td><a href="dht_store.html">DHT put extension</a></td> </tr> <tr> <td><a href="https://sourceforge.net/projects/libtorrent/files/py-libtorrent/">python bindings</a></td> <td><a href="features.html">features</a></td> <td><a href="dht_sec.html">DHT security extension</a></td> </tr> <tr> <td><a href="http://dir.gmane.org/gmane.network.bit-torrent.libtorrent">mailing list archive</a></td> <td><a href="contributing.html">contributing</a></td> <td><a href="streaming.html">streaming</a></td> </tr> <tr> <td><a href="http://code.google.com/p/libtorrent/issues/entry">report a bug</a></td> <td><a href="building.html">building</a></td> <td><a href="bittorrent.pdf">bittorrent slides</a></td> </tr> </table> </div> <div id="filler"></div></div> </div> </body> </html>