forked from premiere/premiere-libtorrent
add DHT extension proposal document
This commit is contained in:
parent
c54ffcbb8d
commit
467f7211c3
|
@ -0,0 +1,308 @@
|
|||
<?xml version="1.0" encoding="utf-8" ?>
|
||||
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
||||
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
|
||||
<head>
|
||||
<script type="text/javascript">
|
||||
/* <![CDATA[ */
|
||||
(function() {
|
||||
var s = document.createElement('script'), t = document.getElementsByTagName('script')[0];
|
||||
|
||||
s.type = 'text/javascript';
|
||||
s.async = true;
|
||||
s.src = 'http://api.flattr.com/js/0.6/load.js?mode=auto';
|
||||
|
||||
t.parentNode.insertBefore(s, t);
|
||||
})();
|
||||
/* ]]> */
|
||||
</script>
|
||||
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
|
||||
<meta name="generator" content="Docutils 0.5: http://docutils.sourceforge.net/" />
|
||||
<title>BitTorrent extension for DHT RSS feeds</title>
|
||||
<meta name="author" content="Arvid Norberg, arvid@rasterbar.com" />
|
||||
<link rel="stylesheet" type="text/css" href="../../css/base.css" />
|
||||
<link rel="stylesheet" type="text/css" href="../../css/rst.css" />
|
||||
<link rel="stylesheet" href="style.css" type="text/css" />
|
||||
<style type="text/css">
|
||||
/* Hides from IE-mac \*/
|
||||
* html pre { height: 1%; }
|
||||
/* End hide from IE-mac */
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="document" id="bittorrent-extension-for-dht-rss-feeds">
|
||||
<div id="container">
|
||||
<div id="headerNav">
|
||||
<ul>
|
||||
<li class="first"><a href="/">Home</a></li>
|
||||
<li><a href="../../products.html">Products</a></li>
|
||||
<li><a href="../../contact.html">Contact</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div id="header">
|
||||
<h1><span>Rasterbar Software</span></h1>
|
||||
<h2><span>Software developement and consulting</span></h2>
|
||||
</div>
|
||||
<div id="main">
|
||||
<h1 class="title">BitTorrent extension for DHT RSS feeds</h1>
|
||||
<table class="docinfo" frame="void" rules="none">
|
||||
<col class="docinfo-name" />
|
||||
<col class="docinfo-content" />
|
||||
<tbody valign="top">
|
||||
<tr><th class="docinfo-name">Author:</th>
|
||||
<td>Arvid Norberg, <a class="last reference external" href="mailto:arvid@rasterbar.com">arvid@rasterbar.com</a></td></tr>
|
||||
<tr><th class="docinfo-name">Version:</th>
|
||||
<td>Draft</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<div class="contents topic" id="table-of-contents">
|
||||
<p class="topic-title first">Table of contents</p>
|
||||
<ul class="simple">
|
||||
<li><a class="reference internal" href="#id1" id="id2">BitTorrent extension for DHT RSS feeds</a></li>
|
||||
<li><a class="reference internal" href="#target-id" id="id3">target ID</a></li>
|
||||
<li><a class="reference internal" href="#messages" id="id4">messages</a></li>
|
||||
<li><a class="reference internal" href="#requesting-items" id="id5">requesting items</a></li>
|
||||
<li><a class="reference internal" href="#request-item-response" id="id6">request item response</a></li>
|
||||
<li><a class="reference internal" href="#announcing-items" id="id7">announcing items</a><ul>
|
||||
<li><a class="reference internal" href="#example" id="id8">Example</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><a class="reference internal" href="#uri-scheme" id="id9">URI Scheme</a></li>
|
||||
<li><a class="reference internal" href="#rationale" id="id10">rationale</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="section" id="id1">
|
||||
<h1>BitTorrent extension for DHT RSS feeds</h1>
|
||||
<p>This is a proposal for an extension to the BitTorrent DHT to allow
|
||||
for decentralized RSS feed like functionality.</p>
|
||||
<p>The intention is to allow the creation of repositories of torrents
|
||||
where only a single identity has the authority to add new content. For
|
||||
this repository to be robust against network failures and resilient
|
||||
to attacks at the source.</p>
|
||||
<p>The target ID under which the repository is stored in the DHT, is the
|
||||
SHA-1 hash of a feed name and the 512 bit public key. This private key
|
||||
in this pair MUST be used to sign every item stored in the repository.
|
||||
Every message that contain signed items MUST also include this key, to
|
||||
allow the receiver to verify the key itself against the target ID as well
|
||||
as the validity of the signatures of the items. Every recipient of a
|
||||
message with feed items in it MUST verify both the validity of the public
|
||||
key against the target ID it is stored under, as well as the validity of
|
||||
the signatures of each individual item.</p>
|
||||
<p>Any peer who is subscribing to a DHT feed SHOULD also participate in
|
||||
regularly re-announcing items that it knows about. Every participant
|
||||
SHOULD store items in long term storage, across sessions, in order to
|
||||
keep items alive for as long as possible, with as few sources as possible.</p>
|
||||
<p>As with normal DHT announces, the write-token mechanism is used to
|
||||
prevent spoof attacks.</p>
|
||||
<p>There are two new proposed messages, <tt class="docutils literal"><span class="pre">announce_item</span></tt> and <tt class="docutils literal"><span class="pre">get_item</span></tt>.
|
||||
Every valid item that is announced, should be stored. In a request to get items,
|
||||
as many items as can fit in a normal UDP packet size should be returned. If
|
||||
there are more items than can fit, a random sub-set should be returned.</p>
|
||||
<p><em>Is there a better heuristic here? Should there be a bias towards newer items?
|
||||
If so, there needs to be a signed timestamp as well, which might get messy</em></p>
|
||||
</div>
|
||||
<div class="section" id="target-id">
|
||||
<h1>target ID</h1>
|
||||
<p>The target, i.e. the ID in the DHT key space feeds are announced to, MUST always
|
||||
be SHA-1(<em>feed_name</em> + <em>public_key</em>). Any request where this condition is not met,
|
||||
MUST be dropped.</p>
|
||||
<p>Using the feed name as part of the target means a feed publisher only needs one
|
||||
public-private keypair for any number of feeds, as long as the feeds have different
|
||||
names.</p>
|
||||
</div>
|
||||
<div class="section" id="messages">
|
||||
<h1>messages</h1>
|
||||
<p>These are the proposed new message formats.</p>
|
||||
</div>
|
||||
<div class="section" id="requesting-items">
|
||||
<h1>requesting items</h1>
|
||||
<pre class="literal-block">
|
||||
{
|
||||
"a":
|
||||
{
|
||||
"filter": <em><variable size bloom-filter></em>,
|
||||
"id": <em><20 byte id of origin node></em>,
|
||||
"key": <em><64 byte public curve25519 key for this feed></em>,
|
||||
"n": <em><feed-name></em>
|
||||
"target": <em><target-id as derived from public key></em>
|
||||
},
|
||||
"q": "get_item",
|
||||
"t": <em><transaction-id></em>,
|
||||
"y": "q",
|
||||
}
|
||||
</pre>
|
||||
<p>The <tt class="docutils literal"><span class="pre">target</span></tt> MUST always be SHA-1(<em>feed_name</em> + <em>public_key</em>). Any request where
|
||||
this condition is not met, MUST be dropped.</p>
|
||||
<p>The <tt class="docutils literal"><span class="pre">n</span></tt> field is the name of this feed. It MUST be UTF-8 encoded string and it
|
||||
MUST match the name of the feed in the receiving node.</p>
|
||||
<p>The bloom filter argument (<tt class="docutils literal"><span class="pre">filter</span></tt>) in the <tt class="docutils literal"><span class="pre">get_item</span></tt> requests is optional.
|
||||
If included in a request, it represents info-hashes that should be excluded from
|
||||
the response. In this case, the response should be a random subset of the non-excluded
|
||||
items, or all of the non-excluded items if they all fit within a packet size.</p>
|
||||
<p>If the bloom filter is specified, its size MUST be an even multiple of 8 bits. The size
|
||||
is implied by the length of the string. For each info-hash to exclude from the response,</p>
|
||||
<p>There are no hash functions for the bloom filter. Since the info-hash is already a
|
||||
hash digest, each pair of bytes, starting with the first bytes (MSB), are used as the
|
||||
results from the imaginary hash functions for the bloom filter. k is 3 in this bloom
|
||||
filter. This means the first 6 bytes of the info-hash is used to set 3 bits in the bloom
|
||||
filter. The pairs of bytes pulled out from the info-hash are interpreted as a big-endian
|
||||
16 bit value.</p>
|
||||
<p>Bits are indexed in bytes from left to right, and within bytes from LSB to MSB. i.e., to
|
||||
set bit 12: <tt class="docutils literal"><span class="pre">bitfield[12/8]</span> <span class="pre">|=</span> <span class="pre">(12</span> <span class="pre">%</span> <span class="pre">8)</span></tt>.</p>
|
||||
<dl class="docutils">
|
||||
<dt>Example:</dt>
|
||||
<dd>To indicate that you are not interested in knowing about the info-hash that
|
||||
starts with 0x4f7d25a... and you choose a bloom filter of size 80 bits. Set bits
|
||||
(0x4f % 80), (0x7d % 80) and (0x25 % 80) in the bloom filter bitmask.</dd>
|
||||
</dl>
|
||||
</div>
|
||||
<div class="section" id="request-item-response">
|
||||
<h1>request item response</h1>
|
||||
<pre class="literal-block">
|
||||
{
|
||||
"r":
|
||||
{
|
||||
"ih":
|
||||
[
|
||||
<em><n * 20 byte(s) info-hash></em>,
|
||||
...
|
||||
],
|
||||
"sig":
|
||||
[
|
||||
<em><64 byte curve25519 signature of info-hash></em>,
|
||||
...
|
||||
],
|
||||
"id": <em><20 byte id of origin node></em>,
|
||||
"token": <em><write-token></em>
|
||||
"nodes": <em><n * compact IPv4-port pair></em>
|
||||
"nodes6": <em><n * compact IPv6-port pair></em>
|
||||
},
|
||||
"t": <em><transaction-id></em>,
|
||||
"y": "r",
|
||||
}
|
||||
</pre>
|
||||
<p>Since the data that's being signed by the public key already is a hash (i.e.
|
||||
an info-hash), the signature of each hash-entry is simply the hash encrypted
|
||||
by the feed's private key.</p>
|
||||
<p>The <tt class="docutils literal"><span class="pre">ih</span></tt> and <tt class="docutils literal"><span class="pre">sig</span></tt> lists MUST have equal number of items. Each item in <tt class="docutils literal"><span class="pre">sig</span></tt>
|
||||
is the signature of the full string in the corresponding item in the <tt class="docutils literal"><span class="pre">ih</span></tt> list.</p>
|
||||
<p>Each item in the <tt class="docutils literal"><span class="pre">ih</span></tt> list may contain any positive number of 20 byte info-hashes.</p>
|
||||
<p>The rationale behind using lists of strings where the strings contain multiple
|
||||
info-hashes is to allow the publisher of a feed to sign multiple info-hashes
|
||||
together, and thus saving space in the UDP packets, allowing nodes to transfer more
|
||||
info-hashes per packet. Original publishers of a feed MAY re-announce items lumped
|
||||
together over time to make the feed more efficient.</p>
|
||||
<p>A client receiving a <tt class="docutils literal"><span class="pre">get_item</span></tt> response MUST verify each signature in the <tt class="docutils literal"><span class="pre">sig</span></tt>
|
||||
list against each corresponding item in the <tt class="docutils literal"><span class="pre">ih</span></tt> list using the feed's public key.
|
||||
Any item whose signature</p>
|
||||
<p><tt class="docutils literal"><span class="pre">nodes</span></tt> and <tt class="docutils literal"><span class="pre">nodes6</span></tt> are optional and have the same semantics as the standard
|
||||
<tt class="docutils literal"><span class="pre">get_peers</span></tt> request. The intention is to be able to use this <tt class="docutils literal"><span class="pre">get_item</span></tt> request
|
||||
in the same way, searching for the nodes responsible for the feed.</p>
|
||||
</div>
|
||||
<div class="section" id="announcing-items">
|
||||
<h1>announcing items</h1>
|
||||
<pre class="literal-block">
|
||||
{
|
||||
"a":
|
||||
{
|
||||
"ih":
|
||||
[
|
||||
<em><n * 20 byte info-hash(es)></em>,
|
||||
...
|
||||
],
|
||||
"sig":
|
||||
[
|
||||
<em><64 byte curve25519 signature of info-hash(es)></em>,
|
||||
...
|
||||
],
|
||||
"id": <em><20 byte node-id of origin node></em>,
|
||||
"key": <em><64 byte public curve25519 key for this feed></em>,
|
||||
"n": <em><feed name></em>
|
||||
"target": <em><target-id as derived from public key></em>,
|
||||
"token": <em><write-token as obtained by previous req.></em>
|
||||
},
|
||||
"y": "q",
|
||||
"q": "announce_item",
|
||||
"t": <em><transaction-id></em>
|
||||
}
|
||||
</pre>
|
||||
<p>An announce can include any number of items, as long as they fit in a packet.</p>
|
||||
<p>Subscribers to a feed SHOULD also announce items that they know of, to the feed.
|
||||
In order to make the repository of torrents as reliable as possible, subscribers
|
||||
SHOULD announce random items from their local repository of items. When re-announcing
|
||||
items, a random subset of all known items should be announced, randomized
|
||||
independently for each node it's announced to. This makes it a little bit harder
|
||||
to determine the IP address an item originated from, since it's a matter of
|
||||
seeing the first announce, and knowing that it wasn't announced anywhere else
|
||||
first.</p>
|
||||
<p>Any subscriber and publisher SHOULD re-announce items every 30 minutes. If
|
||||
a feed does not receive any announced items in 60 minutes, a peer MAY time
|
||||
it out and remove it.</p>
|
||||
<p>Subscribers and publishers SHOULD announce random items.</p>
|
||||
<div class="section" id="example">
|
||||
<h2>Example</h2>
|
||||
<pre class="literal-block">
|
||||
{
|
||||
"a":
|
||||
{
|
||||
"ih":
|
||||
[
|
||||
"7ea94c240691311dc0916a2a91eb7c3db2c6f3e4",
|
||||
"0d92ad53c052ac1f49cf4434afffafa4712dc062e4168d940a48e45a45a0b10808014dc267549624"
|
||||
],
|
||||
"sig":
|
||||
[
|
||||
"980774404e404941b81aa9da1da0101cab54e670cff4f0054aa563c3b5abcb0fe3c6df5dac1ea25266035f09040bf2a24ae5f614787f1fe7404bf12fee5e6101",
|
||||
"3fee52abea47e4d43e957c02873193fb9aec043756845946ec29cceb1f095f03d876a7884e38c53cd89a8041a2adfb2d9241b5ec5d70268714d168b9353a2c01"
|
||||
],
|
||||
"id": "b46989156404e8e0acdb751ef553b210ef77822e",
|
||||
"key": "6bc1de5443d1a7c536cdf69433ac4a7163d3c63e2f9c92d78f6011cf63dbcd5b638bbc2119cdad0c57e4c61bc69ba5e2c08b918c2db8d1848cf514bd9958d307",
|
||||
"n": "my stuff"
|
||||
"target": "b4692ef0005639e86d7165bf378474107bf3a762"
|
||||
"token": "23ba"
|
||||
},
|
||||
"y": "q",
|
||||
"q": "announce_item",
|
||||
"t": "a421"
|
||||
}
|
||||
</pre>
|
||||
<p>Strings are printed in hex for printability, but actual encoding is binary. The
|
||||
response contains 3 feed items, starting with "7ea94c", "0d92ad" and "e4168d".
|
||||
These 3 items are not published optimally. If they were to be merged into a single
|
||||
string in the <tt class="docutils literal"><span class="pre">ih</span></tt> list, more than 64 bytes would be saved (because of having
|
||||
one less signature).</p>
|
||||
<p>Note that <tt class="docutils literal"><span class="pre">target</span></tt> is in fact SHA1('my stuff' + 'key'). The private key
|
||||
used in this example is 980f4cd7b812ae3430ea05af7c09a7e430275f324f42275ca534d9f7c6d06f5b.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="uri-scheme">
|
||||
<h1>URI Scheme</h1>
|
||||
<p>The proposed URI scheme for DHT feeds is:</p>
|
||||
<pre class="literal-block">
|
||||
magnet:?xt=btfd:<em><base16-curve25519-public-key></em> &dn= <em><feed name></em>
|
||||
</pre>
|
||||
<p>Note that a difference from regular torrent magnet links is the <strong>btfd</strong>
|
||||
versus <strong>btih</strong> used in regular magnet links to torrents.</p>
|
||||
<p>The <em>feed name</em> is mandatory since it is used in the request and when
|
||||
calculating the target ID.</p>
|
||||
</div>
|
||||
<div class="section" id="rationale">
|
||||
<h1>rationale</h1>
|
||||
<p>The reason to use <a class="reference external" href="http://cr.yp.to/ecdh.html">curve25519</a> instead of, for instance, RSA is to fit more signatures
|
||||
(i.e. items) in a single DHT packet. One packet is typically restricted to between
|
||||
1280 - 1480 bytes. According to <a class="reference external" href="http://cr.yp.to/">http://cr.yp.to/</a>, curve25519 is free from patent claims
|
||||
and there are open implementations in both C and Java.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div id="footer">
|
||||
<span>Copyright © 2005 Rasterbar Software.</span>
|
||||
</div>
|
||||
</div>
|
||||
<script src="http://www.google-analytics.com/urchin.js" type="text/javascript">
|
||||
</script>
|
||||
<script type="text/javascript">
|
||||
_uacct = "UA-1599045-1";
|
||||
urchinTracker();
|
||||
</script>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
|
@ -0,0 +1,273 @@
|
|||
======================================
|
||||
BitTorrent extension for DHT RSS feeds
|
||||
======================================
|
||||
|
||||
:Author: Arvid Norberg, arvid@rasterbar.com
|
||||
:Version: Draft
|
||||
|
||||
.. contents:: Table of contents
|
||||
:depth: 2
|
||||
:backlinks: none
|
||||
|
||||
BitTorrent extension for DHT RSS feeds
|
||||
--------------------------------------
|
||||
|
||||
This is a proposal for an extension to the BitTorrent DHT to allow
|
||||
for decentralized RSS feed like functionality.
|
||||
|
||||
The intention is to allow the creation of repositories of torrents
|
||||
where only a single identity has the authority to add new content. For
|
||||
this repository to be robust against network failures and resilient
|
||||
to attacks at the source.
|
||||
|
||||
The target ID under which the repository is stored in the DHT, is the
|
||||
SHA-1 hash of a feed name and the 512 bit public key. This private key
|
||||
in this pair MUST be used to sign every item stored in the repository.
|
||||
Every message that contain signed items MUST also include this key, to
|
||||
allow the receiver to verify the key itself against the target ID as well
|
||||
as the validity of the signatures of the items. Every recipient of a
|
||||
message with feed items in it MUST verify both the validity of the public
|
||||
key against the target ID it is stored under, as well as the validity of
|
||||
the signatures of each individual item.
|
||||
|
||||
Any peer who is subscribing to a DHT feed SHOULD also participate in
|
||||
regularly re-announcing items that it knows about. Every participant
|
||||
SHOULD store items in long term storage, across sessions, in order to
|
||||
keep items alive for as long as possible, with as few sources as possible.
|
||||
|
||||
As with normal DHT announces, the write-token mechanism is used to
|
||||
prevent spoof attacks.
|
||||
|
||||
There are two new proposed messages, ``announce_item`` and ``get_item``.
|
||||
Every valid item that is announced, should be stored. In a request to get items,
|
||||
as many items as can fit in a normal UDP packet size should be returned. If
|
||||
there are more items than can fit, a random sub-set should be returned.
|
||||
|
||||
*Is there a better heuristic here? Should there be a bias towards newer items?
|
||||
If so, there needs to be a signed timestamp as well, which might get messy*
|
||||
|
||||
target ID
|
||||
---------
|
||||
|
||||
The target, i.e. the ID in the DHT key space feeds are announced to, MUST always
|
||||
be SHA-1(*feed_name* + *public_key*). Any request where this condition is not met,
|
||||
MUST be dropped.
|
||||
|
||||
Using the feed name as part of the target means a feed publisher only needs one
|
||||
public-private keypair for any number of feeds, as long as the feeds have different
|
||||
names.
|
||||
|
||||
messages
|
||||
--------
|
||||
|
||||
These are the proposed new message formats.
|
||||
|
||||
requesting items
|
||||
----------------
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
{
|
||||
"a":
|
||||
{
|
||||
"filter": *<variable size bloom-filter>*,
|
||||
"id": *<20 byte id of origin node>*,
|
||||
"key": *<64 byte public curve25519 key for this feed>*,
|
||||
"n": *<feed-name>*
|
||||
"target": *<target-id as derived from public key>*
|
||||
},
|
||||
"q": "get_item",
|
||||
"t": *<transaction-id>*,
|
||||
"y": "q",
|
||||
}
|
||||
|
||||
The ``target`` MUST always be SHA-1(*feed_name* + *public_key*). Any request where
|
||||
this condition is not met, MUST be dropped.
|
||||
|
||||
The ``n`` field is the name of this feed. It MUST be UTF-8 encoded string and it
|
||||
MUST match the name of the feed in the receiving node.
|
||||
|
||||
The bloom filter argument (``filter``) in the ``get_item`` requests is optional.
|
||||
If included in a request, it represents info-hashes that should be excluded from
|
||||
the response. In this case, the response should be a random subset of the non-excluded
|
||||
items, or all of the non-excluded items if they all fit within a packet size.
|
||||
|
||||
If the bloom filter is specified, its size MUST be an even multiple of 8 bits. The size
|
||||
is implied by the length of the string. For each info-hash to exclude from the response,
|
||||
|
||||
There are no hash functions for the bloom filter. Since the info-hash is already a
|
||||
hash digest, each pair of bytes, starting with the first bytes (MSB), are used as the
|
||||
results from the imaginary hash functions for the bloom filter. k is 3 in this bloom
|
||||
filter. This means the first 6 bytes of the info-hash is used to set 3 bits in the bloom
|
||||
filter. The pairs of bytes pulled out from the info-hash are interpreted as a big-endian
|
||||
16 bit value.
|
||||
|
||||
Bits are indexed in bytes from left to right, and within bytes from LSB to MSB. i.e., to
|
||||
set bit 12: ``bitfield[12/8] |= (12 % 8)``.
|
||||
|
||||
Example:
|
||||
To indicate that you are not interested in knowing about the info-hash that
|
||||
starts with 0x4f7d25a... and you choose a bloom filter of size 80 bits. Set bits
|
||||
(0x4f % 80), (0x7d % 80) and (0x25 % 80) in the bloom filter bitmask.
|
||||
|
||||
|
||||
request item response
|
||||
---------------------
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
{
|
||||
"r":
|
||||
{
|
||||
"ih":
|
||||
[
|
||||
*<n * 20 byte(s) info-hash>*,
|
||||
...
|
||||
],
|
||||
"sig":
|
||||
[
|
||||
*<64 byte curve25519 signature of info-hash>*,
|
||||
...
|
||||
],
|
||||
"id": *<20 byte id of origin node>*,
|
||||
"token": *<write-token>*
|
||||
"nodes": *<n * compact IPv4-port pair>*
|
||||
"nodes6": *<n * compact IPv6-port pair>*
|
||||
},
|
||||
"t": *<transaction-id>*,
|
||||
"y": "r",
|
||||
}
|
||||
|
||||
Since the data that's being signed by the public key already is a hash (i.e.
|
||||
an info-hash), the signature of each hash-entry is simply the hash encrypted
|
||||
by the feed's private key.
|
||||
|
||||
The ``ih`` and ``sig`` lists MUST have equal number of items. Each item in ``sig``
|
||||
is the signature of the full string in the corresponding item in the ``ih`` list.
|
||||
|
||||
Each item in the ``ih`` list may contain any positive number of 20 byte info-hashes.
|
||||
|
||||
The rationale behind using lists of strings where the strings contain multiple
|
||||
info-hashes is to allow the publisher of a feed to sign multiple info-hashes
|
||||
together, and thus saving space in the UDP packets, allowing nodes to transfer more
|
||||
info-hashes per packet. Original publishers of a feed MAY re-announce items lumped
|
||||
together over time to make the feed more efficient.
|
||||
|
||||
A client receiving a ``get_item`` response MUST verify each signature in the ``sig``
|
||||
list against each corresponding item in the ``ih`` list using the feed's public key.
|
||||
Any item whose signature
|
||||
|
||||
``nodes`` and ``nodes6`` are optional and have the same semantics as the standard
|
||||
``get_peers`` request. The intention is to be able to use this ``get_item`` request
|
||||
in the same way, searching for the nodes responsible for the feed.
|
||||
|
||||
announcing items
|
||||
----------------
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
{
|
||||
"a":
|
||||
{
|
||||
"ih":
|
||||
[
|
||||
*<n * 20 byte info-hash(es)>*,
|
||||
...
|
||||
],
|
||||
"sig":
|
||||
[
|
||||
*<64 byte curve25519 signature of info-hash(es)>*,
|
||||
...
|
||||
],
|
||||
"id": *<20 byte node-id of origin node>*,
|
||||
"key": *<64 byte public curve25519 key for this feed>*,
|
||||
"n": *<feed name>*
|
||||
"target": *<target-id as derived from public key>*,
|
||||
"token": *<write-token as obtained by previous req.>*
|
||||
},
|
||||
"y": "q",
|
||||
"q": "announce_item",
|
||||
"t": *<transaction-id>*
|
||||
}
|
||||
|
||||
An announce can include any number of items, as long as they fit in a packet.
|
||||
|
||||
Subscribers to a feed SHOULD also announce items that they know of, to the feed.
|
||||
In order to make the repository of torrents as reliable as possible, subscribers
|
||||
SHOULD announce random items from their local repository of items. When re-announcing
|
||||
items, a random subset of all known items should be announced, randomized
|
||||
independently for each node it's announced to. This makes it a little bit harder
|
||||
to determine the IP address an item originated from, since it's a matter of
|
||||
seeing the first announce, and knowing that it wasn't announced anywhere else
|
||||
first.
|
||||
|
||||
Any subscriber and publisher SHOULD re-announce items every 30 minutes. If
|
||||
a feed does not receive any announced items in 60 minutes, a peer MAY time
|
||||
it out and remove it.
|
||||
|
||||
Subscribers and publishers SHOULD announce random items.
|
||||
|
||||
Example
|
||||
.......
|
||||
|
||||
::
|
||||
|
||||
{
|
||||
"a":
|
||||
{
|
||||
"ih":
|
||||
[
|
||||
"7ea94c240691311dc0916a2a91eb7c3db2c6f3e4",
|
||||
"0d92ad53c052ac1f49cf4434afffafa4712dc062e4168d940a48e45a45a0b10808014dc267549624"
|
||||
],
|
||||
"sig":
|
||||
[
|
||||
"980774404e404941b81aa9da1da0101cab54e670cff4f0054aa563c3b5abcb0fe3c6df5dac1ea25266035f09040bf2a24ae5f614787f1fe7404bf12fee5e6101",
|
||||
"3fee52abea47e4d43e957c02873193fb9aec043756845946ec29cceb1f095f03d876a7884e38c53cd89a8041a2adfb2d9241b5ec5d70268714d168b9353a2c01"
|
||||
],
|
||||
"id": "b46989156404e8e0acdb751ef553b210ef77822e",
|
||||
"key": "6bc1de5443d1a7c536cdf69433ac4a7163d3c63e2f9c92d78f6011cf63dbcd5b638bbc2119cdad0c57e4c61bc69ba5e2c08b918c2db8d1848cf514bd9958d307",
|
||||
"n": "my stuff"
|
||||
"target": "b4692ef0005639e86d7165bf378474107bf3a762"
|
||||
"token": "23ba"
|
||||
},
|
||||
"y": "q",
|
||||
"q": "announce_item",
|
||||
"t": "a421"
|
||||
}
|
||||
|
||||
Strings are printed in hex for printability, but actual encoding is binary. The
|
||||
response contains 3 feed items, starting with "7ea94c", "0d92ad" and "e4168d".
|
||||
These 3 items are not published optimally. If they were to be merged into a single
|
||||
string in the ``ih`` list, more than 64 bytes would be saved (because of having
|
||||
one less signature).
|
||||
|
||||
Note that ``target`` is in fact SHA1('my stuff' + 'key'). The private key
|
||||
used in this example is 980f4cd7b812ae3430ea05af7c09a7e430275f324f42275ca534d9f7c6d06f5b.
|
||||
|
||||
|
||||
URI Scheme
|
||||
----------
|
||||
|
||||
The proposed URI scheme for DHT feeds is:
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
magnet:?xt=btfd:*<base16-curve25519-public-key>* &dn= *<feed name>*
|
||||
|
||||
Note that a difference from regular torrent magnet links is the **btfd**
|
||||
versus **btih** used in regular magnet links to torrents.
|
||||
|
||||
The *feed name* is mandatory since it is used in the request and when
|
||||
calculating the target ID.
|
||||
|
||||
rationale
|
||||
---------
|
||||
|
||||
The reason to use curve25519_ instead of, for instance, RSA is to fit more signatures
|
||||
(i.e. items) in a single DHT packet. One packet is typically restricted to between
|
||||
1280 - 1480 bytes. According to http://cr.yp.to/, curve25519 is free from patent claims
|
||||
and there are open implementations in both C and Java.
|
||||
|
||||
.. _curve25519: http://cr.yp.to/ecdh.html
|
||||
|
|
@ -69,6 +69,7 @@
|
|||
<li><a class="reference external" href="libtorrent_plugins.html">plugin interface</a></li>
|
||||
<li><a class="reference external" href="dht_extensions.html">DHT extensions</a></li>
|
||||
<li><a class="reference external" href="dht_sec.html">DHT security extension</a></li>
|
||||
<li><a class="reference external" href="dht_rss.html">DHT feed extension</a></li>
|
||||
<li><a class="reference external" href="udp_tracker_protocol.html">UDP tracker protocol</a></li>
|
||||
<li><a class="reference external" href="http://www.getright.com/seedtorrent.html">HTTP seed</a></li>
|
||||
<li><a class="reference external" href="http://bittorrent.org/beps/bep_0012.html">multitracker</a></li>
|
||||
|
|
|
@ -27,6 +27,7 @@ Extensions
|
|||
* `plugin interface`_
|
||||
* `DHT extensions`_
|
||||
* `DHT security extension`_
|
||||
* `DHT feed extension`_
|
||||
* `UDP tracker protocol`_
|
||||
* `HTTP seed`_
|
||||
* multitracker_
|
||||
|
@ -66,6 +67,7 @@ libtorrent
|
|||
.. _`plugin interface`: libtorrent_plugins.html
|
||||
.. _`DHT extensions`: dht_extensions.html
|
||||
.. _`DHT security extension`: dht_sec.html
|
||||
.. _`DHT feed extension`: dht_rss.html
|
||||
.. _`UDP tracker protocol`: udp_tracker_protocol.html
|
||||
.. _`HTTP seed`: http://www.getright.com/seedtorrent.html
|
||||
.. _multitracker: http://bittorrent.org/beps/bep_0012.html
|
||||
|
|
|
@ -4,6 +4,7 @@ WEB_PATH = ~/Documents/rasterbar/web/products/libtorrent
|
|||
|
||||
TARGETS = index \
|
||||
udp_tracker_protocol \
|
||||
dht_rss \
|
||||
client_test \
|
||||
manual \
|
||||
building \
|
||||
|
|
Loading…
Reference in New Issue