wornvirtue
/
premiere-libtorrent
forked from premiere/premiere-libtorrent
1
0
Fork 0
Code Pull Requests Activity

merged DHT fix from RC_0_16

This commit is contained in:
Arvid Norberg 2012-11-16 22:25:39 +00:00
parent 6612d1f88a
commit f12e1c1a3f
5 changed files with 59 additions and 39 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
ChangeLog
View File

@ -5,6 +5,7 @@
* fix uTP edge case where udp socket buffer fills up * fix uTP edge case where udp socket buffer fills up
* fix nagle implementation in uTP * fix nagle implementation in uTP
* fix support for storing arbitrary data in the DHT
* fixed bug in uTP packet circle buffer * fixed bug in uTP packet circle buffer
* fix potential crash when using torrent_handle::add_piece * fix potential crash when using torrent_handle::add_piece
* added missing add_torrent_alert to python binding * added missing add_torrent_alert to python binding

28
docs/dht_store.html
View File

@ -98,14 +98,19 @@ i.e. the node ID of the node sending the message, to maintain the structure of t
network.</p> network.</p>
<p>The <tt class="docutils literal">token</tt> field also has the same semantics as the standard DHT message <tt class="docutils literal">get_peers</tt> <p>The <tt class="docutils literal">token</tt> field also has the same semantics as the standard DHT message <tt class="docutils literal">get_peers</tt>
and <tt class="docutils literal">announce_peer</tt>, when requesting an item and to write an item respectively.</p> and <tt class="docutils literal">announce_peer</tt>, when requesting an item and to write an item respectively.</p>
<p>The <tt class="docutils literal">k</tt> field is the PKCS#1 encoded 2048 bit RSA public key, which the signature
can be authenticated with. When looking up a mutable item, the <tt class="docutils literal">target</tt> field
MUST be the SHA-1 hash of this key.</p>
<p>The distinction between storing mutable and immutable items is the inclusion <p>The distinction between storing mutable and immutable items is the inclusion
of a public key, a sequence number and signature (<tt class="docutils literal">k</tt>, <tt class="docutils literal">seq</tt> and <tt class="docutils literal">sig</tt>). of a public key, a sequence number and signature (<tt class="docutils literal">k</tt>, <tt class="docutils literal">seq</tt> and <tt class="docutils literal">sig</tt>).</p>
The distinction betwewn retrieving a mutable and immutable item is the inclusion of <p><tt class="docutils literal">get</tt> requests for mutable items and immutable items cannot be distinguished from
the public key spill-over (<tt class="docutils literal">k</tt>) in the <tt class="docutils literal">get</tt> request.</p> eachother. An implementation can either store mutable and immutable items in the same
<p>The <tt class="docutils literal">v</tt> key is the <em>value</em> to be stored. It is allowed to be any bencoded type (list, hash table internally, or in separate ones and potentially do two lookups for <tt class="docutils literal">get</tt>
requests.</p>
<p>The <tt class="docutils literal">v</tt> field is the <em>value</em> to be stored. It is allowed to be any bencoded type (list,
dict, string or integer). When it's being hashed (for verifying its signature or to calculate dict, string or integer). When it's being hashed (for verifying its signature or to calculate
its key), its flattened, bencoded, form is used.</p> its key), its flattened, bencoded, form is used.</p>
<p>Storing nodes are SHOULD reject <tt class="docutils literal">put</tt> requests where the bencoded form of <tt class="docutils literal">v</tt> is longer <p>Storing nodes SHOULD reject <tt class="docutils literal">put</tt> requests where the bencoded form of <tt class="docutils literal">v</tt> is longer
than 767 bytes.</p> than 767 bytes.</p>
</div> </div>
<div class="section" id="immutable-items"> <div class="section" id="immutable-items">
@ -172,7 +177,8 @@ there is no need to authenticate the origin of them. This makes immutable items
<h1>mutable items</h1> <h1>mutable items</h1>
<p>Mutable items can be updated, without changing their DHT keys. To authenticate <p>Mutable items can be updated, without changing their DHT keys. To authenticate
that only the original publisher can update an item, it is signed by a private key that only the original publisher can update an item, it is signed by a private key
generated by the original publisher.</p> generated by the original publisher. The target ID mutable items are stored under
is the SHA-1 hash of the public key (as it appears in the <tt class="docutils literal">put</tt> message).</p>
<p>In order to avoid a malicious node to overwrite the list head with an old <p>In order to avoid a malicious node to overwrite the list head with an old
version, the sequence number <tt class="docutils literal">seq</tt> must be monotonically increasing for each update, version, the sequence number <tt class="docutils literal">seq</tt> must be monotonically increasing for each update,
and a node hosting the list node MUST not downgrade a list head from a higher sequence and a node hosting the list node MUST not downgrade a list head from a higher sequence
@ -187,7 +193,7 @@ number and <tt class="docutils literal">v</tt> key. e.g. something like this:: <
&quot;a&quot;: &quot;a&quot;:
{ {
&quot;id&quot;: <em>&lt;20 byte id of sending node (string)&gt;</em>, &quot;id&quot;: <em>&lt;20 byte id of sending node (string)&gt;</em>,
&quot;k&quot;: <em>&lt;RSA-2048 public key (268 bytes string)&gt;</em>, &quot;k&quot;: <em>&lt;RSA-2048 public key (PKCS#1 encoded)&gt;</em>,
&quot;seq&quot;: <em>&lt;monotonically increasing sequence number (integer)&gt;</em>, &quot;seq&quot;: <em>&lt;monotonically increasing sequence number (integer)&gt;</em>,
&quot;sig&quot;: <em>&lt;RSA-2048 signature (256 bytes string)&gt;</em>, &quot;sig&quot;: <em>&lt;RSA-2048 signature (256 bytes string)&gt;</em>,
&quot;token&quot;: <em>&lt;write-token (string)&gt;</em>, &quot;token&quot;: <em>&lt;write-token (string)&gt;</em>,
@ -217,8 +223,7 @@ stored on the node, MUST reject the request.</p>
&quot;a&quot;: &quot;a&quot;:
{ {
&quot;id&quot;: <em>&lt;20 byte id of sending node (string)&gt;</em>, &quot;id&quot;: <em>&lt;20 byte id of sending node (string)&gt;</em>,
&quot;target:&quot; <em>&lt;first 20 bytes of public key (string)&gt;</em>, &quot;target:&quot; <em>&lt;20 byte SHA-1 hash of public key (string)&gt;</em>
&quot;k&quot;: <em>&lt;remaining 248 bytes of public key (string)&gt;</em>
}, },
&quot;t&quot;: <em>&lt;transaction-id (string)&gt;</em>, &quot;t&quot;: <em>&lt;transaction-id (string)&gt;</em>,
&quot;y&quot;: &quot;q&quot;, &quot;y&quot;: &quot;q&quot;,
@ -259,6 +264,11 @@ verification failure if a bencoding serializer alters the order of entries in th
<li>hash the concatenated string with SHA-1</li> <li>hash the concatenated string with SHA-1</li>
<li>sign or verify the hash digest.</li> <li>sign or verify the hash digest.</li>
</ol> </ol>
<p>On the storage node, the signature MUST be verified before accepting the store command. The data
MUST be stored under the SHA-1 hash of the public key (as it appears in the bencoded dict).</p>
<p>On the subscribing nodes, the key they get back from a <tt class="docutils literal">get</tt> request MUST be verified to hash
to the target ID the lookup was made for, as well as verifying the signature. If any of these fail,
the response SHOULD be considered invalid.</p>
</div> </div>
<div class="section" id="expiration"> <div class="section" id="expiration">
<h1>expiration</h1> <h1>expiration</h1>

30
docs/dht_store.rst
View File

@ -44,16 +44,23 @@ network.
The ``token`` field also has the same semantics as the standard DHT message ``get_peers`` The ``token`` field also has the same semantics as the standard DHT message ``get_peers``
and ``announce_peer``, when requesting an item and to write an item respectively. and ``announce_peer``, when requesting an item and to write an item respectively.
The ``k`` field is the PKCS#1 encoded 2048 bit RSA public key, which the signature
can be authenticated with. When looking up a mutable item, the ``target`` field
MUST be the SHA-1 hash of this key.
The distinction between storing mutable and immutable items is the inclusion The distinction between storing mutable and immutable items is the inclusion
of a public key, a sequence number and signature (``k``, ``seq`` and ``sig``). of a public key, a sequence number and signature (``k``, ``seq`` and ``sig``).
The distinction betwewn retrieving a mutable and immutable item is the inclusion of
the public key spill-over (``k``) in the ``get`` request.
The ``v`` key is the *value* to be stored. It is allowed to be any bencoded type (list, ``get`` requests for mutable items and immutable items cannot be distinguished from
eachother. An implementation can either store mutable and immutable items in the same
hash table internally, or in separate ones and potentially do two lookups for ``get``
requests.
The ``v`` field is the *value* to be stored. It is allowed to be any bencoded type (list,
dict, string or integer). When it's being hashed (for verifying its signature or to calculate dict, string or integer). When it's being hashed (for verifying its signature or to calculate
its key), its flattened, bencoded, form is used. its key), its flattened, bencoded, form is used.
Storing nodes are SHOULD reject ``put`` requests where the bencoded form of ``v`` is longer Storing nodes SHOULD reject ``put`` requests where the bencoded form of ``v`` is longer
than 767 bytes. than 767 bytes.
immutable items immutable items
@ -131,7 +138,8 @@ mutable items
Mutable items can be updated, without changing their DHT keys. To authenticate Mutable items can be updated, without changing their DHT keys. To authenticate
that only the original publisher can update an item, it is signed by a private key that only the original publisher can update an item, it is signed by a private key
generated by the original publisher. generated by the original publisher. The target ID mutable items are stored under
is the SHA-1 hash of the public key (as it appears in the ``put`` message).
In order to avoid a malicious node to overwrite the list head with an old In order to avoid a malicious node to overwrite the list head with an old
version, the sequence number ``seq`` must be monotonically increasing for each update, version, the sequence number ``seq`` must be monotonically increasing for each update,
@ -152,7 +160,7 @@ Request:
"a": "a":
{ {
"id": *<20 byte id of sending node (string)>*, "id": *<20 byte id of sending node (string)>*,
"k": *<RSA-2048 public key (268 bytes string)>*, "k": *<RSA-2048 public key (PKCS#1 encoded)>*,
"seq": *<monotonically increasing sequence number (integer)>*, "seq": *<monotonically increasing sequence number (integer)>*,
"sig": *<RSA-2048 signature (256 bytes string)>*, "sig": *<RSA-2048 signature (256 bytes string)>*,
"token": *<write-token (string)>*, "token": *<write-token (string)>*,
@ -187,8 +195,7 @@ Request:
"a": "a":
{ {
"id": *<20 byte id of sending node (string)>*, "id": *<20 byte id of sending node (string)>*,
"target:" *<first 20 bytes of public key (string)>*, "target:" *<20 byte SHA-1 hash of public key (string)>*
"k": *<remaining 248 bytes of public key (string)>*
}, },
"t": *<transaction-id (string)>*, "t": *<transaction-id (string)>*,
"y": "q", "y": "q",
@ -230,6 +237,13 @@ value and sequence number should be done as follows:
3. hash the concatenated string with SHA-1 3. hash the concatenated string with SHA-1
4. sign or verify the hash digest. 4. sign or verify the hash digest.
On the storage node, the signature MUST be verified before accepting the store command. The data
MUST be stored under the SHA-1 hash of the public key (as it appears in the bencoded dict).
On the subscribing nodes, the key they get back from a ``get`` request MUST be verified to hash
to the target ID the lookup was made for, as well as verifying the signature. If any of these fail,
the response SHOULD be considered invalid.
expiration expiration
---------- ----------

7
include/libtorrent/kademlia/node.hpp
View File

@ -131,14 +131,15 @@ struct dht_immutable_item
int size; int size;
}; };
struct rsa_key { char bytes[268]; };
struct dht_mutable_item : dht_immutable_item struct dht_mutable_item : dht_immutable_item
{ {
char sig[256]; char sig[256];
int seq; int seq;
rsa_key key;
}; };
struct rsa_key { char bytes[268]; };
inline bool operator<(rsa_key const& lhs, rsa_key const& rhs) inline bool operator<(rsa_key const& lhs, rsa_key const& rhs)
{ {
return memcmp(lhs.bytes, rhs.bytes, sizeof(lhs.bytes)) < 0; return memcmp(lhs.bytes, rhs.bytes, sizeof(lhs.bytes)) < 0;
@ -184,7 +185,7 @@ class TORRENT_EXTRA_EXPORT node_impl : boost::noncopyable
{ {
typedef std::map<node_id, torrent_entry> table_t; typedef std::map<node_id, torrent_entry> table_t;
typedef std::map<node_id, dht_immutable_item> dht_immutable_table_t; typedef std::map<node_id, dht_immutable_item> dht_immutable_table_t;
typedef std::map<rsa_key, dht_mutable_item> dht_mutable_table_t; typedef std::map<node_id, dht_mutable_item> dht_mutable_table_t;
public: public:
node_impl(alert_dispatcher* alert_disp, udp_socket_interface* sock node_impl(alert_dispatcher* alert_disp, udp_socket_interface* sock

32
src/kademlia/node.cpp
View File

@ -891,8 +891,7 @@ void node_impl::incoming_request(msg const& m, entry& e)
return; return;
#endif #endif
rsa_key target; sha1_hash target = hasher(msg_keys[3]->string_ptr(), msg_keys[3]->string_length()).final();
memcpy(target.bytes, msg_keys[3]->string_ptr(), sizeof(target.bytes));
dht_mutable_table_t::iterator i = m_mutable_table.find(target); dht_mutable_table_t::iterator i = m_mutable_table.find(target);
if (i == m_mutable_table.end()) if (i == m_mutable_table.end())
{ {
@ -916,6 +915,7 @@ void node_impl::incoming_request(msg const& m, entry& e)
memcpy(to_add.sig, msg_keys[4]->string_ptr(), sizeof(to_add.sig)); memcpy(to_add.sig, msg_keys[4]->string_ptr(), sizeof(to_add.sig));
TORRENT_ASSERT(sizeof(to_add.sig) == msg_keys[4]->string_length()); TORRENT_ASSERT(sizeof(to_add.sig) == msg_keys[4]->string_length());
memcpy(to_add.value, buf.first, buf.second); memcpy(to_add.value, buf.first, buf.second);
memcpy(&to_add.key, msg_keys[3]->string_ptr(), sizeof(to_add.key));
boost::tie(i, boost::tuples::ignore) = m_mutable_table.insert( boost::tie(i, boost::tuples::ignore) = m_mutable_table.insert(
std::make_pair(target, to_add)); std::make_pair(target, to_add));
@ -967,14 +967,13 @@ void node_impl::incoming_request(msg const& m, entry& e)
{ {
key_desc_t msg_desc[] = { key_desc_t msg_desc[] = {
{"target", lazy_entry::string_t, 20, 0}, {"target", lazy_entry::string_t, 20, 0},
{"k", lazy_entry::string_t, 268-20, key_desc_t::optional},
}; };
// k is not used for now // k is not used for now
// attempt to parse the message // attempt to parse the message
lazy_entry const* msg_keys[2]; lazy_entry const* msg_keys[1];
if (!verify_message(arg_ent, msg_desc, msg_keys, 2, error_string, sizeof(error_string))) if (!verify_message(arg_ent, msg_desc, msg_keys, 1, error_string, sizeof(error_string)))
{ {
incoming_error(e, error_string); incoming_error(e, error_string);
return; return;
@ -993,27 +992,22 @@ void node_impl::incoming_request(msg const& m, entry& e)
m_table.find_node(target, n, 0); m_table.find_node(target, n, 0);
write_nodes_entry(reply, n); write_nodes_entry(reply, n);
if (msg_keys[1]) dht_immutable_table_t::iterator i = m_immutable_table.find(target);
if (i != m_immutable_table.end())
{ {
rsa_key key; dht_immutable_item const& f = i->second;
memcpy(key.bytes, msg_keys[0]->string_ptr(), 20); reply["v"] = bdecode(f.value, f.value + f.size);
memcpy(key.bytes + 20, msg_keys[1]->string_ptr(), 268-20); }
dht_mutable_table_t::iterator i = m_mutable_table.find(key); else
{
dht_mutable_table_t::iterator i = m_mutable_table.find(target);
if (i != m_mutable_table.end()) if (i != m_mutable_table.end())
{ {
dht_mutable_item const& f = i->second; dht_mutable_item const& f = i->second;
reply["v"] = bdecode(f.value, f.value + f.size); reply["v"] = bdecode(f.value, f.value + f.size);
reply["seq"] = f.seq; reply["seq"] = f.seq;
reply["sig"] = std::string(f.sig, f.sig + 256); reply["sig"] = std::string(f.sig, f.sig + 256);
} reply["k"] = std::string(f.key.bytes, f.key.bytes + sizeof(f.key.bytes));
}
else
{
dht_immutable_table_t::iterator i = m_immutable_table.find(target);
if (i != m_immutable_table.end())
{
dht_immutable_item const& f = i->second;
reply["v"] = bdecode(f.value, f.value + f.size);
} }
} }
} }