BitTorrent extension for arbitrary DHT store
+| Author: | +Arvid Norberg, arvid@rasterbar.com |
|---|---|
| Version: | +Draft |
Table of contents
+ +This is a proposal for an extension to the BitTorrent DHT to allow +storing and retrieving of arbitrary data.
+It supports both storing immutable items, where the key is +the SHA-1 hash of the data itself, and mutable items, where +the key is the public key of the key pair used to sign the data.
+There are two new proposed messages, put and get.
+terminology
+In this document, a storage node refers to the node in the DHT to which +an item is being announced and stored on. A subscribing node refers to +a node which makes look-ups in the DHT to find the storage nodes, to +request items from them, and possibly re-announce those items to keep them +alive.
+messages
+The proposed new messages get and put are similar to the existing get_peers +and announce_peer.
+Responses to get should always include nodes and nodes6 has the same +semantics as in its get_peers response. It should also include a write token, +token, with the same semantics as get_peers.
+The id field in these messages has the same semantics as the standard DHT messages, +i.e. the node ID of the node sending the message, to maintain the structure of the DHT +network.
+The token field also has the same semantics as the standard DHT message get_peers +and announce_peer, when requesting an item and to write an item respectively.
+The distinction between storing mutable and immutable items is the inclusion +of a public key, a sequence number and signature (k, seq and sig). +The distinction betwewn retrieving a mutable and immutable item is the inclusion of +the public key spill-over (k) in the get request.
+The v key is the value to be stored. It is allowed to be any bencoded type (list, +dict, string or integer). When it's being hashed (for verifying its signature or to calculate +its key), its flattened, bencoded, form is used.
+Storing nodes are SHOULD reject put requests where the bencoded form of v is longer +than 767 bytes.
+immutable items
+Immutable items are stored under their SHA-1 hash, and since they cannot be modified, +there is no need to authenticate the origin of them. This makes immutable items simple.
+put message
+Request:
+
+{
+ "a":
+ {
+ "id": <20 byte id of sending node (string)>,
+ "v": <any bencoded type, whose encoded size < 768>
+ },
+ "t": <transaction-id (string)>,
+ "y": "q",
+ "q": "put"
+}
+
+Response:
+
+{
+ "r": { "id": <20 byte id of sending node (string)> },
+ "t": <transaction-id (string)>,
+ "y": "r",
+}
+
+get message
+Request:
+
+{
+ "a":
+ {
+ "id": <20 byte id of sending node (string)>,
+ "target": <SHA-1 hash of item (string)>,
+ },
+ "t": <transaction-id (string)>,
+ "y": "q",
+ "q": "get"
+}
+
+Response:
+
+{
+ "r":
+ {
+ "id": <20 byte id of sending node (string)>,
+ "token": <write token (string)>,
+ "v": <any bencoded type whose SHA-1 hash matches 'target'>,
+ "nodes": <IPv4 nodes close to 'target'>
+ "nodes6": <IPv6 nodes close to 'target'>
+ },
+ "t": <transaction-id>,
+ "y": "r",
+}
+
+mutable items
+Mutable items can be updated, without changing their DHT keys. To authenticate +that only the original publisher can update an item, it is signed by a private key +generated by the original publisher.
+In order to avoid a malicious node to overwrite the list head with an old +version, the sequence number seq must be monotonically increasing for each update, +and a node hosting the list node MUST not downgrade a list head from a higher sequence +number to a lower one, only upgrade.
+The signature is a 2048 bit RSA signature of the SHA-1 hash of the bencoded sequence +number and v key. e.g. something like this:: 3:seqi4e1:v12:Hello world!.
+put message
+Request:
+
+{
+ "a":
+ {
+ "id": <20 byte id of sending node (string)>,
+ "k": <RSA-2048 public key (268 bytes string)>,
+ "seq": <monotonically increasing sequence number (integer)>,
+ "sig": <RSA-2048 signature (256 bytes string)>,
+ "token": <write-token (string)>,
+ "v": <any bencoded type, whose encoded size < 768>
+ },
+ "t": <transaction-id (string)>,
+ "y": "q",
+ "q": "put"
+}
+
+Storing nodes receiving a put request where seq is lower than what's already +stored on the node, MUST reject the request.
+Response:
+
+{
+ "r": { "id": <20 byte id of sending node (string)> },
+ "t": <transaction-id (string)>,
+ "y": "r",
+}
+
+get message
+Request:
+
+{
+ "r":
+ {
+ "id": <20 byte id of sending node (string)>,
+ "target:" <first 20 bytes of public key (string)>,
+ "k": <remaining 248 bytes of public key (string)>
+ },
+ "t": <transaction-id (string)>,
+ "y": "r",
+ "q": "get"
+}
+
+Response:
+
+{
+ "r":
+ {
+ "id": <20 byte id of sending node (string)>,
+ "k": <RSA-2048 public key (268 bytes string)>,
+ "seq": <monotonically increasing sequence number (integer)>,
+ "sig": <RSA-2048 signature (256 bytes string)>,
+ "token": <write-token (string)>,
+ "v": <any bencoded type, whose encoded size < 768>
+
+ },
+ "t": <transaction-id (string)>,
+ "y": "r",
+}
+
+signature verification
+In order to make it maximally difficult to attack the bencoding parser, signing and verification of the +value and sequence number should be done as follows:
+-
+
- encode value and sequence number separately +
- concatenate "3:seqi" seq "e1:v" and the encoded value. +sequence number 1 of value "Hello World!" would be converted to: 3:seqi1e1:v12:Hello World! +In this way it is not possible to convince a node that part of the length is actually part of the +sequence number even if the parser contains certain bugs. Furthermore it is not possible to have a +verification failure if a bencoding serializer alters the order of entries in the dictionary. +
- hash the concatenated string with SHA-1 +
- sign or verify the hash digest. +
expiration
+Without re-announcement, these items MAY expire in 2 hours. In order +to keep items alive, they SHOULD be re-announced once an hour.
+Subscriber nodes MAY help out in announcing items the are interested in to the DHT, +to keep them alive.
+