diff --git a/Jamfile b/Jamfile index 62871a76d..1ae85093d 100755 --- a/Jamfile +++ b/Jamfile @@ -103,6 +103,7 @@ lib torrent multi static # debug:TORRENT_VERBOSE_LOGGING +# TORRENT_ENABLE_EXTENSIONS : debug release ; diff --git a/docs/manual.html b/docs/manual.html index 59fda6317..37a731114 100755 --- a/docs/manual.html +++ b/docs/manual.html @@ -160,7 +160,6 @@ means it can resume a torrent downloaded by any client.

Functions that are yet to be implemented:

    -
  • large file support on linux and Mac OS X.
  • better identification of peers that send bad data
  • ip-filters
  • file-level priority
    • @@ -209,8 +208,8 @@ as a dll too, by typing link=shared abstraction. There's one file_win.cpp which relies on windows file API that supports files larger than 2 Gigabytes. This does not work in vc6 for some reason, possibly because it may require windows NT and above. The other file, file.cpp is the default -implementation that simply relies on the standard library's fstream, and as a result does -not support files larger than 2 Gigabytes.

    +implementation that simply relies on the standard low level io routines (read, write etc.), +but for some reason this implementation doesn't seem to work on windows.

    cygwin and msvc

    Note that if you're building on windows using the msvc toolset, you cannot run it @@ -311,7 +310,7 @@ class session: public boost::noncopyable , const char* listen_interface = 0); torrent_handle add_torrent( - torrent_info const& t + entry const& e , boost::filesystem::path const& save_path , entry const& resume_data = entry()); @@ -360,7 +359,7 @@ the parameters, see listen_on()

    ~session()

    -

    The destructor of session will notify all trackers that our torrents has been shut down. +

    The destructor of session will notify all trackers that our torrents have been shut down. If some trackers are down, they will timout. All this before the destructor of session returns. So, it's adviced that any kind of interface (such as windows) are closed before destructing the sessoin object. Because it can take a few second for it to finish. The @@ -371,7 +370,7 @@ timeout can be set with set_http_settings( 

     torrent_handle add_torrent(
-        torrent_info const& t
+        entry const& e
         , boost::filesystem::path const& save_path
         , entry const& resume_data = entry());
    @@ -420,7 +419,7 @@ of upload rate.

    session_status status() const;
    -

    status() returns session wide statistics and status. The session_status +

    status() returns session wide-statistics and status. The session_status struct has the following members:

     struct session_status
@@ -442,7 +441,7 @@ struct session_status
         int num_peers;
 };
    -

    has_incoming_connections is false as long as no incoming connections has been +

    has_incoming_connections is false as long as no incoming connections have been established on the listening socket. Every time you change the listen port, this will be reset to false.

    upload_rate, download_rate, payload_download_rate and payload_upload_rate @@ -1408,6 +1407,15 @@ sure not to clash with anybody else. Here are some taken id's:

    'MT' Moonlight Torrent +'TS' +Torrent Storm + +'SS' +Swarm Scope + +'XT' +Xan Torrent +

    The major, minor, revision and tag parameters are used to identify the @@ -1540,14 +1548,14 @@ public: alert(severity_t severity, const std::string& msg); virtual ~alert(); - const std::string& msg() const; + std::string const& msg() const; severity_t severity() const; virtual std::auto_ptr<alert> clone() const = 0; };

    This means that all alerts have at least a string describing it. They also -have a severity leve that can be used to sort them or present them to the +have a severity level that can be used to sort them or present them to the user in different ways.

    The specific alerts, that all derives from alert, are:

    @@ -1899,8 +1907,7 @@ int main(int argc, char* argv[]) std::ifstream in(argv[1], std::ios_base::binary); in.unsetf(std::ios_base::skipws); entry e = bdecode(std::istream_iterator<char>(in), std::istream_iterator<char>()); - torrent_info t(e); - s.add_torrent(t, ""); + s.add_torrent(e, ""); // wait for the user to end char a; diff --git a/docs/manual.rst b/docs/manual.rst index fb67a424f..c6133de21 100755 --- a/docs/manual.rst +++ b/docs/manual.rst @@ -218,7 +218,7 @@ The ``session`` class has the following synopsis:: , const char* listen_interface = 0); torrent_handle add_torrent( - torrent_info const& t + entry const& e , boost::filesystem::path const& save_path , entry const& resume_data = entry()); @@ -269,7 +269,7 @@ the parameters, see ``listen_on()`` function. ~session() ---------- -The destructor of session will notify all trackers that our torrents has been shut down. +The destructor of session will notify all trackers that our torrents have been shut down. If some trackers are down, they will timout. All this before the destructor of session returns. So, it's adviced that any kind of interface (such as windows) are closed before destructing the sessoin object. Because it can take a few second for it to finish. The @@ -282,7 +282,7 @@ add_torrent() :: torrent_handle add_torrent( - torrent_info const& t + entry const& e , boost::filesystem::path const& save_path , entry const& resume_data = entry()); @@ -336,7 +336,7 @@ status() session_status status() const; -``status()`` returns session wide statistics and status. The ``session_status`` +``status()`` returns session wide-statistics and status. The ``session_status`` struct has the following members:: struct session_status @@ -358,7 +358,7 @@ struct has the following members:: int num_peers; }; -``has_incoming_connections`` is false as long as no incoming connections has been +``has_incoming_connections`` is false as long as no incoming connections have been established on the listening socket. Every time you change the listen port, this will be reset to false. @@ -1407,6 +1407,12 @@ sure not to clash with anybody else. Here are some taken id's: +----------+-----------------------+ | 'MT' | Moonlight Torrent | +----------+-----------------------+ +| 'TS' | Torrent Storm | ++----------+-----------------------+ +| 'SS' | Swarm Scope | ++----------+-----------------------+ +| 'XT' | Xan Torrent | ++----------+-----------------------+ The ``major``, ``minor``, ``revision`` and ``tag`` parameters are used to identify the @@ -1555,14 +1561,14 @@ is its synopsis:: alert(severity_t severity, const std::string& msg); virtual ~alert(); - const std::string& msg() const; + std::string const& msg() const; severity_t severity() const; virtual std::auto_ptr clone() const = 0; }; This means that all alerts have at least a string describing it. They also -have a severity leve that can be used to sort them or present them to the +have a severity level that can be used to sort them or present them to the user in different ways. The specific alerts, that all derives from ``alert``, are: @@ -1967,8 +1973,7 @@ This is a simple client. It doesn't have much output to keep it simple:: std::ifstream in(argv[1], std::ios_base::binary); in.unsetf(std::ios_base::skipws); entry e = bdecode(std::istream_iterator(in), std::istream_iterator()); - torrent_info t(e); - s.add_torrent(t, ""); + s.add_torrent(e, ""); // wait for the user to end char a; diff --git a/examples/client_test.cpp b/examples/client_test.cpp index a74803afb..9474773ad 100755 --- a/examples/client_test.cpp +++ b/examples/client_test.cpp @@ -226,7 +226,7 @@ int main(int argc, char* argv[]) return 1; } - namespace fs = boost::filesystem; + namespace fs = boost::filesystem; fs::path::default_name_check(fs::no_check); http_settings settings; @@ -243,8 +243,8 @@ int main(int argc, char* argv[]) std::vector handles; session ses(fingerprint("LT", 0, 1, 0, 0)); - ses.listen_on(std::make_pair(100, 110)); - ses.set_upload_rate_limit(50000); + ses.listen_on(std::make_pair(6881, 6889)); + ses.set_upload_rate_limit(100000); // ses.set_download_rate_limit(50000); ses.set_http_settings(settings); ses.set_severity_level(alert::debug); @@ -254,13 +254,26 @@ int main(int argc, char* argv[]) { try { + boost::filesystem::path save_path(""); + + if (std::string(argv[i+1]).substr(0, 7) == "http://") + { + sha1_hash info_hash = boost::lexical_cast(argv[i+2]); + + handles.push_back(ses.add_torrent(argv[i+1], info_hash, save_path)); + handles.back().set_max_connections(60); + handles.back().set_max_uploads(7); + handles.back().set_ratio(1.02f); + ++i; + + continue; + } std::ifstream in(argv[i+1], std::ios_base::binary); in.unsetf(std::ios_base::skipws); entry e = bdecode(std::istream_iterator(in), std::istream_iterator()); torrent_info t(e); t.print(std::cout); - boost::filesystem::path save_path(""); entry resume_data; try { @@ -275,9 +288,9 @@ int main(int argc, char* argv[]) catch (invalid_encoding&) {} catch (boost::filesystem::filesystem_error&) {} - handles.push_back(ses.add_torrent(t, save_path, resume_data)); - handles.back().set_max_connections(60); - handles.back().set_max_uploads(7); + handles.push_back(ses.add_torrent(e, save_path, resume_data)); + handles.back().set_max_connections(200); + handles.back().set_max_uploads(20); handles.back().set_ratio(1.02f); } catch (std::exception& e) @@ -286,6 +299,8 @@ int main(int argc, char* argv[]) } } + if (handles.empty()) return 1; + std::vector peers; std::vector queue; @@ -301,6 +316,8 @@ int main(int argc, char* argv[]) ++i) { torrent_handle h = *i; + if (!h.get_torrent_info().is_valid()) continue; + entry data = h.write_resume_data(); std::stringstream s; s << h.get_torrent_info().name() << ".fastresume"; @@ -396,6 +413,9 @@ int main(int argc, char* argv[]) case torrent_status::connecting_to_tracker: out << "connecting to tracker "; break; + case torrent_status::downloading_metadata: + out << "downloading metadata "; + break; case torrent_status::downloading: out << "downloading "; break; @@ -419,6 +439,7 @@ int main(int argc, char* argv[]) << "u:" << add_suffix(s.upload_rate) << "/s " << "(" << add_suffix(s.total_upload) << ") " << "ratio: " << ratio(s.total_payload_download, s.total_payload_upload) << "\n"; + out << "info-hash: " << i->info_hash() << "\n"; boost::posix_time::time_duration t = s.next_announce; out << "next announce: " << boost::posix_time::to_simple_string(t) << "\n"; diff --git a/examples/simple_client.cpp b/examples/simple_client.cpp index 9a25c5af6..0138a447e 100755 --- a/examples/simple_client.cpp +++ b/examples/simple_client.cpp @@ -62,8 +62,7 @@ int main(int argc, char* argv[]) std::ifstream in(argv[1], std::ios_base::binary); in.unsetf(std::ios_base::skipws); entry e = bdecode(std::istream_iterator(in), std::istream_iterator()); - torrent_info t(e); - s.add_torrent(t, ""); + s.add_torrent(e, ""); // wait for the user to end char a; diff --git a/include/libtorrent/peer_connection.hpp b/include/libtorrent/peer_connection.hpp index a844f071c..4aa0e7276 100755 --- a/include/libtorrent/peer_connection.hpp +++ b/include/libtorrent/peer_connection.hpp @@ -105,6 +105,12 @@ namespace libtorrent , selector& sel , boost::shared_ptr s); + // this function is called once the torrent associated + // with this peer connection has retrieved the meta- + // data. If the torrent was spawned with metadata + // this is called from the constructor. + void init(); + ~peer_connection(); // this adds an announcement in the announcement queue @@ -205,6 +211,18 @@ namespace libtorrent boost::shared_ptr m_logger; #endif + enum extension_index + { + extended_chat_message, + extended_metadata_message, + extended_peer_exchange_message, + extended_listen_port_message, + num_supported_extensions + }; + + bool supports_extension(extension_index ex) const + { return m_extension_messages[ex] != -1; } + // the message handlers are called // each time a recv() returns some new // data, the last time it will be called @@ -225,6 +243,11 @@ namespace libtorrent void on_extension_list(int received); void on_extended(int received); + void on_chat(); + void on_metadata(); + void on_peer_exchange(); + void on_listen_port(); + typedef void (peer_connection::*message_handler)(int received); // the following functions appends messages @@ -240,6 +263,8 @@ namespace libtorrent void send_handshake(); void send_extensions(); void send_chat_message(const std::string& msg); + void send_metadata(int start, int size); + void send_metadata_request(int start, int size); // how much bandwidth we're using, how much we want, // and how much we are allowed to use. @@ -249,10 +274,16 @@ namespace libtorrent private: +#ifndef NDEBUG void check_invariant() const; +#endif bool dispatch_message(int received); + // if we don't have all metadata + // this function will request a part of it + // from this peer + void request_metadata(); // this is called each time this peer generates some // data to be sent. It will add this socket to @@ -336,10 +367,6 @@ namespace libtorrent selector& m_selector; boost::shared_ptr m_socket; - // upload bandwidth used this second. - // Must not exceed m_ul_bandwidth_quota.given. -// int m_ul_bandwidth_quota_used; - // this is the torrent this connection is // associated with. If the connection is an // incoming conncetion, this is set to zero @@ -443,11 +470,6 @@ namespace libtorrent // considered a bad peer and will be banned. int m_trust_points; - enum extension_index - { - extended_chat_message, - num_supported_extensions - }; static const char* extension_names[num_supported_extensions]; int m_extension_messages[num_supported_extensions]; @@ -475,6 +497,10 @@ namespace libtorrent // the time when we sent a not_interested message to // this peer the last time. boost::posix_time::ptime m_became_uninteresting; + + // this is set to the current time each time we get a + // "I don't have metadata" message. + boost::posix_time::ptime m_no_metadata; }; } diff --git a/include/libtorrent/peer_id.hpp b/include/libtorrent/peer_id.hpp index b06534536..11d88591f 100755 --- a/include/libtorrent/peer_id.hpp +++ b/include/libtorrent/peer_id.hpp @@ -36,6 +36,7 @@ POSSIBILITY OF SUCH DAMAGE. #include #include #include +#include #include namespace libtorrent @@ -43,10 +44,17 @@ namespace libtorrent class big_number { + // private type + struct private_pointer {}; // the number of bytes of the number enum { number_size = 20 }; public: + big_number() {} + + // when initialized with 0 + big_number(private_pointer*) { clear(); } + void clear() { std::fill(m_number,m_number+number_size,0); @@ -101,7 +109,7 @@ namespace libtorrent typedef big_number peer_id; typedef big_number sha1_hash; - inline std::ostream& operator<<(std::ostream& os, const big_number& peer) + inline std::ostream& operator<<(std::ostream& os, big_number const& peer) { for (big_number::const_iterator i = peer.begin(); i != peer.end(); @@ -114,6 +122,19 @@ namespace libtorrent return os; } + inline std::istream& operator>>(std::istream& is, big_number& peer) + { + for (big_number::iterator i = peer.begin(); + i != peer.end(); ++i) + { + char c[2]; + is >> c[0] >> c[1]; + *i = ((std::isdigit(c[0])?c[0]-'0':c[0]-'a'+10) << 4) + + (std::isdigit(c[1])?c[1]-'0':c[1]-'a'+10); + } + return is; + } + } #endif // TORRENT_PEER_ID_HPP_INCLUDED diff --git a/include/libtorrent/session.hpp b/include/libtorrent/session.hpp index edc6381f5..448f59ed7 100755 --- a/include/libtorrent/session.hpp +++ b/include/libtorrent/session.hpp @@ -289,9 +289,15 @@ namespace libtorrent // all torrent_handles must be destructed before the session is destructed! torrent_handle add_torrent( - const torrent_info& ti - , const boost::filesystem::path& save_path - , const entry& resume_data = entry()); + entry const& metadata + , boost::filesystem::path const& save_path + , entry const& resume_data = entry()); + + torrent_handle add_torrent( + char const* tracker_url + , sha1_hash const& info_hash + , boost::filesystem::path const& save_path + , entry const& resume_data = entry()); session_status status() const; diff --git a/include/libtorrent/torrent.hpp b/include/libtorrent/torrent.hpp index b079e0d0a..65cb0dd26 100755 --- a/include/libtorrent/torrent.hpp +++ b/include/libtorrent/torrent.hpp @@ -56,7 +56,6 @@ POSSIBILITY OF SUCH DAMAGE. #include "libtorrent/torrent_info.hpp" #include "libtorrent/socket.hpp" #include "libtorrent/policy.hpp" -#include "libtorrent/storage.hpp" #include "libtorrent/tracker_manager.hpp" #include "libtorrent/stat.hpp" #include "libtorrent/alert.hpp" @@ -68,6 +67,8 @@ namespace libtorrent struct logger; #endif + class piece_manager; + std::string escape_string(const char* str, int len); std::string unescape_string(std::string const& s); @@ -75,6 +76,7 @@ namespace libtorrent namespace detail { struct session_impl; + struct piece_checker_data; } // a torrent is a class that holds information @@ -86,12 +88,25 @@ namespace libtorrent torrent( detail::session_impl& ses - , const torrent_info& torrent_file - , const boost::filesystem::path& save_path + , entry const& metadata + , boost::filesystem::path const& save_path + , address const& net_interface); + + // used with metadata-less torrents + // (the metadata is downloaded from the peers) + torrent( + detail::session_impl& ses + , char const* tracker_url + , sha1_hash const& info_hash + , boost::filesystem::path const& save_path , address const& net_interface); ~torrent(); + // this is called when the torrent has metadata. + // it will initialize the storage and the piece-picker + void init(); + // this will flag the torrent as aborted. The main // loop in session_impl will check for this state // on all torrents once every second, and take @@ -219,19 +234,23 @@ namespace libtorrent const std::vector& pieces() const { return m_have_pieces; } + int num_pieces() const { return m_num_pieces; } + // when we get a have- or bitfield- messages, this is called for every // piece a peer has gained. void peer_has(int index) { + assert(m_picker.get()); assert(index >= 0 && index < (signed)m_have_pieces.size()); - m_picker.inc_refcount(index); + m_picker->inc_refcount(index); } // when peer disconnects, this is called for every piece it had void peer_lost(int index) { + assert(m_picker.get()); assert(index >= 0 && index < (signed)m_have_pieces.size()); - m_picker.dec_refcount(index); + m_picker->dec_refcount(index); } int block_size() const { return m_block_size; } @@ -265,14 +284,20 @@ namespace libtorrent } bool is_seed() const - { return m_num_pieces == m_torrent_file.num_pieces(); } + { + return valid_metadata() + && m_num_pieces == m_torrent_file.num_pieces(); + } - boost::filesystem::path save_path() const - { return m_storage.save_path(); } + boost::filesystem::path save_path() const; alert_manager& alerts() const; - piece_picker& picker() { return m_picker; } + piece_picker& picker() + { + assert(m_picker.get()); + return *m_picker; + } policy& get_policy() { return *m_policy; } - piece_manager& filesystem() { return m_storage; } + piece_manager& filesystem(); torrent_info const& torrent_file() const { return m_torrent_file; } torrent_handle get_handle() const; @@ -282,7 +307,7 @@ namespace libtorrent logger* spawn_logger(const char* title); virtual void debug_log(const std::string& line); - void check_invariant(); + void check_invariant() const; #endif // -------------------------------------------- @@ -300,14 +325,16 @@ namespace libtorrent void set_upload_limit(int limit); void set_download_limit(int limit); + bool valid_metadata() const { return m_storage.get() != 0; } + std::vector const& metadata() const { return m_metadata; } + + bool received_metadata(char const* buf, int size, int offset, int total_size); + private: void try_next_tracker(); - // the size of a request block - // each piece is divided into these - // blocks when requested - int m_block_size; + torrent_info m_torrent_file; // is set to true when the torrent has // been aborted. @@ -320,8 +347,15 @@ namespace libtorrent void parse_response(const entry& e, std::vector& peer_list); - torrent_info m_torrent_file; - piece_manager m_storage; + // the size of a request block + // each piece is divided into these + // blocks when requested + int m_block_size; + + // if this pointer is 0, the peer_connection is in + // a state where the metadata hasn't been + // received yet. + std::auto_ptr m_storage; // the time of next tracker request boost::posix_time::ptime m_next_request; @@ -347,7 +381,7 @@ namespace libtorrent // this torrent belongs to. detail::session_impl& m_ses; - piece_picker m_picker; + std::auto_ptr m_picker; // this is an index into m_torrent_file.trackers() int m_last_working_tracker; @@ -399,6 +433,20 @@ namespace libtorrent // can upload per second int m_upload_bandwidth_limit; int m_download_bandwidth_limit; + + // this buffer is filled with the info-section of + // the metadata file while downloading it from + // peers, and while sending it. + std::vector m_metadata; + + // this is a bitfield of size 256, each bit represents + // a piece of the metadata. It is set to one if we + // have that piece. This vector may be empty + // (size 0) if we haven't received any metadata + // or if we already have all metadata + std::vector m_have_metadata; + + boost::filesystem::path m_save_path; }; inline boost::posix_time::ptime torrent::next_announce() const diff --git a/include/libtorrent/torrent_handle.hpp b/include/libtorrent/torrent_handle.hpp index 185b806a7..6b6978569 100755 --- a/include/libtorrent/torrent_handle.hpp +++ b/include/libtorrent/torrent_handle.hpp @@ -92,6 +92,7 @@ namespace libtorrent queued_for_checking, checking_files, connecting_to_tracker, + downloading_metadata, downloading, seeding }; diff --git a/include/libtorrent/torrent_info.hpp b/include/libtorrent/torrent_info.hpp index 3dc9a6eb3..1d184a020 100755 --- a/include/libtorrent/torrent_info.hpp +++ b/include/libtorrent/torrent_info.hpp @@ -86,7 +86,7 @@ namespace libtorrent public: torrent_info(const entry& torrent_file); - torrent_info(int piece_size, const char* name); + torrent_info(int piece_size, const char* name, sha1_hash const& info_hash = sha1_hash(0)); entry create_torrent() const; void set_comment(char const* str); @@ -104,8 +104,10 @@ namespace libtorrent reverse_file_iterator rbegin_files() const { return m_files.rbegin(); } reverse_file_iterator rend_files() const { return m_files.rend(); } - int num_files() const { return (int)m_files.size(); } - const file_entry& file_at(int index) const { assert(index >= 0 && index < (int)m_files.size()); return m_files[index]; } + int num_files() const + { assert(m_piece_length > 0); return (int)m_files.size(); } + const file_entry& file_at(int index) const + { assert(index >= 0 && index < (int)m_files.size()); return m_files[index]; } const std::vector& trackers() const { return m_urls; } @@ -114,12 +116,13 @@ namespace libtorrent // the begining) and return the new index to the tracker. int prioritize_tracker(int index); - size_type total_size() const { return m_total_size; } - size_type piece_length() const { return m_piece_length; } - int num_pieces() const { return (int)m_piece_hash.size(); } + size_type total_size() const { assert(m_piece_length > 0); return m_total_size; } + size_type piece_length() const { assert(m_piece_length > 0); return m_piece_length; } + int num_pieces() const { assert(m_piece_length > 0); return (int)m_piece_hash.size(); } const sha1_hash& info_hash() const { return m_info_hash; } - const std::string& name() const { return m_name; } + const std::string& name() const { assert(m_piece_length > 0); return m_name; } void print(std::ostream& os) const; + bool is_valid() const { return m_piece_length > 0; } void convert_file_names(); @@ -138,6 +141,8 @@ namespace libtorrent const std::string& comment() const { return m_comment; } + void parse_info_section(entry const& e); + private: void read_torrent_info(const entry& libtorrent); @@ -146,6 +151,8 @@ namespace libtorrent std::vector m_urls; // the length of one piece + // if this is 0, the torrent_info is + // in an uninitialized state size_type m_piece_length; // the sha-1 hashes of each piece diff --git a/src/escape_string.cpp b/src/escape_string.cpp index 53dcf16de..4e70d5f2d 100755 --- a/src/escape_string.cpp +++ b/src/escape_string.cpp @@ -54,7 +54,7 @@ namespace libtorrent { if(*i == '+') { - ret+=' '; + ret += ' '; } else if (*i != '%') { @@ -67,9 +67,9 @@ namespace libtorrent throw std::runtime_error("invalid escaped string"); int high; - if(*i >= '0' && *i <= '9') high=*i - '0'; - else if(*i >= 'A' && *i <= 'F') high=*i + 10 - 'A'; - else if(*i >= 'a' && *i <= 'f') high=*i + 10 - 'a'; + if(*i >= '0' && *i <= '9') high = *i - '0'; + else if(*i >= 'A' && *i <= 'F') high = *i + 10 - 'A'; + else if(*i >= 'a' && *i <= 'f') high = *i + 10 - 'a'; else throw std::runtime_error("invalid escaped string"); ++i; @@ -77,9 +77,9 @@ namespace libtorrent throw std::runtime_error("invalid escaped string"); int low; - if(*i >= '0' && *i <= '9') low=*i - '0'; - else if(*i >= 'A' && *i <= 'F') low=*i + 10 - 'A'; - else if(*i >= 'a' && *i <= 'f') low=*i + 10 - 'a'; + if(*i >= '0' && *i <= '9') low = *i - '0'; + else if(*i >= 'A' && *i <= 'F') low = *i + 10 - 'A'; + else if(*i >= 'a' && *i <= 'f') low = *i + 10 - 'a'; else throw std::runtime_error("invalid escaped string"); ret += char(high * 16 + low); diff --git a/src/http_tracker_connection.cpp b/src/http_tracker_connection.cpp index ad862da8a..149ed9976 100755 --- a/src/http_tracker_connection.cpp +++ b/src/http_tracker_connection.cpp @@ -186,7 +186,13 @@ namespace libtorrent } m_send_buffer += "\r\n\r\n"; #ifndef NDEBUG - if (c) c->debug_log("==> TRACKER_REQUEST [ str: " + m_send_buffer + " ]"); + if (c) + { + c->debug_log("==> TRACKER_REQUEST [ str: " + m_send_buffer + " ]"); + std::stringstream info_hash_str; + info_hash_str << req.info_hash; + c->debug_log("info_hash: " + info_hash_str.str() + "\n"); + } #endif m_socket = s; } @@ -513,7 +519,7 @@ namespace libtorrent } catch (const type_error&) {} - int interval = e["interval"].integer(); + int interval = (int)e["interval"].integer(); peer_list.clear(); diff --git a/src/identify_client.cpp b/src/identify_client.cpp index 29716f258..2b483df6a 100755 --- a/src/identify_client.cpp +++ b/src/identify_client.cpp @@ -144,6 +144,42 @@ namespace return boost::optional(ret); } + // checks if a peer id can possibly contain a mainline-style + // identification + boost::optional parse_mainline_style(const peer_id& id) + { + fingerprint ret("..", 0, 0, 0, 0); + peer_id::const_iterator i = id.begin(); + + if (!std::isprint(*i)) return boost::optional(); + ret.id[0] = *i; + ret.id[1] = 0; + ++i; + + if (!std::isdigit(*i)) return boost::optional(); + ret.major_version = *i - '0'; + ++i; + + if (*i != '-') return boost::optional(); + ++i; + + if (!std::isdigit(*i)) return boost::optional(); + ret.minor_version = *i - '0'; + ++i; + + if (*i != '-') return boost::optional(); + ++i; + + if (!std::isdigit(*i)) return boost::optional(); + ret.revision_version = *i - '0'; + ++i; + + if (!std::equal(i, i+1, "--")) return boost::optional(); + + ret.tag_version = 0; + return boost::optional(ret); + } + } // namespace unnamed namespace libtorrent @@ -231,6 +267,26 @@ namespace libtorrent return identity.str(); } + f = parse_mainline_style(p); + if (f) + { + std::stringstream identity; + + // Mainline + if (std::equal(f->id, f->id+1, "M")) + identity << "Mainline "; + + // unknown client + else + identity << std::string(f->id, f->id+1) << " "; + + identity << (int)f->major_version + << "." << (int)f->minor_version + << "." << (int)f->revision_version; + + return identity.str(); + } + // ---------------------- // non standard encodings // ---------------------- diff --git a/src/peer_connection.cpp b/src/peer_connection.cpp index 92491d8de..4a746a3a0 100755 --- a/src/peer_connection.cpp +++ b/src/peer_connection.cpp @@ -62,7 +62,7 @@ namespace libtorrent // the names of the extensions to look for in // the extensions-message const char* peer_connection::extension_names[] = - { "chat" }; + { "chat", "metadata", "peer_exchange", "listen_port" }; const peer_connection::message_handler peer_connection::m_message_handler[] = { @@ -114,6 +114,9 @@ namespace libtorrent , m_disconnecting(false) , m_became_uninterested(boost::posix_time::second_clock::local_time()) , m_became_uninteresting(boost::posix_time::second_clock::local_time()) + , m_no_metadata( + boost::gregorian::date(1970, boost::date_time::Jan, 1) + , boost::posix_time::seconds(0)) { INVARIANT_CHECK; @@ -144,10 +147,11 @@ namespace libtorrent m_recv_buffer.resize(1); // assume the other end has no pieces - m_have_piece.resize(m_torrent->torrent_file().num_pieces()); - std::fill(m_have_piece.begin(), m_have_piece.end(), false); - - send_bitfield(); + if (m_torrent->valid_metadata()) + { + init(); + send_bitfield(); + } } peer_connection::peer_connection( @@ -183,6 +187,9 @@ namespace libtorrent , m_disconnecting(false) , m_became_uninterested(boost::posix_time::second_clock::local_time()) , m_became_uninteresting(boost::posix_time::second_clock::local_time()) + , m_no_metadata( + boost::gregorian::date(1970, boost::date_time::Jan, 1) + , boost::posix_time::seconds(0)) { INVARIANT_CHECK; @@ -220,6 +227,68 @@ namespace libtorrent m_recv_buffer.resize(1); } + void peer_connection::init() + { + assert(m_torrent); + assert(m_torrent->valid_metadata()); + + m_have_piece.resize(m_torrent->torrent_file().num_pieces(), false); + + // now that we have a piece_picker, + // update it with this peers pieces + + // build a vector of all pieces + std::vector piece_list; + for (int i = 0; i < (int)m_have_piece.size(); ++i) + { + if (m_have_piece[i]) + { + ++m_num_pieces; + piece_list.push_back(i); + } + else if (m_have_piece[i]) + { + --m_num_pieces; + m_torrent->peer_lost(i); + } + } + + // shuffle the piece list + std::random_shuffle(piece_list.begin(), piece_list.end()); + + // let the torrent know which pieces the + // peer has, in a shuffled order + bool interesting = false; + for (std::vector::iterator i = piece_list.begin(); + i != piece_list.end(); + ++i) + { + int index = *i; + m_torrent->peer_has(index); + if (!m_torrent->have_piece(index)) + interesting = true; + } + + if (piece_list.size() == m_have_piece.size()) + { +#ifndef NDEBUG + (*m_logger) << " *** THIS IS A SEED ***\n"; +#endif + // if we're a seed too, disconnect + if (m_torrent->is_seed()) + { +#ifndef NDEBUG + (*m_logger) << " we're also a seed, disconnecting\n"; +#endif + throw protocol_error("seed to seed connection redundant, disconnecting"); + } + } + + if (interesting) + m_torrent->get_policy().peer_is_interesting(*this); + + } + peer_connection::~peer_connection() { m_selector.remove(m_socket); @@ -233,6 +302,7 @@ namespace libtorrent void peer_connection::announce_piece(int index) { assert(m_torrent); + assert(m_torrent->valid_metadata()); assert(index >= 0 && index < m_torrent->torrent_file().num_pieces()); m_announce_queue.push_back(index); } @@ -240,6 +310,7 @@ namespace libtorrent bool peer_connection::has_piece(int i) const { assert(m_torrent); + assert(m_torrent->valid_metadata()); assert(i >= 0); assert(i < m_torrent->torrent_file().num_pieces()); return m_have_piece[i]; @@ -337,7 +408,9 @@ namespace libtorrent , 0); // indicate that we support the extension protocol // curently disabled -// m_send_buffer[pos] = 0x80; +#ifdef TORRENT_ENABLE_EXTENSIONS + m_send_buffer[pos+7] = 0x01; +#endif pos += 8; // info hash @@ -364,6 +437,8 @@ namespace libtorrent // and if it can correspond to a request generated by libtorrent. bool peer_connection::verify_piece(const peer_request& p) const { + assert(m_torrent->valid_metadata()); + return p.piece >= 0 && p.piece < m_torrent->torrent_file().num_pieces() && p.length > 0 @@ -543,11 +618,17 @@ namespace libtorrent else { m_have_piece[index] = true; - ++m_num_pieces; - m_torrent->peer_has(index); - if (!m_torrent->have_piece(index) && !is_interesting()) - m_torrent->get_policy().peer_is_interesting(*this); + // only update the piece_picker if + // we have the metadata + if (m_torrent->valid_metadata()) + { + ++m_num_pieces; + m_torrent->peer_has(index); + + if (!m_torrent->have_piece(index) && !is_interesting()) + m_torrent->get_policy().peer_is_interesting(*this); + } if (m_torrent->is_seed() && is_seed()) { @@ -565,14 +646,32 @@ namespace libtorrent INVARIANT_CHECK; assert(received > 0); - if (m_packet_size - 1 != ((int)m_have_piece.size() + 7) / 8) + assert(m_torrent); + // if we don't have the metedata, we cannot + // verify the bitfield size + if (m_torrent->valid_metadata() + && m_packet_size - 1 != ((int)m_have_piece.size() + 7) / 8) throw protocol_error("bitfield with invalid size"); + m_statistics.received_bytes(0, received); if (m_recv_pos < m_packet_size) return; #ifndef NDEBUG (*m_logger) << " <== BITFIELD\n"; #endif + + // if we don't have metadata yet + // just remember the bitmask + // don't update the piecepicker + // (since it doesn't exist yet) + if (!m_torrent->valid_metadata()) + { + m_have_piece.resize((m_packet_size - 1) * 8, false); + for (int i = 0; i < (int)m_have_piece.size(); ++i) + m_have_piece[i] = (m_recv_buffer[1 + (i>>3)] & (1 << (7 - (i&7)))) != 0; + return; + } + // build a vector of all pieces std::vector piece_list; for (int i = 0; i < (int)m_have_piece.size(); ++i) @@ -646,12 +745,37 @@ namespace libtorrent r.start = detail::read_int32(ptr); r.length = detail::read_int32(ptr); + if (!m_torrent->valid_metadata()) + { + // if we don't have valid metadata yet, + // we shouldn't get a request +#ifndef NDEBUG + (*m_logger) << " <== UNEXPECTED_REQUEST [ " + "piece: " << r.piece << " | " + "s: " << r.start << " | " + "l: " << r.length << " | " + "i: " << m_peer_interested << " | " + "t: " << (int)m_torrent->torrent_file().piece_size(r.piece) << " | " + "n: " << m_torrent->torrent_file().num_pieces() << " ]\n"; +#endif + return; + } + if (m_requests.size() > 40) { // don't allow clients to abuse our // memory consumption. // ignore requests if the client // is making too many of them. +#ifndef NDEBUG + (*m_logger) << " <== TOO MANY REQUESTS [ " + "piece: " << r.piece << " | " + "s: " << r.start << " | " + "l: " << r.length << " | " + "i: " << m_peer_interested << " | " + "t: " << (int)m_torrent->torrent_file().piece_size(r.piece) << " | " + "n: " << m_torrent->torrent_file().num_pieces() << " ]\n"; +#endif return; } @@ -896,6 +1020,7 @@ namespace libtorrent { INVARIANT_CHECK; + assert(m_torrent); assert(received > 0); if (m_packet_size > 100 * 1024) { @@ -929,6 +1054,11 @@ namespace libtorrent (*m_logger) << i->first << "\n"; } #endif + if (!m_torrent->valid_metadata() + && supports_extension(extended_metadata_message)) + { + send_metadata_request(0, 255); + } } catch(invalid_encoding&) { @@ -960,52 +1090,155 @@ namespace libtorrent if (m_recv_pos < 5) return; const char* ptr = &m_recv_buffer[1]; - int extended_id = detail::read_int32(ptr); switch (extended_id) { case extended_chat_message: - { - if (m_packet_size > 2 * 1024) - throw protocol_error("CHAT message larger than 2 kB"); - if (m_recv_pos < m_packet_size) return; - try - { - entry d = bdecode(m_recv_buffer.begin()+5, m_recv_buffer.end()); - const std::string& str = d["msg"].string(); - - if (m_torrent->alerts().should_post(alert::critical)) - { - m_torrent->alerts().post_alert( - chat_message_alert( - m_torrent->get_handle() - , m_socket->sender(), str)); - } - - } - catch (invalid_encoding&) - { - throw protocol_error("invalid bencoding in CHAT message"); - } - catch (type_error&) - { - throw protocol_error("invalid types in bencoded CHAT message"); - } - return; - } + on_chat(); break; + case extended_metadata_message: + on_metadata(); break; + case extended_peer_exchange_message: + on_peer_exchange(); break; + case extended_listen_port_message: + on_listen_port(); break; default: throw protocol_error("unknown extended message id"); - }; } + // ----------------------------- + // ----------- CHAT ------------ + // ----------------------------- + void peer_connection::on_chat() + { + if (m_packet_size > 2 * 1024) + throw protocol_error("CHAT message larger than 2 kB"); + if (m_recv_pos < m_packet_size) return; + try + { + entry d = bdecode(m_recv_buffer.begin()+5, m_recv_buffer.end()); + const std::string& str = d["msg"].string(); + if (m_torrent->alerts().should_post(alert::critical)) + { + m_torrent->alerts().post_alert( + chat_message_alert( + m_torrent->get_handle() + , m_socket->sender(), str)); + } + } + catch (invalid_encoding&) + { + // TODO: make these non-fatal errors + // they should just ignore the chat message + // and report the error via an alert + throw protocol_error("invalid bencoding in CHAT message"); + } + catch (type_error&) + { + throw protocol_error("invalid types in bencoded CHAT message"); + } + return; + } + // ----------------------------- + // --------- METADATA ---------- + // ----------------------------- + void peer_connection::on_metadata() + { + assert(m_torrent); + + if (m_packet_size > 500 * 1024) + throw protocol_error("metadata message larger than 500 kB"); + + if (m_recv_pos < m_packet_size) return; + + std::vector::iterator ptr = m_recv_buffer.begin()+5; + int type = detail::read_uint8(ptr); + + switch (type) + { + case 0: // request + { + int start = detail::read_uint8(ptr); + int size = detail::read_uint8(ptr); + + if (start + size > 255 || start + size <= 0 || m_packet_size != 8) + { + // invalid metadata request + throw protocol_error("invalid metadata request"); + } + + send_metadata(start, size); + } + break; + case 1: // data + { + int total_size = detail::read_int32(ptr); + int offset = detail::read_int32(ptr); + int data_size = m_packet_size - 5 - 9; + + if (total_size > 500 * 1024) + throw protocol_error("metadata size larger than 500 kB"); + if (offset > total_size) + throw protocol_error("invalid metadata offset"); + if (offset + data_size > total_size) + throw protocol_error("invalid metadata message"); + + m_torrent->received_metadata(&m_recv_buffer[5+9], data_size, offset, total_size); + } + break; + case 2: // have no data + m_no_metadata = boost::posix_time::second_clock::local_time(); + break; + } + + } + + // ----------------------------- + // ------ PEER EXCHANGE -------- + // ----------------------------- + + void peer_connection::on_peer_exchange() + { + + } + + // ----------------------------- + // ------- LISTEN PORT --------- + // ----------------------------- + + void peer_connection::on_listen_port() + { + assert(m_torrent); + + if (m_packet_size != 7) + throw protocol_error("invalid listen_port message"); + + if (is_local()) + { +#ifndef NDEBUG + (*m_logger) << "<== LISTEN_PORT [ UNEXPECTED ]\n"; +#endif + return; + } + + const char* ptr = &m_recv_buffer[5]; + unsigned short port = detail::read_uint16(ptr); + +#ifndef NDEBUG + (*m_logger) << "<== LISTEN_PORT [ port: " << port << " ]\n"; +#endif + + address adr = m_socket->sender(); + adr.port = port; + m_torrent->get_policy().peer_from_tracker(adr, m_peer_id); + } void peer_connection::disconnect() @@ -1050,6 +1283,8 @@ namespace libtorrent { INVARIANT_CHECK; + assert(m_torrent->valid_metadata()); + assert(block.piece_index >= 0); assert(block.piece_index < m_torrent->torrent_file().num_pieces()); assert(block.block_index >= 0); @@ -1100,6 +1335,7 @@ namespace libtorrent { INVARIANT_CHECK; + assert(m_torrent->valid_metadata()); assert(block.piece_index >= 0); assert(block.piece_index < m_torrent->torrent_file().num_pieces()); assert(block.block_index >= 0); @@ -1151,12 +1387,82 @@ namespace libtorrent send_buffer_updated(); } + void peer_connection::send_metadata(int start, int size) + { + assert(start >= 0); + assert(size > 0); + assert(start <= 255); + assert(start + size <= 255); + assert(m_torrent); + INVARIANT_CHECK; + + // abort if the peer doesn't support the metadata extension + if (!supports_extension(extended_metadata_message)) return; + + std::back_insert_iterator > ptr(m_send_buffer); + + if (m_torrent->valid_metadata()) + { + int offset = start * (int)m_torrent->metadata().size() / 255; + int metadata_size + = (start + size) * (int)m_torrent->metadata().size() / 255 - offset; + + // yes, we have metadata, send it + assert(size <= (int)m_torrent->metadata().size()); + detail::write_uint32(5 + 9 + metadata_size, ptr); + detail::write_uint8(msg_extended, ptr); + detail::write_int32(m_extension_messages[extended_metadata_message], ptr); + // means 'data packet' + detail::write_uint8(1, ptr); + detail::write_uint32((int)m_torrent->metadata().size(), ptr); + detail::write_uint32(offset, ptr); + std::vector const& metadata = m_torrent->metadata(); + std::copy(&metadata[offset], &metadata[offset + metadata_size], ptr); + } + else + { + // we don't have the metadata, reply with + // don't have-message + detail::write_uint32(1 + 4 + 1, ptr); + detail::write_uint8(msg_extended, ptr); + detail::write_int32(m_extension_messages[extended_metadata_message], ptr); + // means 'have no data' + detail::write_uint8(2, ptr); + } + send_buffer_updated(); + } + + void peer_connection::send_metadata_request(int start, int size) + { + assert(start >= 0); + assert(size > 0); + assert(start < 255); + assert(start + size <= 255); + assert(m_torrent); + assert(!m_torrent->valid_metadata()); + INVARIANT_CHECK; + + // abort if the peer doesn't support the metadata extension + if (!supports_extension(extended_metadata_message)) return; + + std::back_insert_iterator > ptr(m_send_buffer); + + detail::write_uint32(1 + 4 + 3, ptr); + detail::write_uint8(msg_extended, ptr); + detail::write_int32(m_extension_messages[extended_metadata_message], ptr); + // means 'request data' + detail::write_uint8(0, ptr); + detail::write_uint8(start, ptr); + detail::write_uint8(size, ptr); + send_buffer_updated(); + } + void peer_connection::send_chat_message(const std::string& msg) { INVARIANT_CHECK; assert(msg.length() <= 1 * 1024); - if (m_extension_messages[extended_chat_message] == -1) return; + if (!supports_extension(extended_chat_message)) return; entry e(entry::dictionary_t); e["msg"] = msg; @@ -1176,6 +1482,8 @@ namespace libtorrent { INVARIANT_CHECK; + if (m_torrent->num_pieces() == 0) return; + #ifndef NDEBUG (*m_logger) << " ==> BITFIELD\n"; #endif @@ -1290,6 +1598,7 @@ namespace libtorrent void peer_connection::send_have(int index) { + assert(m_torrent->valid_metadata()); assert(index >= 0); assert(index < m_torrent->torrent_file().num_pieces()); INVARIANT_CHECK; @@ -1550,10 +1859,10 @@ namespace libtorrent // these 8 bytes would be used to describe the // extensions available on the other side // currently disabled -// if (m_recv_buffer[0] & 0x80) -// { -// m_supports_extensions = true; -// } +#ifdef TORRENT_ENABLE_EXTENSIONS + if (m_recv_buffer[7] & 0x01) + m_supports_extensions = true; +#endif if (m_torrent == 0) { @@ -1583,8 +1892,11 @@ namespace libtorrent throw protocol_error("connection rejected by paused torrent"); } + if (m_torrent->valid_metadata()) init(); + // assume the other end has no pieces - m_have_piece.resize(m_torrent->torrent_file().num_pieces()); + // if we don't have valid metadata yet, + // leave the vector unallocated std::fill(m_have_piece.begin(), m_have_piece.end(), false); // yes, we found the torrent @@ -1747,6 +2059,7 @@ namespace libtorrent && ((int)m_send_buffer.size() < m_torrent->block_size()) && !m_choked) { + assert(m_torrent->valid_metadata()); peer_request& r = m_requests.front(); assert(r.piece >= 0); @@ -1796,6 +2109,7 @@ namespace libtorrent if (!m_announce_queue.empty()) { + assert(m_torrent->valid_metadata()); for (std::vector::iterator i = m_announce_queue.begin(); i != m_announce_queue.end(); ++i) @@ -1887,6 +2201,7 @@ namespace libtorrent void peer_connection::check_invariant() const { assert(can_write() == m_selector.is_writability_monitored(m_socket)); + /* assert(m_num_pieces == std::count( m_have_piece.begin() diff --git a/src/session.cpp b/src/session.cpp index 81bb62062..a7a227108 100755 --- a/src/session.cpp +++ b/src/session.cpp @@ -805,10 +805,12 @@ namespace libtorrent // current platform. // if the torrent already exists, this will throw duplicate_torrent torrent_handle session::add_torrent( - const torrent_info& ti - , const boost::filesystem::path& save_path - , const entry& resume_data) + entry const& metadata + , boost::filesystem::path const& save_path + , entry const& resume_data) { + torrent_info ti(metadata); + if (ti.begin_files() == ti.end_files()) throw std::runtime_error("no files in torrent"); @@ -834,7 +836,7 @@ namespace libtorrent // the checker thread and store it before starting // the thread boost::shared_ptr torrent_ptr( - new torrent(m_impl, ti, save_path, m_impl.m_listen_interface)); + new torrent(m_impl, metadata, save_path, m_impl.m_listen_interface)); detail::piece_checker_data d; d.torrent_ptr = torrent_ptr; @@ -854,6 +856,44 @@ namespace libtorrent return torrent_handle(&m_impl, &m_checker_impl, ti.info_hash()); } + torrent_handle session::add_torrent( + char const* tracker_url + , sha1_hash const& info_hash + , boost::filesystem::path const& save_path + , entry const& resume_data) + { + { + // lock the session + boost::mutex::scoped_lock l(m_impl.m_mutex); + + // is the torrent already active? + if (m_impl.find_torrent(info_hash)) + throw duplicate_torrent(); + } + + { + // lock the checker_thread + boost::mutex::scoped_lock l(m_checker_impl.m_mutex); + + // is the torrent currently being checked? + if (m_checker_impl.find_torrent(info_hash)) + throw duplicate_torrent(); + } + + // create the torrent and the data associated with + // the checker thread and store it before starting + // the thread + boost::shared_ptr torrent_ptr( + new torrent(m_impl, tracker_url, info_hash, save_path, m_impl.m_listen_interface)); + + boost::mutex::scoped_lock l(m_impl.m_mutex); + m_impl.m_torrents.insert( + std::make_pair(info_hash, torrent_ptr)).first; + + return torrent_handle(&m_impl, &m_checker_impl, info_hash); + } + + void session::remove_torrent(const torrent_handle& h) { if (h.m_ses != &m_impl) return; diff --git a/src/storage.cpp b/src/storage.cpp index 76207139d..d5a36a834 100755 --- a/src/storage.cpp +++ b/src/storage.cpp @@ -341,6 +341,7 @@ namespace libtorrent if (file_offset + read_bytes > file_iter->size) read_bytes = static_cast(file_iter->size - file_offset); + // TODO: this assert will be hit if a file has size 0 assert(read_bytes > 0); // in.read(buf + buf_pos, read_bytes); @@ -784,6 +785,7 @@ namespace libtorrent hasher small_digest; small_digest.update(&piece_data[0], last_piece_size); hasher large_digest(small_digest); + assert(piece_size - last_piece_size >= 0); if (piece_size - last_piece_size > 0) { large_digest.update( diff --git a/src/torrent.cpp b/src/torrent.cpp index 62e97cf2d..f5ae879b0 100755 --- a/src/torrent.cpp +++ b/src/torrent.cpp @@ -146,21 +146,20 @@ namespace libtorrent { torrent::torrent( detail::session_impl& ses - , const torrent_info& torrent_file - , const boost::filesystem::path& save_path + , entry const& metadata + , boost::filesystem::path const& save_path , address const& net_interface) - : m_block_size(calculate_block_size(torrent_file)) + : m_torrent_file(metadata) , m_abort(false) , m_paused(false) , m_event(tracker_request::started) - , m_torrent_file(torrent_file) - , m_storage(m_torrent_file, save_path) + , m_block_size(0) + , m_storage(0) , m_next_request(boost::posix_time::second_clock::local_time()) , m_duration(1800) , m_policy(new policy(this)) // warning: uses this in member init list , m_ses(ses) - , m_picker(static_cast(torrent_file.piece_length() / m_block_size), - static_cast((torrent_file.total_size()+m_block_size-1)/m_block_size)) + , m_picker(0) , m_last_working_tracker(-1) , m_currently_trying_tracker(0) , m_failed_trackers(0) @@ -174,8 +173,42 @@ namespace libtorrent , m_upload_bandwidth_limit(std::numeric_limits::max()) , m_download_bandwidth_limit(std::numeric_limits::max()) { - assert(torrent_file.begin_files() != torrent_file.end_files()); - m_have_pieces.resize(torrent_file.num_pieces(), false); + bencode(std::back_inserter(m_metadata), metadata["info"]); + init(); + } + + torrent::torrent( + detail::session_impl& ses + , char const* tracker_url + , sha1_hash const& info_hash + , boost::filesystem::path const& save_path + , address const& net_interface) + : m_torrent_file(0, 0, info_hash) + , m_abort(false) + , m_paused(false) + , m_event(tracker_request::started) + , m_block_size(0) + , m_storage(0) + , m_next_request(boost::posix_time::second_clock::local_time()) + , m_duration(1800) + , m_policy(new policy(this)) // warning: uses this in member init list + , m_ses(ses) + , m_picker(0) + , m_last_working_tracker(-1) + , m_currently_trying_tracker(0) + , m_failed_trackers(0) + , m_time_scaler(0) + , m_priority(.5) + , m_num_pieces(0) + , m_got_tracker_response(false) + , m_ratio(0.f) + , m_total_failed_bytes(0) + , m_net_interface(net_interface.ip(), address::any_port) + , m_upload_bandwidth_limit(std::numeric_limits::max()) + , m_download_bandwidth_limit(std::numeric_limits::max()) + , m_save_path(save_path) + { + m_torrent_file.add_tracker(tracker_url); } torrent::~torrent() @@ -184,6 +217,18 @@ namespace libtorrent if (m_ses.m_abort) m_abort = true; } + void torrent::init() + { + assert(m_torrent_file.is_valid()); + + m_have_pieces.resize(m_torrent_file.num_pieces(), false); + m_storage.reset(new piece_manager(m_torrent_file, m_save_path)); + m_block_size = calculate_block_size(m_torrent_file); + m_picker.reset(new piece_picker( + static_cast(m_torrent_file.piece_length() / m_block_size) + , static_cast((m_torrent_file.total_size()+m_block_size-1)/m_block_size))); + } + void torrent::use_interface(const char* net_interface) { m_net_interface = address(net_interface, address::any_port); @@ -246,11 +291,17 @@ namespace libtorrent size_type torrent::bytes_left() const { + // if we don't have the metadata yet, we + // cannot tell how big the torrent is. + if (!valid_metadata()) return -1; return m_torrent_file.total_size() - bytes_done(); } size_type torrent::bytes_done() const { + if (!valid_metadata()) return 0; + + assert(m_picker.get()); const int last_piece = m_torrent_file.num_pieces()-1; size_type total_done @@ -266,7 +317,7 @@ namespace libtorrent } const std::vector& dl_queue - = m_picker.get_download_queue(); + = m_picker->get_download_queue(); const int blocks_per_piece = static_cast(m_torrent_file.piece_length() / m_block_size); @@ -285,7 +336,7 @@ namespace libtorrent // correction if this was the last piece // and if we have the last block if (i->index == last_piece - && i->finished_blocks[m_picker.blocks_in_last_piece()-1]) + && i->finished_blocks[m_picker->blocks_in_last_piece()-1]) { total_done -= m_block_size; total_done += m_torrent_file.piece_size(last_piece) % m_block_size; @@ -304,7 +355,7 @@ namespace libtorrent { if (m_have_pieces[p->piece_index]) continue; - if (m_picker.is_finished(piece_block(p->piece_index, p->block_index))) + if (m_picker->is_finished(piece_block(p->piece_index, p->block_index))) continue; total_done += p->bytes_downloaded; @@ -316,6 +367,8 @@ namespace libtorrent void torrent::piece_failed(int index) { + assert(m_storage.get()); + assert(m_picker.get()); assert(index >= 0); assert(index < m_torrent_file.num_pieces()); @@ -329,7 +382,7 @@ namespace libtorrent m_total_failed_bytes += m_torrent_file.piece_size(index); std::vector
    downloaders; - m_picker.get_downloaders(downloaders, index); + m_picker->get_downloaders(downloaders, index); // decrease the trust point of all peers that sent // parts of this piece. @@ -366,8 +419,8 @@ namespace libtorrent // if some clients has sent more than one piece // start with redownloading the pieces that the client // that has sent the least number of pieces - m_picker.restore_piece(index); - m_storage.mark_failed(index); + m_picker->restore_piece(index); + m_storage->mark_failed(index); assert(m_have_pieces[index] == false); } @@ -375,11 +428,12 @@ namespace libtorrent void torrent::announce_piece(int index) { + assert(m_picker.get()); assert(index >= 0); assert(index < m_torrent_file.num_pieces()); std::vector
    downloaders; - m_picker.get_downloaders(downloaders, index); + m_picker->get_downloaders(downloaders, index); // increase the trust point of all peers that sent // parts of this piece. @@ -392,7 +446,7 @@ namespace libtorrent p->second->received_valid_data(); } - m_picker.we_have(index); + m_picker->we_have(index); for (peer_iterator i = m_connections.begin(); i != m_connections.end(); ++i) i->second->announce_piece(index); } @@ -416,6 +470,7 @@ namespace libtorrent req.downloaded = m_stat.total_payload_download(); req.uploaded = m_stat.total_payload_upload(); req.left = bytes_left(); + if (req.left == -1) req.left = 1000; req.event = m_event; req.url = m_torrent_file.trackers()[m_currently_trying_tracker].url; req.num_want = std::max( @@ -443,26 +498,29 @@ namespace libtorrent i != p->download_queue().end(); ++i) { - m_picker.abort_download(*i); + m_picker->abort_download(*i); } - std::vector piece_list; - const std::vector& pieces = p->get_bitfield(); - - for (std::vector::const_iterator i = pieces.begin(); - i != pieces.end(); - ++i) + if (valid_metadata()) { - if (*i) piece_list.push_back(static_cast(i - pieces.begin())); - } + std::vector piece_list; + const std::vector& pieces = p->get_bitfield(); - std::random_shuffle(piece_list.begin(), piece_list.end()); + for (std::vector::const_iterator i = pieces.begin(); + i != pieces.end(); + ++i) + { + if (*i) piece_list.push_back(static_cast(i - pieces.begin())); + } - for (std::vector::iterator i = piece_list.begin(); - i != piece_list.end(); - ++i) - { - peer_lost(*i); + std::random_shuffle(piece_list.begin(), piece_list.end()); + + for (std::vector::iterator i = piece_list.begin(); + i != piece_list.end(); + ++i) + { + peer_lost(*i); + } } m_policy->connection_closed(*p); @@ -576,16 +634,14 @@ namespace libtorrent void torrent::check_files(detail::piece_checker_data& data, boost::mutex& mutex) { - m_storage.check_pieces(mutex, data, m_have_pieces); + assert(m_storage.get()); + m_storage->check_pieces(mutex, data, m_have_pieces); m_num_pieces = std::accumulate( m_have_pieces.begin() , m_have_pieces.end() , 0); - m_picker.files_checked(m_have_pieces, data.unfinished_pieces); -#ifndef NDEBUG - m_picker.integrity_check(this); -#endif + m_picker->files_checked(m_have_pieces, data.unfinished_pieces); } alert_manager& torrent::alerts() const @@ -593,6 +649,19 @@ namespace libtorrent return m_ses.m_alerts; } + boost::filesystem::path torrent::save_path() const + { + assert(m_storage.get()); + return m_storage->save_path(); + } + + piece_manager& torrent::filesystem() + { + assert(m_storage.get()); + return *m_storage; + } + + torrent_handle torrent::get_handle() const { return torrent_handle(&m_ses, 0, m_torrent_file.info_hash()); @@ -601,13 +670,13 @@ namespace libtorrent #ifndef NDEBUG - void torrent::check_invariant() + void torrent::check_invariant() const { assert(m_num_pieces == std::count(m_have_pieces.begin(), m_have_pieces.end(), true)); assert(m_priority >= 0.f && m_priority < 1.f); - assert(m_block_size > 0); - assert((m_torrent_file.piece_length() % m_block_size) == 0); + assert(!valid_metadata() || m_block_size > 0); + assert(!valid_metadata() || (m_torrent_file.piece_length() % m_block_size) == 0); } #endif @@ -710,13 +779,14 @@ namespace libtorrent bool torrent::verify_piece(int piece_index) { + assert(m_storage.get()); assert(piece_index >= 0); assert(piece_index < m_torrent_file.num_pieces()); int size = static_cast(m_torrent_file.piece_size(piece_index)); std::vector buffer(size); assert(size > 0); - m_storage.read(&buffer[0], piece_index, 0, size); + m_storage->read(&buffer[0], piece_index, 0, size); hasher h; h.update(&buffer[0], size); @@ -748,12 +818,7 @@ namespace libtorrent torrent_status st; - if (m_last_working_tracker >= 0) - { - st.current_tracker - = m_torrent_file.trackers()[m_last_working_tracker].url; - } - + st.num_peers = (int)m_connections.size(); st.paused = m_paused; st.total_done = bytes_done(); @@ -776,16 +841,35 @@ namespace libtorrent st.download_payload_rate = m_stat.download_payload_rate(); st.upload_payload_rate = m_stat.upload_payload_rate(); - st.progress = st.total_done - / static_cast(m_torrent_file.total_size()); - st.next_announce = next_announce() - boost::posix_time::second_clock::local_time(); if (st.next_announce.is_negative()) st.next_announce = boost::posix_time::seconds(0); st.announce_interval = boost::posix_time::seconds(m_duration); - st.num_peers = (int)m_connections.size(); + // if we don't have any metadata, stop here + + if (!valid_metadata()) + { + if (m_got_tracker_response == false) + st.state = torrent_status::connecting_to_tracker; + else + st.state = torrent_status::downloading_metadata; + // TODO: implement progress + st.progress = 0.f; + return st; + } + + // fill in status that depends on metadata + + if (m_last_working_tracker >= 0) + { + st.current_tracker + = m_torrent_file.trackers()[m_last_working_tracker].url; + } + + st.progress = st.total_done + / static_cast(m_torrent_file.total_size()); st.pieces = &m_have_pieces; @@ -799,6 +883,89 @@ namespace libtorrent return st; } + bool torrent::received_metadata(char const* buf, int size, int offset, int total_size) + { + INVARIANT_CHECK; + + if (valid_metadata()) return false; + + if ((int)m_metadata.size() < total_size) + m_metadata.resize(total_size); + + std::copy( + buf + , buf + size + , &m_metadata[offset]); + + if (m_have_metadata.empty()) + m_have_metadata.resize(255, false); + + int start = offset * 255 / (int)m_metadata.size(); + if ((offset * 255) % (int)m_metadata.size() != 0) + throw protocol_error("unaligned metadata message offset"); + + int block_size = size * (255 - offset) / (int)m_metadata.size() - start; + + assert(start >= 0); + assert(block_size > 0); + assert(start < 256); + assert(start + block_size <= 256); + + std::fill( + m_have_metadata.begin() + start + , m_have_metadata.begin() + start + block_size + , true); + + bool have_all = std::count(m_have_metadata.begin(), m_have_metadata.end(), true) == 255; + if (!have_all) return false; + + hasher h; + h.update(&m_metadata[0], m_metadata.size()); + sha1_hash info_hash = h.final(); + + if (info_hash != m_torrent_file.info_hash()) + { + std::fill( + m_have_metadata.begin() + , m_have_metadata.end() + start + block_size + , false); + // TODO: rerequest + assert(false); + return false; + } + + m_torrent_file.parse_info_section(bdecode(m_metadata.begin(), m_metadata.end())); + + init(); + + boost::mutex m; + detail::piece_checker_data d; + d.abort = false; + // TODO: this check should be moved to the checker thread + check_files(d, m); + + // all peer connections have to initialize themselves now that the metadata + // is available + for (std::map::iterator i = m_connections.begin(); + i != m_connections.end(); ++i) + { + i->second->init(); + } + +#ifndef NDEBUG + m_picker->integrity_check(this); +#endif + + + // clear the storage for the bitfield + { + std::vector t; + m_have_metadata.swap(t); + } + + return true; + } + void torrent::tracker_request_timed_out() { #ifndef NDEBUG diff --git a/src/torrent_handle.cpp b/src/torrent_handle.cpp index 6064f2220..04da4e7fb 100755 --- a/src/torrent_handle.cpp +++ b/src/torrent_handle.cpp @@ -321,7 +321,9 @@ namespace libtorrent torrent* t = m_ses->find_torrent(m_info_hash); if (t == 0) throw invalid_handle(); - + + if (!t->valid_metadata()) return entry(); + t->filesystem().export_piece_map(piece_index); entry ret(entry::dictionary_t); @@ -400,6 +402,9 @@ namespace libtorrent { // we cannot save remote connection // since we don't know their listen port + // TODO: iterate the peers in the policy + // instead, since peers may be remote + // but still connectable if (!i->second->is_local()) continue; address ip = i->second->get_socket()->sender(); @@ -552,13 +557,14 @@ namespace libtorrent { INVARIANT_CHECK; - queue.clear(); - if (m_ses == 0) throw invalid_handle(); boost::mutex::scoped_lock l(m_ses->m_mutex); torrent* t = m_ses->find_torrent(m_info_hash); + + queue.clear(); if (t == 0) return; + if (!t->valid_metadata()) return; const piece_picker& p = t->picker(); diff --git a/src/torrent_info.cpp b/src/torrent_info.cpp index 94d6ccd8c..fe1400530 100755 --- a/src/torrent_info.cpp +++ b/src/torrent_info.cpp @@ -114,15 +114,70 @@ namespace libtorrent // just the necessary to use it with piece manager torrent_info::torrent_info( int piece_size - , const char* name) + , const char* name + , sha1_hash const& info_hash) : m_piece_length(piece_size) , m_total_size(0) + , m_info_hash(info_hash) , m_creation_date(second_clock::local_time()) { - m_info_hash.clear(); } + void torrent_info::parse_info_section(entry const& info) + { + // encode the info-field in order to calculate it's sha1-hash + std::vector buf; + bencode(std::back_inserter(buf), info); + hasher h; + h.update(&buf[0], (int)buf.size()); + m_info_hash = h.final(); + + // extract piece length + m_piece_length = (int)info["piece length"].integer(); + + // extract file name (or the directory name if it's a multifile libtorrent) + m_name = info["name"].string(); + + // extract file list + entry const* i = info.find_key("files"); + if (i == 0) + { + // if there's no list of files, there has to be a length + // field. + file_entry e; + e.path = m_name; + e.size = info["length"].integer(); + m_files.push_back(e); + } + else + { + extract_files(i->list(), m_files); + } + + // calculate total size of all pieces + m_total_size = 0; + for (std::vector::iterator i = m_files.begin(); i != m_files.end(); ++i) + m_total_size += i->size; + + // extract sha-1 hashes for all pieces + // we want this division to round upwards, that's why we have the + // extra addition + + int num_pieces = static_cast((m_total_size + m_piece_length - 1) / m_piece_length); + m_piece_hash.resize(num_pieces); + const std::string& hash_string = info["pieces"].string(); + + if ((int)hash_string.length() != num_pieces * 20) + throw invalid_torrent_file(); + + for (int i = 0; i < num_pieces; ++i) + std::copy( + hash_string.begin() + i*20 + , hash_string.begin() + (i+1)*20 + , m_piece_hash[i].begin()); + } + // extracts information from a libtorrent file and fills in the structures in // the torrent object void torrent_info::read_torrent_info(const entry& torrent_file) @@ -197,57 +252,7 @@ namespace libtorrent if (i == 0) throw invalid_torrent_file(); entry const& info = *i; - // encode the info-field in order to calculate it's sha1-hash - std::vector buf; - bencode(std::back_insert_iterator >(buf), info); - hasher h; - h.update(&buf[0], (int)buf.size()); - m_info_hash = h.final(); - std::copy(m_info_hash.begin(), m_info_hash.end(), std::ostream_iterator(std::cout)); - - // extract piece length - m_piece_length = (int)info["piece length"].integer(); - - // extract file name (or the directory name if it's a multifile libtorrent) - m_name = info["name"].string(); - - // extract file list - i = info.find_key("files"); - if (i == 0) - { - // if there's no list of files, there has to be a length - // field. - file_entry e; - e.path = m_name; - e.size = info["length"].integer(); - m_files.push_back(e); - } - else - { - extract_files(i->list(), m_files); - } - - // calculate total size of all pieces - m_total_size = 0; - for (std::vector::iterator i = m_files.begin(); i != m_files.end(); ++i) - m_total_size += i->size; - - // extract sha-1 hashes for all pieces - // we want this division to round upwards, that's why we have the - // extra addition - - int num_pieces = static_cast((m_total_size + m_piece_length - 1) / m_piece_length); - m_piece_hash.resize(num_pieces); - const std::string& hash_string = info["pieces"].string(); - - if ((int)hash_string.length() != num_pieces * 20) - throw invalid_torrent_file(); - - for (int i = 0; i < num_pieces; ++i) - std::copy( - hash_string.begin() + i*20 - , hash_string.begin() + (i+1)*20 - , m_piece_hash[i].begin()); + parse_info_section(info); } boost::optional @@ -306,6 +311,8 @@ namespace libtorrent entry torrent_info::create_torrent() const { + assert(m_piece_length > 0); + using namespace boost::gregorian; using namespace boost::posix_time;