added initial support for share-mode

This commit is contained in:
Arvid Norberg 2010-09-05 16:01:36 +00:00
parent a8cc326df6
commit b6f92aa981
16 changed files with 377 additions and 17 deletions

View File

@ -1,3 +1,4 @@
* added share-mode feature for improving share ratios
* merged all proxy settings into a single one * merged all proxy settings into a single one
* improved SOCKS5 support by proxying hostname lookups * improved SOCKS5 support by proxying hostname lookups
* improved support for multi-homed clients * improved support for multi-homed clients

View File

@ -48,6 +48,8 @@ extensions
* support for IPv6, including `BEP 7`_ and `BEP 24`_. * support for IPv6, including `BEP 7`_ and `BEP 24`_.
* support for merkle hash tree torrents. This makes the size of torrent files * support for merkle hash tree torrents. This makes the size of torrent files
scale well with the size of the content. scale well with the size of the content.
* share-mode. This is a special mode torrents can be put in to optimize share
ratio rather than downloading the torrent.
.. _extensions: manual.html#extensions .. _extensions: manual.html#extensions
.. _`http seeding`: manual.html#http-seeding .. _`http seeding`: manual.html#http-seeding
@ -252,6 +254,17 @@ and hence decreasing the likelihood of slow peers blocking the completion of pie
The piece picker can also be set to download pieces in sequential order. The piece picker can also be set to download pieces in sequential order.
share mode
----------
The share mode feature in libtorrent is intended for users who are only interested in
helping out swarms, not downloading the torrents.
It works by predicting the demand for pieces, and only download pieces if there is enough
demand. New pieces will only be downloaded once the share ratio has hit a certain target.
This feature is especially useful when combined with RSS, so that a client can be set up
to provide additional bandwidth to an entire feed.
merkle hash tree torrents merkle hash tree torrents
------------------------- -------------------------

View File

@ -393,6 +393,7 @@ add_torrent()
bool override_resume_data; bool override_resume_data;
bool upload_mode; bool upload_mode;
std::vector<boost::uint8_t> const* file_priorities; std::vector<boost::uint8_t> const* file_priorities;
bool share_mode;
}; };
torrent_handle add_torrent(add_torrent_params const& params); torrent_handle add_torrent(add_torrent_params const& params);
@ -503,6 +504,22 @@ on disk I/O errors, and if the torrent is also auto managed, it will be taken ou
of this state periodically. This mode can be used to avoid race conditions when of this state periodically. This mode can be used to avoid race conditions when
adjusting priorities of pieces before allowing the torrent to start downloading. adjusting priorities of pieces before allowing the torrent to start downloading.
``share_mode`` determines if the torrent should be added in *share mode* or not.
Share mode indicates that we are not interested in downloading the torrent, but
merlely want to improve our share ratio (i.e. increase it). A torrent started in
share mode will do its best to never download more than it uploads to the swarm.
If the swarm does not have enough demand for upload capacity, the torrent will
not download anything. This mode is intended to be safe to add any number of torrents
to, without manual screening, without the risk of downloading more than is uploaded.
A torrent in share mode sets the priority to all pieces to 0, except for the pieces
that are downloaded, when pieces are decided to be downloaded. This affects the progress
bar, which might be set to "100% finished" most of the time. Do not change file or piece
priorities for torrents in share mode, it will make it not work.
The share mode has one setting, the share ratio target, see ``session_settings::share_mode_target``
for more info.
``file_priorities`` can be set to control the initial file priorities when adding ``file_priorities`` can be set to control the initial file priorities when adding
a torrent. The semantics are the same as for ``torrent_handle::prioritize_files()``. a torrent. The semantics are the same as for ``torrent_handle::prioritize_files()``.
@ -2058,6 +2075,7 @@ Its declaration looks like this::
void force_recheck() const; void force_recheck() const;
void clear_error() const; void clear_error() const;
void set_upload_mode(bool m) const; void set_upload_mode(bool m) const;
void set_share_mode(bool m) const;
void flush_cache() const; void flush_cache() const;
@ -2559,6 +2577,18 @@ are automatically put in upload mode whenever they encounter a disk write error.
To test if a torrent is in upload mode, call ``torrent_handle::status()`` and inspect To test if a torrent is in upload mode, call ``torrent_handle::status()`` and inspect
``torrent_status::upload_mode``. ``torrent_status::upload_mode``.
set_share_mode()
----------------
::
void set_share_mode(bool m) const;
Enable or disable share mode for this torrent. When in share mode, the torrent will
not necessarily be downloaded, especially not the whole of it. Only parts that are likely
to be distributed to more than 2 other peers are downloaded, and only if the previous
prediction was correct.
resolve_countries() resolve_countries()
------------------- -------------------
@ -3129,8 +3159,8 @@ It contains the following fields::
int sparse_regions; int sparse_regions;
bool seed_mode; bool seed_mode;
bool upload_mode; bool upload_mode;
bool share_mode;
int priority; int priority;
@ -3363,6 +3393,9 @@ been resolved. If the torrent is not auto-managed, you have to explicitly
take it out of the upload mode by calling `set_upload_mode()`_ on the take it out of the upload mode by calling `set_upload_mode()`_ on the
torrent_handle_. torrent_handle_.
``share_mode`` is true if the torrent is currently in share-mode, i.e.
not downloading the torrent, but just helping the swarm out.
``added_time`` is the posix-time when this torrent was added. i.e. what ``added_time`` is the posix-time when this torrent was added. i.e. what
``time(NULL)`` returned at the time. ``time(NULL)`` returned at the time.
@ -3929,6 +3962,7 @@ session_settings
bool ignore_resume_timestamps; bool ignore_resume_timestamps;
bool anonymous_mode; bool anonymous_mode;
int tick_interval; int tick_interval;
int share_mode_target;
}; };
``version`` is automatically set to the libtorrent version you're using ``version`` is automatically set to the libtorrent version you're using
@ -4590,6 +4624,15 @@ peers. It should not be more than one second (i.e. 1000 ms). Setting this
to a low value (around 100) means higher resolution bandwidth quota distribution, to a low value (around 100) means higher resolution bandwidth quota distribution,
setting it to a higher value saves CPU cycles. setting it to a higher value saves CPU cycles.
``share_mode_target`` specifies the target share ratio for share mode torrents.
This defaults to 3, meaning we'll try to upload 3 times as much as we download.
Setting this very high, will make it very conservative and you might end up
not downloading anything ever (and not affecting your share ratio). It does
not make any sense to set this any lower than 2. For instance, if only 3 peers
need to download the rarest piece, it's impossible to download a single piece
and upload it more than 3 times. If the share_mode_target is set to more than 3,
nothing is downloaded.
pe_settings pe_settings
=========== ===========

View File

@ -530,6 +530,8 @@ std::string outgoing_interface = "";
int poll_interval = 5; int poll_interval = 5;
int max_connections_per_torrent = 50; int max_connections_per_torrent = 50;
bool share_mode = false;
using boost::bind; using boost::bind;
// monitored_dir is true if this torrent is added because // monitored_dir is true if this torrent is added because
@ -560,6 +562,7 @@ void add_torrent(libtorrent::session& ses
printf("%s\n", t->name().c_str()); printf("%s\n", t->name().c_str());
add_torrent_params p; add_torrent_params p;
p.share_mode = share_mode;
lazy_entry resume_data; lazy_entry resume_data;
std::string filename = combine_path(save_path, t->name() + ".resume"); std::string filename = combine_path(save_path, t->name() + ".resume");
@ -810,7 +813,7 @@ int main(int argc, char* argv[])
//settings.announce_to_all_trackers = true; //settings.announce_to_all_trackers = true;
settings.optimize_hashing_for_speed = false; settings.optimize_hashing_for_speed = false;
settings.disk_cache_algorithm = session_settings::largest_contiguous; settings.disk_cache_algorithm = session_settings::largest_contiguous;
settings.volatile_read_cache = true; settings.volatile_read_cache = false;
proxy_settings ps; proxy_settings ps;
@ -868,6 +871,7 @@ int main(int argc, char* argv[])
from_hex(argv[i], 40, (char*)&info_hash[0]); from_hex(argv[i], 40, (char*)&info_hash[0]);
add_torrent_params p; add_torrent_params p;
p.share_mode = share_mode;
p.tracker_url = argv[i] + 41; p.tracker_url = argv[i] + 41;
p.info_hash = info_hash; p.info_hash = info_hash;
p.save_path = save_path; p.save_path = save_path;
@ -908,6 +912,7 @@ int main(int argc, char* argv[])
case 'U': torrent_upload_limit = atoi(arg) * 1000; break; case 'U': torrent_upload_limit = atoi(arg) * 1000; break;
case 'D': torrent_download_limit = atoi(arg) * 1000; break; case 'D': torrent_download_limit = atoi(arg) * 1000; break;
case 'm': monitor_dir = arg; break; case 'm': monitor_dir = arg; break;
case 'M': share_mode = true; --i; break;
case 'b': bind_to_interface = arg; break; case 'b': bind_to_interface = arg; break;
case 'w': settings.urlseed_wait_retry = atoi(arg); break; case 'w': settings.urlseed_wait_retry = atoi(arg); break;
case 't': poll_interval = atoi(arg); break; case 't': poll_interval = atoi(arg); break;
@ -1053,6 +1058,7 @@ int main(int argc, char* argv[])
if (std::strstr(i->c_str(), "magnet:") == i->c_str()) if (std::strstr(i->c_str(), "magnet:") == i->c_str())
{ {
add_torrent_params p; add_torrent_params p;
p.share_mode = share_mode;
p.save_path = save_path; p.save_path = save_path;
p.storage_mode = (storage_mode_t)allocation_mode; p.storage_mode = (storage_mode_t)allocation_mode;
printf("adding MANGET link: %s\n", i->c_str()); printf("adding MANGET link: %s\n", i->c_str());
@ -1404,7 +1410,7 @@ int main(int argc, char* argv[])
out += str; out += str;
} }
if (print_piece_bar && s.progress_ppm < 1000000 && s.progress > 0) if (print_piece_bar && s.state != torrent_status::seeding)
{ {
out += " "; out += " ";
out += piece_bar(s.pieces, terminal_width - 7); out += piece_bar(s.pieces, terminal_width - 7);

View File

@ -62,6 +62,7 @@ namespace libtorrent
, override_resume_data(false) , override_resume_data(false)
, upload_mode(false) , upload_mode(false)
, file_priorities(0) , file_priorities(0)
, share_mode(false)
{} {}
// libtorrent version. Used for forward binary compatibility // libtorrent version. Used for forward binary compatibility
@ -82,6 +83,7 @@ namespace libtorrent
bool override_resume_data; bool override_resume_data;
bool upload_mode; bool upload_mode;
std::vector<boost::uint8_t> const* file_priorities; std::vector<boost::uint8_t> const* file_priorities;
bool share_mode;
}; };
} }

View File

@ -102,7 +102,7 @@ namespace libtorrent
void start(); void start();
enum { upload_only_msg = 2 }; enum { upload_only_msg = 2, share_mode_msg = 3 };
~bt_peer_connection(); ~bt_peer_connection();
@ -205,6 +205,7 @@ namespace libtorrent
#ifndef TORRENT_DISABLE_EXTENSIONS #ifndef TORRENT_DISABLE_EXTENSIONS
void write_extensions(); void write_extensions();
void write_upload_only(); void write_upload_only();
void write_share_mode();
#endif #endif
void write_metadata(std::pair<int, int> req); void write_metadata(std::pair<int, int> req);
void write_metadata_request(std::pair<int, int> req); void write_metadata_request(std::pair<int, int> req);
@ -365,6 +366,10 @@ private:
// 0 if not supported // 0 if not supported
int m_upload_only_id; int m_upload_only_id;
// the message ID for share mode message
// 0 if not supported
int m_share_mode_id;
char m_reserved_bits[8]; char m_reserved_bits[8];
// this is set to true if the handshake from // this is set to true if the handshake from
// the peer indicated that it supports the // the peer indicated that it supports the

View File

@ -274,6 +274,10 @@ namespace libtorrent
bool can_read(char* state = 0) const; bool can_read(char* state = 0) const;
bool is_seed() const; bool is_seed() const;
int num_have_pieces() const { return m_num_pieces; }
void set_share_mode(bool m);
bool share_mode() const { return m_share_mode; }
void set_upload_only(bool u); void set_upload_only(bool u);
bool upload_only() const { return m_upload_only; } bool upload_only() const { return m_upload_only; }
@ -1043,7 +1047,10 @@ namespace libtorrent
// the http-downloader, to request whole pieces // the http-downloader, to request whole pieces
// at a time. // at a time.
bool m_request_large_blocks:1; bool m_request_large_blocks:1;
// set to true if this peer is in share mode
bool m_share_mode:1;
// set to true when this peer is only uploading // set to true when this peer is only uploading
bool m_upload_only:1; bool m_upload_only:1;

View File

@ -95,6 +95,8 @@ namespace libtorrent
{ {
public: public:
struct piece_pos;
enum enum
{ {
// the number of priority levels // the number of priority levels
@ -162,7 +164,7 @@ namespace libtorrent
// the number of blocks in the requested state // the number of blocks in the requested state
boost::int16_t requested; boost::int16_t requested;
}; };
piece_picker(); piece_picker();
void get_availability(std::vector<int>& avail) const; void get_availability(std::vector<int>& avail) const;
@ -292,7 +294,13 @@ namespace libtorrent
// returns information about the given piece // returns information about the given piece
void piece_info(int index, piece_picker::downloading_piece& st) const; void piece_info(int index, piece_picker::downloading_piece& st) const;
piece_pos const& piece_stats(int index) const
{
TORRENT_ASSERT(index >= 0 && index < int(m_piece_map.size()));
return m_piece_map[index];
}
// if a piece had a hash-failure, it must be restored and // if a piece had a hash-failure, it must be restored and
// made available for redownloading // made available for redownloading
void restore_piece(int index); void restore_piece(int index);
@ -359,6 +367,8 @@ namespace libtorrent
std::pair<int, int> expand_piece(int piece, int whole_pieces std::pair<int, int> expand_piece(int piece, int whole_pieces
, bitfield const& have) const; , bitfield const& have) const;
public:
struct piece_pos struct piece_pos
{ {
piece_pos() {} piece_pos() {}
@ -452,6 +462,8 @@ namespace libtorrent
}; };
private:
BOOST_STATIC_ASSERT(sizeof(piece_pos) == sizeof(char) * 4); BOOST_STATIC_ASSERT(sizeof(piece_pos) == sizeof(char) * 4);
void update_pieces() const; void update_pieces() const;

View File

@ -221,6 +221,7 @@ namespace libtorrent
, anonymous_mode(false) , anonymous_mode(false)
, tick_interval(100) , tick_interval(100)
, report_web_seed_downloads(true) , report_web_seed_downloads(true)
, share_mode_target(3)
{} {}
// libtorrent version. Used for forward binary compatibility // libtorrent version. Used for forward binary compatibility
@ -843,6 +844,9 @@ namespace libtorrent
// specifies whether downloads from web seeds is reported to the // specifies whether downloads from web seeds is reported to the
// tracker or not. Defaults to on // tracker or not. Defaults to on
bool report_web_seed_downloads; bool report_web_seed_downloads;
// this is the target share ratio for share-mode torrents
int share_mode_target;
}; };
#ifndef TORRENT_DISABLE_DHT #ifndef TORRENT_DISABLE_DHT

View File

@ -178,8 +178,12 @@ namespace libtorrent
void start_announcing(); void start_announcing();
void stop_announcing(); void stop_announcing();
void send_share_mode();
void send_upload_only(); void send_upload_only();
void set_share_mode(bool s);
bool share_mode() const { return m_share_mode; }
void set_upload_mode(bool b); void set_upload_mode(bool b);
bool upload_mode() const { return m_upload_mode; } bool upload_mode() const { return m_upload_mode; }
bool is_upload_only() const bool is_upload_only() const
@ -502,6 +506,8 @@ namespace libtorrent
// -------------------------------------------- // --------------------------------------------
// PIECE MANAGEMENT // PIECE MANAGEMENT
void recalc_share_mode();
void update_sparse_piece_prio(int piece, int cursor, int reverse_cursor); void update_sparse_piece_prio(int piece, int cursor, int reverse_cursor);
void get_suggested_pieces(std::vector<int>& s) const; void get_suggested_pieces(std::vector<int>& s) const;
@ -525,8 +531,8 @@ namespace libtorrent
int num_have() const int num_have() const
{ {
return has_picker() return has_picker()
?m_picker->num_have() ? m_picker->num_have()
:m_torrent_file->num_pieces(); : m_torrent_file->num_pieces();
} }
// when we get a have message, this is called for that piece // when we get a have message, this is called for that piece
@ -1210,6 +1216,9 @@ namespace libtorrent
// slots. // slots.
bool m_auto_managed:1; bool m_auto_managed:1;
// this is set when the torrent is in share-mode
bool m_share_mode:1;
// m_num_verified = m_verified.count() // m_num_verified = m_verified.count()
boost::uint16_t m_num_verified; boost::uint16_t m_num_verified;

View File

@ -123,6 +123,7 @@ namespace libtorrent
, sparse_regions(0) , sparse_regions(0)
, seed_mode(false) , seed_mode(false)
, upload_mode(false) , upload_mode(false)
, share_mode(false)
, priority(0) , priority(0)
, added_time(0) , added_time(0)
, completed_time(0) , completed_time(0)
@ -306,6 +307,9 @@ namespace libtorrent
// write operation failing // write operation failing
bool upload_mode; bool upload_mode;
// this is true if the torrent is in share-mode
bool share_mode;
// the priority of this torrent // the priority of this torrent
int priority; int priority;
@ -468,6 +472,7 @@ namespace libtorrent
void pause() const; void pause() const;
void resume() const; void resume() const;
void set_upload_mode(bool b) const; void set_upload_mode(bool b) const;
void set_share_mode(bool b) const;
void flush_cache() const; void flush_cache() const;
void force_recheck() const; void force_recheck() const;

View File

@ -98,6 +98,7 @@ namespace libtorrent
, m_state(read_protocol_identifier) , m_state(read_protocol_identifier)
#ifndef TORRENT_DISABLE_EXTENSIONS #ifndef TORRENT_DISABLE_EXTENSIONS
, m_upload_only_id(0) , m_upload_only_id(0)
, m_share_mode_id(0)
, m_supports_extensions(false) , m_supports_extensions(false)
#endif #endif
, m_supports_dht_port(false) , m_supports_dht_port(false)
@ -1496,6 +1497,13 @@ namespace libtorrent
return; return;
} }
if (extended_id == share_mode_msg)
{
if (!packet_finished()) return;
set_share_mode(detail::read_uint8(recv_buffer.begin));
return;
}
disconnect(errors::invalid_message, 2); disconnect(errors::invalid_message, 2);
return; return;
} }
@ -1564,6 +1572,9 @@ namespace libtorrent
if (root.dict_find_int_value("upload_only", 0)) if (root.dict_find_int_value("upload_only", 0))
set_upload_only(true); set_upload_only(true);
if (root.dict_find_int_value("share_mode", 0))
set_share_mode(true);
std::string myip = root.dict_find_string_value("yourip"); std::string myip = root.dict_find_string_value("yourip");
if (!myip.empty()) if (!myip.empty())
{ {
@ -1591,7 +1602,8 @@ namespace libtorrent
// if we're finished and this peer is uploading only // if we're finished and this peer is uploading only
// disconnect it // disconnect it
if (t->is_finished() && upload_only() if (t->is_finished() && upload_only()
&& t->settings().close_redundant_connections) && t->settings().close_redundant_connections
&& !t->share_mode())
disconnect(errors::upload_upload_connection); disconnect(errors::upload_upload_connection);
} }
@ -1662,6 +1674,7 @@ namespace libtorrent
boost::shared_ptr<torrent> t = associated_torrent().lock(); boost::shared_ptr<torrent> t = associated_torrent().lock();
if (m_upload_only_id == 0) return; if (m_upload_only_id == 0) return;
if (t->share_mode()) return;
char msg[7] = {0, 0, 0, 3, msg_extended}; char msg[7] = {0, 0, 0, 3, msg_extended};
char* ptr = msg + 5; char* ptr = msg + 5;
@ -1669,6 +1682,20 @@ namespace libtorrent
detail::write_uint8(t->is_upload_only(), ptr); detail::write_uint8(t->is_upload_only(), ptr);
send_buffer(msg, sizeof(msg)); send_buffer(msg, sizeof(msg));
} }
void bt_peer_connection::write_share_mode()
{
INVARIANT_CHECK;
boost::shared_ptr<torrent> t = associated_torrent().lock();
if (m_share_mode_id == 0) return;
char msg[7] = {0, 0, 0, 3, msg_extended};
char* ptr = msg + 5;
detail::write_uint8(m_share_mode_id, ptr);
detail::write_uint8(t->share_mode(), ptr);
send_buffer(msg, sizeof(msg));
}
#endif #endif
void bt_peer_connection::write_keepalive() void bt_peer_connection::write_keepalive()
@ -1900,19 +1927,25 @@ namespace libtorrent
TORRENT_ASSERT(t); TORRENT_ASSERT(t);
m["upload_only"] = upload_only_msg; m["upload_only"] = upload_only_msg;
m["share_mode"] = share_mode_msg;
int complete_ago = -1; int complete_ago = -1;
if (t->last_seen_complete() > 0) complete_ago = t->time_since_complete(); if (t->last_seen_complete() > 0) complete_ago = t->time_since_complete();
handshake["complete_ago"] = complete_ago; handshake["complete_ago"] = complete_ago;
// if we're using lazy bitfields or if we're super seeding, don't say // if we're using lazy bitfields or if we're super seeding, don't say
// we're upload only, since it might make peers disconnect // we're upload only, since it might make peers disconnect
if (t->is_upload_only() && (!m_ses.settings().lazy_bitfields // don't tell anyone we're upload only when in share mode
// we want to stay connected to seeds
if (t->is_upload_only() && !t->share_mode() && (!m_ses.settings().lazy_bitfields
#ifndef TORRENT_DISABLE_ENCRYPTION #ifndef TORRENT_DISABLE_ENCRYPTION
|| m_encrypted || m_encrypted
#endif #endif
)) ))
handshake["upload_only"] = 1; handshake["upload_only"] = 1;
if (t->share_mode())
handshake["share_mode"] = 1;
if (!m_ses.m_settings.anonymous_mode) if (!m_ses.m_settings.anonymous_mode)
{ {
tcp::endpoint ep = m_ses.get_ipv6_interface(); tcp::endpoint ep = m_ses.get_ipv6_interface();

View File

@ -156,7 +156,7 @@ namespace libtorrent { namespace
virtual void add_handshake(entry& h) virtual void add_handshake(entry& h)
{ {
entry& messages = h["m"]; entry& messages = h["m"];
messages["lt_tex"] = 3; messages["lt_tex"] = 19;
h["tr"] = m_tp.list_hash().to_string(); h["tr"] = m_tp.list_hash().to_string();
} }
@ -186,7 +186,7 @@ namespace libtorrent { namespace
virtual bool on_extended(int length virtual bool on_extended(int length
, int extended_msg, buffer::const_interval body) , int extended_msg, buffer::const_interval body)
{ {
if (extended_msg != 3) return false; if (extended_msg != 19) return false;
if (m_message_index == 0) return false; if (m_message_index == 0) return false;
if (!m_pc.packet_finished()) return true; if (!m_pc.packet_finished()) return true;

View File

@ -144,6 +144,7 @@ namespace libtorrent
, m_connecting(true) , m_connecting(true)
, m_queued(true) , m_queued(true)
, m_request_large_blocks(false) , m_request_large_blocks(false)
, m_share_mode(false)
, m_upload_only(false) , m_upload_only(false)
, m_snubbed(false) , m_snubbed(false)
, m_bitfield_received(false) , m_bitfield_received(false)
@ -282,6 +283,7 @@ namespace libtorrent
, m_connecting(false) , m_connecting(false)
, m_queued(false) , m_queued(false)
, m_request_large_blocks(false) , m_request_large_blocks(false)
, m_share_mode(false)
, m_upload_only(false) , m_upload_only(false)
, m_snubbed(false) , m_snubbed(false)
, m_bitfield_received(false) , m_bitfield_received(false)
@ -513,7 +515,7 @@ namespace libtorrent
void peer_connection::update_interest() void peer_connection::update_interest()
{ {
boost::shared_ptr<torrent> t = m_torrent.lock(); boost::shared_ptr<torrent> t = m_torrent.lock();
TORRENT_ASSERT(t); if (!t) return;
// if m_have_piece is 0, it means the connections // if m_have_piece is 0, it means the connections
// have not been initialized yet. The interested // have not been initialized yet. The interested
@ -1794,6 +1796,9 @@ namespace libtorrent
boost::shared_ptr<torrent> t = m_torrent.lock(); boost::shared_ptr<torrent> t = m_torrent.lock();
if (!t) return; if (!t) return;
// don't close connections in share mode, we don't know if we need them
if (t->share_mode()) return;
if (m_upload_only && t->is_finished()) if (m_upload_only && t->is_finished())
{ {
disconnect(errors::upload_upload_connection); disconnect(errors::upload_upload_connection);
@ -4025,6 +4030,7 @@ namespace libtorrent
INVARIANT_CHECK; INVARIANT_CHECK;
#endif #endif
bool sent_a_piece = false;
boost::shared_ptr<torrent> t = m_torrent.lock(); boost::shared_ptr<torrent> t = m_torrent.lock();
if (!t) return; if (!t) return;
@ -4070,7 +4076,11 @@ namespace libtorrent
m_reading_bytes += r.length; m_reading_bytes += r.length;
m_requests.erase(m_requests.begin()); m_requests.erase(m_requests.begin());
sent_a_piece = true;
} }
if (t->share_mode() && sent_a_piece)
t->recalc_share_mode();
} }
void peer_connection::on_disk_read_complete(int ret, disk_io_job const& j, peer_request r) void peer_connection::on_disk_read_complete(int ret, disk_io_job const& j, peer_request r)
@ -5143,7 +5153,8 @@ namespace libtorrent
if (t->ready_for_connections() && m_initialized) if (t->ready_for_connections() && m_initialized)
TORRENT_ASSERT(t->torrent_file().num_pieces() == int(m_have_piece.size())); TORRENT_ASSERT(t->torrent_file().num_pieces() == int(m_have_piece.size()));
if (m_ses.settings().close_redundant_connections) // in share mode we don't close redundant connections
if (m_ses.settings().close_redundant_connections && !t->share_mode())
{ {
// make sure upload only peers are disconnected // make sure upload only peers are disconnected
if (t->is_finished() && m_upload_only) if (t->is_finished() && m_upload_only)
@ -5333,6 +5344,14 @@ namespace libtorrent
return m_num_pieces == (int)m_have_piece.size() && m_num_pieces > 0 && t && t->valid_metadata(); return m_num_pieces == (int)m_have_piece.size() && m_num_pieces > 0 && t && t->valid_metadata();
} }
void peer_connection::set_share_mode(bool u)
{
// if the peer is a seed, ignore share mode messages
if (is_seed()) return;
m_share_mode = u;
}
void peer_connection::set_upload_only(bool u) void peer_connection::set_upload_only(bool u)
{ {
// if the peer is a seed, don't allow setting // if the peer is a seed, don't allow setting

View File

@ -378,6 +378,7 @@ namespace libtorrent
, m_allow_peers(!p.paused) , m_allow_peers(!p.paused)
, m_upload_mode(p.upload_mode) , m_upload_mode(p.upload_mode)
, m_auto_managed(p.auto_managed) , m_auto_managed(p.auto_managed)
, m_share_mode(p.share_mode)
, m_num_verified(0) , m_num_verified(0)
, m_last_scrape(0) , m_last_scrape(0)
, m_last_download(0) , m_last_download(0)
@ -418,7 +419,6 @@ namespace libtorrent
m_trackers.back().source = announce_entry::source_magnet_link; m_trackers.back().source = announce_entry::source_magnet_link;
m_torrent_file->add_tracker(p.tracker_url); m_torrent_file->add_tracker(p.tracker_url);
} }
} }
void torrent::start() void torrent::start()
@ -533,6 +533,19 @@ namespace libtorrent
} }
} }
void torrent::send_share_mode()
{
#ifndef TORRENT_DISABLE_EXTENSIONS
for (std::set<peer_connection*>::iterator i = m_connections.begin()
, end(m_connections.end()); i != end; ++i)
{
if ((*i)->type() != peer_connection::bittorrent_connection) continue;
bt_peer_connection* p = (bt_peer_connection*)*i;
p->write_share_mode();
}
#endif
}
void torrent::send_upload_only() void torrent::send_upload_only()
{ {
#ifndef TORRENT_DISABLE_EXTENSIONS #ifndef TORRENT_DISABLE_EXTENSIONS
@ -546,6 +559,20 @@ namespace libtorrent
#endif #endif
} }
void torrent::set_share_mode(bool s)
{
if (s == m_share_mode) return;
m_share_mode = s;
// in share mode, all pieces have their priorities initialized to 0
std::fill(m_file_priority.begin(), m_file_priority.end(), !m_share_mode);
update_piece_priorities();
if (m_share_mode) recalc_share_mode();
}
void torrent::set_upload_mode(bool b) void torrent::set_upload_mode(bool b)
{ {
if (b == m_upload_mode) return; if (b == m_upload_mode) return;
@ -855,6 +882,16 @@ namespace libtorrent
m_picker->init(blocks_per_piece, blocks_in_last_piece, m_torrent_file->num_pieces()); m_picker->init(blocks_per_piece, blocks_in_last_piece, m_torrent_file->num_pieces());
} }
if (m_share_mode)
{
// in share mode, all pieces have their priorities initialized to 0
std::fill(m_file_priority.begin(), m_file_priority.end(), 0);
}
// in case file priorities were passed in via the add_torrent_params
// ans also in the case of share mode, we need to update the priorities
update_piece_priorities();
std::vector<std::string> const& url_seeds = m_torrent_file->url_seeds(); std::vector<std::string> const& url_seeds = m_torrent_file->url_seeds();
for (std::vector<std::string>::const_iterator i = url_seeds.begin() for (std::vector<std::string>::const_iterator i = url_seeds.begin()
, end(url_seeds.end()); i != end; ++i) , end(url_seeds.end()); i != end; ++i)
@ -2335,6 +2372,9 @@ namespace libtorrent
} }
m_last_download = 0; m_last_download = 0;
if (m_share_mode)
recalc_share_mode();
} }
void torrent::piece_failed(int index) void torrent::piece_failed(int index)
@ -3744,6 +3784,7 @@ namespace libtorrent
m_file_priority[i] = file_priority->list_int_value_at(i, 1); m_file_priority[i] = file_priority->list_int_value_at(i, 1);
update_piece_priorities(); update_piece_priorities();
} }
lazy_entry const* piece_priority = rd.dict_find_string("piece_priority"); lazy_entry const* piece_priority = rd.dict_find_string("piece_priority");
if (piece_priority && piece_priority->string_length() if (piece_priority && piece_priority->string_length()
== m_torrent_file->num_pieces()) == m_torrent_file->num_pieces())
@ -4365,6 +4406,9 @@ namespace libtorrent
} }
#endif #endif
if (m_share_mode)
recalc_share_mode();
return peerinfo->connection; return peerinfo->connection;
} }
@ -4494,6 +4538,10 @@ namespace libtorrent
#if defined TORRENT_DEBUG && !defined TORRENT_DISABLE_INVARIANT_CHECKS #if defined TORRENT_DEBUG && !defined TORRENT_DISABLE_INVARIANT_CHECKS
m_policy.check_invariant(); m_policy.check_invariant();
#endif #endif
if (m_share_mode)
recalc_share_mode();
return true; return true;
} }
@ -4961,7 +5009,7 @@ namespace libtorrent
if (!i->not_wanted && !i->timed_out) ++num_requests[i->block]; if (!i->not_wanted && !i->timed_out) ++num_requests[i->block];
if (!p.is_choked() && !p.ignore_unchoke_slots()) ++num_uploads; if (!p.is_choked() && !p.ignore_unchoke_slots()) ++num_uploads;
torrent* associated_torrent = p.associated_torrent().lock().get(); torrent* associated_torrent = p.associated_torrent().lock().get();
if (associated_torrent != this) if (associated_torrent != this && associated_torrent != 0)
TORRENT_ASSERT(false); TORRENT_ASSERT(false);
} }
TORRENT_ASSERT(num_uploads == m_num_uploads); TORRENT_ASSERT(num_uploads == m_num_uploads);
@ -5862,6 +5910,152 @@ namespace libtorrent
m_stat.second_tick(tick_interval_ms); m_stat.second_tick(tick_interval_ms);
} }
void torrent::recalc_share_mode()
{
TORRENT_ASSERT(share_mode());
if (is_seed()) return;
int pieces_in_torrent = m_torrent_file->num_pieces();
int num_seeds = 0;
int num_peers = 0;
int num_downloaders = 0;
int missing_pieces = 0;
int num_interested = 0;
for (std::set<peer_connection*>::iterator i = m_connections.begin()
, end(m_connections.end()); i != end; ++i)
{
peer_connection* p = *i;
if (p->is_connecting()) continue;
++num_peers;
if (p->is_seed())
{
++num_seeds;
continue;
}
if (p->share_mode()) continue;
if ((*i)->is_peer_interested()) ++num_interested;
++num_downloaders;
missing_pieces += pieces_in_torrent - p->num_have_pieces();
}
if (num_peers == 0) return;
if (num_seeds * 100 / num_peers > 50
&& (num_peers * 100 / m_max_connections > 90
|| num_peers > 20))
{
// we are connected to more than 90% seeds (and we're beyond
// 90% of the max number of connections). That will
// limit our ability to upload. We need more downloaders.
// disconnect some seeds so that we don't have more than 50%
int to_disconnect = num_seeds - num_peers / 2;
std::vector<peer_connection*> seeds;
seeds.reserve(num_seeds);
for (std::set<peer_connection*>::iterator i = m_connections.begin()
, end(m_connections.end()); i != end; ++i)
{
peer_connection* p = *i;
if (p->is_seed()) seeds.push_back(p);
}
std::random_shuffle(seeds.begin(), seeds.end());
TORRENT_ASSERT(to_disconnect <= int(seeds.size()));
for (int i = 0; i < to_disconnect; ++i)
seeds[i]->disconnect(errors::upload_upload_connection);
}
if (num_downloaders == 0) return;
// assume that the seeds are about as fast as us. During the time
// we can download one piece, and upload one piece, each seed
// can upload two pieces.
missing_pieces -= 2 * num_seeds;
if (missing_pieces <= 0) return;
// missing_pieces represents our opportunity to download pieces
// and share them more than once each
// now, download at least one piece, otherwise download one more
// piece if our downloaded (and downloading) pieces is less than 50%
// of the uploaded bytes
int num_downloaded_pieces = (std::max)(m_picker->num_have()
, pieces_in_torrent - m_picker->num_filtered());
if (num_downloaded_pieces * m_torrent_file->piece_length()
* settings().share_mode_target > m_total_uploaded
&& num_downloaded_pieces > 0)
return;
// don't have more pieces downloading in parallel than 5% of the total
// number of pieces we have downloaded
if (m_picker->get_download_queue().size() > num_downloaded_pieces / 20)
return;
// one more important property is that there are enough pieces
// that more than one peer wants to download
// make sure that there are enough downloaders for the rarest
// piece. Go through all pieces, figure out which one is the rarest
// and how many peers that has that piece
std::vector<int> rarest_pieces;
int num_pieces = m_torrent_file->num_pieces();
int rarest_rarity = INT_MAX;
bool prio_updated = false;
for (int i = 0; i < num_pieces; ++i)
{
piece_picker::piece_pos const& pp = m_picker->piece_stats(i);
if (pp.peer_count == 0) continue;
if (pp.filtered() && (pp.have() || pp.downloading))
{
m_picker->set_piece_priority(i, 1);
prio_updated = true;
continue;
}
// don't count pieces we already have or are downloading
if (!pp.filtered() || pp.have()) continue;
if (pp.peer_count > rarest_rarity) continue;
if (pp.peer_count == rarest_rarity)
{
rarest_pieces.push_back(i);
continue;
}
rarest_pieces.clear();
rarest_rarity = pp.peer_count;
rarest_pieces.push_back(i);
}
if (prio_updated)
m_policy.recalculate_connect_candidates();
// now, rarest_pieces is a list of all pieces that are the rarest ones.
// and rarest_rarity is the number of peers that have the rarest pieces
// if there's only a single peer that doesn't have the rarest piece
// it's impossible for us to download one piece and upload it
// twice. i.e. we cannot get a positive share ratio
if (num_peers - rarest_rarity < settings().share_mode_target) return;
// we might be able to do better than a share ratio of 2 if there are
// enough downloaders of the pieces we already have.
// TODO: go through the pieces we have and count the total number
// of downloaders we have. Only count peers that are interested in us
// since some peers might not send have messages for pieces we have
// it num_interested == 0, we need to pick a new piece
// now, pick one of the rarest pieces to download
int pick = rand() % rarest_pieces.size();
bool was_finished = is_finished();
m_picker->set_piece_priority(rarest_pieces[pick], 1);
update_peer_interest(was_finished);
m_policy.recalculate_connect_candidates();
}
void torrent::refresh_explicit_cache(int cache_size) void torrent::refresh_explicit_cache(int cache_size)
{ {
TORRENT_ASSERT(m_ses.is_network_thread()); TORRENT_ASSERT(m_ses.is_network_thread());
@ -6404,6 +6598,7 @@ namespace libtorrent
st.completed_time = m_completed_time; st.completed_time = m_completed_time;
st.last_scrape = m_last_scrape; st.last_scrape = m_last_scrape;
st.share_mode = m_share_mode;
st.upload_mode = m_upload_mode; st.upload_mode = m_upload_mode;
st.up_bandwidth_queue = 0; st.up_bandwidth_queue = 0;
st.down_bandwidth_queue = 0; st.down_bandwidth_queue = 0;

View File

@ -363,6 +363,12 @@ namespace libtorrent
TORRENT_ASYNC_CALL(pause); TORRENT_ASYNC_CALL(pause);
} }
void torrent_handle::set_share_mode(bool b) const
{
INVARIANT_CHECK;
TORRENT_ASYNC_CALL1(set_share_mode, b);
}
void torrent_handle::set_upload_mode(bool b) const void torrent_handle::set_upload_mode(bool b) const
{ {
INVARIANT_CHECK; INVARIANT_CHECK;