deprecate explicit cache feature

This commit is contained in:
arvidn 2016-03-18 12:15:03 -04:00
parent 7070492a5f
commit 3c98868f6c
11 changed files with 30 additions and 47 deletions

View File

@ -238,42 +238,17 @@ In order to increase the possibility of read cache hits, set the
``session_settings::cache_expiry`` to a large number. This won't degrade anything as ``session_settings::cache_expiry`` to a large number. This won't degrade anything as
long as the client is only seeding, and not downloading any torrents. long as the client is only seeding, and not downloading any torrents.
In order to increase the disk cache hit rate, you can enable suggest messages based on There's a *guided cache* mode. This means the size of the read cache line that's
what's in the read cache. To do this, set ``session_settings::suggest_mode`` to stored in the cache is determined based on the upload rate to the peer that
``session_settings::suggest_read_cache``. This will send suggest messages to peers triggered the read operation. The idea being that slow peers don't use up a
for the most recently used pieces in the read cache. This is especially useful if you disproportional amount of space in the cache. This is enabled through
also enable explicit read cache, by settings ``session_settings::explicit_read_cache`` ``session_settings::guided_read_cache``.
to the number of pieces to keep in the cache. The explicit read cache will make the
disk read cache stick, and not be evicted by cache misses. The explicit read cache
will automatically pull in the rarest pieces in the read cache.
Assuming that you seed much more data than you can keep in the cache, to a large
numbers of peers (so that the read cache wouldn't be useful anyway), this may be a
good idea.
When peers first connect, libtorrent will send them a number of allow-fast messages,
which lets the peers download certain pieces even when they are choked, since peers
are choked by default, this often triggers immediate requests for those pieces. In the
case of using explicit read cache and suggesting those pieces, allowing fast pieces
should be disabled, to not systematically trigger requests for pieces that are not cached
for all peers. You can turn off allow-fast by settings ``session_settings::allowed_fast_set_size``
to 0.
As an alternative to the explicit cache and suggest messages, there's a *guided cache*
mode. This means the size of the read cache line that's stored in the cache is determined
based on the upload rate to the peer that triggered the read operation. The idea being
that slow peers don't use up a disproportional amount of space in the cache. This
is enabled through ``session_settings::guided_read_cache``.
In cases where the assumption is that the cache is only used as a read-ahead, and that no In cases where the assumption is that the cache is only used as a read-ahead, and that no
other peer will ever request the same block while it's still in the cache, the read other peer will ever request the same block while it's still in the cache, the read
cache can be set to be *volatile*. This means that every block that is requested out of cache can be set to be *volatile*. This means that every block that is requested out of
the read cache is removed immediately. This saves a significant amount of cache space the read cache is removed immediately. This saves a significant amount of cache space
which can be used as read-ahead for other peers. This mode should **never** be combined which can be used as read-ahead for other peers. To enable volatile read cache, set
with either ``explicit_read_cache`` or ``suggest_read_cache``, since those uses opposite
strategies for the read cache. You don't want to on one hand attract peers to request
the same pieces, and on the other hand assume that they won't request the same pieces
and drop them when the first peer requests it. To enable volatile read cache, set
``session_settings::volatile_read_cache`` to true. ``session_settings::volatile_read_cache`` to true.
SSD as level 2 cache SSD as level 2 cache

View File

@ -933,6 +933,7 @@ namespace libtorrent
// torrents. // torrents.
int m_auto_scrape_time_scaler; int m_auto_scrape_time_scaler;
#ifndef TORRENT_NO_DEPRECATED
// the index of the torrent that we'll // the index of the torrent that we'll
// refresh the next time // refresh the next time
int m_next_explicit_cache_torrent; int m_next_explicit_cache_torrent;
@ -941,6 +942,7 @@ namespace libtorrent
// the next time the read cache is rotated, if we're // the next time the read cache is rotated, if we're
// using an explicit read read cache. // using an explicit read read cache.
int m_cache_rotation_timer; int m_cache_rotation_timer;
#endif
// the index of the torrent that we'll // the index of the torrent that we'll
// refresh the next time // refresh the next time

View File

@ -451,6 +451,7 @@ namespace libtorrent
// once we want to calculate the piece hash // once we want to calculate the piece hash
bool dont_flush_write_cache; bool dont_flush_write_cache;
#ifndef TORRENT_NO_DEPRECATE
// defaults to 0. If set to something greater than 0, the disk read cache // defaults to 0. If set to something greater than 0, the disk read cache
// will not be evicted by cache misses and will explicitly be controlled // will not be evicted by cache misses and will explicitly be controlled
// based on the rarity of pieces. Rare pieces are more likely to be // based on the rarity of pieces. Rare pieces are more likely to be
@ -468,6 +469,7 @@ namespace libtorrent
// the cache, so that subsequent refreshes only swaps in pieces that are // the cache, so that subsequent refreshes only swaps in pieces that are
// rarer than whatever is in the cache at the time. // rarer than whatever is in the cache at the time.
int explicit_cache_interval; int explicit_cache_interval;
#endif
// the buffer modes to use for reading and writing. Set // the buffer modes to use for reading and writing. Set
// session_settings::disk_io_read_mode and disk_io_write_mode to one of // session_settings::disk_io_read_mode and disk_io_write_mode to one of

View File

@ -275,6 +275,7 @@ namespace libtorrent
// hash // hash
dont_flush_write_cache, dont_flush_write_cache,
#ifndef TORRENT_NO_DEPRECATED
// ``explicit_read_cache`` defaults to 0. If set to something greater // ``explicit_read_cache`` defaults to 0. If set to something greater
// than 0, the disk read cache will not be evicted by cache misses and // than 0, the disk read cache will not be evicted by cache misses and
// will explicitly be controlled based on the rarity of pieces. Rare // will explicitly be controlled based on the rarity of pieces. Rare
@ -284,6 +285,9 @@ namespace libtorrent
// actual read cache can't fit as many, it will essentially be // actual read cache can't fit as many, it will essentially be
// clamped. // clamped.
explicit_read_cache, explicit_read_cache,
#else
deprecated10,
#endif
// allocate separate, contiguous, buffers for read and write calls. // allocate separate, contiguous, buffers for read and write calls.
// Only used where writev/readv cannot be used will use more RAM but // Only used where writev/readv cannot be used will use more RAM but
@ -924,6 +928,7 @@ namespace libtorrent
cache_buffer_chunk_size, cache_buffer_chunk_size,
cache_expiry, cache_expiry,
#ifndef TORRENT_NO_DEPRECATED
// ``explicit_cache_interval`` is the number of seconds in between // ``explicit_cache_interval`` is the number of seconds in between
// each refresh of a part of the explicit read cache. Torrents take // each refresh of a part of the explicit read cache. Torrents take
// turns in refreshing and this is the time in between each torrent // turns in refreshing and this is the time in between each torrent
@ -933,6 +938,9 @@ namespace libtorrent
// subsequent refreshes only swaps in pieces that are rarer than // subsequent refreshes only swaps in pieces that are rarer than
// whatever is in the cache at the time. // whatever is in the cache at the time.
explicit_cache_interval, explicit_cache_interval,
#else
deprecated11,
#endif
// determines how files are opened when they're in read only mode // determines how files are opened when they're in read only mode
// versus read and write mode. The options are: // versus read and write mode. The options are:

View File

@ -676,14 +676,16 @@ namespace libtorrent
void get_peer_info(std::vector<peer_info>& v); void get_peer_info(std::vector<peer_info>& v);
void get_download_queue(std::vector<partial_piece_info>* queue) const; void get_download_queue(std::vector<partial_piece_info>* queue) const;
#ifndef TORRENT_NO_DEPRECATED
void refresh_explicit_cache(int cache_size); void refresh_explicit_cache(int cache_size);
#endif
void add_suggest_piece(int piece); void add_suggest_piece(int piece);
void update_suggest_piece(int index, int change); void update_suggest_piece(int index, int change);
void update_auto_sequential(); void update_auto_sequential();
void refresh_suggest_pieces(); void refresh_suggest_pieces();
void do_refresh_suggest_pieces(); void do_refresh_suggest_pieces();
void on_cache_info(disk_io_job const* j);
// -------------------------------------------- // --------------------------------------------
// TRACKER MANAGEMENT // TRACKER MANAGEMENT

View File

@ -231,7 +231,6 @@ namespace libtorrent
// download rate // download rate
set.set_int(settings_pack::max_queued_disk_bytes, 7 * 1024 * 1024); set.set_int(settings_pack::max_queued_disk_bytes, 7 * 1024 * 1024);
set.set_bool(settings_pack::explicit_read_cache, false);
// prevent fast pieces to interfere with suggested pieces // prevent fast pieces to interfere with suggested pieces
// since we unchoke everyone, we don't need fast pieces anyway // since we unchoke everyone, we don't need fast pieces anyway
set.set_int(settings_pack::allowed_fast_set_size, 0); set.set_int(settings_pack::allowed_fast_set_size, 0);

View File

@ -393,8 +393,10 @@ namespace aux {
, m_optimistic_unchoke_time_scaler(0) , m_optimistic_unchoke_time_scaler(0)
, m_disconnect_time_scaler(90) , m_disconnect_time_scaler(90)
, m_auto_scrape_time_scaler(180) , m_auto_scrape_time_scaler(180)
#ifndef TORRENT_NO_DEPRECATED
, m_next_explicit_cache_torrent(0) , m_next_explicit_cache_torrent(0)
, m_cache_rotation_timer(0) , m_cache_rotation_timer(0)
#endif
, m_next_suggest_torrent(0) , m_next_suggest_torrent(0)
, m_suggest_timer(0) , m_suggest_timer(0)
, m_peak_up_rate(0) , m_peak_up_rate(0)
@ -3333,6 +3335,7 @@ retry:
++m_next_suggest_torrent; ++m_next_suggest_torrent;
} }
#ifndef TORRENT_NO_DEPRECATED
// -------------------------------------------------------------- // --------------------------------------------------------------
// refresh explicit disk read cache // refresh explicit disk read cache
// -------------------------------------------------------------- // --------------------------------------------------------------
@ -3368,6 +3371,7 @@ retry:
least_recently_refreshed->second->refresh_explicit_cache(cache_size); least_recently_refreshed->second->refresh_explicit_cache(cache_size);
++m_next_explicit_cache_torrent; ++m_next_explicit_cache_torrent;
} }
#endif
// -------------------------------------------------------------- // --------------------------------------------------------------
// connect new peers // connect new peers

View File

@ -152,7 +152,7 @@ namespace libtorrent
SET(use_read_cache, true, 0), SET(use_read_cache, true, 0),
DEPRECATED_SET(use_write_cache, true, 0), DEPRECATED_SET(use_write_cache, true, 0),
SET(dont_flush_write_cache, false, 0), SET(dont_flush_write_cache, false, 0),
SET(explicit_read_cache, false, 0), DEPRECATED_SET(explicit_read_cache, false, 0),
SET(coalesce_reads, false, 0), SET(coalesce_reads, false, 0),
SET(coalesce_writes, false, 0), SET(coalesce_writes, false, 0),
SET(auto_manage_prefer_seeds, false, 0), SET(auto_manage_prefer_seeds, false, 0),
@ -251,7 +251,7 @@ namespace libtorrent
SET(cache_size, 1024, 0), SET(cache_size, 1024, 0),
SET(cache_buffer_chunk_size, 0, &session_impl::update_cache_buffer_chunk_size), SET(cache_buffer_chunk_size, 0, &session_impl::update_cache_buffer_chunk_size),
SET(cache_expiry, 300, 0), SET(cache_expiry, 300, 0),
SET(explicit_cache_interval, 30, 0), DEPRECATED_SET(explicit_cache_interval, 30, 0),
SET(disk_io_write_mode, settings_pack::enable_os_cache, 0), SET(disk_io_write_mode, settings_pack::enable_os_cache, 0),
SET(disk_io_read_mode, settings_pack::enable_os_cache, 0), SET(disk_io_read_mode, settings_pack::enable_os_cache, 0),
SET(outgoing_port, 0, 0), SET(outgoing_port, 0, 0),

View File

@ -10576,6 +10576,7 @@ namespace libtorrent
update_want_peers(); update_want_peers();
} }
#ifndef TORRENT_NO_DEPRECATED
// TODO: 2 this should probably be removed // TODO: 2 this should probably be removed
void torrent::refresh_explicit_cache(int cache_size) void torrent::refresh_explicit_cache(int cache_size)
{ {
@ -10674,6 +10675,7 @@ namespace libtorrent
} }
} }
} }
#endif
void torrent::sent_bytes(int bytes_payload, int bytes_protocol) void torrent::sent_bytes(int bytes_payload, int bytes_protocol)
{ {

View File

@ -48,13 +48,12 @@ void test_swarm(int flags)
using namespace libtorrent; using namespace libtorrent;
namespace lt = libtorrent; namespace lt = libtorrent;
fprintf(stderr, "\n\n ==== TEST SWARM === %s%s%s%s%s%s ===\n\n\n" fprintf(stderr, "\n\n ==== TEST SWARM === %s%s%s%s%s ===\n\n\n"
, (flags & super_seeding) ? "super-seeding ": "" , (flags & super_seeding) ? "super-seeding ": ""
, (flags & strict_super_seeding) ? "strict-super-seeding ": "" , (flags & strict_super_seeding) ? "strict-super-seeding ": ""
, (flags & seed_mode) ? "seed-mode ": "" , (flags & seed_mode) ? "seed-mode ": ""
, (flags & time_critical) ? "time-critical ": "" , (flags & time_critical) ? "time-critical ": ""
, (flags & suggest) ? "suggest ": "" , (flags & suggest) ? "suggest ": ""
, (flags & explicit_cache) ? "explicit-cache ": ""
); );
// in case the previous run was terminated // in case the previous run was terminated
@ -89,15 +88,6 @@ void test_swarm(int flags)
if (flags & suggest) if (flags & suggest)
pack.set_int(settings_pack::suggest_mode, settings_pack::suggest_read_cache); pack.set_int(settings_pack::suggest_mode, settings_pack::suggest_read_cache);
if (flags & explicit_cache)
pack.set_bool(settings_pack::explicit_read_cache, true);
if (flags & explicit_cache)
{
pack.set_bool(settings_pack::explicit_read_cache, true);
pack.set_int(settings_pack::explicit_cache_interval, 5);
}
// this is to avoid everything finish from a single peer // this is to avoid everything finish from a single peer
// immediately. To make the swarm actually connect all // immediately. To make the swarm actually connect all
// three peers before finishing. // three peers before finishing.

View File

@ -39,7 +39,6 @@ enum test_flags_t
seed_mode = 4, seed_mode = 4,
time_critical = 8, time_critical = 8,
suggest = 16, suggest = 16,
explicit_cache = 32
}; };
void EXPORT test_swarm(int flags = 0); void EXPORT test_swarm(int flags = 0);