premiere
/
premiere-libtorrent
2
2
Fork 1
Code Issues Pull Requests Releases Wiki Activity

added support for non-compact storage layout. i.e. all the files are allocated on disk before downloading starts

This commit is contained in:
Arvid Norberg 2005-05-13 00:39:39 +00:00
parent 4de9a08166
commit 9707b6aeb0
9 changed files with 144 additions and 70 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
docs/manual.html
View File

@ -333,13 +333,15 @@ class session: public boost::noncopyable
torrent_handle add_torrent(
entry const& e
, boost::filesystem::path const& save_path
, entry const& resume_data = entry());
, entry const& resume_data = entry()
, bool compact_mode = true);
torrent_handle add_torrent(
char const* tracker_url
, sha1_hash const& info_hash
, boost::filesystem::path const& save_path
, entry const& resume_data = entry());
, entry const& resume_data = entry()
, bool compact_mode = true);
void remove_torrent(torrent_handle const& h);
@ -403,12 +405,14 @@ timeout can be set with <tt class="docutils literal"><span class="pre">set_http_
torrent_handle add_torrent(
entry const&amp; e
, boost::filesystem::path const&amp; save_path
, entry const&amp; resume_data = entry());
, entry const&amp; resume_data = entry()
, bool compact_mode = true);
torrent_handle add_torrent(
char const* tracker_url
, sha1_hash const&amp; info_hash
, boost::filesystem::path const&amp; save_path
, entry const&amp; resume_data = entry());
, entry const&amp; resume_data = entry()
, bool compact_mode = true);
</pre>
</blockquote>
<p>You add torrents through the <tt class="docutils literal"><span class="pre">add_torrent()</span></tt> function where you give an
@ -418,9 +422,15 @@ structure in the torrent-file.</p>
<p>If the torrent you are trying to add already exists in the session (is either queued
for checking, being checked or downloading) <tt class="docutils literal"><span class="pre">add_torrent()</span></tt> will throw
<a class="reference" href="#duplicate-torrent">duplicate_torrent</a> which derives from <tt class="docutils literal"><span class="pre">std::exception</span></tt>.</p>
<p>The optional last parameter, <tt class="docutils literal"><span class="pre">resume_data</span></tt> can be given if up to date fast-resume data
<p>The optional parameter, <tt class="docutils literal"><span class="pre">resume_data</span></tt> can be given if up to date fast-resume data
is available. The fast-resume data can be acquired from a running torrent by calling
<tt class="docutils literal"><span class="pre">torrent_handle::write_resume_data()</span></tt>. See <a class="reference" href="#fast-resume">fast resume</a>.</p>
<p>The <tt class="docutils literal"><span class="pre">compact_mode</span></tt> paramater refers to the layout of the storage for this torrent. If
set to true (default), the storage will grow as more pieces are downloaded, and pieces
are rearranged to finally be in their correct places once the entire torrent has been
downloaded. If it is false, the entire storage is allocated before download begins. I.e.
the files contained in the torrent are filled with zeroes, and each downloaded piece
is put in its final place directly when downloaded.</p>
<p>The <a class="reference" href="#torrent-handle">torrent_handle</a> returned by <tt class="docutils literal"><span class="pre">add_torrent()</span></tt> can be used to retrieve information
about the torrent's progress, its peers etc. It is also used to abort a torrent.</p>
<p>The second overload that takes a tracker url and an info-hash instead of metadata (<tt class="docutils literal"><span class="pre">entry</span></tt>)
@ -742,6 +752,8 @@ reverse_file_iterator rend_files() const;
in the torrent, you can use <tt class="docutils literal"><span class="pre">begin_files()</span></tt>, <tt class="docutils literal"><span class="pre">end_files()</span></tt>,
<tt class="docutils literal"><span class="pre">rbegin_files()</span></tt> and <tt class="docutils literal"><span class="pre">rend_files()</span></tt>. These will give you standard vector
iterators with the type <tt class="docutils literal"><span class="pre">file_entry</span></tt>.</p>
<p>The <tt class="docutils literal"><span class="pre">path</span></tt> is the full (relative) path of each file. i.e. if it is a multi-file
torrent, all the files starts with a directory with the same name as <tt class="docutils literal"><span class="pre">torrent_info::name()</span></tt>.</p>
<pre class="literal-block">
struct file_entry
{

21
docs/manual.rst
View File

@ -217,13 +217,15 @@ The ``session`` class has the following synopsis::
torrent_handle add_torrent(
entry const& e
, boost::filesystem::path const& save_path
, entry const& resume_data = entry());
, entry const& resume_data = entry()
, bool compact_mode = true);
torrent_handle add_torrent(
char const* tracker_url
, sha1_hash const& info_hash
, boost::filesystem::path const& save_path
, entry const& resume_data = entry());
, entry const& resume_data = entry()
, bool compact_mode = true);
void remove_torrent(torrent_handle const& h);
@ -291,12 +293,14 @@ add_torrent()
torrent_handle add_torrent(
entry const& e
, boost::filesystem::path const& save_path
, entry const& resume_data = entry());
, entry const& resume_data = entry()
, bool compact_mode = true);
torrent_handle add_torrent(
char const* tracker_url
, sha1_hash const& info_hash
, boost::filesystem::path const& save_path
, entry const& resume_data = entry());
, entry const& resume_data = entry()
, bool compact_mode = true);
You add torrents through the ``add_torrent()`` function where you give an
object representing the information found in the torrent file and the path where you
@ -307,10 +311,17 @@ If the torrent you are trying to add already exists in the session (is either qu
for checking, being checked or downloading) ``add_torrent()`` will throw
duplicate_torrent_ which derives from ``std::exception``.
The optional last parameter, ``resume_data`` can be given if up to date fast-resume data
The optional parameter, ``resume_data`` can be given if up to date fast-resume data
is available. The fast-resume data can be acquired from a running torrent by calling
``torrent_handle::write_resume_data()``. See `fast resume`_.
The ``compact_mode`` paramater refers to the layout of the storage for this torrent. If
set to true (default), the storage will grow as more pieces are downloaded, and pieces
are rearranged to finally be in their correct places once the entire torrent has been
downloaded. If it is false, the entire storage is allocated before download begins. I.e.
the files contained in the torrent are filled with zeroes, and each downloaded piece
is put in its final place directly when downloaded.
The torrent_handle_ returned by ``add_torrent()`` can be used to retrieve information
about the torrent's progress, its peers etc. It is also used to abort a torrent.

6
include/libtorrent/session.hpp
View File

@ -311,13 +311,15 @@ namespace libtorrent
torrent_handle add_torrent(
entry const& metadata
, boost::filesystem::path const& save_path
, entry const& resume_data = entry());
, entry const& resume_data = entry()
, bool compact_mode = true);
torrent_handle add_torrent(
char const* tracker_url
, sha1_hash const& info_hash
, boost::filesystem::path const& save_path
, entry const& resume_data = entry());
, entry const& resume_data = entry()
, bool compact_mode = true);
session_status status() const;

7
include/libtorrent/storage.hpp
View File

@ -117,14 +117,15 @@ namespace libtorrent
piece_manager(
const torrent_info& info
, const boost::filesystem::path& path);
, const boost::filesystem::path& path);
~piece_manager();
void check_pieces(
boost::mutex& mutex
, detail::piece_checker_data& data
, std::vector<bool>& pieces);
, detail::piece_checker_data& data
, std::vector<bool>& pieces
, bool compact_mode);
void release_files();

9
include/libtorrent/torrent.hpp
View File

@ -94,7 +94,8 @@ namespace libtorrent
detail::session_impl& ses
, entry const& metadata
, boost::filesystem::path const& save_path
, address const& net_interface);
, address const& net_interface
, bool compact_mode);
// used with metadata-less torrents
// (the metadata is downloaded from the peers)
@ -103,7 +104,8 @@ namespace libtorrent
, char const* tracker_url
, sha1_hash const& info_hash
, boost::filesystem::path const& save_path
, address const& net_interface);
, address const& net_interface
, bool compact_mode);
~torrent();
@ -493,6 +495,9 @@ namespace libtorrent
std::vector<int> m_requested_metadata;
boost::filesystem::path m_save_path;
// determines the storage state for this torrent.
const bool m_compact_mode;
};
inline boost::posix_time::ptime torrent::next_announce() const

32
src/piece_picker.cpp
View File

@ -90,8 +90,7 @@ namespace libtorrent
- std::accumulate(pieces.begin(), pieces.end(), 0));
for (std::vector<bool>::const_iterator i = pieces.begin();
i != pieces.end();
++i)
i != pieces.end(); ++i)
{
if (*i) continue;
int index = static_cast<int>(i - pieces.begin());
@ -103,8 +102,7 @@ namespace libtorrent
// add the pieces to the piece_picker
for (std::vector<int>::iterator i = piece_list.begin();
i != piece_list.end();
++i)
i != piece_list.end(); ++i)
{
int index = *i;
assert(index >= 0);
@ -124,9 +122,7 @@ namespace libtorrent
if (!unfinished.empty())
{
for (std::vector<downloading_piece>::const_iterator i
= unfinished.begin();
i != unfinished.end();
++i)
= unfinished.begin(); i != unfinished.end(); ++i)
{
address peer;
for (int j = 0; j < m_blocks_per_piece; ++j)
@ -151,8 +147,7 @@ namespace libtorrent
assert((int)m_piece_map.size() == t->torrent_file().num_pieces());
for (std::vector<piece_pos>::const_iterator i = m_piece_map.begin();
i != m_piece_map.end();
++i)
i != m_piece_map.end(); ++i)
{
int index = static_cast<int>(i - m_piece_map.begin());
@ -160,8 +155,7 @@ namespace libtorrent
{
int actual_peer_count = 0;
for (torrent::const_peer_iterator peer = t->begin();
peer != t->end();
++peer)
peer != t->end(); ++peer)
{
if (peer->second->has_piece(index)) actual_peer_count++;
}
@ -199,12 +193,10 @@ namespace libtorrent
// with this index. (there shouldn't
// be since the piece_map is 0xfffff)
for (std::vector<std::vector<int> >::const_iterator i = m_piece_info.begin();
i != m_piece_info.end();
++i)
i != m_piece_info.end(); ++i)
{
for (std::vector<int>::const_iterator j= i->begin();
j != i->end();
++j)
j != i->end(); ++j)
{
assert(*j != index);
}
@ -215,8 +207,7 @@ namespace libtorrent
++i)
{
for (std::vector<int>::const_iterator j = i->begin();
j != i->end();
++j)
j != i->end(); ++j)
{
assert(*j != index);
}
@ -489,8 +480,7 @@ namespace libtorrent
assert(num_blocks > 0);
for (std::vector<int>::const_iterator i = piece_list.begin();
i != piece_list.end();
++i)
i != piece_list.end(); ++i)
{
assert(*i >= 0);
assert(*i < (int)m_piece_map.size());
@ -771,8 +761,7 @@ namespace libtorrent
{
int counter = 0;
for (std::vector<downloading_piece>::const_iterator i = m_downloads.begin();
i != m_downloads.end();
++i)
i != m_downloads.end(); ++i)
{
counter += (int)i->finished_blocks.count();
}
@ -780,3 +769,4 @@ namespace libtorrent
}
}

11
src/session.cpp
View File

@ -934,7 +934,8 @@ namespace libtorrent
torrent_handle session::add_torrent(
entry const& metadata
, boost::filesystem::path const& save_path
, entry const& resume_data)
, entry const& resume_data
, bool compact_mode)
{
assert(!save_path.empty());
torrent_info ti(metadata);
@ -962,7 +963,8 @@ namespace libtorrent
// the checker thread and store it before starting
// the thread
boost::shared_ptr<torrent> torrent_ptr(
new torrent(m_impl, metadata, save_path, m_impl.m_listen_interface));
new torrent(m_impl, metadata, save_path, m_impl.m_listen_interface
, compact_mode));
detail::piece_checker_data d;
d.torrent_ptr = torrent_ptr;
@ -983,7 +985,8 @@ namespace libtorrent
char const* tracker_url
, sha1_hash const& info_hash
, boost::filesystem::path const& save_path
, entry const&)
, entry const&
, bool compact_mode)
{
// TODO: support resume data in this case
assert(!save_path.empty());
@ -1012,7 +1015,7 @@ namespace libtorrent
// the thread
boost::shared_ptr<torrent> torrent_ptr(
new torrent(m_impl, tracker_url, info_hash, save_path
, m_impl.m_listen_interface));
, m_impl.m_listen_interface, compact_mode));
m_impl.m_torrents.insert(
std::make_pair(info_hash, torrent_ptr)).first;

96
src/storage.cpp
View File

@ -557,12 +557,13 @@ namespace libtorrent
impl(
const torrent_info& info
, const path& path);
, const path& path);
void check_pieces(
boost::mutex& mutex
, detail::piece_checker_data& data
, std::vector<bool>& pieces);
, detail::piece_checker_data& data
, std::vector<bool>& pieces
, bool compact_mode);
void release_files();
@ -603,9 +604,13 @@ namespace libtorrent
void export_piece_map(std::vector<int>& p) const;
private:
void post_check(boost::mutex& mutex
, detail::piece_checker_data& data);
// returns the slot currently associated with the given
// piece or assigns the given piece_index to a free slot
int identify_data(
const std::vector<char>& piece_data
, int current_slot
@ -621,6 +626,17 @@ namespace libtorrent
#endif
storage m_storage;
// if this is true, pieces are always allocated at the
// lowest possible slot index. If it is false, pieces
// are always written to their final place immediately
bool m_compact_mode;
// if this is true, pieces that haven't been downloaded
// will be filled with zeroes. Not filling with zeroes
// will not work in some cases (where a seek cannot pass
// the end of the file).
bool m_fill_mode;
// a bitmask representing the pieces we have
std::vector<bool> m_have_piece;
@ -663,6 +679,8 @@ namespace libtorrent
const torrent_info& info
, const path& save_path)
: m_storage(info, save_path)
, m_compact_mode(false)
, m_fill_mode(true)
, m_info(info)
, m_save_path(complete(save_path))
, m_allocating(false)
@ -703,16 +721,14 @@ namespace libtorrent
p.clear();
std::vector<int>::const_reverse_iterator last;
for (last = m_slot_to_piece.rbegin();
last != m_slot_to_piece.rend();
++last)
last != m_slot_to_piece.rend(); ++last)
{
if (*last != unallocated) break;
}
for (std::vector<int>::const_iterator i =
m_slot_to_piece.begin();
i != last.base();
++i)
i != last.base(); ++i)
{
p.push_back(*i);
}
@ -970,8 +986,7 @@ namespace libtorrent
// already been assigned
int free_piece = unassigned;
for (std::vector<int>::iterator i = matching_pieces.begin();
i != matching_pieces.end();
++i)
i != matching_pieces.end(); ++i)
{
if (have_pieces[*i]) continue;
free_piece = *i;
@ -993,10 +1008,33 @@ namespace libtorrent
}
}
void piece_manager::impl::post_check(boost::mutex& mutex
, detail::piece_checker_data& data)
{
if (!m_compact_mode)
{
// if we're not in compact mode, make sure the
// pieces are spread out and placed at their
// final position.
int num_slots = (int)m_unallocated_slots.size();
for (int i = 0; i < num_slots; ++i)
{
allocate_slots(1);
boost::mutex::scoped_lock lock(mutex);
data.progress = (float)i / num_slots;
if (data.abort || (data.torrent_ptr && data.torrent_ptr->is_aborted()))
return;
}
}
}
void piece_manager::impl::check_pieces(
boost::mutex& mutex
, detail::piece_checker_data& data
, std::vector<bool>& pieces)
, detail::piece_checker_data& data
, std::vector<bool>& pieces
, bool compact_mode)
{
assert(m_info.piece_length() > 0);
// synchronization ------------------------------------------------------
@ -1005,6 +1043,8 @@ namespace libtorrent
INVARIANT_CHECK;
m_compact_mode = compact_mode;
// This will corrupt the storage
// use while debugging to find
// states that cannot be scanned
@ -1058,6 +1098,9 @@ namespace libtorrent
{
m_unallocated_slots.push_back(i);
}
post_check(mutex, data);
return;
}
@ -1309,15 +1352,19 @@ namespace libtorrent
return;
}
}
post_check(mutex, data);
// TODO: sort m_free_slots and m_unallocated_slots?
}
void piece_manager::check_pieces(
boost::mutex& mutex
, detail::piece_checker_data& data
, std::vector<bool>& pieces)
, detail::piece_checker_data& data
, std::vector<bool>& pieces
, bool compact_mode)
{
m_pimpl->check_pieces(mutex, data, pieces);
m_pimpl->check_pieces(mutex, data, pieces, compact_mode);
}
int piece_manager::impl::allocate_slot_for_piece(int piece_index)
@ -1491,29 +1538,30 @@ namespace libtorrent
const int piece_size = static_cast<int>(m_info.piece_length());
std::vector<char> zeros(piece_size, 0);
std::vector<char> buffer(piece_size, 0);
for (int i = 0;
i < num_slots && !m_unallocated_slots.empty();
++i)
for (int i = 0; i < num_slots && !m_unallocated_slots.empty(); ++i)
{
int pos = m_unallocated_slots.front();
// int piece_pos = pos;
bool write_back = false;
int new_free_slot = pos;
if (m_piece_to_slot[pos] != has_no_slot)
{
assert(m_piece_to_slot[pos] >= 0);
m_storage.read(&zeros[0], m_piece_to_slot[pos], 0, static_cast<int>(m_info.piece_size(pos)));
m_storage.read(&buffer[0], m_piece_to_slot[pos], 0, static_cast<int>(m_info.piece_size(pos)));
new_free_slot = m_piece_to_slot[pos];
m_slot_to_piece[pos] = pos;
m_piece_to_slot[pos] = pos;
write_back = true;
}
m_unallocated_slots.erase(m_unallocated_slots.begin());
m_slot_to_piece[new_free_slot] = unassigned;
m_free_slots.push_back(new_free_slot);
m_storage.write(&zeros[0], pos, 0, static_cast<int>(m_info.piece_size(pos)));
if (write_back || m_fill_mode)
m_storage.write(&buffer[0], pos, 0, static_cast<int>(m_info.piece_size(pos)));
}
assert(m_free_slots.size() > 0);
@ -1546,8 +1594,7 @@ namespace libtorrent
assert((int)m_slot_to_piece.size() == m_info.num_pieces());
for (std::vector<int>::const_iterator i = m_free_slots.begin();
i != m_free_slots.end();
++i)
i != m_free_slots.end(); ++i)
{
assert(*i < (int)m_slot_to_piece.size());
assert(*i >= 0);
@ -1555,8 +1602,7 @@ namespace libtorrent
}
for (std::vector<int>::const_iterator i = m_unallocated_slots.begin();
i != m_unallocated_slots.end();
++i)
i != m_unallocated_slots.end(); ++i)
{
assert(*i < (int)m_slot_to_piece.size());
assert(*i >= 0);

10
src/torrent.cpp
View File

@ -150,7 +150,8 @@ namespace libtorrent
detail::session_impl& ses
, entry const& metadata
, boost::filesystem::path const& save_path
, address const& net_interface)
, address const& net_interface
, bool compact_mode)
: m_torrent_file(metadata)
, m_abort(false)
, m_paused(false)
@ -179,6 +180,7 @@ namespace libtorrent
, m_upload_bandwidth_limit(std::numeric_limits<int>::max())
, m_download_bandwidth_limit(std::numeric_limits<int>::max())
, m_save_path(complete(save_path))
, m_compact_mode(compact_mode)
{
m_uploads_quota.min = 2;
m_connections_quota.min = 2;
@ -198,7 +200,8 @@ namespace libtorrent
, char const* tracker_url
, sha1_hash const& info_hash
, boost::filesystem::path const& save_path
, address const& net_interface)
, address const& net_interface
, bool compact_mode)
: m_torrent_file(info_hash)
, m_abort(false)
, m_paused(false)
@ -226,6 +229,7 @@ namespace libtorrent
, m_upload_bandwidth_limit(std::numeric_limits<int>::max())
, m_download_bandwidth_limit(std::numeric_limits<int>::max())
, m_save_path(complete(save_path))
, m_compact_mode(compact_mode)
{
m_uploads_quota.min = 2;
m_connections_quota.min = 2;
@ -760,7 +764,7 @@ namespace libtorrent
boost::mutex& mutex)
{
assert(m_storage.get());
m_storage->check_pieces(mutex, data, m_have_pieces);
m_storage->check_pieces(mutex, data, m_have_pieces, m_compact_mode);
m_num_pieces = std::accumulate(
m_have_pieces.begin()
, m_have_pieces.end()