Storage

Author: Arvid Norberg, arvid@rasterbar.com
Version: 1.0.0

Table of contents

file_entry

Declared in "libtorrent/file_storage.hpp"

information about a file in a file_storage

struct file_entry
{
   std::string path;
   std::string symlink_path;
   size_type offset;
   size_type size;
   size_type file_base;
   std::time_t mtime;
   sha1_hash filehash;
   bool pad_file:1;
   bool hidden_attribute:1;
   bool executable_attribute:1;
   bool symlink_attribute:1;
};
path
the full path of this file. The paths are unicode strings encoded in UTF-8.
symlink_path
the path which this is a symlink to, or empty if this is not a symlink. This field is only used if the symlink_attribute is set.
offset
the offset of this file inside the torrent
size
the size of the file (in bytes) and offset is the byte offset of the file within the torrent. i.e. the sum of all the sizes of the files before it in the list.
file_base
the offset in the file where the storage should start. The normal case is to have this set to 0, so that the storage starts saving data at the start if the file. In cases where multiple files are mapped into the same file though, the file_base should be set to an offset so that the different regions do not overlap. This is used when mapping "unselected" files into a so-called part file.
mtime
the modification time of this file specified in posix time.
filehash
a sha-1 hash of the content of the file, or zeroes, if no file hash was present in the torrent file. It can be used to potentially find alternative sources for the file.
pad_file
set to true for files that are not part of the data of the torrent. They are just there to make sure the next file is aligned to a particular byte offset or piece boundry. These files should typically be hidden from an end user. They are not written to disk.
hidden_attribute
true if the file was marked as hidden (on windows).
executable_attribute
true if the file was marked as executable (posix)
symlink_attribute
true if the file was a symlink. If this is the case the symlink_index refers to a string which specifies the original location where the data for this file was found.

file_slice

Declared in "libtorrent/file_storage.hpp"

represents a window of a file in a torrent.

The file_index refers to the index of the file (in the torrent_info). To get the path and filename, use file_at() and give the file_index as argument. The offset is the byte offset in the file where the range starts, and size is the number of bytes this range is. The size + offset will never be greater than the file size.

struct file_slice
{
   int file_index;
   size_type offset;
   size_type size;
};
file_index
the index of the file
offset
the offset from the start of the file, in bytes
size
the size of the window, in bytes

file_storage

Declared in "libtorrent/file_storage.hpp"

The file_storage class represents a file list and the piece size. Everything necessary to interpret a regular bittorrent storage file structure.

class file_storage
{
   bool is_valid () const;
   void reserve (int num_files);
   void add_file (std::string const& p, size_type size, int flags = 0
      , std::time_t mtime = 0, std::string const& s_p = "");
   void add_file (file_entry const& e, char const* filehash = 0);
   void rename_file (int index, std::string const& new_filename);
   void rename_file_borrow (int index, char const* new_filename, int len);
   std::vector<file_slice> map_block (int piece, size_type offset
      , int size) const;
   peer_request map_file (int file, size_type offset, int size) const;
   int num_files () const;
   file_entry at (int index) const;
   size_type total_size () const;
   void set_num_pieces (int n);
   int num_pieces () const;
   void set_piece_length (int l);
   int piece_length () const;
   int piece_size (int index) const;
   void set_name (std::string const& n);
   const std::string& name () const;
   void swap (file_storage& ti);
   void optimize (int pad_file_limit = -1, int alignment = -1);
   size_type file_size (int index) const;
   sha1_hash hash (int index) const;
   std::string file_name (int index) const;
   size_type file_offset (int index) const;
   time_t mtime (int index) const;
   bool pad_file_at (int index) const;
   std::string const& symlink (int index) const;
   std::string file_path (int index, std::string const& save_path = "") const;
   int file_flags (int index) const;
   void set_file_base (int index, size_type off);
   size_type file_base (int index) const;
   int file_index_at_offset (size_type offset) const;
   char const* file_name_ptr (int index) const;
   int file_name_len (int index) const;

   enum flags_t
   {
      pad_file,
      attribute_hidden,
      attribute_executable,
      attribute_symlink,
   };

   enum file_flags_t
   {
      flag_pad_file,
      flag_hidden,
      flag_executable,
      flag_symlink,
   };
};

is_valid()

bool is_valid () const;

returns true if the piece length has been initialized on the file_storage. This is typically taken as a proxy of whether the file_storage as a whole is initialized or not.

reserve()

void reserve (int num_files);

allocates space for num_files in the internal file list. This can be used to avoid reallocating the internal file list when the number of files to be added is known up-front.

add_file()

void add_file (std::string const& p, size_type size, int flags = 0
      , std::time_t mtime = 0, std::string const& s_p = "");
void add_file (file_entry const& e, char const* filehash = 0);

Adds a file to the file storage. The flags argument sets attributes on the file. The file attributes is an extension and may not work in all bittorrent clients.

For possible file attributes, see file_storage::flags_t.

If more files than one are added, certain restrictions to their paths apply. In a multi-file file storage (torrent), all files must share the same root directory.

That is, the first path element of all files must be the same. This shared path element is also set to the name of the torrent. It can be changed by calling set_name.

The built in functions to traverse a directory to add files will make sure this requirement is fulfilled.

rename_file()

void rename_file (int index, std::string const& new_filename);

renames the file at index to new_filename. Keep in mind that filenames are expected to be UTF-8 encoded.

rename_file_borrow()

void rename_file_borrow (int index, char const* new_filename, int len);

this is a low-level function that sets the name of a file by making it reference a buffer that is not owned by the file_storage. it's an optimization used when loading .torrent files, to not duplicate names in memory.

map_block()

std::vector<file_slice> map_block (int piece, size_type offset
      , int size) const;

returns a list of file_slice objects representing the portions of files the specified piece index, byte offset and size range overlaps. this is the inverse mapping of map_file().

map_file()

peer_request map_file (int file, size_type offset, int size) const;

returns a peer_request representing the piece index, byte offset and size the specified file range overlaps. This is the inverse mapping ove map_block().

num_files()

int num_files () const;

returns the number of files in the file_storage

at()

file_entry at (int index) const;

returns a file_entry with information about the file at index. Index must be in the range [0, num_files() ).

total_size()

size_type total_size () const;

returns the total number of bytes all the files in this torrent spans

num_pieces() set_num_pieces()

void set_num_pieces (int n);
int num_pieces () const;

set and get the number of pieces in the torrent

piece_length() set_piece_length()

void set_piece_length (int l);
int piece_length () const;

set and get the size of each piece in this torrent. This size is typically an even power of 2. It doesn't have to be though. It should be divisible by 16kiB however.

piece_size()

int piece_size (int index) const;

returns the piece size of index. This will be the same as piece_length(), except for the last piece, which may be shorter.

set_name() name()

void set_name (std::string const& n);
const std::string& name () const;

set and get the name of this torrent. For multi-file torrents, this is also the name of the root directory all the files are stored in.

swap()

void swap (file_storage& ti);

swap all content of this with ti.

optimize()

void optimize (int pad_file_limit = -1, int alignment = -1);

if pad_file_limit >= 0, files larger than that limit will be padded, default is to not add any padding (-1). The alignment specifies the alignment files should be padded to. This defaults to the piece size (-1) but it may also make sense to set it to 16 kiB, or something divisible by 16 kiB. If pad_file_limit is 0, every file will be padded (except empty ones).

file_flags()

int file_flags (int index) const;

returns a bitmask of flags from file_flags_t that apply to file at index.

file_base() set_file_base()

void set_file_base (int index, size_type off);
size_type file_base (int index) const;

The file base of a file is the offset within the file on the filsystem where it starts to write. For the most part, this is always 0. It's possible to map several files (in the torrent) into a single file on the filesystem by making them all point to the same filename, but with different file bases, so that they don't overlap. torrent_info::remap_files() can be used to use a new file layout.

file_index_at_offset()

int file_index_at_offset (size_type offset) const;

returns the index of the file at the given offset in the torrent

file_name_len() file_name_ptr()

char const* file_name_ptr (int index) const;
int file_name_len (int index) const;

low-level function. returns a pointer to the internal storage for the filename. This string may not be null terinated! the file_name_len() function returns the length of the filename.

enum flags_t

Declared in "libtorrent/file_storage.hpp"

name value description
pad_file 1 the file is a pad file. It's required to contain zeroes at it will not be saved to disk. Its purpose is to make the following file start on a piece boundary.
attribute_hidden 2 this file has the hidden attribute set. This is primarily a windows attribute
attribute_executable 4 this file has the executable attribute set.
attribute_symlink 8 this file is a symbilic link. It should have a link target string associated with it.

enum file_flags_t

Declared in "libtorrent/file_storage.hpp"

name value description
flag_pad_file 1 this file is a pad file. The creator of the torrent promises the file is entirely filled with zeroes and does not need to be downloaded. The purpose is just to align the next file to either a block or piece boundary.
flag_hidden 2 this file is hiddent (sets the hidden attribute on windows)
flag_executable 4 this file is executable (sets the executable bit on posix like systems)
flag_symlink 8 this file is a symlink. The symlink target is specified in a separate field

default_storage_constructor()

Declared in "libtorrent/storage_defs.hpp"

storage_interface* default_storage_constructor (
   file_storage const&, file_storage const* mapped, std::string const&, file_pool&
   , std::vector<boost::uint8_t> const&);

the constructor function for the regular file storage. This is the default value for add_torrent_params::storage.

disabled_storage_constructor()

Declared in "libtorrent/storage_defs.hpp"

storage_interface* disabled_storage_constructor (
   file_storage const&, file_storage const* mapped, std::string const&, file_pool&
   , std::vector<boost::uint8_t> const&);

the constructor function for the disabled storage. This can be used for testing and benchmarking. It will throw away any data written to it and return garbage for anything read from it.

enum storage_mode_t

Declared in "libtorrent/storage_defs.hpp"

name value description
storage_mode_allocate 0 All pieces will be written to their final position, all files will be allocated in full when the torrent is first started. This is done with fallocate() and similar calls. This mode minimizes fragmentation.
storage_mode_sparse 1 All pieces will be written to the place where they belong and sparse files will be used. This is the recommended, and default mode.