Commit Graph

305 Commits

Author SHA1 Message Date
Arvid Norberg 9edb348093 read time is only cache misses 2010-03-04 03:49:06 +00:00
Arvid Norberg a942890e8d added more detailed instrumentation of the disk I/O thread 2010-03-03 07:09:04 +00:00
Arvid Norberg fdb68ab784 attempt to not make m_buffer_queue_size to ever get negative 2010-03-03 04:32:06 +00:00
Arvid Norberg 053d11ea3e fixed mutex lock typo 2010-02-24 00:04:28 +00:00
Arvid Norberg 278c1f91c0 fix typo in previous check-in 2010-02-21 09:03:56 +00:00
Arvid Norberg 59deff7a44 fixed issue with disk read cache not being cleared when removing torrents 2010-02-21 08:52:26 +00:00
Arvid Norberg c263f8daf7 volatile cache improvement 2010-02-06 17:56:58 +00:00
Arvid Norberg 7b0555ae45 fixed disk cache bug 2010-02-05 04:30:57 +00:00
Arvid Norberg c1b8124d20 volatile cache tuning 2010-02-01 01:11:10 +00:00
Arvid Norberg 238f7e38f5 improve disk stats logging to include cache hits 2010-01-31 21:13:52 +00:00
Arvid Norberg ca20d2d600 fixed broken build 2010-01-31 20:22:35 +00:00
Arvid Norberg 5b747f58f7 add guided read cache and update tuning docs 2010-01-31 19:14:00 +00:00
Arvid Norberg 53144f6f14 fixes to the new disk cache and the volatile cache feature 2010-01-31 16:29:52 +00:00
Arvid Norberg ccf8b0278a added volatile read cache option 2010-01-30 03:50:17 +00:00
Arvid Norberg f6c0d5af35 optimized disk cache to work with large caches 2010-01-27 04:25:45 +00:00
Arvid Norberg a046850130 support disk I/O priority setting 2010-01-23 03:02:32 +00:00
Arvid Norberg c9b594fde1 support sending suggest messages based on what's in the read cache. support 'explicit read cache' 2010-01-15 16:45:42 +00:00
Arvid Norberg 2948b3b375 fixed file checking bug introduced by fancy disk IO optimizations 2010-01-13 23:37:23 +00:00
Arvid Norberg a1299c3a10 optimize disk I/O elevator algorithm to spend less time picking job 2010-01-12 01:56:48 +00:00
Arvid Norberg d498c129ab clear sparse flag on files that complete on windows 2010-01-09 18:40:05 +00:00
Arvid Norberg ecb770eca7 always unlock while waiting for disk-IO 2009-12-25 16:13:35 +00:00
Arvid Norberg 6298322b1d service disk-read requests that hit the cache immediately and circumvent the elevator ordering 2009-12-16 10:49:15 +00:00
Arvid Norberg 1f5859d31f error_code fix. added is_error_code_enum template specialization and using the abbreviated syntax to build error_codes 2009-11-29 07:06:38 +00:00
Arvid Norberg ab4b7f99ad fixed error handling in read_into_piece 2009-11-28 03:14:08 +00:00
Arvid Norberg a5586d154d more header dependency optimizations 2009-11-25 06:55:34 +00:00
Arvid Norberg 6343fe6a6d optimized header dependencies to lower build time 2009-11-23 08:38:50 +00:00
Arvid Norberg 562e2512b9 some vs2010 fixes 2009-11-22 23:55:54 +00:00
Arvid Norberg c418165e07 replaced boost.filesystem with custom functions (improves efficiency and drops unnecessary dependencies and improves libtorrent portability) 2009-10-26 01:29:39 +00:00
Arvid Norberg 8a5b7d5d36 got rid of the last recursive mutexes. abstracted the threading primitives (and switched over to use asio's internal ones). 2009-10-20 02:49:56 +00:00
Arvid Norberg 6d01cb475c msvc build fixes 2009-09-06 00:57:01 +00:00
Arvid Norberg bec481acdf improved disk read performance and fixed bug in storage_interface's backwards compatibility functions 2009-09-05 07:21:10 +00:00
Arvid Norberg 49f54a6f7f file alignment and disk_io_thread bug fixes 2009-09-02 16:42:33 +00:00
Arvid Norberg 7933132cd9 fixed assert when built with disk stats 2009-08-05 19:02:51 +00:00
Arvid Norberg 63368c1383 added dummy storage class to disable disk I/O and an option to disable hash checking (for performance testing and simulation) 2009-08-02 06:40:45 +00:00
Arvid Norberg 838a246f6b disk stats fix 2009-08-01 22:47:07 +00:00
Arvid Norberg 73a8673cfc queued disk bytes fix 2009-07-26 19:54:16 +00:00
Arvid Norberg ecdd7231b8 removed redundant code in disk_io_thread 2009-07-21 04:32:27 +00:00
Arvid Norberg e9bd8f6e1f turned fast-resume errors into error_codes. fixed typo in documentation 2009-06-28 00:36:41 +00:00
Arvid Norberg 306b75e6b1 boost 1.35 fixes 2009-06-19 16:42:33 +00:00
Arvid Norberg 3df15ad39e fixed memory leak in disk io thread when not using the cache 2009-06-14 22:48:07 +00:00
Arvid Norberg 62a7946968 disk stats fix 2009-06-14 22:20:23 +00:00
Arvid Norberg f0134c1b11 improved disk error handling and expanded use of error_code in error reporting 2009-06-10 08:30:55 +00:00
Arvid Norberg e438a3ccfc more boost-1.34.1 fixes 2009-06-03 07:46:50 +00:00
Arvid Norberg a9a12a2391 fix build with boost-1.34.1 2009-06-03 07:22:43 +00:00
Arvid Norberg bd33814376 improve disk error handling 2009-05-31 22:41:53 +00:00
Arvid Norberg 0283f07aed more buffer categories to help debugging 2009-05-31 22:38:49 +00:00
Arvid Norberg b2c5d8a563 include the piece index that failed in disk errors 2009-05-31 19:44:56 +00:00
Arvid Norberg 3e7bd46706 fixed handling of unaligned block requests 2009-05-31 19:33:54 +00:00
Arvid Norberg d45e6cb118 fix bug in disk_io_thread 2009-05-31 10:15:45 +00:00
Arvid Norberg bc83387583 fix uninitialized variable in disk_io_thread 2009-05-30 04:02:03 +00:00
Arvid Norberg 3ea52b240f fixed memory leak in disk cache 2009-05-28 01:19:48 +00:00
Arvid Norberg 065bd283cf timer fixes 2009-05-27 17:06:50 +00:00
Arvid Norberg c66dda1c9f fixed bug in disk cache flush, largest contiguous function 2009-05-27 16:50:46 +00:00
Arvid Norberg 2f627a4b00 attempt to mitigate read cache thrashing 2009-05-24 16:49:01 +00:00
Arvid Norberg 29d56f9a8b improved read cache memory utilization 2009-05-24 15:32:14 +00:00
Arvid Norberg a89d6cced1 clean up some disk cache duplication of logic and added another memory usage optimization for the write cache 2009-05-24 00:12:53 +00:00
Arvid Norberg ae8488fb91 improved read cache memory efficiency 2009-05-23 19:27:27 +00:00
Arvid Norberg e322b318ae improved handling of out-of-memory conditions in disk I/O thread 2009-05-23 18:39:55 +00:00
Arvid Norberg e197f5cb85 disk_io_thread assert fixes 2009-05-23 17:17:47 +00:00
Arvid Norberg f9c12ef589 added more disk io asserts 2009-05-23 15:50:38 +00:00
Arvid Norberg c1d9198dc3 added largest_contiguous cache flush algorithm 2009-05-23 07:35:45 +00:00
Arvid Norberg 1392b14fec fixed disk_io error handling bugs 2009-05-23 03:05:21 +00:00
Arvid Norberg e36ea4ba5d added instrumentation for disk access. Useful to benchmark disk cache performance 2009-05-23 03:03:52 +00:00
Arvid Norberg 0127edac14 added disk-checking delay, to leave room for other disk operations while checking a torrent 2009-05-22 06:32:39 +00:00
Arvid Norberg cb9c3cb37d memory optimization for checking torrents 2009-05-21 16:15:05 +00:00
Arvid Norberg c047f66454 improved disk buffer logging and added extra asserts when logging disk buffer stats 2009-05-19 07:00:05 +00:00
Arvid Norberg 062e1796c0 fixed crash when shutting down while checking a torrent 2009-05-13 17:17:33 +00:00
Arvid Norberg 85bf463e11 added session setting to control disk buffer pool allocation sizes 2009-05-12 18:52:05 +00:00
Arvid Norberg 039ee486af use alloca for iovec arrays 2009-05-12 17:56:12 +00:00
Arvid Norberg 89e4c41ea2 fixed indentation 2009-05-11 21:19:03 +00:00
Arvid Norberg 1eb2799a03 added more allocation logging for statistics builds. Identified and fixed some memory waste 2009-05-07 20:30:20 +00:00
Arvid Norberg 838df44184 documented file_rename_failed_alert and file_renamed_alert. they were previously not documented. Fixed bug where the storage would be left in an error state (and eventually pause the torrent) when a rename_file() failed. The error is now only reported back through the alert 2009-05-07 06:41:41 +00:00
Arvid Norberg ff3db926fb allocates 10 disk buffers at a time instead of 32. Fixed bug where connection candidates would be too high 2009-05-06 07:06:26 +00:00
Arvid Norberg cf6dc377c9 define fixes for disk stats build configurations 2009-05-04 06:42:24 +00:00
Arvid Norberg 641abc6c4a slight memory optimization and some header cleanup 2009-05-03 20:21:24 +00:00
Arvid Norberg 807c1fc397 added validation check to disk_buffer_pool (and disk_io_thread in 0.14). This should catch any reference to the disk_io_thread after it has been destructed 2009-05-03 03:16:15 +00:00
Arvid Norberg e4ddb13b6f made the cache_size setting include send and receive buffers 2009-05-02 06:52:57 +00:00
Arvid Norberg 033fd5bd22 count send and receive buffers against the cache size limit 2009-05-01 08:00:58 +00:00
Arvid Norberg 509c8ed402 timed_wait does not seem to work as expected, revert to regular wait() 2009-04-15 16:32:05 +00:00
Arvid Norberg 91ba36a396 added compile time option for mlock support 2009-03-31 08:15:21 +00:00
Arvid Norberg c6017f9366 reverted bug introduced a few check-ins ago. Fixed case when recursive_copy fails (don't remove the original files). Cleaned up callback dispatching in disk_io_thread in trunk. Improved unit tests for move_storage 2009-03-31 08:05:46 +00:00
Arvid Norberg 36eb9c4f51 move storage fix 2009-03-30 04:55:23 +00:00
Arvid Norberg af10b55254 wakes up the disk io thread once a second to flush caches 2009-03-14 09:24:58 +00:00
Arvid Norberg 7ddffbac64 fixed windows build 2009-02-06 17:59:49 +00:00
Arvid Norberg 5c12db28d2 added option to lock disk cache in physical memory 2009-02-06 09:46:13 +00:00
Arvid Norberg 1ac8f806fa added seed mode support (lazy hash checking) 2009-02-03 07:46:24 +00:00
Arvid Norberg 26910eff1d update_settings fixes for disk_io_thread 2009-01-28 07:09:10 +00:00
Arvid Norberg b3a17b1787 added disk buffer instrumentation and log parser 2009-01-23 09:13:31 +00:00
Arvid Norberg 00808473e7 factor out disk_buffer_pool from disk_io_thread. refactored the file open modes to be platform independent. gave the disk_io_thread its own copy of session_settings that it shares with storage. added an unaligned_read implementation to storage. Added options to session_settings on when to open files in unbuffered mode. Added unit tests for unaligned reads 2009-01-21 07:31:49 +00:00
Arvid Norberg 113d1f3557 moved page_aligned_allocator to its own file and uses it in set_piece_hashes to not pass in unaligned buffers to storage read 2009-01-15 17:09:36 +00:00
Arvid Norberg 630601aedf use & instead of % 2009-01-15 04:25:07 +00:00
Arvid Norberg b9d7538ba8 fixed storage to generate a proper error when the read operation 2009-01-14 03:05:35 +00:00
Arvid Norberg 3f0fa60164 fixed typo preventing pool-allocators=off build 2009-01-13 06:49:40 +00:00
Arvid Norberg 7592ad4aee updated disk IO to support unbuffered files 2009-01-11 02:02:34 +00:00
Arvid Norberg cfd116ffad made disk_io_settings be synchronized through the job queue 2009-01-10 05:46:02 +00:00
Arvid Norberg f1d229aae6 first pass of replacing coalesce writes and reads by writev and readv (no windows support yet) 2009-01-03 08:11:31 +00:00
Arvid Norberg 3bf9fa239d fixed incorrect failure test in storage. Affected reading when not coalescing reads 2009-01-02 10:31:26 +00:00
Arvid Norberg 06da1f8e8a added some session logging. added more disk-io-thread stats logging and parse script 2008-12-30 08:20:25 +00:00
Arvid Norberg 28cb451eac expire read cache as well (same time out as write cache) 2008-12-27 02:38:14 +00:00
Arvid Norberg 2d4bdbca7d fixed disk_io_thread over-allocation 2008-12-03 06:42:53 +00:00
Arvid Norberg 1022ba45fc fixed bug in disk IO thread which would make it sometimes0 exit the thread if an error occurred 2008-11-29 21:38:34 +00:00
Arvid Norberg 09b3a865ef fixed compiler flags in pkg-config file and made header files not depend on the NDEBUG define 2008-11-29 21:33:21 +00:00
Arvid Norberg f503bbfe86 fix for when aborting torrents that are being checked. Fix in client_test when removing torrents that are checking. 2008-11-17 01:19:46 +00:00
Arvid Norberg 49a3ac9710 disk_io_thread shutdown fix 2008-11-11 09:32:51 +00:00
Arvid Norberg 4e9124019b added accessor to query for the max number of connections 2008-11-08 07:40:55 +00:00
Arvid Norberg 5f6351146a removed assert that could cause dead-lock in debug mode 2008-10-05 19:47:55 +00:00
Arvid Norberg 0f6ec06e12 fixed race condition in invariant check in disk io thread (only affected debug builds) 2008-09-30 01:16:19 +00:00
Arvid Norberg 26dea3c222 fixed stats miscount when deleting files 2008-09-17 02:29:05 +00:00
Arvid Norberg 658eef82a5 get rid of msvc warning 2008-09-15 13:58:38 +00:00
Arvid Norberg 8488d5eb59 fixed bug in disk_io_thread read cache management 2008-08-07 12:30:36 +00:00
Arvid Norberg 58978048e8 fixed assert in disk_io_thread 2008-07-30 08:05:04 +00:00
Arvid Norberg 21dff61cb5 clears read cache when paused. Saves memory 2008-07-18 15:31:22 +00:00
Arvid Norberg 3cb1369385 improved error reporting of file errors (uses boost.system's error_code). Now permission errors are properly reported when checking files 2008-07-17 23:41:46 +00:00
Arvid Norberg 9c9c13c7bb exposed read cache info as well as write cache 2008-07-11 10:29:26 +00:00
Arvid Norberg 9c84908bb9 switched resume data parsing over to use lazy_bdecode. Improves memory allocation performance, especially noticable when heap allocations are expensive. Makes it more practical to run with malloc debug. Changed resume data interface to take a vector as opposed to a parsed structure 2008-06-30 23:14:31 +00:00
Arvid Norberg 52f6204288 simplified the disk-io mutexes and made it more efficient. separated the disk io mutex into a buffer mutex, queue mutex and the piece cache mutex. 2008-06-12 04:40:37 +00:00
Arvid Norberg 6e69480176 disk_io_thread fixes. Properly destructs torrent objects while disk io mutex is not locked. Properly flushes disk cache when closing, and aborting read and file check jobs. Fixes #334 2008-06-09 04:46:34 +00:00
Arvid Norberg 3910fe78de separated file_storage from torrent_info and create_torrent 2008-05-28 08:44:40 +00:00
Arvid Norberg f3d0eb3c11 fixed boost-1.35 build 2008-05-01 16:31:19 +00:00
Arvid Norberg 644d3aa66c disk IO thread error handling 2008-04-13 21:26:57 +00:00
Arvid Norberg 3fea2080fd added an asynchronous save_resume_data to make it easier to synchronize with the disk IO thread 2008-04-13 18:54:36 +00:00
Arvid Norberg d4cfa126aa storage fixes 2008-04-12 22:08:07 +00:00
Arvid Norberg 093d912e9a receive buffer optimization. added receive_buffer_size and used_receive_buffer to peer_info. changed plugin api to make use of new disk_buffer_holder type 2008-04-10 10:03:23 +00:00
Arvid Norberg 5f35d170b0 disk io thread cleanup (all read operations are now read into disk buffers, no custom buffers) 2008-04-10 09:11:54 +00:00
Arvid Norberg 75ef4ec1f1 added option to not use pool allocators (useful when using memory debugging tools) 2008-04-09 05:19:11 +00:00
Arvid Norberg 4e9ed2dc96 fixed incorrect flushing of disk cache 2008-03-16 10:51:25 +00:00
Arvid Norberg e5a98d1b7e more invariant checks in disk_io_thread 2008-03-10 08:19:31 +00:00
Arvid Norberg f53cfa7eeb removed checker thread 2008-03-08 06:06:31 +00:00
Arvid Norberg c689c0b5fc replaced deque with list in disk_io_thread 2008-02-26 20:08:33 +00:00
Arvid Norberg 1d1398ed7f read cache fix 2008-02-24 23:14:10 +00:00
Arvid Norberg 19462007d1 read cache fix 2008-02-22 08:22:57 +00:00
Arvid Norberg ef9ef674d5 added read cache. Not fully configurable yet 2008-02-22 04:11:04 +00:00
Arvid Norberg 7e83c3fc51 changed storage interface to not require exceptions 2008-02-14 03:48:20 +00:00
Arvid Norberg 752149e341 fixed bug in write cache flush logic 2008-02-10 20:36:48 +00:00
Arvid Norberg 6c552db68e added disk cache expiration 2008-02-10 00:58:25 +00:00
Arvid Norberg ea7253c575 increased default cache size to 8 MB 2008-02-09 21:09:29 +00:00
Arvid Norberg 8cf0510144 added disk cache for write operations 2008-02-08 10:22:05 +00:00
Arvid Norberg f7d34794a9 improved error handling 2008-01-14 23:51:04 +00:00
Arvid Norberg e0d1951fac fixes #234. properly shuts down the disk io thread 2007-12-24 08:15:10 +00:00
Arvid Norberg 9b54043478 attempt to fix potential deadlock in disk_io_thread 2007-12-13 04:05:25 +00:00
Arvid Norberg 1a280e31fa cleanup. changed the connection map into a set of peer_connections. moved the policy object into the torrent (as opposed to being a pointer). Fixes issues with multiple peers on the same IP. Reduces some lookups. 2007-10-31 09:48:20 +00:00
Arvid Norberg a1a564ef51 fixed debug issue with disk io thread 2007-10-23 22:11:17 +00:00
Arvid Norberg b032c26bf0 adds invariant check to make sure pieces that are complete are also checked. adds a debug function in the disk_io_thread. Fixes a problem with a failing disk write that would not clear the block for re-download. Adds some extra session logging 2007-10-18 00:32:16 +00:00
Arvid Norberg 59c7f5bfbb added option to delete files from disk as a torrent is removed 2007-10-13 03:33:33 +00:00
Arvid Norberg 966800cd5b replaced the assert macro to TORRENT_ASSERT instead of hi-jacking assert. 2007-10-05 00:30:00 +00:00
Arvid Norberg f4f30f5b19 disk io priority fix 2007-10-03 22:30:40 +00:00
Arvid Norberg b48948e231 changed the double send buffers to a chained buffer using iovec send operations. buffers are recycled (saves memory and cpu) and memory copies are avoided (saves cpu). More statistics is added to track buffer usage 2007-09-29 16:14:03 +00:00
Arvid Norberg b5a2373bff added an option to generate a disk io log file 2007-09-17 08:15:54 +00:00
Arvid Norberg 9364a2d01d added priority to disk io jobs 2007-09-16 01:34:06 +00:00
Arvid Norberg 94f5b93d21 made it possible to pass in custom buffers to piece_manager::async_read, increased the maximum buffer size on bottled http_requests 2007-09-11 17:45:20 +00:00
Arvid Norberg 192ce4b463 added some extra checks for buffer overrun in disk_io_thread 2007-08-02 01:42:19 +00:00
Arvid Norberg bde5a9be0a fixed intended behavior for async_move_storage (2nd attempt) 2007-07-17 08:33:14 +00:00
Arvid Norberg fa37b9f3df fixed intended behavior for async_move_storage 2007-07-17 08:12:32 +00:00
Arvid Norberg eb03851276 fixed mutex issue in disk io thread 2007-06-17 05:58:12 +00:00
Arvid Norberg 3b8670626a merged back async_io branch into trunk 2007-06-10 20:46:09 +00:00