Commit Graph

356 Commits

Author SHA1 Message Date
Arvid Norberg 79bd8fcd52 made the default cache size depend on the amount of physical RAM. and fixed disk thread settings update when loading through load_state 2010-03-10 07:14:10 +00:00
Arvid Norberg 9edb348093 read time is only cache misses 2010-03-04 03:49:06 +00:00
Arvid Norberg a942890e8d added more detailed instrumentation of the disk I/O thread 2010-03-03 07:09:04 +00:00
Arvid Norberg fdb68ab784 attempt to not make m_buffer_queue_size to ever get negative 2010-03-03 04:32:06 +00:00
Arvid Norberg 053d11ea3e fixed mutex lock typo 2010-02-24 00:04:28 +00:00
Arvid Norberg 278c1f91c0 fix typo in previous check-in 2010-02-21 09:03:56 +00:00
Arvid Norberg 59deff7a44 fixed issue with disk read cache not being cleared when removing torrents 2010-02-21 08:52:26 +00:00
Arvid Norberg c263f8daf7 volatile cache improvement 2010-02-06 17:56:58 +00:00
Arvid Norberg 7b0555ae45 fixed disk cache bug 2010-02-05 04:30:57 +00:00
Arvid Norberg c1b8124d20 volatile cache tuning 2010-02-01 01:11:10 +00:00
Arvid Norberg 238f7e38f5 improve disk stats logging to include cache hits 2010-01-31 21:13:52 +00:00
Arvid Norberg ca20d2d600 fixed broken build 2010-01-31 20:22:35 +00:00
Arvid Norberg 5b747f58f7 add guided read cache and update tuning docs 2010-01-31 19:14:00 +00:00
Arvid Norberg 53144f6f14 fixes to the new disk cache and the volatile cache feature 2010-01-31 16:29:52 +00:00
Arvid Norberg ccf8b0278a added volatile read cache option 2010-01-30 03:50:17 +00:00
Arvid Norberg f6c0d5af35 optimized disk cache to work with large caches 2010-01-27 04:25:45 +00:00
Arvid Norberg a046850130 support disk I/O priority setting 2010-01-23 03:02:32 +00:00
Arvid Norberg c9b594fde1 support sending suggest messages based on what's in the read cache. support 'explicit read cache' 2010-01-15 16:45:42 +00:00
Arvid Norberg 2948b3b375 fixed file checking bug introduced by fancy disk IO optimizations 2010-01-13 23:37:23 +00:00
Arvid Norberg a1299c3a10 optimize disk I/O elevator algorithm to spend less time picking job 2010-01-12 01:56:48 +00:00
Arvid Norberg d498c129ab clear sparse flag on files that complete on windows 2010-01-09 18:40:05 +00:00
Arvid Norberg ecb770eca7 always unlock while waiting for disk-IO 2009-12-25 16:13:35 +00:00
Arvid Norberg 6298322b1d service disk-read requests that hit the cache immediately and circumvent the elevator ordering 2009-12-16 10:49:15 +00:00
Arvid Norberg 1f5859d31f error_code fix. added is_error_code_enum template specialization and using the abbreviated syntax to build error_codes 2009-11-29 07:06:38 +00:00
Arvid Norberg ab4b7f99ad fixed error handling in read_into_piece 2009-11-28 03:14:08 +00:00
Arvid Norberg a5586d154d more header dependency optimizations 2009-11-25 06:55:34 +00:00
Arvid Norberg 6343fe6a6d optimized header dependencies to lower build time 2009-11-23 08:38:50 +00:00
Arvid Norberg 562e2512b9 some vs2010 fixes 2009-11-22 23:55:54 +00:00
Arvid Norberg c418165e07 replaced boost.filesystem with custom functions (improves efficiency and drops unnecessary dependencies and improves libtorrent portability) 2009-10-26 01:29:39 +00:00
Arvid Norberg 8a5b7d5d36 got rid of the last recursive mutexes. abstracted the threading primitives (and switched over to use asio's internal ones). 2009-10-20 02:49:56 +00:00
Arvid Norberg 6d01cb475c msvc build fixes 2009-09-06 00:57:01 +00:00
Arvid Norberg bec481acdf improved disk read performance and fixed bug in storage_interface's backwards compatibility functions 2009-09-05 07:21:10 +00:00
Arvid Norberg 49f54a6f7f file alignment and disk_io_thread bug fixes 2009-09-02 16:42:33 +00:00
Arvid Norberg 7933132cd9 fixed assert when built with disk stats 2009-08-05 19:02:51 +00:00
Arvid Norberg 63368c1383 added dummy storage class to disable disk I/O and an option to disable hash checking (for performance testing and simulation) 2009-08-02 06:40:45 +00:00
Arvid Norberg 838a246f6b disk stats fix 2009-08-01 22:47:07 +00:00
Arvid Norberg 73a8673cfc queued disk bytes fix 2009-07-26 19:54:16 +00:00
Arvid Norberg ecdd7231b8 removed redundant code in disk_io_thread 2009-07-21 04:32:27 +00:00
Arvid Norberg e9bd8f6e1f turned fast-resume errors into error_codes. fixed typo in documentation 2009-06-28 00:36:41 +00:00
Arvid Norberg 306b75e6b1 boost 1.35 fixes 2009-06-19 16:42:33 +00:00
Arvid Norberg 3df15ad39e fixed memory leak in disk io thread when not using the cache 2009-06-14 22:48:07 +00:00
Arvid Norberg 62a7946968 disk stats fix 2009-06-14 22:20:23 +00:00
Arvid Norberg f0134c1b11 improved disk error handling and expanded use of error_code in error reporting 2009-06-10 08:30:55 +00:00
Arvid Norberg e438a3ccfc more boost-1.34.1 fixes 2009-06-03 07:46:50 +00:00
Arvid Norberg a9a12a2391 fix build with boost-1.34.1 2009-06-03 07:22:43 +00:00
Arvid Norberg bd33814376 improve disk error handling 2009-05-31 22:41:53 +00:00
Arvid Norberg 0283f07aed more buffer categories to help debugging 2009-05-31 22:38:49 +00:00
Arvid Norberg b2c5d8a563 include the piece index that failed in disk errors 2009-05-31 19:44:56 +00:00
Arvid Norberg 3e7bd46706 fixed handling of unaligned block requests 2009-05-31 19:33:54 +00:00
Arvid Norberg d45e6cb118 fix bug in disk_io_thread 2009-05-31 10:15:45 +00:00
Arvid Norberg bc83387583 fix uninitialized variable in disk_io_thread 2009-05-30 04:02:03 +00:00
Arvid Norberg 3ea52b240f fixed memory leak in disk cache 2009-05-28 01:19:48 +00:00
Arvid Norberg 065bd283cf timer fixes 2009-05-27 17:06:50 +00:00
Arvid Norberg c66dda1c9f fixed bug in disk cache flush, largest contiguous function 2009-05-27 16:50:46 +00:00
Arvid Norberg 2f627a4b00 attempt to mitigate read cache thrashing 2009-05-24 16:49:01 +00:00
Arvid Norberg 29d56f9a8b improved read cache memory utilization 2009-05-24 15:32:14 +00:00
Arvid Norberg a89d6cced1 clean up some disk cache duplication of logic and added another memory usage optimization for the write cache 2009-05-24 00:12:53 +00:00
Arvid Norberg ae8488fb91 improved read cache memory efficiency 2009-05-23 19:27:27 +00:00
Arvid Norberg e322b318ae improved handling of out-of-memory conditions in disk I/O thread 2009-05-23 18:39:55 +00:00
Arvid Norberg e197f5cb85 disk_io_thread assert fixes 2009-05-23 17:17:47 +00:00
Arvid Norberg f9c12ef589 added more disk io asserts 2009-05-23 15:50:38 +00:00
Arvid Norberg c1d9198dc3 added largest_contiguous cache flush algorithm 2009-05-23 07:35:45 +00:00
Arvid Norberg 1392b14fec fixed disk_io error handling bugs 2009-05-23 03:05:21 +00:00
Arvid Norberg e36ea4ba5d added instrumentation for disk access. Useful to benchmark disk cache performance 2009-05-23 03:03:52 +00:00
Arvid Norberg 0127edac14 added disk-checking delay, to leave room for other disk operations while checking a torrent 2009-05-22 06:32:39 +00:00
Arvid Norberg cb9c3cb37d memory optimization for checking torrents 2009-05-21 16:15:05 +00:00
Arvid Norberg c047f66454 improved disk buffer logging and added extra asserts when logging disk buffer stats 2009-05-19 07:00:05 +00:00
Arvid Norberg 062e1796c0 fixed crash when shutting down while checking a torrent 2009-05-13 17:17:33 +00:00
Arvid Norberg 85bf463e11 added session setting to control disk buffer pool allocation sizes 2009-05-12 18:52:05 +00:00
Arvid Norberg 039ee486af use alloca for iovec arrays 2009-05-12 17:56:12 +00:00
Arvid Norberg 89e4c41ea2 fixed indentation 2009-05-11 21:19:03 +00:00
Arvid Norberg 1eb2799a03 added more allocation logging for statistics builds. Identified and fixed some memory waste 2009-05-07 20:30:20 +00:00
Arvid Norberg 838df44184 documented file_rename_failed_alert and file_renamed_alert. they were previously not documented. Fixed bug where the storage would be left in an error state (and eventually pause the torrent) when a rename_file() failed. The error is now only reported back through the alert 2009-05-07 06:41:41 +00:00
Arvid Norberg ff3db926fb allocates 10 disk buffers at a time instead of 32. Fixed bug where connection candidates would be too high 2009-05-06 07:06:26 +00:00
Arvid Norberg cf6dc377c9 define fixes for disk stats build configurations 2009-05-04 06:42:24 +00:00
Arvid Norberg 641abc6c4a slight memory optimization and some header cleanup 2009-05-03 20:21:24 +00:00
Arvid Norberg 807c1fc397 added validation check to disk_buffer_pool (and disk_io_thread in 0.14). This should catch any reference to the disk_io_thread after it has been destructed 2009-05-03 03:16:15 +00:00
Arvid Norberg e4ddb13b6f made the cache_size setting include send and receive buffers 2009-05-02 06:52:57 +00:00
Arvid Norberg 033fd5bd22 count send and receive buffers against the cache size limit 2009-05-01 08:00:58 +00:00
Arvid Norberg 509c8ed402 timed_wait does not seem to work as expected, revert to regular wait() 2009-04-15 16:32:05 +00:00
Arvid Norberg 91ba36a396 added compile time option for mlock support 2009-03-31 08:15:21 +00:00
Arvid Norberg c6017f9366 reverted bug introduced a few check-ins ago. Fixed case when recursive_copy fails (don't remove the original files). Cleaned up callback dispatching in disk_io_thread in trunk. Improved unit tests for move_storage 2009-03-31 08:05:46 +00:00
Arvid Norberg 36eb9c4f51 move storage fix 2009-03-30 04:55:23 +00:00
Arvid Norberg af10b55254 wakes up the disk io thread once a second to flush caches 2009-03-14 09:24:58 +00:00
Arvid Norberg 7ddffbac64 fixed windows build 2009-02-06 17:59:49 +00:00
Arvid Norberg 5c12db28d2 added option to lock disk cache in physical memory 2009-02-06 09:46:13 +00:00
Arvid Norberg 1ac8f806fa added seed mode support (lazy hash checking) 2009-02-03 07:46:24 +00:00
Arvid Norberg 26910eff1d update_settings fixes for disk_io_thread 2009-01-28 07:09:10 +00:00
Arvid Norberg b3a17b1787 added disk buffer instrumentation and log parser 2009-01-23 09:13:31 +00:00
Arvid Norberg 00808473e7 factor out disk_buffer_pool from disk_io_thread. refactored the file open modes to be platform independent. gave the disk_io_thread its own copy of session_settings that it shares with storage. added an unaligned_read implementation to storage. Added options to session_settings on when to open files in unbuffered mode. Added unit tests for unaligned reads 2009-01-21 07:31:49 +00:00
Arvid Norberg 113d1f3557 moved page_aligned_allocator to its own file and uses it in set_piece_hashes to not pass in unaligned buffers to storage read 2009-01-15 17:09:36 +00:00
Arvid Norberg 630601aedf use & instead of % 2009-01-15 04:25:07 +00:00
Arvid Norberg b9d7538ba8 fixed storage to generate a proper error when the read operation 2009-01-14 03:05:35 +00:00
Arvid Norberg 3f0fa60164 fixed typo preventing pool-allocators=off build 2009-01-13 06:49:40 +00:00
Arvid Norberg 7592ad4aee updated disk IO to support unbuffered files 2009-01-11 02:02:34 +00:00
Arvid Norberg cfd116ffad made disk_io_settings be synchronized through the job queue 2009-01-10 05:46:02 +00:00
Arvid Norberg f1d229aae6 first pass of replacing coalesce writes and reads by writev and readv (no windows support yet) 2009-01-03 08:11:31 +00:00
Arvid Norberg 3bf9fa239d fixed incorrect failure test in storage. Affected reading when not coalescing reads 2009-01-02 10:31:26 +00:00
Arvid Norberg 06da1f8e8a added some session logging. added more disk-io-thread stats logging and parse script 2008-12-30 08:20:25 +00:00
Arvid Norberg 28cb451eac expire read cache as well (same time out as write cache) 2008-12-27 02:38:14 +00:00
Arvid Norberg 2d4bdbca7d fixed disk_io_thread over-allocation 2008-12-03 06:42:53 +00:00
Arvid Norberg 1022ba45fc fixed bug in disk IO thread which would make it sometimes0 exit the thread if an error occurred 2008-11-29 21:38:34 +00:00
Arvid Norberg 09b3a865ef fixed compiler flags in pkg-config file and made header files not depend on the NDEBUG define 2008-11-29 21:33:21 +00:00
Arvid Norberg f503bbfe86 fix for when aborting torrents that are being checked. Fix in client_test when removing torrents that are checking. 2008-11-17 01:19:46 +00:00
Arvid Norberg 49a3ac9710 disk_io_thread shutdown fix 2008-11-11 09:32:51 +00:00
Arvid Norberg 4e9124019b added accessor to query for the max number of connections 2008-11-08 07:40:55 +00:00
Arvid Norberg 5f6351146a removed assert that could cause dead-lock in debug mode 2008-10-05 19:47:55 +00:00
Arvid Norberg 0f6ec06e12 fixed race condition in invariant check in disk io thread (only affected debug builds) 2008-09-30 01:16:19 +00:00
Arvid Norberg 26dea3c222 fixed stats miscount when deleting files 2008-09-17 02:29:05 +00:00
Arvid Norberg 658eef82a5 get rid of msvc warning 2008-09-15 13:58:38 +00:00
Arvid Norberg 8488d5eb59 fixed bug in disk_io_thread read cache management 2008-08-07 12:30:36 +00:00
Arvid Norberg 58978048e8 fixed assert in disk_io_thread 2008-07-30 08:05:04 +00:00
Arvid Norberg 21dff61cb5 clears read cache when paused. Saves memory 2008-07-18 15:31:22 +00:00
Arvid Norberg 3cb1369385 improved error reporting of file errors (uses boost.system's error_code). Now permission errors are properly reported when checking files 2008-07-17 23:41:46 +00:00
Arvid Norberg 9c9c13c7bb exposed read cache info as well as write cache 2008-07-11 10:29:26 +00:00
Arvid Norberg 9c84908bb9 switched resume data parsing over to use lazy_bdecode. Improves memory allocation performance, especially noticable when heap allocations are expensive. Makes it more practical to run with malloc debug. Changed resume data interface to take a vector as opposed to a parsed structure 2008-06-30 23:14:31 +00:00
Arvid Norberg 52f6204288 simplified the disk-io mutexes and made it more efficient. separated the disk io mutex into a buffer mutex, queue mutex and the piece cache mutex. 2008-06-12 04:40:37 +00:00
Arvid Norberg 6e69480176 disk_io_thread fixes. Properly destructs torrent objects while disk io mutex is not locked. Properly flushes disk cache when closing, and aborting read and file check jobs. Fixes #334 2008-06-09 04:46:34 +00:00
Arvid Norberg 3910fe78de separated file_storage from torrent_info and create_torrent 2008-05-28 08:44:40 +00:00
Arvid Norberg f3d0eb3c11 fixed boost-1.35 build 2008-05-01 16:31:19 +00:00
Arvid Norberg 644d3aa66c disk IO thread error handling 2008-04-13 21:26:57 +00:00
Arvid Norberg 3fea2080fd added an asynchronous save_resume_data to make it easier to synchronize with the disk IO thread 2008-04-13 18:54:36 +00:00
Arvid Norberg d4cfa126aa storage fixes 2008-04-12 22:08:07 +00:00
Arvid Norberg 093d912e9a receive buffer optimization. added receive_buffer_size and used_receive_buffer to peer_info. changed plugin api to make use of new disk_buffer_holder type 2008-04-10 10:03:23 +00:00
Arvid Norberg 5f35d170b0 disk io thread cleanup (all read operations are now read into disk buffers, no custom buffers) 2008-04-10 09:11:54 +00:00
Arvid Norberg 75ef4ec1f1 added option to not use pool allocators (useful when using memory debugging tools) 2008-04-09 05:19:11 +00:00
Arvid Norberg 4e9ed2dc96 fixed incorrect flushing of disk cache 2008-03-16 10:51:25 +00:00
Arvid Norberg e5a98d1b7e more invariant checks in disk_io_thread 2008-03-10 08:19:31 +00:00
Arvid Norberg f53cfa7eeb removed checker thread 2008-03-08 06:06:31 +00:00
Arvid Norberg c689c0b5fc replaced deque with list in disk_io_thread 2008-02-26 20:08:33 +00:00
Arvid Norberg 1d1398ed7f read cache fix 2008-02-24 23:14:10 +00:00
Arvid Norberg 19462007d1 read cache fix 2008-02-22 08:22:57 +00:00
Arvid Norberg ef9ef674d5 added read cache. Not fully configurable yet 2008-02-22 04:11:04 +00:00
Arvid Norberg 7e83c3fc51 changed storage interface to not require exceptions 2008-02-14 03:48:20 +00:00
Arvid Norberg 752149e341 fixed bug in write cache flush logic 2008-02-10 20:36:48 +00:00
Arvid Norberg 6c552db68e added disk cache expiration 2008-02-10 00:58:25 +00:00
Arvid Norberg ea7253c575 increased default cache size to 8 MB 2008-02-09 21:09:29 +00:00
Arvid Norberg 8cf0510144 added disk cache for write operations 2008-02-08 10:22:05 +00:00
Arvid Norberg f7d34794a9 improved error handling 2008-01-14 23:51:04 +00:00
Arvid Norberg e0d1951fac fixes #234. properly shuts down the disk io thread 2007-12-24 08:15:10 +00:00
Arvid Norberg 9b54043478 attempt to fix potential deadlock in disk_io_thread 2007-12-13 04:05:25 +00:00
Arvid Norberg 1a280e31fa cleanup. changed the connection map into a set of peer_connections. moved the policy object into the torrent (as opposed to being a pointer). Fixes issues with multiple peers on the same IP. Reduces some lookups. 2007-10-31 09:48:20 +00:00
Arvid Norberg a1a564ef51 fixed debug issue with disk io thread 2007-10-23 22:11:17 +00:00
Arvid Norberg b032c26bf0 adds invariant check to make sure pieces that are complete are also checked. adds a debug function in the disk_io_thread. Fixes a problem with a failing disk write that would not clear the block for re-download. Adds some extra session logging 2007-10-18 00:32:16 +00:00
Arvid Norberg 59c7f5bfbb added option to delete files from disk as a torrent is removed 2007-10-13 03:33:33 +00:00
Arvid Norberg 966800cd5b replaced the assert macro to TORRENT_ASSERT instead of hi-jacking assert. 2007-10-05 00:30:00 +00:00
Arvid Norberg f4f30f5b19 disk io priority fix 2007-10-03 22:30:40 +00:00
Arvid Norberg b48948e231 changed the double send buffers to a chained buffer using iovec send operations. buffers are recycled (saves memory and cpu) and memory copies are avoided (saves cpu). More statistics is added to track buffer usage 2007-09-29 16:14:03 +00:00
Arvid Norberg b5a2373bff added an option to generate a disk io log file 2007-09-17 08:15:54 +00:00
Arvid Norberg 9364a2d01d added priority to disk io jobs 2007-09-16 01:34:06 +00:00
Arvid Norberg 94f5b93d21 made it possible to pass in custom buffers to piece_manager::async_read, increased the maximum buffer size on bottled http_requests 2007-09-11 17:45:20 +00:00
Arvid Norberg 192ce4b463 added some extra checks for buffer overrun in disk_io_thread 2007-08-02 01:42:19 +00:00
Arvid Norberg bde5a9be0a fixed intended behavior for async_move_storage (2nd attempt) 2007-07-17 08:33:14 +00:00
Arvid Norberg fa37b9f3df fixed intended behavior for async_move_storage 2007-07-17 08:12:32 +00:00
Arvid Norberg eb03851276 fixed mutex issue in disk io thread 2007-06-17 05:58:12 +00:00
Arvid Norberg 3b8670626a merged back async_io branch into trunk 2007-06-10 20:46:09 +00:00