Commits : Individual Commit
Analyzed 9 days
ago based on code collected 9 days ago.
Commit ID
|
Contributor: | stsp | Files Modified: | 11 |
| Date: | 15-December-2011 at 11:03 | Lines Added: | 1149 | |
| Repository: | http://svn.apache.org/repos/asf/subversion/trunk /subversion/trunk | Lines Removed: | 230 | |
| Commit Comment: | New and improved implementation of 'hotcopy' for FSFS. This is based on the old implementation, adds support for cancellation, supports an incremental hotcopy mode (addresses issue #3815, "incremental hotcopy, fsfs-only"), has better input parameter verification, and always uses absolute paths internally (if called from the repos layer). The incremental mode is invoked via "svnadmin hotcopy --incremental". In incremental mode, only files which are new or have changed in type (e.g. file->symlink), size, or mtime since the last hotcopy are copied to the destination. This mode supports writing to existing hotcopies of the repository. Before copying it verifies that the UUID of the destination matches the UUID of the source, that the destination has no newer revisions than the source, and that the filesystem format and configuration of the source and destination are equal. It also removes stale locks from the destination (by removing all locks from the destination and then copying all locks from source to destination -- this is racy and may need to be revisited). Performance improvements of incremental mode seem to be quite significant. Copying the ASF repository (a copy from r0 to r421895, about 15GB in size), using the original hotcopy implementation, took about 150 minutes (source and destination residing on the same disk volume of a software RAID1 configuration on an OpenBSD FFS2 filesystem with softupdates disabled). A subsequent incremental copy that doesn't copy any changed files from source to destination completes in about 2 minutes. Behaviour of the non-incremental mode is affected slightly. It still refuses to copy to an existing destination. However, readers can peek at the destination filesystem while the hotcopy is in progress and see newer revisions appearing as the hotcopy progresses. This change was made to avoid having two separate implementations of hotcopy in FSFS, one for incremental and one for non-incremental mode. A write lock is now always held on the destination while a hotcopy is in progress. In the old implementation it was possible for a client to commit to the destination while a hotcopy was in progress, racing the hotcopy operation. The BDB hotcopy implementation is not affected, except that it throws an error if incremental hotcopy mode is requested. Cancellation support is only added at the API level of the BDB implementation in this commit, but is not actually implemented. With lots of suggestions from danielsh, thanks. * subversion/svnadmin/main.c (options_table): Adjust description of --incremental option. (cmd_table): Update help text of 'svnadmin hotcopy'. (subcommand_hotcopy): Call svn_repos_hotcopy2(). * subversion/include/svn_fs.h (svn_fs_hotcopy2): Declare. * subversion/include/svn_repos.h (svn_repos_hotcopy2): Declare. * subversion/libsvn_fs_fs/fs.c (fs_hotcopy): Adjust for new vtable 'hotcopy' prototype, adding source and destination filesystem parameters (instead of just their paths), the new 'incremental' parameter, and a cancellation function. * subversion/libsvn_fs_fs/fs_fs.c (svn_fs_fs__hotcopy): Moved down to the near the bottom of this file and mostly reimplemented. (hotcopy_io_dir_file_copy, entry_name_to_utf8, hotcopy_io_copy_dir_recursively): New helpers. Based on similar functions in libsvn_subr/io.c. (hotcopy_copy_shard_file, hotcopy_copy_packed_shard, hotcopy_update_current, hotcopy_remove_rev_files, hotcopy_incremental_check_preconditions, hotcopy_create_empty_dest): New helpers. (hotcopy_body_baton, hotcopy_body): New. This function contains those parts of the hotcopy code which run with the FSFS write-lock held on the destination filesystem. * subversion/libsvn_fs_fs/fs_fs.h (svn_fs_fs__hotcopy): Update declaration, adding parameters for source and destination filesystem, a boolean to enabled incremental mode, and cancellation support. * subversion/libsvn_fs/fs-loader.h (fs_library_vtable_t): Update declaration of 'hotcopy' as in previous. * subversion/libsvn_fs/fs-loader.c (svn_fs_hotcopy2): New. Adds support for incremental mode and cancellation. * subversion/tests/cmdline/svnadmin_tests.py (hotcopy_incremental, test_list): New, fairly simple, test. The hotcopy tests are currently few in number. None of the existing and new tests verify the complete result of the hotcopy but run 'svnadmin dump' on the destination after the hotcopy operation. This doesn't verify if all files in the repository were copied correctly (rings a bell wrt r905303). These deficiencies in testing will be addressed in a later commit. * subversion/libsvn_fs_base/fs.c (base_hotcopy): Change signature in accordance with API changes made above. But don't change the implementation. Refuse to operate in incremental mode. * subversion/libsvn_repos/repos.c (hotcopy_ctx_t): Add new parameters for incremental mode and cancellation. (hotcopy_structure): Add cancellation support. Tolerate existing destination directory in incremental hotcopy mode. (svn_repos_hotcopy2): Add support for cancellation and incremental mode. Convert input source and destionation paths to absolute paths. Verify that source and destination parameters are different directories. Apart from the general hilarity of copying things onto themselves this avoids a dead-lock when trying to hotcopy the rep-cache sqlite db onto itself. Tolerate some existing directories in incremental mode. (svn_repos_hotcopy): Reimplement as wrapper around svn_repos_hotcopy2(). |
|||
Changes by Language
Changes by File
Showing page 1 of 2
| File | Language | Code Added | Code Removed | Comments Added | Comment Removed | Blanks Added | Blanks Removed |
|---|---|---|---|---|---|---|---|
| subversion/include/svn_fs.h | C | 8 | 0 | 13 | 0 | 1 | 0 |
| subversion/include/svn_repos.h | C++ | 8 | 0 | 12 | 0 | 1 | 0 |
| subversion/libsvn_fs_base/fs.c | C | 10 | 1 | 0 | 0 | 1 | 0 |
| subversion/libsvn_fs_fs/fs.c | C | 13 | 3 | 3 | 1 | 0 | 0 |
| subversion/libsvn_fs_fs/fs_fs.c | C | 632 | 163 | 175 | 35 | 90 | 0 |
| subversion/libsvn_fs_fs/fs_fs.h | C | 6 | 1 | 3 | 2 | 0 | 0 |
| subversion/libsvn_fs/fs-loader.c | C | 57 | 6 | 0 | 0 | 5 | 0 |
| subversion/libsvn_fs/fs-loader.h | C | 5 | 2 | 0 | 0 | 0 | 0 |
| subversion/libsvn_repos/repos.c | C | 63 | 11 | 0 | 0 | 4 | 0 |
| subversion/svnadmin/main.c | C | 8 | 5 | 0 | 0 | 0 | 0 |
Copyright
©
2013
Black Duck Software, Inc.
and its contributors, Some Rights Reserved. Unless otherwise marked, this work is licensed under a
Creative Commons Attribution 3.0 Unported License
. Ohloh
®
and the Ohloh logo are trademarks of
Black Duck Software, Inc.
in the United States and/or other jurisdictions. All other trademarks are the property of their respective holders.