|Date:||15-December-2011 at 11:03||Lines Added:||1149|
|Repository:||http://svn.apache.org/repos/asf/subversion/trunk /subversion/trunk||Lines Removed:||230|
|Commit Comment:||New and improved implementation of 'hotcopy' for FSFS.
This is based on the old implementation, adds support for cancellation,
supports an incremental hotcopy mode (addresses issue #3815, "incremental
hotcopy, fsfs-only"), has better input parameter verification, and always
uses absolute paths internally (if called from the repos layer).
The incremental mode is invoked via "svnadmin hotcopy --incremental".
In incremental mode, only files which are new or have changed in type
(e.g. file->symlink), size, or mtime since the last hotcopy are copied
to the destination. This mode supports writing to existing hotcopies of
the repository. Before copying it verifies that the UUID of the destination
matches the UUID of the source, that the destination has no newer revisions
than the source, and that the filesystem format and configuration of the
source and destination are equal.
It also removes stale locks from the destination (by removing all locks
from the destination and then copying all locks from source to
destination -- this is racy and may need to be revisited).
Performance improvements of incremental mode seem to be quite significant.
Copying the ASF repository (a copy from r0 to r421895, about 15GB in size),
using the original hotcopy implementation, took about 150 minutes (source and
destination residing on the same disk volume of a software RAID1 configuration
on an OpenBSD FFS2 filesystem with softupdates disabled).
A subsequent incremental copy that doesn't copy any changed files from source
to destination completes in about 2 minutes.
Behaviour of the non-incremental mode is affected slightly. It still
refuses to copy to an existing destination. However, readers can peek
at the destination filesystem while the hotcopy is in progress and
see newer revisions appearing as the hotcopy progresses.
This change was made to avoid having two separate implementations of
hotcopy in FSFS, one for incremental and one for non-incremental mode.
A write lock is now always held on the destination while a hotcopy
is in progress. In the old implementation it was possible for a client
to commit to the destination while a hotcopy was in progress, racing
the hotcopy operation.
The BDB hotcopy implementation is not affected, except that it throws
an error if incremental hotcopy mode is requested. Cancellation support
is only added at the API level of the BDB implementation in this commit,
but is not actually implemented.
With lots of suggestions from danielsh, thanks.
(options_table): Adjust description of --incremental option.
(cmd_table): Update help text of 'svnadmin hotcopy'.
(subcommand_hotcopy): Call svn_repos_hotcopy2().
(fs_hotcopy): Adjust for new vtable 'hotcopy' prototype, adding source
and destination filesystem parameters (instead of just their paths),
the new 'incremental' parameter, and a cancellation function.
(svn_fs_fs__hotcopy): Moved down to the near the bottom of this file and
hotcopy_io_copy_dir_recursively): New helpers. Based on similar functions
(hotcopy_copy_shard_file, hotcopy_copy_packed_shard, hotcopy_update_current,
hotcopy_create_empty_dest): New helpers.
(hotcopy_body_baton, hotcopy_body): New. This function contains those
parts of the hotcopy code which run with the FSFS write-lock held on
the destination filesystem.
(svn_fs_fs__hotcopy): Update declaration, adding parameters for source
and destination filesystem, a boolean to enabled incremental mode,
and cancellation support.
(fs_library_vtable_t): Update declaration of 'hotcopy' as in previous.
(svn_fs_hotcopy2): New. Adds support for incremental mode and cancellation.
(hotcopy_incremental, test_list): New, fairly simple, test. The hotcopy
tests are currently few in number. None of the existing and new tests
verify the complete result of the hotcopy but run 'svnadmin dump' on
the destination after the hotcopy operation. This doesn't verify if all
files in the repository were copied correctly (rings a bell wrt r905303).
These deficiencies in testing will be addressed in a later commit.
(base_hotcopy): Change signature in accordance with API changes made above.
But don't change the implementation. Refuse to operate in incremental mode.
(hotcopy_ctx_t): Add new parameters for incremental mode and cancellation.
(hotcopy_structure): Add cancellation support. Tolerate existing destination
directory in incremental hotcopy mode.
(svn_repos_hotcopy2): Add support for cancellation and incremental mode.
Convert input source and destionation paths to absolute paths.
Verify that source and destination parameters are different directories.
Apart from the general hilarity of copying things onto themselves this
avoids a dead-lock when trying to hotcopy the rep-cache sqlite db onto
itself. Tolerate some existing directories in incremental mode.
(svn_repos_hotcopy): Reimplement as wrapper around svn_repos_hotcopy2().
|File||Language||Code Added||Code Removed||Comments Added||Comment Removed||Blanks Added||Blanks Removed|
Copyright © 2013 Black Duck Software, Inc. and its contributors, Some Rights Reserved. Unless otherwise marked, this work is licensed under a Creative Commons Attribution 3.0 Unported License . Ohloh ® and the Ohloh logo are trademarks of Black Duck Software, Inc. in the United States and/or other jurisdictions. All other trademarks are the property of their respective holders.