python-dolly

What is PollyPolly stands for Python-Dolly. It is a program inspired by Dolly+. Dolly+ is a program to clone huge files on to a large number of nodes, named after the world first cloned lamb 'Dolly'. Dolly+ uses 'ring' of network connection, instead of 'tree' or 'star'. Each node will get data from upstream, write it to disk, and foward it to downstream. In theory, copying time will be 'constant' for any number of nodes, since everything there is performed in pipeline fashion.

Polly is a python version of the program. It is designed to be 0-installation and 0-configuration, leveraging the power of scripting language. It uses ssh to form the 'ring' and send the copy of itselt via ssh connection and invoke it.

How to use itInstallJust check out (or cut and paste from the Source Browse page) the code. No compilation required.

Usage$ polly.py HOSTA [HOSTB HOSTC ...] -files FILEA [FILEB FILEC ...]The files will be copied to the /tmp directory on each node. ssh key have to be propery configured, so that no password/passphrase is required to login.

Tags

  cluster ssh filecloning python

Share