There were many days since the new SVN repo was enlisted in Arch Linux Packages project. I've seen some progress numbers a couple of times, but it always returns to 'Step 1 of 3: Downloading source code history (Waiting in queue)' after some time. Could you please look in logs why it cannot download the code, so if it's a problem on our side we could fix it.
This looks like a problem on our end. The particular server which has been trying to download this code has been having some problems.
I've reassigned the work to a different server. It looks like it's running quickly now. I'll keep an eye on it.
Hi Robin, there seem to be the exact same problem on project freecad (hosted on sourceforge)... Last update (failed) is from 10 months ago, I just reset the svn url, but it continues the same... Can you have a look too? Thanks Yorik
@ Robin: thank you for looking into that.
Unfortunately there does not seem to be much progress.
For the past few hours I see the same status message: 'Step 1 of 3: Downloading source code history (Running 0/2)'.
I wonder why it's /2 only, because I do remember it was many thousands during unsuccessful tries to download the source few days before.
There's more going on behind the scenes that I didn't explain.
Our "regular" Subversion importer downloads each revision individually, so there is a progress bar step for each revision. Although this is a robust strategy, the process is very slow.
We have also a "fast" Subversion importer that is based on
svnsync. It's much, much faster. However, not all servers work well with this importer, so we only use it by default with Google Code and SourceForge.
Because this is a large repository and because it was giving us trouble, I switched us over to use the
svnsync strategy to see if that would work better in this case.
The downside of using
svnsync is that it's not trivial to see the progress, so our downloader only shows two steps: one for us to create a local mirror, and another step for us to create a backup of that mirror.
As it turns out, the first step (create a local mirror) has gone well, and we've already downloaded all 45000+ revisions.
We're currently on the second step, creating a backup.
I'm not sure what the problem may have been with FreeCAD.
By deleting the old enlistment and replacing it with a new one, you've discarded any debugging information we might have had about the old repository.
In any case, things seem to be running normally now -- I don't see any problems. I'll keep an eye on it.
@ Robin: thank you for the detailed explanation!
Thanks Robin, it seems to be working, sorry I deleted the logs! Yorik
@ Robin: the souce was successfully downloaded, but the next step failed:
Step 3 of 3: Counting lines of source code (Failed about 8 hours ago.)
I think it failed to count the number of lines again because now I get:
Step 3 of 3: Counting lines of source code (Failed about 4 hours ago.)
I've been watching this job. It looks like we may have some kind of new bug that is causing failures. Give us a few days to investigate.
This repository was so large that it uncovered a memory leak in our analysis tools.
I've deployed the fix, and the repository now seems to be making progress.
It looks like it will take a few days for the analysis to go through, but I think we are now successfully past the worst of it.
Thank you for the support, Robin!
Hm.. It's been 13 days since Ohloh finished SVN checkout. Since that time it tried to sync from CVS repo (we're in process of fixing it to be accessible again), but never tried to sync from SVN. Is it normal?
As long as any repository is in a failed state, our system won't attempt to update any of the others. One failed repository will block the entire project.
If the CVS is going to down for some time, you might want to remove it from the project temporarily.
Okay, thank you.
Well, I've removed the CVS repository that failed but now the svnsync-based importer fails:
svn://svn.archlinux.org/srv/svn-packages Subversion (via SvnSync) (Failed 7 days ago.)
Though non-svnsync importer seems to be okay, just waiting for the other enlistenment to sync:
svn://aur.archlinux.org/srv/svn-packages Subversion Ohloh update completed 15 days ago.
Unfortunately, it looks like this project update is now blocked on a crashing bug in our line counter. This may or may not be a duplicate of another crashing bug that we are already working on. I will open a new ticket, and a fix should be in place soon.
The bug blocking progress is now fixed and we're again making progress.
Hmm.. I still see: svn://svn.archlinux.org/srv/svn-packages Subversion (via SvnSync) (Failed about 20 hours ago.)
Sorry about the repeated failures. We did finish about 1/3 of the work remaining and I've rescheduled it again.
we're now having the following trouble:
~/d$ svn info -r HEAD --no-auth-cache 'svn://svn.archlinux.org/srv/svn-packages@HEAD'
svn: No repository found in 'svn://svn.archlinux.org/srv/svn-packages'
This is because we completely reinstalled our server in past two days, and the URL is now changed. I've replaced the URLs now.
Seems to download fine so far (even the repo that was workin only via svnsync previously).
Subversion Step 1 of 3: Downloading source code history (Running 49/1775)
Subversion Step 1 of 3: Downloading source code history (Running 39/50874)
Hm, looks like download was restarted, because now it is:
Subversion Step 1 of 3: Downloading source code history (Running 17/1496)
Subversion Step 1 of 3: Downloading source code history (Running 23/50628)
Maybe try to switch both enlistenments to use svnsync again?
Copyright © 2013 Black Duck Software, Inc. and its contributors, Some Rights Reserved. Unless otherwise marked, this work is licensed under a Creative Commons Attribution 3.0 Unported License . Ohloh ® and the Ohloh logo are trademarks of Black Duck Software, Inc. in the United States and/or other jurisdictions. All other trademarks are the property of their respective holders.