How often does ohloh update a repository?

From the possibly-more-than-you-ever-wanted-to-know dept:

Projects and analyses are separate. A project can have many analyses laying around in our database, although we only show you the "best" one.

The project timestamp records the time of the last change of any kind to a project, whether it be a new description, tag changes, or license info.

An analysis timestamp records the time of our last query to the source control system.

So when you sort the project list by "updated at", you're seeing projects sorted by any change at all, not necessarily the last time we updated the analysis.

Importantly, it's possible that we have queried the source control more recently than the project was updated. This can happen if there haven't been any changes to the code in a while. In this case, we can continue using the same old analysis for the project. We mark the analysis timestamp to show that we've checked the source control, and the project continues on completely unchanged.

(It's also possible that a project contains no code, or the source control URLs are broken, or we simply have a bug that prevents us from downloading the code. These projects also never change.)

So the answer to question one is yes. There are a lot of projects whose timestamps haven't been updated in a year, yet their analyses are still being timestamped every week.

Question two: well, there could always be a bug, but the scheduler doesn't actually pick projects, the scheduler picks repositories. For every repository, we know the last time we queried the source control. This is what dictates priority. And just so we don't all go crazy, if we update any one repository in a project, we update them all. So projects are scheduled for update according to the time we last checked the source control for any of its repositories.

Last Friday we had a backlog of about 70 new projects, and more new ones are coming in all the time. This morning, we're down to 5 new projects in the queue, and soon we should be completely caught up -- and will hopefully stay that way [until the next boost in Ohloh traffic.] I really appreciate everyone's patience. I'm optimistic that our recent changes are going to make a big difference.

Well, famous last words I guess. Some kind souls have added three separate linux distributions to our system recently, which kept us rather busy this week. We have broken through the log jam, though, and we are currently running updates on existing projects.

Recent changes to our system really will speed things up in the long run -- among other things, it allows us to install a few more servers over the next few days.

Meanwhile, I'll schedule swingx. It should run today.

Thanks, Robin

Hi Peter,

Our jobs are running pretty quickly but I wasn't sure how much longer rockbox might take. So I went ahead and scheduled it. The update finished and should be available now.

Thanks!

Hi Ciprian,

If you add all of your branches in one project, your report will show too many lines of code. Ohloh will count the lines of code in each branch separately and then add them all up. With 4 or 5 different branches added to Ohloh, the lines of code calculations will be 4 or 5 times too high.

However, if you want all of your commits to show, you don't have much choice. It's a design limitation in Ohloh. We're not smart enough (yet) to recognize that branches are mostly the same code repeated. So unfortunately, when you use Ohloh you have to choose between (a) showing all of your commits across all branches (but having line counts that are too high), or (b) showing a precise line count but missing all of the activity that happens on branches.

KudoRank is only updated once per day, so changes to projects are not immediately reflected in your KudoRank.

Also, it's not enough to write a lot of code. Someone has to use that code. When people add a project to their stack, you will receive Kudo credit if you have made commits to that project. So adding more commits to a project doesn't necessarily help you unless many people have stacked that project.

Don't worry about the Subversion module names. Subversion does not create branches in the same way that other source control systems do. Subversion creates branches by simply copying files to new directories. The module name is always blank for Subversion. As long as the directory name shown in the URL is correct, Ohloh will find the correct code.

We have a bit of a status page here.

Right now, most project reports are about 2 weeks old. We seem to be in an endless arms race between the number of download servers we are maintaining vs. the number of new projects coming into the system. It's great that the Ohloh service has become very popular, but it has been an operational challenge.

Yes, we've talked about EC2 a bit.

We haven't looked at it very deeply yet for a couple of reasons. Although our application is well distributed across a lot of download servers, those servers are very tightly coupled with our central database (a very big computer), and these download servers require a lot of storage, bandwidth, and CPU. This would seem to make us a poor fit for EC2 right now. We don't have a lot of development time to make the architectural changes it would require.

However, in the long run, we are definitely interested in decoupling our download servers from the database, and creating a much more scalable system. At that point, EC2 might be a reasonable option. We just have so many things on our plate already that it will be a while before we get there.

Robin

Hi mytto - That is unusual. This project name is familiar to me, as if we had some problems in the past, but from looking through our logs I can't see any reason why this project isn't updating. I've rescheduled the updates, and I'll keep an eye on it today. I'll let you know if something odd comes up.

Thanks, Robin

Hi zeroflag,

Sorry, there's been a lot going on around here, and we are again falling behind on updates. I've scheduled an update for this project.

Robin