Upgrade Ohloh's Bazaar?

Hi Jonathan,

I agree, it looks like Ohloh is going to have to upgrade its version of Bazaar.

I don't think this can happen immediately -- We will have to free up some time for testing and deployment. Perhaps we can get this done next week.

Thanks, Robin

BTW congrats on opening up Launchpad, that's great news. :-)

Hi Jonathan,

We have made the code changes required so that we are compatible with bzr 1.17. It's working on development machines.

Now we need to actually deploy bzr 1.17 onto our server cluster. I'm going to take a stab at this today, and I'll let you know how it turns out.

Ohloh is now running bzr 1.17 across the board.

The tests are all looking good. Let me know if you discover any troubles!

Thanks, Robin

Hi Jonathan,

The move to bzr 1.17 has not been smooth.

In Launchpad's case, we are trying to get bzr to peform this enormous task:

bzr log --long --show-id --forward --include-merges -r 1.. -v

Each attempt takes upwards of an hour before it fails. The failures occur in various, non-repeatable ways.

Sometime bzr complains that some data is missing (I don't believe it), and we see an error like this one:

bzr: ERROR: The file id "x_Stuart_Bishop__Sat_SepW24_14:41:02_2005_18800.0" is not present in the tree .

Other times, we are seeing general exceptions, with bzr complaining that it has encountered an "internal error" with an exception stack.

We're sort of between a rock and a hard place. It seems like bzr 1.17 is unstable, with unrepeatable failures, yet older versions of the bzr are not compatible with new repositories.

There are some new Ohloh bugs that turned up over the weekend. bzr 1.17 is different enough from older versions, that our code is breaking in some subtle ways. I'm going to work through some bug fixes on some smaller projects, and perhaps those fixes will help the situation on larger repositories like Launchpad.

However, our initial problem is just a basic inability to fetch the log without error, and I don't know yet how to fix this on our end.

Well, this is something of a pleasant surprise -- it looks like an alternate repository URL was supplied, and our download from this URL completed successfully.

We were also able to extract the log from the repository.

Now we're in the arduous step of examining every commit to determine the file changes that occurred. For Bazaar repositories, this takes a very, very long time. It might take literally months.

I'm not sure why Ohloh's performance is so terrible with Bazaar -- we use the same strategy we're using with other source control systems. We seem to be OK with smaller repositories, but once the repository reaches a certain size, things really get out of hand.

I haven't had time to really investigate the source of the poor Bazaar performance, so I'm not sure if it's something stupid on Ohloh's side, or if Bazaar just isn't well suited to Ohloh's typical strategy, and we should be trying a different method.

I'll be keeping an eye on it. As long as it keeps moving forward (even slowly) I'll leave it be.

Thanks, Robin