Ohloh Blog 
Permalink
Lately we've been receiving a lot of patches to Ohcount, the tool we use to count lines of code. This has been really exciting for us, and we're very thankful to the contributors who've been helping us fix bugs and add new languages.
Historically, Ohloh has never distinguished between C and C++. We simply lumped them together and called them "C/C++". This annoyed a lot of people. :-)
A major change that we're adopting today is to (finally) distinguish between C and C++. This is thanks to the efforts of Ciaran McCreesh, who submitted a patch to do this earlier this week.
This change poses something of a challenge for Ohloh: we have literally billions of lines of C and C++ in our system, and it's going to take us a very long time to recount all this code. It doesn't help that our current server farm is already overworked.
This means that for the next few months, you're going to be seeing an increasing amount of "C" and "C++" as separate languages, but you're also going to see a lot of legacy "C/C++" code. These might coexist in the same reports.
I can't predict how long it will take us to recount and sort out all of the existing "C/C++", but I estimate it will take a few months. We are expecting to install more server hardware soon, and that may help speed up the counts.
If you are eager to see your own project recounted, just let us know and I'll be happy to bump you to the front of the queue.
Thanks to everyone for the contributions, and here's to C and C++.
Woke up to interesting news today: GigaOm launched a new open source directory called OStatic.
I took a look around the site and found it interesting. It's definitely a great initial launch for a website - very polished for a 1.0 release. Congrats!
While they are appear to be a mix of a blog and a directory, there's some obvious feature overlap with Ohloh. They provide a directory of projects and let the community edit the tags, licences, etc... Unlike Ohloh, they don't do any metrics and don't provide any visibility into the people building (or using, really) the software.
Here's are some of their feature that stood out for me:
Alternatives: They let users provide recommendations for every project. So on the 'Vim' project you could add 'Emacs' as a viable alternative.
Proprietary Software: They also list 'closed'-source software on their site. This has a benefit of letting people discover discover open source alternatives while browsing a proprietary-sourced project page.
Categorization: They have a classification for their software (beyond tags), which lets you find related software fairly easily.
Questions: You can ask specific questions about projects. For example, on Vim, you might ask: "Will this edit a 1GB file?". The community can provide answers to these questions, a little like 'Yahoo Answers'.
I might have missed something but that's my quick take. Oh - one last thing, they seemed to have populated their directory with existing data from Wikipedia, FlossMole, etc... The upside is that they claim to have over 100k projects. The downside is that the data is somewhat suspect.
Personally, I like the first 3 features. I think the 'Questions' feature is a tricky one to pull off. It tends to duplicate the purpose of the actual projects' own forums. If you have anything deeper than a superficial question, I suspect you'll be better served asking it in the actual projects' forum.
I am excited to see more players in this space. It's interesting to analyze similarities and differences and, in the end, nothing focuses the mind like a little competition.
I'm curious to get your opinions on their service. Do you think Ohloh could benefit from implementing anything similar? Or rather, perhaps you think there are features Ohloh should not emulate?
We made a big change to our site's UI last week. Now that the dust has settled, I'd like to take some time to describe the "what" and the "why"'s behind this change.
A little background
Although we launched 1.5 years ago, our codebase is almost 2 years old. When we began this effort, we made a bet on using Ruby, which we chose because of what I perceived as its power & elegance. Two years passed and my opinion hasn't changed - it's easily the most enjoyable and rewarding language to program in that I've had the pleasure to use.
Challenges
A few months ago, however, I started noticing that our pace of innovation had slowed down. Adding features and modifying existing ones was getting harder and taking longer. What was going on here? After some introspection, I found that it boiled down to 2 things:
Web UI
Writing features means writing new UI. With minor work, it might just be a little html. Other times, it might mean firing up photoshop, or getting our hands dirty in javascript and css. Now, as it may surprise no-one, we don't have a dedicated web designer on our site (shocking to no-one, I suspect ;-). As a small outfit, all us developers wear multiple hats. On a practical note, this means every minute spent on UI is a minute we don't spend on improving our back-end scalability or analysis depth. Figuring out a way to lighten our load here would help us out greatly. Ergo, our first challenge: how to reduce the amount of time we spend on our interface.
Code Size & Rot
Secondly, the code rot. The good news is that our Ruby development skills have improved significantly. The bad news our earlier-written code was now ugly (by our new-and-improved standards) - and sometimes tricky to edit and extend.
My favorite aspect of Ruby is its conciseness. As our Ruby chops improved, we found that our earlier code was "smelly". By now we've gotten quite good at identifying the symptoms. Functions with vague names (and often vague purposes). Functions with more than a dozen lines of code. Duplicated code. These are the easier signs to identify.
By backporting our new-and-improved coding patterns to our earlier code we were hoping to shrink our codebase down significantly. I've become a big fan of small LOCs (lines of code). After all, there are only so many lines of code a developer can maintain (yet alone extend).
Solutions
In early January we took the plunge. To help out our UI challenges, we decided to make some pretty big changes:
Web UI Solutions
Grid-Based Css: Our prior UI was mostly developed in an ad-hoc manner. Instead, we developed our own web CSS framework with much inspiration from Blueprint. Using a grid approach reduced a great amount of guess-work involved in how to layout our pages. Not only did it reduce the amount of custom CSS required for every page, it also allowed us to spend less time inventing layouts.
Sparse, Whitespace Friendly Layout: We've removed a lot of our web-2.0 styling elements. No more rounded corners and a lot less 3d-ish panels. We've received some negative reactions to this (see this thread). However, by reducing the visual clutter, we are enabling our pages to become denser with data (rather than styling) AND we're reducing the amount of time we spend on UI. These gains have not been fully realized because we have not redesigned many pages of our UI (yet).
JQuery: Rails has a very "close" relationship with a javascript library called Prototype. It's really cool and we've had very few complaints with it. There's been a proliferation of Javascript Frameworks, to the point where they have practically become commodities. However, I was smitten with the genius behind JQuery's chaining model. We decided to port our Javascript from Prototype to JQuery as a result. We saw a reduction of LOC's, but not as much as I was expecting. However, we also benefitted from simply being better Javascript developers.
Haml && Sass: We've changed how we write our html markup from the vanilla Rails-based approach (called Erb) to instead use Haml. It has radically shortened our view files and made them much easier to read and modify. Similarly, we've ditched our hand-edited css files and instead adopted Sass (comes with Haml). Again, our total number of css-equivalent lines of code (now written in Sass instead) - down significantly.
Code Rot Solutions
Rails Plugin-ization: We took many of our common patterns and created simple plugins out of them. This cleaned up our codebase and reduced the amount of unit/functional tests required. No magic here, just common sense.
MakeResourceful: Score another one for Hampton Catlin, the genius behind Haml and Sass. MakeResourceful is a relatively simple plugin that seems completely obvious after you try it out. It works by abstracting all the common code from your RESTful controllers, while still giving you enough hooks to modify any aspect of the behavior you want. In plain english, it shrinks your Controller code drastically. Yes - we saw amazing reduction of our LOCs. More importantly, badly-written controllers now stick out like a sore thumb. We find ourselves opening up a Controller file and either seeing very little code, or saying "hey, why is this file over 20 lines of code? Aw man - we should clean this up!". This plugin belongs in Rails core. Kudos to nex3 one of the main devs on this project.
On the fence: RSpec. It might be coincidence, but it's interesting to notice that while RSpec is really cool, one effect it doesn't have is to shrink your LOCs. It's a testing framework that mostly replaces the existing Rails-centric test framework. We are using this on a different codebase we'll be releasing soon, but I have to admit that even after embracing mock-based, behaviour-driven testing, I'm still a little skeptical. We've written a lot of BDD tests - and they run very quickly, which is awesome. But even after all this discipline, I'm never sure the app really worked - i find myself thinking that I'm just testing that my mocks are being called in the way I expect them too. For now, at least on Ohloh, we'll stick with good ol' Test::Unit.
Conclusion
My first conclusion is that I shouldn't write long posts like this. It tires me out.
Beyond this, however, is that I keep sounding like broken record when I say that there is simply a ton of cool, innovative work going on in the Open Source World. At the beginning of Ohloh, I felt a little lost trying to deal with the chaos of the open source world. Then I learned to effectively "deal with it". I've since transcended into the "loving & thriving in it" camp. Where I perceived chaos, I now see strength. Diversity and competition of ideas and technologies are are healthy symptoms of adaptability and progress. The species appear and evolve, and yes, sometimes die off. But the ecosystem as a whole continues to grow and solidify.
Lastly, we're back to moving Ohloh features forward. We will be turning our attention to our back-end for a while and see how far we can push our scalability and performance. We'd like to see how close to we can get to real-time metrics (for the record, I consider hourly-updates to be real-time enough).
Thanks for everyone's help and feedback.
We've been busy cooking up a big update to Ohloh. In fact, we were going to deploy it today. However, that would have been a bad idea. You see, our servers are currently busy being slashdotted.
Once this quiets down, we'll release.
I've been engaged in an interesting discussion on another blog.
Initially there was an assertion that Ohloh violates FOSS developers' privacy.
I (and others) responded with the following basic arguments:
- Transparency is one of the strengths of FOSS.
- This is publicly available information that people publish freely on the Internet.
- If you don't want to reveal your activities on the Internet then don't attach specific personal information to your contributions such as your name.
- Ohloh and every other participant in the Internet community has the privilege of free speech insomuch as we can comment upon or analyze publicly available information.
But it became increasingly clear that the discussion was not only about privacy but about being compared to other FOSS developers. This brings up an interesting point. Should Ohloh supply a KudoRank only for users that have registered on Ohloh and discontinue KudoRank for non-registered aliases on specific projects?
I would love to get comments on the KudoRank issue as well as the privacy issue.