[112 total ]
Posted
14 days
ago
by
Gregory Brown
I'm happy to announce that the Prawn PDF library has hit another milestone on the Ruby Mendicant project roadmap. This time we'll look at Prawn's shiny new table drawing support, as well as some of the other features that have...
Posted
about 1 month
ago
by
Gregory Brown
Continuing on a theme, this post brings you the latest news from Prawn, the Ruby PDF library currently being developed under the auspices of the Ruby Mendicant project. The news this time around focuses on m17n progress, TTF font embedding,...
Posted
about 1 month
ago
by
Gregory Brown
I really like the open command on OS X, but I was too lazy to look for its Linux equivalent. Actually, my solution probably took less time than sifting through a google search: system(case ARGV[0] when /\.pdf/ "epdfview" when /\.html/...
Posted
about 1 month
ago
by
Gregory Brown
Several weeks ago I wrote about the first development checkpoint for Prawn, the PDF library currently being developed for the Ruby Mendicant project. I've finally reached the second checkpoint, which involves basic text rendering operations, and this post describes these...
Posted
2 months
ago
by
Daniel Berger
One of the rather unique features of the Fortress1 programming language is that it has builtin support for Unicode operators. For example, instead of using "==" you would use U 2261. After reading the spec, and a recent thread on ruby-core,...
Posted
3 months
ago
by
Gregory Brown
Back in March, I announced the Ruby Mendicant project after several readers of this blog encouraged me to pursue the idea. For those who didn't see the follow up details elsewhere, here's the readers digest version: Thanks to 70 donors,...
Posted
3 months
ago
by
Gregory Brown
While working on Prawn, I ran into this (not-so) fun little gotcha: >> 1.to_sym => nil >> 101241.to_sym => nil Anyone cool enough to tell me what this feature is all about? Update: I guess it isn't totally clear what...
Posted
3 months
ago
by
Timothy M. O'Brien
Sorry, it isn't entirely Ruby related... it is Python Django to be specific, but it is a message aimed at you ("the wiki-worker types") from Lawrence Lessig recruiting people to tag members of Congress at Change-Congress.org: Today we're launching the...
Posted
3 months
ago
by
Timothy M. O'Brien
I'm sure you don't believe it, doesn't seem like NetBeans is going to take the Ruby developer world by storm, but Sun seems to be pouring money into Ruby support. I'm skeptical that the Ruby community is going to embrace...
Posted
3 months
ago
by
Eric Berry
The long-awaited mod_rails for apache was released yesterday by Phusion, a Netherlands based IT company. As many of you have already done, I hopped on board to see how it worked and was amazed on how easy it was to...
Posted
4 months
ago
by
Gregory Brown
Reposting from the official announcement on RubyTalk Gobi version 1.0.0 has been released! * http://gobi.stonecode.org * http://metametta.blogspot.com * gregory.t.brown@gmail.com I am happy to announce the first release of my new fork of Ruby called Gobi. The goal of Gobi is...
Posted
4 months
ago
by
Phlip
CruiseControl.rb, by ThoughtWorks, is an elegant Continuous Integration dashboard for Ruby projects. This article shows how to call the RoR command "rake stats" once per build, capture the results, and chart them to track trends over time.
Posted
4 months
ago
by
Gregory Brown
Though I'm typing this from a Mac right now, I'm hardly a fan boy. I spend a little less than half of my time on OS X, the rest spent on cheap PC hardware and ArchLinux, and honestly, I'm happy...
Posted
4 months
ago
by
Gregory Brown
Though I'm typing this from a Mac right now, I'm hardly a fan boy. I spend a little less than half of my time on OS X, the rest spent on cheap PC hardware and ArchLinux, and honestly, I'm happy...
Posted
4 months
ago
by
Gregory Brown
When I wrote the post I'd love to quit my job! (sort of), I was mainly aiming to start a conversation. Still, enough people took me seriously, and now I can announce that the project is officially under way. I'm...
Posted
5 months
ago
by
Phlip
This project upgrades an online forum to add a search engine, using
Test
Driven Development. Our tools are
RoR's
Beast,
Sphinx, and
assert{ 2.0 }.
Posted
5 months
ago
by
Gregory Brown
UPDATE: Though my original post was 95% a fantasy, I've received some funding offers that have brought it down to 75% fantasy. I will be documenting any planning I'm doing towards via a wiki called RubyMendicant. If you'd like to...
Posted
5 months
ago
by
Daniel Berger
The Problem I find Ruby's current warning system, if you can call it that, lacking. Warnings are controlled by the -W flag on the command line, and are generated via the Kernel#warn method within code. There are a host of...
Posted
5 months
ago
by
Gregory Brown
Interesting: I asked a question on Phlip's last post about whether assert { 2.0 } was really necessary, given that it's mostly just assert_block from Test::Unit. I noticed this question disappeared from the post, because I guess it was offensive...
Posted
5 months
ago
by
Phlip_
This is a problem in all of unit testing - the cobbler's
own children always get the worst shoes! Our platform knows
everything we know about assert x == 42, but it
can't tell us everything for one reason: Languages
... [More]
optimize
for the needs of production code, not test code. So our
customers will always get better tools than we get!
Ruby supplies just enough reflection for an assertion to
reconstruct a block of code. [Less]
Posted
6 months
ago
by
Gregory Brown
The title of this post is the title of a talk I'll be giving at NYC Ruby on February 12th. Aside from blatent self promotion, I'm actually posting in search of opinions and thoughts to incorporate into this discussion, so...
Posted
6 months
ago
by
Daniel Berger
With the addition of Java Native Access (JNA) to JRuby, systems programmers using JRuby now have greater flexibility in terms of interfacing with underlying operating system.
Some Ruby users are familiar with the ‘Win32API’
... [More]
library that ships as part of the Ruby standard library. That library lets you interface with the Windows API by defining function pointers from specific DLL’s that you later call. With JRuby’s JNA interface you can now interface with Windows in a similar fashion.
Getting Started
The first thing we need to do is require the ‘java’ library.
require 'java'
With that out of the way, we’ll then want to get an instance of the DLL we need to export functions from. I’ll use Kernel32 for our purposes.
Kernel32 = com.sun.jna.NativeLibrary.getInstance('kernel32')
If you’re familiar with the Windows API, we’ve effectively called the LoadLibrary() function which maps the kernel32 Windows module into our address space, wrapped by our ‘Kernel32′ variable. From here we can define any functions that the kernel32.dll exports. For demonstration purposes I’ll define the GetCurrentDirectoryA() function:
GetCurrentDirectoryA = Kernel32.getFunction('GetCurrentDirectoryA')
This effectively calls the Windows GetProcAddress() function behind the scenes, returning a function pointer address, wrapped in our ‘GetCurrentDirectoryA’ variable.
The Nitty Gritty
Now that we have a function pointer, we want to call it. This is trickier than the standard Win32API library for two reasons. First, you must know the return type of the GetCurrentDirectoryA() function in order to know how to properly invoke it. I know from looking at the documentation that it returns a DWORD, which is a 32 bit number1. Looking at the JNA documentation2 we can see there’s an invokeInt() function. Let’s try it.
buf = 0.chr * 256
num = GetCurrentDirectoryA.invokeInt(buf.length, buf)
puts buf.strip
Oops! That won’t work because in Java strings are immutable. We have to convert our Ruby buffer to a ByteBuffer first, then convert it back to a regular Ruby string once we’re ready using the String#from_java_bytes method. Also, the invokeInt() function takes a Java array. We need to explicitly coerce the Ruby array using the .to_java method.3 Armed with this knowledge, let’s try again:
buf = java.nio.ByteBuffer.allocate(256)
num = GetCurrentDirectoryA.invokeInt([256, buf].to_java)
buf = String.from_java_bytes(buf.array)
puts buf.strip
That will return a string like ‘C:\Documents and Settings\some_user’, or whatever your current directory happens to be.
That’s all for now folks. In the future I plan to publish a JRuby gem of the win32-api library. Stay tuned.4
1I originally said invokeLong() here, but it was pointed out that a 32 bit unsigned long is considered an integer in Java.
2The JNA home page is at https://jna.dev.java.net/
3You can find more information at http://www.coffee-bytes.com/node/558
4I have a preliminary version in CVS. You can find the win32-api library at http://www.rubyforge.org/projects/win32utils. [Less]
Posted
6 months
ago
by
Daniel Berger
With the addition of Java Native Access (JNA) to JRuby, systems programmers using JRuby now have greater flexibility in terms of interfacing with underlying operating system.
Some Ruby users are familiar with the ‘Win32API’
... [More]
library that ships as part of the Ruby standard library. That library lets you interface with the Windows API by defining function pointers from specific DLL’s that you later call. With JRuby’s JNA interface you can now interface with Windows in a similar fashion.
Getting Started
The first thing we need to do is require the ‘java’ library.
require 'java'
With that out of the way, we’ll then want to get an instance of the DLL we need to export functions from. I’ll use Kernel32 for our purposes.
Kernel32 = com.sun.jna.NativeLibrary.getInstance('kernel32')
If you’re familiar with the Windows API, we’ve effectively called the LoadLibrary() function which maps the kernel32 Windows module into our address space, wrapped by our ‘Kernel32′ variable. From here we can define any functions that the kernel32.dll exports. For demonstration purposes I’ll define the GetCurrentDirectoryA() function:
GetCurrentDirectoryA = Kernel32.getFunction('GetCurrentDirectoryA')
This effectively calls the Windows GetProcAddress() function behind the scenes, returning a function pointer address, wrapped in our ‘GetCurrentDirectoryA’ variable.
The Nitty Gritty
Now that we have a function pointer, we want to call it. This is trickier than the standard Win32API library for two reasons. First, you must know the return type of the GetCurrentDirectoryA() function in order to know how to properly invoke it. I know from looking at the documentation that it returns a long (DWORD). Looking at the JNA documentation1 we can see there’s an invokeLong() function. Let’s try it.
buf = 0.chr * 256
num = GetCurrentDirectory.invokeLong(buf.length, buf)
puts buf.strip
Oops! That won’t work because in Java strings are immutable. We have to convert our Ruby buffer to a ByteBuffer first, then convert it back to a regular Ruby string once we’re ready using the String#from_java_bytes method. Also, the invokeLong() function takes a Java array. We need to explicitly coerce the Ruby array using the .to_java method.2 Armed with this knowledge, let’s try again:
buf = java.nio.ByteBuffer.allocate(256)
num = GetCurrentDirectory.invokeLong([256, buf].to_java)
buf = String.from_java_bytes(buf.array)
puts buf.strip
That will return a string like ‘C:\Documents and Settings\some_user’, or whatever your current directory happens to be.
That’s all for now folks. In the future I plan to publish a JRuby gem of the win32-api library. Stay tuned.3
1The JNA home page is at https://jna.dev.java.net/
2You can find more information at http://www.coffee-bytes.com/node/558
3I have a preliminary version in CVS. You can find the win32-api library at http://www.rubyforge.org/projects/win32utils. [Less]
Posted
6 months
ago
by
Gregory Brown
When I first got involved in free software development, I didn’t really know why I was doing it. It just seemed reasonably fun and challenging, which was enough to let it steal up every spare minute of my time in the form of a not-so-mini
... [More]
obsession. I didn’t so much think in terms of community, or how whether of my work would be useful to people, I was mostly just hacking for hack’s sake.
Eventually, I came to realize what fueled my work in open source, and that was the ability to learn from some truly amazing people, and later, return the favor by doing the same for others. If you like any of the work I’ve done in Ruby, you have exactly one person to thank for getting me started (James Edward Gray II), but hundreds to thank for keeping me going.
Here I’d like to talk a bit about my experience with the Ruby community and how it compares to something completely different, the community surrounding the board game Go. As I played in this past weekend’s North American Oza tournament, the idea for this article came to mind, and hopefully it doesn’t sound much worse in type than it did in my head. I must warn you, if you’re looking for technical depth, you’re not going to find it here, this is mostly just wishy-washy feelings and general observations that’ll only be interesting for those that have an obsession with community dynamics.
If that doesn’t scare you away, feel free to read on.
Time flies when you’re having fun (and coding Ruby)
By the time I started attending conferences and user groups meetings, I was basically a ‘Ruby guy’. This isn’t because I have the same kind of sick obsession some people get with one particular language (which leads them to consistently try to fit square pegs into round holes), but more because for most of the work I’ve needed to do, Ruby has fit the bill.
During my first RubyConf in 2005, I had already gained some cred in the community by helping James work on Highline, and also, by working on one of the RubyCentral codefest grant projects, Gambit. Though I was clearly not a rockstar at the time, I already had enough Ruby experience to not be a n00b. Amazingly, I had just begun on Ruport during this time, but if you go and download the 0.1.0 release, I don’t think you’ll see anything that screams Ruby 1337 in there.
Within a year, through several explosions of good luck and accidental successes (along with some hard work), I found myself speaking at RubyConf 2006 about Ruport. If you go and download the 0.6.0 release, you probably still won’t find much masterful code. However, by this time, I found myself at least in possession of *ideas* interesting enough to attract the attention of other hackers, and I knew enough where I could be helpful to others, which means that the hallway track started to get a whole lot more interesting.
By RubyConf 2007, after giving talks at a number of other conferences, I found myself too fatigued to even submit a talk. If you go and download Ruport 1.4, I think you’ll find some really excellent stuff, which is at the cost of mine and at least a few other people’s sanity over time. It was amazing how much two years had changed things. I found myself promising to spend time with more people than I was able to, and feeling generally spread too thin, and I realized at that point that maybe I had gone as far as I wanted with Ruby, maybe even too far.
Before this sounds like a long egotistical rant, I might as well come to my point. Over the last few years, I went from counting lines of code to counting downloads to counting users. Only within the last six month have I started counting the minutes of quality time I’ve spent with people I find to be extremely bright and talented, which turns out to be most of the folks I’ve met in the Ruby community. I’d say if anything, despite the criticisms, the Ruby community suffers from an embarrassment of riches when it comes to smart, nice, and generally interesting people.
At any rate, I guess what happened is that somewhere along the way I started to believe that Ruby had a monopoly on all the smart people, and in order to keep up with the Jones’s, the only solution was to keep doing more and more Ruby. This of course, is an absurd thing to think. In fact, it’s an absurd idea even when applied to programmers in general.
For this reason, I decided to seek out other hotbeds of intellectual prosperity, and the one that was most appealing to me ended up to be the one surrounding the game of Go.
Go is truly a hacker’s game
If you haven’t played Go before, I can’t decide whether to tell you to start immediately or never pick it up. Like programming, for those with the mindset, it can become an extreme addiction, and is wholly deep enough to spend two lifetimes mastering.
Like programming, I didn’t get involved with Go at first for the community. I mostly just found the game interesting, and then watched my time increasingly getting eaten up by games.
Eventually, through playing on KGS and also through studying with a strong player in my hometown, I realized that again, I was in it for the people. I took great joy in learning from players who would review my games and give me ideas of how i could improve. Like with programming, as soon as I was just good enough to be helpful to others, I started sharing my advice with them. Though I’m nowhere near mastering the game, it is a good feeling to be able to teach others what I know.
This experience alone was sufficient to make me believe that something I once thought (however naively) was unique to programming could be found other places as well. However, this past weekend taught me that there is a whole lot more in common between Go and Ruby when it comes to community than I ever ventured to guess.
How a Ruby Conference is like a Go tournament
Everyone is accessible
One thing that impressed me about my first Ruby conference is how willing people are to intermingle. Though there are obviously some cliques, they seem to form more accidentally than intentionally. The simple fact remains to this day, if you have a problem with a certain package, and that maintainer is at the conference, you can just go up and ask them a question.
I guess this shouldn’t come as a surprise, but it’s definitely a good thing. I like that by and large, there isn’t a cloud of pretense permeating the community. Maybe others have had different experiences with this, but it’s always seemed to me like there are very few obstacles that keep you from talking to more or less whoever you want.
At the North American Oza Tournament, I felt a lot of the same sort of love. More or less any player sitting nearby would happily do reviews and answer questions about strategy, and even offer teaching games to help you get stronger. In fact, it almost felt like this was to be expected, and reminds me a whole lot of when strangers start looking over your shoulder at a RubyConf and ask if you wouldn’t mind some suggestions. This kind of stuff makes everyone better, and I was happy to see this alive and well in the Go community since I enjoyed it so much in the Ruby community.
Friendly Competition is alive and well
I guess it’s more natural in a community of strategy game players than in a community of programmers for there to be some competitiveness, but I suppose the manner in which this manifests itself in both the Ruby and Go communities is a bit uncanny.
In both communities, a strong meritocracy is present. It’s fine to say ‘oh, that sucks’, but you need to have the stones to back it up, usually through something demonstrable.
I like that people don’t blindly accept advice in either of these communities, but need to see the correctness of any suggestion for themselves. This general atmosphere leaves room for competition. The bold can challenge common assumptions or ignorance, and if they are skillful, prevail.
Though ego plays some role in this, being proven wrong is almost always beneficial in some way to the person who loses the argument, so long as they’re willing to listen.
Bluntness
I like knowing I can go up to someone and say “What you’re doing there is terrible, you should never do that.” I find that this works both in Ruby and in Go. Sometimes I can be wrong, and if I am, I say “Oh, well, I guess I didn’t think of *that* reason for doing it”. This sort of admission of error is sufficient, and I don’t come off as an asshole if I play my cards right.
The ability to have blunt, no holds barred discussion that doesn’t include a ton of weasel words or random empty phrases that beat around the bush is really valuable. It lets you focus on technical details, and allows you to express passion about your assertions. This is the kind of stuff that gets me in trouble in my social life, but seems to be universally understood among hackers and Go players. Of course, there are exceptions, and it’s worth getting to know someone a bit to make sure you don’t surprise them with some rude comment.
I guess perhaps you could get away with this in a few other places, but I always found it to be a refreshing change of pace from day to day dialog where being diplomatic is far more important. To me, the sense of urgency and directness in technical discussions is a good thing, and I am glad to see that you don’t need to be a programmer to appreciate it.
Sense of humor and playfulness
Though the tournament games themselves were intense enough to be downright unnerving, most of the informal games that went on at Oza were a lot of fun, and usually had some sort of hackish elements to them.
I remember one player suggesting it was impossible to cheat at Go, and then 4 or 5 of us showing about 10 different ways to cheat, ranging from the realistic to absurd. There was also a game in which a player had about 8 people telling him a certain move wouldn’t work, and he played it anyway ‘just to see what would happen’.
This reminds me a lot of the sort of playful hacking that goes on at Ruby conferences, people sit down and toy around with some code, without any real goal in mind. Sometimes the goal is to do something silly or ridiculous, and I think that’s great.
A good sense of humor within a community prevents the dryness of technical content from being overwhelming. It creates a good sense of balance, and makes participation enjoyable, rather than feeling too much like work.
Though I can go on, these are the real similarities I’ve found. I guess they’re highly subjective, but I found them interesting anyway. Now, before I wrap up, I’d like to cover some noticeable differences.
How a Ruby Conference is NOT like a Go Tournament
The Oza tournament had a healthier mix of race and gender
It’s slightly entertaining that both Go and Ruby share a higher concentration than normal of Japanese community members, but in both cases, this is for obvious reasons.
Nevertheless, there is a point here. When I walk into a Ruby conference, though things may be getting a little better over time, I still see what is essentially a sea of 25 to 35 year old white males. Let’s not get into the details about this in fear of naysayers who claim there isn’t really a problem, but what I can say is that I didn’t notice any lack of diversity at the North American Oza Tournament.
There were probably more males than females, more whites than non-whites, etc, but nothing stood out as being surprising. What was surprising in a more positive light was that age distribution was really interesting at the Go tournament, I think you could find anyone from 4 years old to 80 years old there. Though it’s quite embarassing to see so many young children with higher ratings than me, in general I thought this was pretty neat.
Of course, the difference in crowd diversity may be mainly due to the difference in scope of the gatherings. I doubt there are many systematic problems in place that discourage certain people from playing Go, where programming is a whole lot murkier of an area.
Ruby Conferences are more fun
This is purely subjective, and almost ironic. A programming conference that is to some extent a training session for work, ends up being more enjoyable than a gathering around a board game. I think part of this is that I’m new to Go, and hadn’t made too many friends before the tournament within the community, but I also feel like there is just a looser tie to general fun in the Go world.
The age distribution has something to do with this. Though I learned a lot from some of the 14 year olds I met, I couldn’t very well thank them with a beer (at least not legally). Also sadly, Werewolf was not played.
I think maybe a more general event like the US Go Congress might score more points of the fun factor, but I won’t know until I attend one. I think this is also perhaps because at a tournament everyone is studying, whereas at a conference a lot of attendees are just along for the ride.
It’s strange, I guess Ruby has me quite spoiled on the fun factor. I had anticipated wild partying at Oza, and the best option I had was that the playing room was open late at night. But after a full day of Go, a beer among friends or some general debauchery would have been most welcome.
Vertical rather than Horizontal expertise
When I go to a Ruby conference, I tend to assume most people there are some kind of expert. I think this makes a slightly more equalized community, in the sense that the lines which separate hackers by skill level are quite fuzzy. Someone can be an expert in one aspect of things, and a n00b in others. This makes learning from each other quite easy, and somewhat laid back.
In Go, you have an actual strength rating. Though some players manage to play above their rank and some below, they generally show pretty accurately who you can beat and who you can’t. There is a ton to be learned horizontally in go, but expertise certainly stacks vertically.
This means that with the exception of folks within a few ranks of you, you can easily classify everyone as being better or worse than you at the game. This necessarily creates a more structured community, and to some extent, makes it a bit complicated to avoid elitism. Though I feel like people do a pretty good job of seeming reasonably humble about their skill, it’s clear that it takes a conscious effort.
A different kind of hero worship
Fanboys in software development tend to make my stomach turn. Hell, even comments like “I could never do what you’re doing” have made me feel bad at Ruby conferences before. Usually, these come from folks who have a family and a healthier social life than mine, and my response is usually, “It’s really that you don’t have to do what I’m doing”.
In Go, it’s a little bit different. With a somewhat empirical way of determining people’s strength, there are some folks in the Go world that it’s safe to say that barring some herculean effort, you will never catch up to them. For this reason, it’s kind of interesting that there was a separate ’strong players room’ for the top few boards in the tournament.
Walking into that room was like walking into some sort of alternate universe. Though it’s not true that dan-players glow in real life, the tension in the air was overwhelming. Though I *still* don’t believe hero worship is productive, I can see a whole lot more why it could happen in Go than I can in the much more subjective software development world.
So I guess the key difference is that one man’s hero is another’s annoyance in Ruby, but in Go… well, if you’re not strong enough to beat them, the high dan players are pretty universally amazing.
What I learned from all of this
I guess the big personal awakening here for me is that it’s not what we do with our intelligence, but how we use it and who we do it with that makes all the difference. It’s refreshing to know that no one topic has a monopoly on smartness, and maybe this is something to keep in mind all the time.
Dave Thomas says that you should learn one new programming language each year, to improve your skills and change the way you think. I say that we can generalize this a bit, and that you should learn one new skill and participate in a new community each year.
I can safely say my Go playing has improved my programming and vice-versa, and I find it remarkably interesting that it is possible to gain life experience through a ton of different means, and still see a lot of similarities within it all.
If you’ve had a similar experience as mine, with Go or something else in your life, I’d like to hear about it. It’d be interesting to see how the hacker spirit crops up in less obvious places than programming conferences. [Less]
Posted
6 months
ago
by
Phlip_
To ensure your test cases call efficient MySQL…
def test_my_case
assert_efficient_sql do
# just wrap them in this block!
end
end
The assertion intercepts and copies out your MySQL
... [More]
SELECT statements, then
calls EXPLAIN
on each one, and inspects the results for common problems.
The goal is test cases that resist database pessimization, even as you change
your data relations, to add new features. If you run your tests after
every few changes, you can easily detect which change broke your database’s
indices and relations.
This article is a reference for this assertion’s options. The techniques should be ported to any database with an EXPLAIN or similar system.
Credits to Jay Pipe
for his excellent little survey of ways to detect pessimizations!
Install
piston is the best way
to install this assertion:
piston import svn://rubyforge.org/var/svn/efficient-sql/ \
vendor/plugins/efficient-sql
The documented Rails way is the worst:
script/plugin install -x svn://rubyforge.org/var/svn/efficient-sql
The -x option bonds your Subversion repository to rubyforge.org. Each time you svn
update your own code, you will also automatically get the latest
version of the plugin, whether you need it or not. piston avoids
these fun issues. You can update on command, with piston
update…
Diagnose
If your SQL is already efficient, use :verbose to diagnose
why it‘s efficient:
def test_verbose
assert_stdout /select_type/ do
assert_efficient_sql :verbose do
Foo.find_by_id(42)
end
end
end
It prints a table like this to STDOUT:
query for SELECT * FROM foos WHERE (foos.`id` = 42) LIMIT 1
Foo Load
select_type | key_len | type | id | Extra | possible_keys | table | rows | ref | key
-------------------------------------------------------------------------------------------
SIMPLE | 4 | const | 1 | | PRIMARY | foos | 1 | const | PRIMARY
Reading a table‘s primary key is naturally efficient, so the
type field is a healthy const.
See the MySQL document Optimizing
Queries with EXPLAIN to learn what those results mean.
Optimize
This test shows assert_efficient_sql failing:
def test_assert_inefficient_sql
assert_flunked /Pessimistic.*
full.table.scan.*
Foo.Load/mx do
assert_efficient_sql do
Foo.find_by_sql('select * from foos a')
end
end
end
Its failure would look like this:
1) Failure:
test_assert_inefficient_sql(AssertEfficientSqlTest)
[./lib/assert_efficient_sql.rb:220:in `analyze_efficiency'
./test/assert_efficient_sql_test.rb:67:in `test_assert_inefficient_sql']:
Pessimistic
query for select * from foos a
full table scan
Foo Load
select_type | key_len | type | id | Extra | possible_keys | table | rows | ref | key
------------------------------------------------------------------------------------
SIMPLE | | ALL | 1 | | | a | 43 | | .
7 tests, 9 assertions, 1 failures, 0 errors
Most queries should not blindly read ALL rows in a database table.
If that query used a WHERE condition that selects fewer records,
but if EXPLAIN produced a query type of ALL,
this would indicate MySQL might read the entire database table just to
apply the WHERE. The best fix, in that situation, is to add an
index (a MySQL "KEY") to the table, and put the fields
from that WHERE condition into it.
That technique allows MySQL to read the index first, then chop directly
thru the table to the records you need.
Pessimize
Sometimes you need an ALL, even while other
assert_efficient_sql checks must pass. To positively declare we
like ALL, pass in :ALL => true.
def test_assert_all
assert_efficient_sql :ALL => true do
Foo.find(:all)
end
end
Assertions use positive reinforcement; they state conditions we like. They
don‘t try to deny conditions we don‘t like. If :ALL => false
meant "false to warn about ALL", the assertion would
confuse developers. So we use :ALL => true to allow queries to
scan all rows.
Options
If assert_efficient_sql (generally) dislikes your arguments, it
will print out its default options, each with an explanation.
def test_help
assert_stdout /invalid.*argument.*
verbose.*=\>.*false/mx do
assert_efficient_sql(:help){}
end
end
Nesting
You can also nest the assertion, to provide different options for different
blocks. The assertion allows this because your test might also have some
other reason to use blocks.
def test_nest
outer_result = assert_efficient_sql do
inner_result = assert_efficient_sql :ALL => true do
Foo.find(:all)
end
assert_no_match /where/i, inner_result[0][0]
Foo.find(42)
end
assert_match /where/i, outer_result[0][0]
end
Using filesort
If your WHERE and ORDER clauses are too complex, MySQL
might need to write a file (or worse), just to satisfy a query.
assert_efficient_sql detects this pernicious situation:
def test_prevent_filesorts
_exec %[ CREATE TABLE `t1` (
`a` int(11) NOT NULL DEFAULT '0',
`b` blob NOT NULL,
`c` text NOT NULL,
PRIMARY KEY (`a`,`b`(255),`c`(255)),
KEY `t1ba` (`b`(10),`a`)
) ENGINE=InnoDB ]
assert_flunked /Using.filesort/ do
assert_efficient_sql do
Foo.find_by_sql('SELECT a FROM t1 ORDER BY b')
end
end
ensure
_exec 'drop table t1'
end
If you want to use the assertion without faulting on filesorts, pass the
option :Using_filesort => true.
Throttle
One common pessimization is a query that reads thousands of rows just to
return a few. assert_efficient_sql counts the rows hit in each
phase of an SQL SELECT, and faults if any row count exceeds
1,000.
Adjust this count with :throttle => 42.
def test_throttle
101.times{|x| Foo.create :name => "foo_#{ x }" }
assert_flunked /Pessimistic.*
more.than.*100.*
Foo.Load/mx do
assert_efficient_sql :throttle => 100, :ALL => true do
Foo.find(:all)
end
end
end
SHOW SESSION STATUS
assert_efficient_sql calls SHOW SESSION STATUS before and
after its sampled block. If you are seeking an advanced pessimization, such
as Created_tmp_disk_tables, pass :Created_tmp_disk_tables
=> 0. The assertion will compare difference in STATUS before and
after calling its block. A difference greater than the allowed difference
will trigger a fault.
To test this, we simply detect a STATUS variable which is not a warning.
def test_declare_futile_war_on_Innodb_rows_read
assert_flunked /just.for.test.*
Innodb_rows_read/mx do
assert_efficient_sql :diagnostic => 'just for test!',
:Innodb_rows_read => 0 do
Foo.find(:all)
end
end
end
Inconclusion
assert_inefficient_sql is available at
http://efficient-sql.rubyforge.org/.
Watch that space (and this) for more tuning and pessimization detections. [Less]
Posted
7 months
ago
by
Gregory Brown
If you don’t mind ‘bad’ language and a little bit of hate, go check out Zed Shaw’s rant,Rails is a Ghetto.
I’m not quite sure what his intent was with the article, I’m not sure that even matters. What I
... [More]
know is that it has shock value, it made me laugh, and that a number of things in it ring true with me, even if a bit magnified.
Zed and I have had at least a couple conversations about the Ruby / Rails communities before, and I’d say philosophically, we’re mostly on the same page. The key difference is that Zed is… umm… Zed.
One thing that he pointed out as a difference between us is that I don’t really challenge the status quo. This is true, I prefer to slowly bend the rules rather than shatter them, and see where that takes me. My response though was that I like to challenge the folks who try to suppress those who would challenge the status quo.
So here’s my challenge at a few responses, because everyone loves to stir things up!
Zed is so Ghetto:
However, such a hate-filled nasty person should not be allowed to terrorize the nice, pleasant, and generous Ruby community. Sure there are bad guys out there, there are in any community, but this type of rant is pure crap. Such arrogance and hate are self-destructive. I hope Zed’s self-destruction happens far away from my world, but I wish it would hurry up so the rest of us can get back to enjoying writing code.
Wait a second. Zed is getting in the way of you enjoying writing code? Is Zed Shaw standing there threatening Roundhouse kicks to the head while you try to type in some Ruby code?
Or is Zed distracting you because you’re reading his blog and getting offended? If so… it seems that the solution is the same as if you don’t like something on TV. Change the channel. Go outside and take a walk.
Wear some garlic, I heard Zed hates garlic. Something about how it steals his magic powers.
On a serious note though, saying that hating on people in public is self destructive is like saying that smoking crack is self-destructive. If you think that Zed doesn’t know what he’s doing, you’ve been duped.
I was going to go on and challenge a few more of Zed’s haters. But you know what, I haven’t found any. The closest thing is Giles Bowkett being concerned about how anger is bad for Zed’s heart. There was also something from a consulting company which mostly agreed with Zed’s points, with only minor disagreements.
I’m not defending Zed. As you can see from the posts, he’ll physically defend himself if need be, so he doesn’t need that. I’m also not much of a fanboy, I like mongrel but I can remember when it was just something we were chatting about and playing with at NYC.rb. Still, the real thing that I’d like to confirm here is that he has a point.
If the Ruby community starts to get up in arms when they see rants like this, that’s even more of an indication of the fact that the lines between the Ruby and Rails communities have blurred, and that we’re going to increasingly turn away our best hackers who are interested in real community, beginning a slow slip all the way to JavaOne.
The good news of course is that Technorati lists a number of links to Zed’s rant with comments like “entertaining”, “hilarious”, and other such praises. Every strong community needs heroes and anti-heroes. My question is only who will replace Zed. [Less]
Posted
7 months
ago
by
Gregory Brown
Some of you may know that I’ve been working on The Ruport Book, a free-content book about Ruby Reports. Mike Milner and I have really learned through trial by fire some of the pros and cons of DIY technical publishing in the last few months.
... [More]
I’d like to share some of my thoughts here in a mini-ramble in hopes of stirring up conversation and ideas.
The idea of open source books is certainly appealing. This is especially true when you’re talking about books that cover open source software. Unlike certain technologies that have reached some stable, rock solid point, most open source projects are in flux, and the same is true for their documentation. I’ve seen a couple examples I can think of as success stories, or at least as projects that really stand out as interesting from the reader’s perspective.
One especially well done book is Karl Fogel’s Producing Open Source Software, which is actually produced by O’Reilly under the Creative Commons Attribution-Sharealike License, with copyright retained by the author.
The other is The Django Book, published by Apress under the GNU FDL.
Of course, for two big publishers, both of which are quite friendly to open source and more progressive publishing models, this kind of work is still quite noticeably the exception and not the rule. I think the reason for this might be a pragmatic one: The resources that mainstream publishers are geared to provide are tuned for high-volume general audiences, and not even a majority of worthwhile open source projects fit in this category.
Still, it’s really exciting to see things in the Ruby community such as PeepCode Press which offers a really fair deal to authors, along with a generally hip feel that seems to jive great for a number of niche Rails topics. Nevertheless, an E-Book , no matter how well done, can’t quite address the same needs that a dead-tree copy of a book can. I mean… you can’t very well impress people with a large collection of PDFs, unless you’re a pirate, but a wall of books looks nice pretty much anywhere. Of course, there are also more practical reasons to have printed books, even for short works, but who needs those?
So we’ve got things like Lulu. Word on the street is that they don’t quite have the same feel of a mainstream publisher’s final product, and to be honest, anyone who says that is basically trying to tell you that a hang glider doesn’t quite have the same transportation merits as a 747. Though I haven’t heard this comment from O’Reilly, some other publishers use this to discourage authors from self publication, and my feelings on this attitude can be summed up in one word. Lame.
But you know what? We felt a little shaky at first deciding to go this approach with the Ruport Book because of that common criticism. Amazingly, _why rescued us without ever knowing it. Nobody Knows Shoes is awesome. If _why’s doing it… it must be cool.
Anyway, enough ranting and link-blogging, and a little more plotting and thinking and prodding.
Here are the things I absolutely love about the fact we decided to self publish the Ruport book:
We own the book, and have the right to share it as we please with whoever we’d like, as well as allow direct community contributions to the content
We can publish an extremely up to date book. Ruport 1.4 isn’t even out yet, and the Ruport Book covers it. Our first pre-orders will ship within a week of its release
We are able to do whatever we want. Well… maybe not whatever we want, but close to it. Unfortunately, the idea of putting pacman ghosts on the front cover of the book were dashed by copyright violation concerns
We decided to offer up 25% of our revenue to a nice charity called Engineers Without Borders. I doubt we could have made this fly with a major publisher
No need to be secretive, our development has been transparent from the top down. You can even check the book content out of SVN
By making the book available to our users as we’ve developed it, we’ve not only gotten some feedback about the work itself, our users have become more skilled with Ruport, and have made tons of interesting contributions. Questions that are answered in the book are becoming increasingly rare on our mailing lists, as well.
But those things come at a hefty price, one that makes us sometimes wonder if the seemingly small royalty percentage mainstream publishers offer is worth it after all:
My friend Brad sent his chapters for Advanced Rails off to the ORA production department when he was ready for them to be formatted for print. The Ruport Book ‘production department’ is named Dinko, and he has a life outside of this book. :)
We had to mooch from our friends, project sponsors, and from the community just to cover book expenses, let alone even think about advances to cover our time spent on the project
We want to ship the book by the end of the year, and just decided to think about cover art, um… today
Our editing department consisted of #ruport and the Ruby Reports mailing list, and numerous emails traded off-list.
Our marketing department consisted of Mike and I mentioning the book at RubyConf and RubyEast
And this list could go on, but hopefully illustrates a point. :)
Now I’d like to leave you with the questions in my head as we prepare for this great experiment of publishing a book by ourselves. Maybe these will be interesting discussion points, or maybe you’ll think I’m crazy. Either way, I’d like to hear from you.
Are developers willing to accept a work with a production quality of ‘as best as we can muster up’ in exchange for complete transparency, direct project involvement, and the willingness to improve the work over time?
Are people sympathetic to the fact that print-on-demand services have some issues that you just wouldn’t see with a major publisher… such as $8 a pop production costs revenue sharing cuts?
To what extent is a ‘brand name’ important on a book? Do people buy on author name, publisher name, or a combination of both?
Will the fact that the book is open source genuinely help people make the most out of the work? Can we hope to see translations, mashups, and derived works?
How do we get the same benefit of a major publication in terms of reaching out to people who don’t already know about our project?
What publishers can we lean on in hopes of eventually getting support from without changing the spirit of the project? Can maybe the Ruport book eventually have a nice animal on the front cover?
These questions are of course a little geared towards my project, but I think they apply to any reasonably popular but somewhat niche open source Ruby project out there. A dream of mine is that wherever it makes sense to do so, we could have free documentation. A hallucination of mine is that somehow we can make it so that authors, publishers, and readers can all benefit and get a fair deal, while still putting out bleeding edge content that looks and reads nice.
What are your thoughts? [Less]
Posted
8 months
ago
by
Daniel Berger
Got a favorite gem you want to tell the world about? Or one you want to warn other people about? You can do both with Gemtacular!
Gemtacular (http://www.gemtacular.com) is a place to rate and review Ruby gems. It’s a great place to not
... [More]
only find opinions on various gems, but also to see the most recent gem uploads, find the most highly rated gems, or just search for existing gems.
Some quick guidelines:
* Gemtacular is not a place to report bugs. Use the project page for that.
* Be nice. If you have a problem with a particular gem, please explain why without getting nasty.
* Don’t be lame and rate your own gems.1
1I already have a feature request in to try and prevent this at http://rubyforge.org/tracker/index.php?func=detail&aid=10063&group_id=2863&atid=11067 [Less]
Posted
9 months
ago
by
Gregory Brown
The most important thing I’ve picked up with each RubyConf I’ve attended is a new outlook on my work. Each year the talks invite people to have their minds bent a bit, both technically and philosophically. Though I didn’t take
... [More]
notes and my attention span is usually too low to keep a full talk in my head, the following discussion is largely based on ideas from various talks I attended, especially from Nathaniel Talbott, Eric Hodel, Ryan Davis and Evan Phoenix. It also has some sprinklings from the hallway track, as well as some delusions from the back of my mind. Sorting all that out is up to you.
So what is real productivity? Maybe we can look at it as the coding nirvana that we’re all striving for, but I think it’s even more simple than that. True productivity just involves eliminating apathy from your life, leaving you with nothing left but things you care about, and no choice but to take care of them. What follows is a simple extension of that general idea into software practices.
Know yourself and your needs.
We can learn practices by rote memorization, or we can fake the funk and pretend that some idea we’re holding on to is helping us when it’s really getting in our way. Neither of these things will end up helping us become better programmers, much less happier people.
Here are plenty of examples of things from my own life that maybe some of you can relate to:
It took me over a year to ‘get’ why TDD/BDD is a good idea.
I refused to use TextMate for years because I believed there was one true editor (vim).
I’ve followed community models designed for projects with 50 active commiters for projects that only had me and casual contributors involved.
I used to think it was inconceivable to not host all your project services on RubyForge
I used to care whether or not my projects worked without RubyGems
We probably can’t stop ourselves from getting caught up in this cycle now and again, but when we miss the next bit, we’ve really got ourselves in trouble.
Listen to the world around you
If eight people call you an ass, there might be a pattern forming. Same goes for any serious criticism that pops up again and again. Listening is more difficult than hearing and responding. If you’re skilled at debate, you can present yourself as being correct, justified in your actions, and even untouchable.
But the bottom line is, if you’re still an ass at the end of the day, you’ve only succeeding at saving an ass-face. The funny thing is, this isn’t necessarily a permanent state, some really nice people will surprise you with how bull-headed they can become if they get involved in a flame war on a blog or mailing list. I’m not sure if I qualify as ‘really nice’, but I’m probably ‘nice enough’ and if you search some Ruby archives, you can find more than a few instances where there is no evidence of that whatsoever.
This almost always is the result of failing to really listen. All it takes is a second of hearing what you wanted to hear instead of what someone actually said, and you’re off on a dead end path.
But this isn’t supposed to be a lecture on kindness or ethics, it’s about productivity. Getting mad about technical or philosophical issues rarely results in growth, so cutting it out of your life will inherently make you more productive.
Here’s what’s more: In this global community of hackers, we have limitless access to criticism from very smart people. That’s a very powerful tool, if used right.
Be meticulous about your practice
Some of you may know that I spend a lot of my non-coding time studying Buddhism. Though this is certainly not the forum for religious discourse, there is an especially good concept I found in one of my Dharma books which I’ve applied to all aspects of my life, including software development. This one is so simple, but kind of hard to keep up with: Just be very rigorous about what you do.
We try so hard to discipline ourselves, to teach different mind tricks that let us be more productive: The various GTD systems are an example of this. However, it’s worth realizing that the core of all of this is just paying attention to what’s going on in your life.
If you find yourself stumbling over a bug more than once and you haven’t written a test for it, write the test. If you can’t do that right away or you don’t know how you’ll do it, write that down somewhere. Noticing that the last hour of your working day always goes way slower than the rest? Look at your schedule and try to figure out why.
Whatever problems you’re having in your code, don’t let anything slip through the cracks without at least noticing it. There is a tremendous difference between thinking “Oh, I should learn how to fix this problem”, and “This is the 20th time this problem has come up, and I still haven’t taken the time to learn it”. The latter lets you know exactly where you stand and how these things effect you, the former is the ‘maybe someday’ trap.
Take care of the small annoyances, and ignore the big things
Whatever happiness we have, it’s mostly relative to how happy we just were. I’ve been playing a lot of the board game Go lately, and it’s really taught me how a single mistake can throw off your mental state and send you tumbling through a downward spiral. Seeing this while playing a game, has allowed me to see it in my work.
Ruport provides a great source to practice this idea with for me. I spent over 2 years focusing on how the whole system would come together and how we’d address the problems of our users and even our mythical potential users, completely ignoring a bunch of the small annoyances.
Recently, I’ve turned that on its head. Now, whenever I see a method that throws back an absurd error that’s completely incomprehensible, I throw in a fix that handles the case better. Each one of those shaves a few seconds off of my thinking time every single time I encounter the problem again.
Sure, maybe there is some big shiny thing in the future that will make my life better, but it’s probably expensive, and it probably won’t pay off right now. The small stuff does.
That’s all for now
Here is where I start to run out of steam. Maybe it’s because I’m tired, or maybe it’s because if I keep on rolling, I’ll deviate completely from technology and start talking about the divine theoretical toaster of doom, or something even more absurd. But really, what I’m interested is what kinds of things influence your lives as developers, and some of the thoughts and practices that drive you to be productive and happy.
So… what is real productivity to you? [Less]
Posted
9 months
ago
by
Daniel Berger
Did you know you can do this with Ruby out of the box?
# A real lambda
λ { puts ‘Hello’ }.call => ‘Hello’
# Sigma - sum of all elements
∑(1,2,3) => 6
# Square root
√ 49 =>
... [More]
7.0
How difficult was this to implement? Keep reading!
# Be sure to run with the "-Ku" flag!
module Kernel
alias λ proc
def ∑(*args)
sum = 0
args.each{ |e| sum = e }
sum
end
def √(root)
Math.sqrt(root)
end
end
Pretty tricky, eh?
Just remember the “-Ku”. :) [Less]
