Adapting The Joel Test To Web Development
Out Of Date Warning
Languages change. Perspectives are different. Ideas move on. This article was published on November 4, 2009 which is more than two years ago. It may be out of date. You should verify that technical information in this article is still current before relying upon it for your own purposes.
- Avoiding Notices: When to Use isset() and empty()
- Configuring PHP: Essential INI Settings
- Accessing Databases with PDO: A Primer
- To The New PHP Programmers…
- How To Write A Function In PHP
- Five Cool PHP Array Functions
- Micro Optimizations That Don’t Matter
- Adapting The Joel Test To Web Development
- Exceptional PHP: Introduction to Exceptions
- Suhosin: The Invisible Hand Of PHP
- Why You Should Replace ENUM With Something Else
Nearly a decade ago, Joel Spolsky came up with a method by which to evaluate software development shops that has come to be known as the Joel Test. This crucial test evaluates a software development company on the basis of twelve criteria points; Spoksly said that “a score of 12 is perfect, 11 is tolerable, but 10 or lower and you’ve got serious problems. The truth is that most software organizations are running with a score of 2 or 3, and they need serious help, because companies like Microsoft run at 12 full-time.”
When Joel wrote the test, there wasn’t much development for the web; the little that was being done wasn’t being done in any of the modern languages that we write in today. In fact, Facebook, Myspace, Twitter, Gmail, and LinkedIn hadn’t even been invented yet.
Today’s world makes heavy use of web-based software (the term “software as as a service” keeps floating around). And so, it is necessary to update Joel’s test in order to properly apply it to web development. This has been done by some folks, but I will do it again, mostly because I disagree with them.
Note: I will not be reprinting the original Joel Test; you can read that on your own.
The Joel Test – 2009 Edition
- Do you use source control?
- Can you make a shippable version of your software in one step?
- Do you use continuous integration?
- Do you have a bug database?
- Do you fix bugs before writing new code?
- Do you have an up-to-date schedule?
- Do you have a spec?
- Do programmers have quiet working conditions?
- Do you use the best tools, and spend money on those tools that are necessary and expensive?
- Do you write unit tests?
- Do new candidates demonstrate their code knowledge in some tangible way during the job interview?
- Do you have testers?
Again, just like the original, you can answer a quick yes or no to these questions in about five minutes. You should know how your operation scores without doing much deep thinking.
Let’s look at each point separately.
1. Do you use source control?
This is completely unchanged from the original test, because it is one of the most crucial components of professional software development. Whether you’re a team of one or one hundred, you need source control. Since 2000, lots of new options have been introduced: Mercurial, Git, Subversion, to name a few. If you’re not running version control you may as well hang a shingle that says “your code is not safe with us.” Because it isn’t.
2. Can you make a shippable version of your software in one step?
There’s a lot of argument about how or whether this applies to web development. It does.
Web developers need to have a build process in place. That build process needs to do things like run the unit tests, create a tarball or package version for distribution, and run other tasks. My personal build process builds a manifest of all the classes for the autoloader and strips comments out of my code.
Someone will inevitably argue that the nature of the web allows for deployment of code without creating a build. This is true, but it also lends itself to bad development practices. If you simply do an “svn up” whenever you’re ready to release new features, you never actually go through the process of reviewing, tagging and releasing something; you’re always updating from trunk, and this makes it hard to version and fix bugs. Yes, there is some overhead incurred, but this is necessary to the art of software development.
Do you use continuous integration?
The time to find out that one of your developers broke the build or failed a unit test is within hours of their commit, not at 2 am in the morning on the day after the release was due at 5 pm.
Continuous integration (there’s phpUnderControl and XincM for PHP projects) allows you to regularly build your application, test your application, and write the documentation for your application. If the tests or the build fails it can alert you to problems before you’re ready to roll a release. Use it. It works well, it’s easy to set up, and it’s invaluable.
Do you have a bug database?
This point also remains unchanged, due to the fundamental necessity to software development. There are hundreds of bug tracking software packages that will integrate with Subversion or other VCS programs.
Tracking bugs provides the ability to see what issues remain, who is working on them, and why they have yet to be resolved. It also allows for the measurement of metrics (e.g. knowing how long the average ticket is open). Tracking bugs is crucial, because if you ship a product repeatedly with the same bug, your customers will get antsy and frustrated. “Oh, we forgot about that one” stops working after using it for the third time.
5. Do you fix bugs before writing new code?
When deciding how to work on a release, you should always fix existing bugs before adding new functionality. Always.
This might seem counter-intuitive but consider: your customers will be more annoyed by persistent bugs that hurt their ability to use your software than they will by a lacking function that they want you to integrate.
Consider the reality that a customer who sees a bug automatically has a lower level of trust in an application. If they’ve reported it and the bug persists between versions, that trust will be even lower. This is certainly true of me: the WordPress cron task has been broken since WordPress 2.7.0 and remains broken; I have less trust in WordPress as a result.
Additionally, the longer you wait to fix a bug the more expensive it will be. That bug may affect six features (all of which you added in the last version) instead of one, necessitating a complete rewrite of all six features (and requiring you to do the work twice). Don’t do that to yourself.
6. Do you have an up-to-date schedule?
A schedule is critical for business strategy and planning. But it’s also crucial for keeping the programmers from being surprised. Developers don’t like surprises. And neither do business people.
A schedule is never set in stone; it must be fluid and respond to the needs of the development process. For example, if the schedule is slipping there are really two choices: relax the schedule or reduce the requirements. By having a schedule you avoid scope creep, because items not on the schedule can safely be ignored. And having a schedule that is up-to-date means that everyone knows when, how and where the deliverables are, at any given point.
6. Do you have a spec?
Joel Spolsky spends a good deal of time writing about specs in a four-part series. Apparently when people see “spec” they think of 600-page documents that no one ever reads.
A spec is as simple as a wireframe that describes the functionality and the views that the application is going to have. A spec is as simple as defining the scope of a feature. A spec is as simple as a bullet-point list that describes how an application will work.
Specs allow you to plan. They allow you to more accurately estimate. They allow you to ensure that the customer is getting what they want. They allow you to ensure that all the points have been hit before you start writing code. Before you start writing code. That’s the big point: specs must be done before code is written.
By putting a spec on paper, you also avoid feature creep and last-minute adjustments by the client. They’ve signed a piece of paper saying “yes, these are all of our needs for this version” which you can then take back to them, with a promise that you’ll integrate their request in the next iteration. Your developers don’t get angry at you for springing a new component (and lots of overtime) on them at the last minute, and your clients stay happy because they get what they ask for (even if they have to wait longer when they change their minds).
8. Do programmers have quiet working conditions?
This Joel Test point is often contentious amongst those who think that a big, collaborative atmosphere is best. Joel argues that developers should get their own offices. Having worked in both situations, I agree with Joel. Developers should work in offices, not in cubicles or big spaces.
In order to program effectively, you have to be able to hold lots of data in your head at the same time. If you’re interrupted, that “RAM” gets cleared and you start over from scratch. I’ve lost many a day because I just couldn’t get in the zone: there were meetings, phone calls, emails to reply to, and other issues to resolve.
Programmers are expensive, and they need to be treated like any precious resource. Place them in their own space, which they can decorate, and give them a real door that shuts. Give them the ability to shut out the world while they work. The code will be better for it, as will the end product.
9. Do you use the best tools, and spend money on those tools that are necessary and expensive?
When the Joel Test was written, many of the open source languages we use weren’t popular and some didn’t even exist. But they exist now and this point should be modified to reflect that.
When open source tools are available, and they are the best, you should by all means make use of them. Many languages, applications and platforms are available for free through the open source community, and they are just as good or, in some cases, better than the paid tools you can buy (feature set for PHP versus ASP or .NET, anyone?).
But when it comes to paid tools that are both necessary and expensive, are you willing to shell out the cash? Do you buy your developers awesome machines? Do you cater lunch for them? Do you take care of them with software and books and tools? If you don’t or are unwilling to spend the money to do so, this is a problem.
10. Do you write unit tests?
In order for tools like continuous integration and a build system to be effective, you must unit test your code. Unit tests are an effective way to ensure that your code is accurate, that refactoring doesn’t break the expected results, and that your release is ready to go.
Lots of projects don’t use unit tests. But more projects should. Writing unit tests is a critical component of identifying early on that something is wrong. In fact, one development methodology called Test Driven Development relies solely on writing tests first, before any other code is written.
Unit tests are necessary to ensuring the integrity of your application.
11. Do new candidates demonstrate their code knowledge in some tangible way during the job interview?
I opted to change this point due to the fact that some people were asking too much in interviews.
You want to ask candidates to demonstrate their code knowledge – either by writing some simple functions or discussing programming concepts and topics with you. You might show them some code and have them debug it; this is also a great way to find out if a candidate is qualified.
Try not to make this into too much of a quiz scenario; coding knowledge is displayed in logic, thought process, and architecture skills, not in the memorization of function argument orders and obscure data. The general rule of thumb is that if you can find it in the documentation, don’t ask about it in an interview. Google is a great resource when on the job.
12. Do you have testers?
Why have I dropped hallway testing and opted for testers instead? The reason for me is simple: in an excellent development shop there is an entire team devoted to UI, who should be responsible for the data getting into the system and formatting it when it comes out. The programmers should very rarely interface with the UI team’s work.
This is not to say that developers should have no user experience training. They should be competent enough to chime in on the discussions and potentially work with the user interface if necessary. But the hallway testing should take place on the UI team side of the house.
Testers are another matter altogether. They are required for both UI and development. Someone needs to bang on code and figure out how it’s broken, and send it back for fixing. The testing department can be shared with the UI team and the programmers, but should be independent and able to veto any release of an application that doesn’t meet the spec and function well.
This update of the Joel Test for the modern web world should serve you well when looking for a new job or rating your existing shop. It integrates the most important components of the original test while adding new components and deleting dated ones, as well as adapting the test to a web-centric development process. Good luck!