Moving the Blog to Octopress

I’ve decided to switch the blog from to an Octopress site hosted on Github Pages. From now on there will be no more new posts on WordPress. I’ve also registered the domain and will be using this address for the blog in future.


Test Automation

Update: This blog is now at and this post has been moved to here.

Wherever possible I think that automating your testing is essential. For this project there is no way that I can manually rerun the same tests over and over again – the time commitment is just too much. I need to be able to click a button and have a framework run all my tests automatically and report back to me the number of tests passing, failing etc. I know that this won’t catch all the bugs in the code and that I’ll still need to do some manual testing for each release. However, the more I can push into this automated testing, the more time I’ll save on the manual testing later.

Based on my previous post I’ve mentally divided up the code into different layers which build on top of each other. The app GUI is one layer. Then we have the model classes which in turn depend on utility classes etc. These different levels of code require different levels of testing.

For the utility and model layers I plan to use an automated unit testing framework such as JUnit. Xcode provides OCUnit which is a JUnit like framework for implementing unit tests. This framework provides the usual fail and assert macros and functions. There are some annoying things about running unit tests in Xcode. One is that it doesn’t have a separate test runner panel like JUnit has in Eclipse. Instead it treats the unit test errors as build failures. All the print output for the tests is dumped into the same console output which can make it tricky to identify which output is for which test. Also there is no way to selectively run these tests. You have to rerun them all together every time. There are other frameworks out there e.g. GHUnit but for the moment I’ll stick with the default.

For the higher level GUI code, unit testing is not useful so I’ll need a different testing approach and toolset. There seem to be number of options out there. I’m not sure what is available and what is the best to use. Apple have an Instruments application which allows you do perform UI testing using Javascript. I’ve also seen these Frank and Zucchini frameworks. I haven’t evaluated any of these yet as I don’t have a fully functional app to test it on. Once I get a nearly complete app I can see how much my unit testing provides, compare that to what I have left to test, and then make a judgement on where to go from there.

Devising my Unit Testing Philosophy

Update: This blog is now at and this post has been moved to here.

Up to now my approach to unit testing has been dictated by who I was writing the code for. For instance in college I never wrote unit tests. In the real world some of my employers required unit testing, with certain branch and code coverage goals, while other employers didn’t require much in the way of unit testing, as long as other types of testing were carried out. In practice I found that the as the code became more lower level, the demands for unit tests increased.

For this project I need to define for myself what level of testing I will perform. My initial assumption was that of course I would do unit testing. Personally I thought that the benefits of unit testing were

  • the confidence that it gives me in my code. I know that I can call this piece of code from somewhere else and there is a high probability that it will work as intended.
  • by catching errors at an early stage, I spend less time in the debugger later.
  • refactoring should go easier as the unit tests will catch a lot of regression issues
  • I can be confident that any new bugs will be in the new code rather than the older highly unit tested modules, thus cutting down the amount of code required to search in order to fix the bug.

However I read an interesting and thought provoking article by Will Shipley outlining why he doesn’t do unit testing. My immediate reaction was astonishment. How can you not have unit tests? However instead of unit tests, he beta tests the product. This finds the type of bugs that the actual end users will find. Also he writes his code in a manner that doesn’t require unit tests.

My programming philosophy is “less code is better code.” Unit tests take a lot of code, and in my projects I don’t find that they find very many bugs. Part of this is because I tend not to modify my individual methods much once they’ve been written; if my “append string to string” method works, it’s really not going to get revisited anytime soon unless it has a bug.

Part of it is because, in fact, I do do integrated testing of a form, and I probably should have talked about that more. Yes, I use “NSAssert” all over the place, and I perform sanity checks and raise errors if there’s a problem.

I don’t think I’ve come across “Not modifying methods once they are written” before but looking at it now it makes sense. By not changing existing working methods and treating them as black boxes, we should have fewer regression errors in future. I’m not sure how to reconcile this with refactoring though – it’s something to think about. I’d imagine a coding style that favours smaller, more focused methods would be more suited to this approach.

His use of sanity checks and asserts also struck a chord especially when he says

Debugging most errors, once found, should take very little time in a properly-written program, because the error should always be from one of your own sanity checks, and you know where those are in the code, so you just go there and figure out what went wrong.

This was eye-opening for me. I’ve never tracked this before so I’m not sure if the bugs I’ve written have evaded my sanity checks and asserts. I would imagine a lot of them did. I’d never have seen that as important before but looking at it now it seems to me that this is a failure on my behalf. Not catching these errors means that I’m not fully aware of how my code can go wrong and what it is doing in all cases. I think a useful analysis I should run on each bug I find is – did this escape my coding checks/asserts and if so why? I should learn from this and in future code should check for this condition.

In a response to Will’s article, a follow up post by bbum shows how unit testing was of great benefit on infrastructure code, in this case the Core Data library. This would suggest to me that the utility and model code would be amenable to unit testing but that it’s not useful for the GUI code.

Another article on codeville challenged my assumptions that unit tests were about finding bugs. Instead the type of testing that finds bugs is manual testing or automated integration testing. According to him, unit testing is not about finding bugs (except during refactoring) but more about designing robust software components.

TDD is a design process, not a testing process. TDD is a robust way of designing software components (“units”) interactively so that their behaviour is specified through unit tests

Also it’s not enough to just write unit tests. The tests need to be either true unit tests which design a single component or else integration tests which automate the entire system. Anything in between are

dirty hybrids. Unclear goal. High maintenance, don’t prove much

Otherwise when we refactor code, we end up breaking lots of seemingly unrelated hybrid tests. If you change a unit, then its unit tests should change but no other unit tests should.

After all that I’ve decided that I want to do some level of unit testing but I realize that I need to focus it on the right areas. I think it will be useful for the utility type code that I write but that I need to do more research into how I want to implement integration testing and UI testing.

Objective C Categories and Monkey Patching

Update: This blog is now at and this post has been moved to here.

One of my favorite features of Objective C are categories. Categories allow you to add your own functionality to existing classes without the need for inheritance. You don’t even need the source code of the modified class. You are only allowed add extra methods or modify existing methods. Adding extra instance variables is not permitted. This feature is generally known as monkey patching (what a great name!) in languages such as Ruby.

Creating a category is straightforward. The interface and implementation files look the same as normal interface and implementation files except the category name is placed in brackets after the name of the class we are extending. For example to create a category MyCategory on MyClass the interface file looks like

#import "MyClass.h"

@interface MyClass (MyCategory)
    // Declare Methods here

and the implementation like this

#import "MyClass+MyCategory.h"

@implementation MyClass (MyCategory)
    // Implement methods here

The standard naming convention for category files is ClassToExtend+Category.(h|m). So for the above we would have both MyClass+MyCategory.h and MyClass+MyCategory.m files. The header file needs to be imported in any file that uses the category. For common extensions we can place them in the precompiled header (pch) files.

This has a number of applications. It allows you to add methods to classes (or indeed to alter existing methods) instead of having to create utility classes comprised of static methods. Thus the code is more clearly associated with the class to which it belongs. It allows you to create object-oriented code that favours composition over inheritance.

To give a concrete example, for my current project I wanted to be able to shuffle an array. Instead of having to create a ShuffleableArray subclass or implement an ArrayUtils.shuffle helper method, I was able to add a shuffle method to the NSArray type via a category.

Not only does the class to which you add the category get the new method, but all its subclasses gain the method also. This can be very useful if you are extending a class high up in the hierarchy. For example, another use I made of categories was to add debug methods to the UIView class e.g. to print data about itself and its subviews. Given that lots of the UI widget classes extend UIView this means that everything on screen got my new methods. I find trace methods like these very useful during unit testing and debugging. I tend to prefer adding print code where possible rather than step through the debugger, and for that case the ability to add functionality to existing classes is invaluable.

Naturally like all programming language features categories can be greatly abused. Changing the behaviour of existing functions can break code which assumes the older behaviour. I’d imaging this would be especially so with system classes such as NSArray etc. However done right, they look like a very useful feature which I’m sure I’ll use a great deal.