Tuesday, December 7, 2010

Parallel processing in a project team

How important is it to prioritize?

Lets assume we have 4 ideal projects A, B, C, D, and an ideal project team. Each of these projects takes 3 months to complete. They are all equally important. We work on them in parallel. When will the projects be ready? Project A will be ready after 12 months. Project B will be ready in 12 months. Project C will be ready in 12 months. And Project D will be ready in 12 months.

Now, what happens when we prioritize and work on the projects in alphabetical order? When will the projects be ready now? Project A will be ready after 3 months. Project B will be ready after 6 months. Project C will be ready after 9 months. And project D will be ready after 12 months. Everyone except the customer of project D will be better off! Fantastic, isn't it?

Unfortunately, this is not the perception of your customers. Each of them sees a 3 month project and wants it finished in 3 months. The customer in project D does not want to hear "we will start on your project in 9 months". All too often, priorities therefore change. Every month, at a project team review, you will be forced to reprioritize. What is the effect? Project A will be ready after 9 months. Project B will be ready after 10 months. Project C will be ready after 11 months. Project D will be ready after 12 months.

What a waste. Don't fall into this trap. Prioritize, and stick with the choice.

[Image credit: goldstardeputy]

Saturday, June 19, 2010

The difference between what people want and what they ask for

A software shop like ours should deliver what customers want... but it may be difficult, because they often do not ask what they want. This is because customers think they know what causes a problem and they think they know the best way to solve it. They then formulate their request in an attempt to help us.

An example: I once had customers asking me whether I could change my software so that it would round the numbers that it would use to position a robot. It would have been easy to satisfy that request, but I decided to ask why? This proved to be a good idea. I found out the customers were copying the numbers into some other software package. Rather than doing what the customers asked, I ended up writing a direct interface to the other software. This made life of the users much easier yet, without limiting the possibilities of the robot.

We can not blame customers for not knowing what is easy and what is difficult to implement. Both ways. They can think that something is very easy, when in fact it is fundamentally very hard. But it also happens that they do not dare to ask a question they think is hard, when in fact it would be very easy.

If you want to make the best possible software, you need to keep asking "why" until your user's report has been changed to "If I do A, I get B. But instead of B I would like to see C (because I need D)". This will help you to decide how customer satisfaction can be maximized. The maximum may be much higher than your customers expect.

Saturday, April 24, 2010

Lets go build some obsolete tools.... and prevent being blamed.

One of the first stages in the development of a new tool (software or hardware) is a functional specification. The functional specification matures in discussions between the developers (department of R&D) and the customer representatives (often the departments of marketing and sales).

Of course a functional specification is useful: it is very hard to develop something new without an idea of how the new tool will be used and what it will be compared with. However, defining a functional specification can also be taken too far. In some organizations, the functional specifications are spelled out in the tiniest details. At the end of a long formal procedure, the book of specifications is signed like a contract between marketing and development. The development can only be started when the list of signatures is complete and will be performed in splendid isolation from the world of potential users. Why do organizations do specifications this way? It is often an attempt to separate the responsibilities of the departments, so that if anything fails the appropriate party can be blamed. If the final product does not meet the functional specifications, this can be blamed on the developers. And if the product does not succeed even though it meets all the functional specifications, this will be blamed on marketing.

I have a serious problem with this approach: using this procedure, how can one ensure that the product will be useful? After two years without interaction, development may produce exactly what marketing asked for (making all deadlines and within budget), but the market has changed and does no longer need the designed product. Or technology has changed, and better specifications would have been within reach and are offered by the competition. Or maybe marketing truly made a mistake, and asked for something the world is not waiting for. In these cases, clearly the development department can not be blamed! But if you develop an obsolete product this way, where does that leave the organization as a whole? And if this is not the best solution for the organization as a whole, will it be good for the development department? Even though everyone did exactly what was expected, people may be laid off because development costs can not be recovered.

The solution is, as often, to keep a middle road. Using e.g. Agile or LEAN development methods,  developers can stay in constant communication with marketing. Iterative  and modular design procedures can be used to verify that the new tool does what it should, without relying on the capacity of people to describe specifications in words beforehand. And because the communication with the market is not lost during the development process, the tool will have a significantly higher chance of actually being useful at the moment of introduction.

Image (by QuiteLucid on Flickr): "a camel is a racing horse designed by a committee".

Monday, April 19, 2010

Abstract or concrete? It all depends on your point of view.

Our team is working with a number of task forces in bioinformatics. Each of those task forces was started to collaborate on the development of a platform for their sub-field: a set of software tools that work together to solve the problems that everyone in the field needs to solve. Developing the platform does not require any new bioinformatics developments: the purpose is to put existing tools together.

The advantage of having these platforms available is obvious:
  • to a biologist the advantage consist of having all the de facto standard tools available under the press of a button.
  • to a specialist bioinformatics researcher working on a new tool the advantage is that he does not have to deal with the intricacies of all the other tools, and is able to plug his new tool into the platform using well described protocols.
To get to the development of such a platform there is a bootstrapping problem. The situation is like a table with biologists sitting on one side, bioinformaticians at the other side. Above the table, a thick (volcanic?) fog. The layout of the platform is drawn in diagrams on the table: all the tools making up the common work flow, with all their relations. On the side of the bioinformaticians, the diagram shows the concrete tools. Through the fog, they can vaguely see the workflow on the other side of the table. For the biologists, the situation looks completely different: they have a clear view on the concrete workflow they need, but the tools are vague entities that are only visible through the thick fog.

Without good support from a project leader that can listen to people on both sides of the table, the bioinformaticians will try to solve the very concrete problems they encounter on their very concrete individual tools. A little optimization here, a better data storage facility there. None of this is visible for the biologists.

This is why we put project leaders from our engineering team into each of the task forces. They will direct the focus of the bioinformaticians towards more visible changes. Work on common data formats. Work on (common) user interfaces.

Getting things to work together will bootstrap the true collaborative advantages. It will blow away the fog. Suddenly the biologists will be able to see what is going on. They will be able to provide directed feedback. And the bioinformaticians will be able to see the workflow even from their side, and build upon it.

Image credit: Three views of three tables, by EJP Photo on Flickr.

Saturday, March 20, 2010

Who would go welding without a drawing?

Imagine a mechanical workshop. You come with a problem that they will solve. What do you expect? You expect to work together with them, roughly in the following five-step procedure:
  1. Specify. You will describe the problem in the form of a functional specification
  2. Design. They will make a drawing
  3. Pieces. They (possibly multiple people in parallel) will use the design to select and manufacture pieces
  4. Product. They will weld the pieces together into the tool you need
  5. Test. You test it, and it may require minor adjustments
Now imagine a software workshop. You come with a problem that they will solve. What do you expect?
  1. Specify. You will describe the problem in the form of a functional specification
  2. Product. They will make the software tool
  3. Test. You alpha test, they fix, you beta test, they fix. Repeat until satisfactory
How come this list contains only three instead of five steps? And why is the last step always causing so much pain? Could this difference be the origin of why so many software products fail? What is the real difference between software and hardware development?

Rather than trying to solve the software problem at once, first imagine a hardware workshop that works like this:
  1. Specify. You will describe the problem in the form of a functional specification
  2. Product. They will weld materials together into a tool
  3. Test. You test, they fix, you test again, they fix. Repeat until satisfactory
How likely would it be that this procedure is faster than the five step procedure? How obvious is it which pieces need to be welded together? Will this allow multiple people to work together on the product? And how much work is the testing for you? How much waste is produced in the process? You would not accept this kind of quackery! And my point: you should not accept it from a software workshop either.

A good software workshop would use the same five steps the good mechanical workshop uses:
  1. Specify. You will describe the problem in the form of a functional specification
  2. Design. They will make a (modular) design
  3. Pieces. They (possibly multiple programmers in parallel) will use the design to build and select software modules 
  4. Product. They will use the modules to build the software you need
  5. Test. You test it, and it may require minor adjustments
Investment in a design will result in a solution that will truly solve your stated problem, and not require endless iterations to get right. It may look like a slow solution, but that is deception: you will actually get a result that gets you where you want to be, also when you have new additional requirements in the future.

I'd like to thank a former colleague at Bruker AXS that gave me this nice comparison. I know he wishes to remain anonymous on the 'net. Image credits: Dystopos on Flickr.

Wednesday, February 24, 2010

Radical change rarely brings immediate improvement

After every radical change in an organization, there is a need for a phase of quiet thoughtful improvements. Expecting miracles from huge corporate reorganizations is a fallacy that leads to reorganization upon reorganization potentially resulting in complete destruction of the organization.

Have you ever played a game of Pac-man? It is a simple game where you control a little eater eating dots on the screen, while ghosts are chasing you. The game is an excellent mirror of business life in a changing environment:
  • In Pac-man, you are trying to improve your health at every step by eating a dot and staying out of the way of the ghosts.
  • In business life, you are making small changes to your products and procedures to sell more and stay out of the way of your competitors.
There is a further analogy:
  • In Pac-man, sometimes things get stuck. Ghosts are closing in from all sides, and there is no escape. At such a point, you can use the teleport: a panic key that takes you to a random spot in the scene in an instant.
  • In business life, sometimes things get stuck. Competitors are closing in and it seems there is no way out. At such a point the CEO will call (often quickly without consulting all those that are involved) for a radical reorganization.
In business there is an important lesson we can learn from the teleport feature in Pac-man: A teleport is far from a guaranteed save! It can bring you into a very dangerous situation. The goal of the teleport is not an immediate improvement in the flow of the game, it is to escape from a hopelessly stuck situation, from impending disaster. Directly after a teleport, you have to act and make steps to regain control. Similarly, in business a radical reorganization will rarely take you to a better situation immediately. A reorganization is meant to shake up the bowl and escape from a hopelessly stuck situation (often invisible to many of the employees). After the relatively thoughtless jump that has to be executed quickly to avoid an immediate game over, the organization will need to go into a thoughtful phase in which small improvements are made to optimize the situation.

If you realize that a reorganization has not brought you immediate gains, try to refrain from making further reorganizations. Instead, look for opportunities for small changes, and give it some time.

Tuesday, February 23, 2010

Is my software any good?

If you are not getting any user feedback for your software, there are two possible reasons.
  1. It is bad. Nobody uses it.
  2. It is good. Everyone is happy.
If this happens to you, think back. Did you ever get any feedback before? How did you react?
  • Did you listen to your users and fix their problems?
  • Did you teach your users the way your software should be used?
By answering these two questions you can figure out for yourself why you no longer get feedback. If you listened, and the stream of questions stopped, this probably means the users are now happy. If you attempted to correct their usage, most likely nobody uses it any more.

You did remember to include your contact details, did you?