No thinking beyond this point

My priority is higher than yours....

2011-11-27T11:50:00.000+01:00

Don't chop up your work

When more than one task must be completed, it is always better to do one before the other. I discussed that earlier.

An individual customer with a new question could think that it is even quicker if you fit his project in between. We have to make them aware that if we allow interruptions of any kind in an agile sprint, nothing will ever be completely done. What can we tell them? There will be other people with new requests while we are working on your task.... should we honor their requests to interrupt our work too?

They should understand: agile sprints are not interrupted. They are either completed, or canceled.

Image from Flickr by brittgow

Decision tree for scientific programmers in bioinformatics

2011-11-11T23:34:00.001+01:00

This is one of the syndromes we're trying to fight in BioAssist...

Lessons from social media in a distributed project environment

2011-09-11T15:21:00.000+02:00

The project management podcast episode 189, talks about what projects can learn from social media. I recognized many points relevant to our organization!

Power? Manage like a CEO!

2011-08-28T20:24:00.001+02:00

In an organization with more than two people, most likely there is a hierarchy. Some people are managing others.

On the other hand, project leaders are not necessarily managers of the members of the project team. Project leaders can not give orders to get things done. In our organization it is even worse: project team members are employed by different universities and hospitals, and the project leaders are in our separate group. Orders will not work.

A question that comes up sometimes is: how can one manage a project and its team members without giving orders? Sometimes this comes in the form: for you things are much easier, you are the boss and can tell everyone what to do; I do not have such power over the project team. My answer to that is: how often do I order you what to do? And if I'm not ordering you what to do, how do we run a coherent organization?

The clue is: don't tell people what to do, convince them instead. Build trust (this takes a while), and then use your expertise to convince the members of the team to align.

In fact in larger organizations, managers do not use orders a whole lot. Trust (and expertise) are much more effective. And this gets stronger higher up in the organization, up to the CEO. The biggest problem with managerial power is that it wears off when you use it. Trust grows when used properly.

Don't manage by giving orders. Act like the big company CEO.

For more inspiration, listen to this Manager Tools cast.

[Image credit: Giving orders by lincolndisplayimages.com on flickr]

Two meanings of the word chemistry, with different connotations

2011-06-25T15:57:00.000+02:00

Chemistry as a noun has two completely distinct meanings in every day life:

A good social relationship:

"It was visible that there was chemistry between those two people"

Something related to a compound that is supposedly bad for people or the environment. "Chemical" is often used as synomymous with poisonous:

"A chemical leaked from the container into the sea, endangering the fish"

How come these two meanings of the same word have such extremely different connotations? After all, the scientific word chemistry represents any kind of reaction between two compounds and does not have any positive nor negative meaning in itself. Water is a chemical. Life is chemistry.

As a chemist, I wish I could change the negative connotation of molecular chemistry in the news. But if I really do not succeed, maybe I can influence the social meaning of chemistry to make things consistent:

"There was chemistry between those two! When they first met, she tried to poison him. As soon as he recovered he exploded in anger."

Somehow I feel this would not be as satisfying.

[image credit: Nic McPhee on flickr]

Never forget the real purpose of Agile! Bureaucracy is your enemy.

2011-03-19T12:53:00.000+01:00

Agile in software development primarily means that the customer must feel that you can move immediately.

To be able to deliver results quickly, guaranteed quality is essential. You must make sure that nothing you change breaks existing functionality.

However, if you try to insure your quality by adding release regulations and code review procedures or by increasing formalities by making someone into a release manager, this defies the agility goal. A two minute fix of a bug becomes a two week ordeal.

Instead, you should ensure quality through automated testing. Continuous integration tools never get tired of running the same tests over and over again. They are always available when you need them, never make mistakes, and are fast. They are the agile quality guarantee.

Scrum is for unknown roads

2011-02-19T18:40:00.001+01:00

There are two ways of car navigation: non-agile and agile. Non agile navigation is when you take a sheet of paper with you with instructions like this:

Drive 5km. Take a left. At the third light to the right, then before the blue building to the left. Stop when you arrive at a gas station.

Non-agile navigation works fine if there is a path to the destination that is well known, and where you can't make a mistake. If anything changes on the path, if there are roadworks, if you make any mistake along the route: there is no contingency; it will be impossible to reach the destination. Furthermore: If there is nobody around that has been to the same destination (even several times), it will be impossible to get a detailed description of the route.

Agile navigation is using road signs. To get to the San Francisco Museum of Modern Art, you first follow signs to California, then San Francisco, and when you arrive there, you follow signs for SFMOMA. The only thing you need to know is where you're going, and roughly where that is (recursively, if you want). This method of navigation is very robust: even if a street becomes a one-way street, or if the blue building is torn down, you will still be able to get to your destination. What's even better is that if, along the way, your exact goal changes, you will still be able to change your plans. On your way to San Francisco, you may still decide that you want to visit the Golden Gate instead.

Now, of course, I am not trying to teach you how to navigate a car. I just want to use this as a metaphor for the development of software.

How is software development like navigation? You normally do not exactly know what you need to develop. And only rarely can you exactly follow the tracks of somebody else. Software development is not like the known path that is suitable to non-agile navigation.

Nevertheless, a lot of software is still developed using non-agile methods. Specifications are fixed completely before the development is started. The exact steps required are written up in detail. Contracts are signed. And then road blocks occur, and the project goes over budget and falls behind schedule. And when the project is delivered, the customer is not happy with the functionality because either his ideas have changed, or he has not been able to express himself accurately enough in the specifications.

Agile software development has all of the advantages of agile navigation. You can go to unknown places. You can even change the details of your plans along the way. And it is very robust against unexpected road blocks. It is clearly the right choice.

Agile software development is not easy. But it is less prone to utter failure than the traditional method. Let's learn to navigate on the road signs!

Remote meeting participants? We use Skype and VNC

2011-01-30T01:02:00.002+01:00

We did experiment quite a bit with different ways of having our group meetings. At most of our meetings some of the group members are joining us from remote locations, and we want the remote members to be full meeting participants.

For a little while we only shared the sound with the remote participants, using skype. That worked rather well, but it brought the difficulty that the remote participants could not see the same screen that we were projecting on the wall in our meeting room. If we had only one remote participant, we could transmit a lo-fi image through video skype, but this was hardly better than no screen at all.

We then started trying different web-meeting software: a free trial of gotomeeting, and the free version of dimdim (when it was still available). Gotomeeting had the problem that it is not supporting Windows, Mac and Linux equally. Dimdim had better support for different operating systems, but had much worse sound quality and the video was sometimes completely black or delayed by as much as 10 seconds. These "official" solutions did therefore not lead to better communication for us.

Our problem was solved when we realized that we could use skype for audio, and all share the same screen by using VNC. We installed a VNC server with (readonly) java client on one of our linux servers. During a meeting, the meeting facilitator is in control via a read-write VNC session while the other participants only watch. Audio connection is done via skype as before, delivering good quality audio through a special external speaker/microphone combination.

This made us very happy, without monthly subscription costs. I'd like to hear in the comments about other alternatives you have explored.

Update: David van Enckevort explains in a posting on his blog how to set up a VNC server like we did.

Parallel processing in a project team

2010-12-07T22:14:00.000+01:00

How important is it to prioritize?

Lets assume we have 4 ideal projects A, B, C, D, and an ideal project team. Each of these projects takes 3 months to complete. They are all equally important. We work on them in parallel. When will the projects be ready? Project A will be ready after 12 months. Project B will be ready in 12 months. Project C will be ready in 12 months. And Project D will be ready in 12 months.

Now, what happens when we prioritize and work on the projects in alphabetical order? When will the projects be ready now? Project A will be ready after 3 months. Project B will be ready after 6 months. Project C will be ready after 9 months. And project D will be ready after 12 months. Everyone except the customer of project D will be better off! Fantastic, isn't it?

Unfortunately, this is not the perception of your customers. Each of them sees a 3 month project and wants it finished in 3 months. The customer in project D does not want to hear "we will start on your project in 9 months". All too often, priorities therefore change. Every month, at a project team review, you will be forced to reprioritize. What is the effect? Project A will be ready after 9 months. Project B will be ready after 10 months. Project C will be ready after 11 months. Project D will be ready after 12 months.

What a waste. Don't fall into this trap. Prioritize, and stick with the choice.

[Image credit: goldstardeputy]

The difference between what people want and what they ask for

2010-06-19T22:41:00.000+02:00

A software shop like ours should deliver what customers want... but it may be difficult, because they often do not ask what they want. This is because customers think they know what causes a problem and they think they know the best way to solve it. They then formulate their request in an attempt to help us.

An example: I once had customers asking me whether I could change my software so that it would round the numbers that it would use to position a robot. It would have been easy to satisfy that request, but I decided to ask why? This proved to be a good idea. I found out the customers were copying the numbers into some other software package. Rather than doing what the customers asked, I ended up writing a direct interface to the other software. This made life of the users much easier yet, without limiting the possibilities of the robot.

We can not blame customers for not knowing what is easy and what is difficult to implement. Both ways. They can think that something is very easy, when in fact it is fundamentally very hard. But it also happens that they do not dare to ask a question they think is hard, when in fact it would be very easy.

If you want to make the best possible software, you need to keep asking "why" until your user's report has been changed to "If I do A, I get B. But instead of B I would like to see C (because I need D)". This will help you to decide how customer satisfaction can be maximized. The maximum may be much higher than your customers expect.

Lets go build some obsolete tools.... and prevent being blamed.

2010-04-24T20:43:00.000+02:00

One of the first stages in the development of a new tool (software or hardware) is a functional specification. The functional specification matures in discussions between the developers (department of R&D) and the customer representatives (often the departments of marketing and sales).

Of course a functional specification is useful: it is very hard to develop something new without an idea of how the new tool will be used and what it will be compared with. However, defining a functional specification can also be taken too far. In some organizations, the functional specifications are spelled out in the tiniest details. At the end of a long formal procedure, the book of specifications is signed like a contract between marketing and development. The development can only be started when the list of signatures is complete and will be performed in splendid isolation from the world of potential users. Why do organizations do specifications this way? It is often an attempt to separate the responsibilities of the departments, so that if anything fails the appropriate party can be blamed. If the final product does not meet the functional specifications, this can be blamed on the developers. And if the product does not succeed even though it meets all the functional specifications, this will be blamed on marketing.

I have a serious problem with this approach: using this procedure, how can one ensure that the product will be useful? After two years without interaction, development may produce exactly what marketing asked for (making all deadlines and within budget), but the market has changed and does no longer need the designed product. Or technology has changed, and better specifications would have been within reach and are offered by the competition. Or maybe marketing truly made a mistake, and asked for something the world is not waiting for. In these cases, clearly the development department can not be blamed! But if you develop an obsolete product this way, where does that leave the organization as a whole? And if this is not the best solution for the organization as a whole, will it be good for the development department? Even though everyone did exactly what was expected, people may be laid off because development costs can not be recovered.

The solution is, as often, to keep a middle road. Using e.g. Agile or LEAN development methods, developers can stay in constant communication with marketing. Iterative and modular design procedures can be used to verify that the new tool does what it should, without relying on the capacity of people to describe specifications in words beforehand. And because the communication with the market is not lost during the development process, the tool will have a significantly higher chance of actually being useful at the moment of introduction.

Image (by QuiteLucid on Flickr): "a camel is a racing horse designed by a committee".

Abstract or concrete? It all depends on your point of view.

2010-04-19T22:36:00.000+02:00

Our team is working with a number of task forces in bioinformatics. Each of those task forces was started to collaborate on the development of a platform for their sub-field: a set of software tools that work together to solve the problems that everyone in the field needs to solve. Developing the platform does not require any new bioinformatics developments: the purpose is to put existing tools together.

The advantage of having these platforms available is obvious:

to a biologist the advantage consist of having all the de facto standard tools available under the press of a button.
to a specialist bioinformatics researcher working on a new tool the advantage is that he does not have to deal with the intricacies of all the other tools, and is able to plug his new tool into the platform using well described protocols.

To get to the development of such a platform there is a bootstrapping problem. The situation is like a table with biologists sitting on one side, bioinformaticians at the other side. Above the table, a thick (volcanic?) fog. The layout of the platform is drawn in diagrams on the table: all the tools making up the common work flow, with all their relations. On the side of the bioinformaticians, the diagram shows the concrete tools. Through the fog, they can vaguely see the workflow on the other side of the table. For the biologists, the situation looks completely different: they have a clear view on the concrete workflow they need, but the tools are vague entities that are only visible through the thick fog.

Without good support from a project leader that can listen to people on both sides of the table, the bioinformaticians will try to solve the very concrete problems they encounter on their very concrete individual tools. A little optimization here, a better data storage facility there. None of this is visible for the biologists.

This is why we put project leaders from our engineering team into each of the task forces. They will direct the focus of the bioinformaticians towards more visible changes. Work on common data formats. Work on (common) user interfaces.

Getting things to work together will bootstrap the true collaborative advantages. It will blow away the fog. Suddenly the biologists will be able to see what is going on. They will be able to provide directed feedback. And the bioinformaticians will be able to see the workflow even from their side, and build upon it.

Image credit: Three views of three tables, by EJP Photo on Flickr.

Who would go welding without a drawing?

2010-03-20T09:02:00.000+01:00

Imagine a mechanical workshop. You come with a problem that they will solve. What do you expect? You expect to work together with them, roughly in the following five-step procedure:

Specify. You will describe the problem in the form of a functional specification
Design. They will make a drawing
Pieces. They (possibly multiple people in parallel) will use the design to select and manufacture pieces
Product. They will weld the pieces together into the tool you need
Test. You test it, and it may require minor adjustments

Now imagine a software workshop. You come with a problem that they will solve. What do you expect?

Specify. You will describe the problem in the form of a functional specification
Product. They will make the software tool
Test. You alpha test, they fix, you beta test, they fix. Repeat until satisfactory

How come this list contains only three instead of five steps? And why is the last step always causing so much pain? Could this difference be the origin of why so many software products fail? What is the real difference between software and hardware development?

Rather than trying to solve the software problem at once, first imagine a hardware workshop that works like this:

Specify. You will describe the problem in the form of a functional specification
Product. They will weld materials together into a tool
Test. You test, they fix, you test again, they fix. Repeat until satisfactory

How likely would it be that this procedure is faster than the five step procedure? How obvious is it which pieces need to be welded together? Will this allow multiple people to work together on the product? And how much work is the testing for you? How much waste is produced in the process? You would not accept this kind of quackery! And my point: you should not accept it from a software workshop either.

A good software workshop would use the same five steps the good mechanical workshop uses:

Specify. You will describe the problem in the form of a functional specification
Design. They will make a (modular) design
Pieces. They (possibly multiple programmers in parallel) will use the design to build and select software modules
Product. They will use the modules to build the software you need
Test. You test it, and it may require minor adjustments

Investment in a design will result in a solution that will truly solve your stated problem, and not require endless iterations to get right. It may look like a slow solution, but that is deception: you will actually get a result that gets you where you want to be, also when you have new additional requirements in the future.

I'd like to thank a former colleague at Bruker AXS that gave me this nice comparison. I know he wishes to remain anonymous on the 'net. Image credits: Dystopos on Flickr.

Radical change rarely brings immediate improvement

2010-02-24T20:42:00.000+01:00

After every radical change in an organization, there is a need for a phase of quiet thoughtful improvements. Expecting miracles from huge corporate reorganizations is a fallacy that leads to reorganization upon reorganization potentially resulting in complete destruction of the organization.

Have you ever played a game of Pac-man? It is a simple game where you control a little eater eating dots on the screen, while ghosts are chasing you. The game is an excellent mirror of business life in a changing environment:

In Pac-man, you are trying to improve your health at every step by eating a dot and staying out of the way of the ghosts.
In business life, you are making small changes to your products and procedures to sell more and stay out of the way of your competitors.

There is a further analogy:

In Pac-man, sometimes things get stuck. Ghosts are closing in from all sides, and there is no escape. At such a point, you can use the teleport: a panic key that takes you to a random spot in the scene in an instant.
In business life, sometimes things get stuck. Competitors are closing in and it seems there is no way out. At such a point the CEO will call (often quickly without consulting all those that are involved) for a radical reorganization.

In business there is an important lesson we can learn from the teleport feature in Pac-man: A teleport is far from a guaranteed save! It can bring you into a very dangerous situation. The goal of the teleport is not an immediate improvement in the flow of the game, it is to escape from a hopelessly stuck situation, from impending disaster. Directly after a teleport, you have to act and make steps to regain control. Similarly, in business a radical reorganization will rarely take you to a better situation immediately. A reorganization is meant to shake up the bowl and escape from a hopelessly stuck situation (often invisible to many of the employees). After the relatively thoughtless jump that has to be executed quickly to avoid an immediate game over, the organization will need to go into a thoughtful phase in which small improvements are made to optimize the situation.

If you realize that a reorganization has not brought you immediate gains, try to refrain from making further reorganizations. Instead, look for opportunities for small changes, and give it some time.

Is my software any good?

2010-02-23T22:31:00.001+01:00

If you are not getting any user feedback for your software, there are two possible reasons.

It is bad. Nobody uses it.
It is good. Everyone is happy.

If this happens to you, think back. Did you ever get any feedback before? How did you react?

Did you listen to your users and fix their problems?
Did you teach your users the way your software should be used?

By answering these two questions you can figure out for yourself why you no longer get feedback. If you listened, and the stream of questions stopped, this probably means the users are now happy. If you attempted to correct their usage, most likely nobody uses it any more.

You did remember to include your contact details, did you?

It is a dirty job but somebody has got to do it

2009-11-15T19:49:00.000+01:00

Not everybody likes the same kind of work. This is a great opportunity for people involved in a group, if it is recognized, and if communication about what everyone likes and dislikes is open.

I once heard a terrifying story of an old couple that celebrated their 40th wedding anniversary. As one of the parts of the celebration they made the appointment that they would communicate for once about something they dislike about the other. The man starts: "my dearest, I love almost everything you do for me. But if there is one thing that I would like to change it is the way you cut the bread. You always serve me the heels of the bread, and I much prefer a normal center slice". Upon this, the woman nearly fainted. She had served her husband the heels of the bread for 40 years, only because this is the part she likes best.

In any enterprise it is important that all jobs are done, including the dirty ones. Sometimes we just have to do things we do not like. But in a collaborative effort, it is important to communicate about which parts of the work we like and dislike. Maybe, just maybe, a colleague would love to take over part of the work that you despise.

Academic software

2009-10-25T12:07:00.001+01:00

A lot of very good software in academia is developed by PhD students or post-docs. Both groups have temporary contracts, and very often the software dies at the end of the contract.

One can compare academic software with a Formula 1 race car: it is very fast, beautiful, technologically advanced. But it is very difficult to drive and needs frequent service. The only way of keeping it on the road is to have the driver and the designer working closely together.

To make it possible to use the technology that is developed for the Formula 1 race car in practice, the car must be redesigned:

It must be possible for anyone with a normal drivers license to drive it.
It has to be possible to drive on different types of road, not only a polished race track.
It must be possible for a regular trained service engineer to service it.
The service interval must be raised from 200 km to 25000 km.

It is obvious that in car design, these changes are not the responsibility of the designers in the Formula 1 team. There are other designers that are specialized in normal cars.

The equivalent concepts in software design are:

The program must be made user-friendly. It must be usable by casual users that are not also computer hackers.
It must be robust enough to work on real data as encountered by real users.
Running and installation must be well documented.
It must be able to run unsupervised for many data sets and not break down after limited time e.g. through a seemingly unrelated upgrade to the operating system or a minor change to an underlying web service.

Funny enough, in many larger projects it is expected that the designers of the Formula 1 software will also do this redesign. This is a strong contrast with car design! In NBIC-II we will not make this mistake. Taking software the extra step will be one of the important tasks of our Central Engineering Team. And this team will contain specialists in industrial strength software engineering.

We will turn the Ferraris into Volkswagens. And we will be proud of them!

We have a business plan!

2009-10-24T22:25:00.001+02:00

This week, the Management Team of NBIC has agreed to a new operation plan for the BioAssist unit. This now puts the BioAssist Engineering Team on firm ground. We will have three important tasks:

Guarantee interoperability of biological data
Be consultant and project leader for the academic task forces within BioAssist
Accept academic software and make it usable by others

I am excited to get going.

New job; the purpose of this blog

2009-09-05T21:24:00.001+02:00

Last Tuesday, September 1st 2009, I started a new job as CTO of NBIC. NBIC is a Dutch not-for-profit organization (foundation) that has as prime target to bring BioInformatics in the Netherlands to a higher standard.

Until now, NBIC has been run as a small administrative office in Nijmegen, coordinating research ("BioRange" program), education ("BioWise" program) and support ("BioAssist" program) at several different academic locations in the Netherlands.

We are now setting up a virtual workshop as part of the NBIC office, operating as part of the BioAssist program. This office will be working on data and on software development. We will start where the research groups end. We will take what is produced, and continue development by applying industrial design standards to it. Above the virtual door to our virtual office there is a virtual sign:

No thinking beyond this point.

Sometimes, however, in the office, we can not help having our own thoughts. I will use this blog to write about the way we run our office together, including obstacles. And occasionally, I may add a little thought I have had.