We had a little debate at the office the other day about how to split projects among different development streams. During the chat we exposed some examples and I thought of cataloguing here some patterns that I’ve seem for future reference.

Motivation

Agile is not about armies, it is about Commando Teams. Instead of throwing ten thousand people to the mission we send a small and efficient group. Although I’ve heard happy stories about large agile teams, in my experience it simply doesn’t work for a group above a certain number of people -probably something between 7 and 10.

Management is often responsible for an inappropriate team size. When your clients are very anxious to get the software done it is very common for some managers to fight any problem just by throwing more people at it. In this case you can always quote Fred Brooks:

The bearing of a child takes nine months, no matter how many women are assigned.

But it is very difficult for some people to believe that software development won’t scale this way (maybe due to the flat and parallel characteristics of agile development itself). I’d say that he problem is mainly a communication issue. In agile teams you value direct communication over heavily formal and structured approval/review processes but that direct communication won’t scale well for a large group. It is very hard to get a big group communicating with the required efficiency.

Some people use stand-up meetings as the main point of communication. This is not a good idea, stand-ups are about broadcasting information and commitment and not a report of every single task performed in the previous day. It is very easy to forget to talk about something relevant during a stand-up.

Others assume that rotating pairs frequently is good enough. It isn’t. Pair rotation is useful but it won’t prevent that two pairs write exactly the same code -and just realize in the next stand-up. I’ve seemed this happening more than once.

Other very important argument for split teams is that it is easier to deal smaller problems. With a smaller code base you feel more confident to refactor and improve. When you have to make sure that your change won’t break code from 30 other developers before you check-in you loose your courage and avoid significant changes –or even basic cleanups.

Who is Guilty?

Splitting a team may result in some kind of us-against-them syndrome. Some teams tend to assume a completely defensive position instead of thinking about the overall project. For me, team members must be committed to the project as a whole. If she sees something wrong in other stream she has to raise an issue about it.

There are several strategies to fight this problem. One is to have a whole-project retrospective in a regular basis, say twice a month. In the stream-specific retrospectives the topics are more specific to what that team is building, but in the bigger retrospective the focus is in the collaboration between streams. Other useful technique is rotation: A person should not be kept into a stream forever. He should be expected to work on different streams over iterations.

Alternative: Serialized Smaller Projects

Before trying any of the software development techniques below I’d suggest something more radical: instead of a single huge project it probably would be better to have a pipeline of smaller projects. Notice that this is not exactly the same as release early, release often. You can keep the one-big-project approach and still do small releases during the development –actually, you must do this. What I’m saying here is to break the project into smaller projects so you could use a smaller team.

In the Web 2.0 world is not that hard to convince the business to adopt an strategy like this, but it can be really difficult to convince conservative clients that instead of a single project they will have to wait a dozen little project to be finished.

In my experience, breaking a huge project into a pipeline of small projects won’t increase or decrease the needed effort but the projects will be obviously more manageable and therefore less risky. Having frequent releases of end-to-end projects instead of frequent releases of isolated features is an advantage as well.

Subsystems

Down to the technical level, this is probably the easiest way of splitting a project. A huge system often can be divided into modules or subsystems and in some cases you can have different teams developing each one of those.

There are basically two issues to be handled in this approach: shared code and dependencies among modules.

Been from the same system, the modules will very likely share some concepts, modeled as classes or the like. This code has to be handled in a different way. Eric Evans calls this kind of grey zone Shared Kernel:

Designate some subset of the domain model that the two teams agree to share. […] This explicitly shared stuff has special status, and shouldn’t be changed without consultation with the other team.

Integrate a functional system frequently, but somewhat less often than the pace of CONTINUOUS INTEGRATION within the teams. At these integrations, run the tests of both teams.

Other main issue is about dependencies. Subsystems should be layered -so their communication protocol is well-defined as top-down events- and should publish contracts that express what the clients layer should expect from it.

I like this approach very much and when the architecture allows it is often a good option. In a billing system I’ve worked we adopted this strategy. The system was huge and the company got a really big team of good developers to finish it on schedule. After stepping on everyone’s toes we decided to split the project into different streams. A team got the main module and two others got some of the modules that would integrate to other systems in the network –we had to communicate over something about 10 awkward/proprietary/not-documented protocols in those modules.

At the beginning we had lots of problems, the initial architecture didn’t think about modules we had too much shared code between streams. After some iterations we got to a point where every module publishes interfaces to others and the shared code has shrink to the objects used in those interfaces.

Data Production vs. Data Consumption

Some systems have a clear distinction of the code that produces data from the code that consumes it. This is pretty common on websites where there is an editor that inputs data consumed by the users, and it may be a great point to split development streams.

I think that the principal concern in this approach is how to split stories -that are supposed to deliver business value- among teams.

Eric Evans calls this “Customer/Supplier Development Teams”.

Establish a clear customer/supplier relationship between the two teams. In planning sessions, make the downstream team play the customer role to the upstream team. Negotiate and budget tasks for downstream requirements so that everyone understands the commitment and schedule.

He also recommends that you define a test suite that would perform like acceptance tests from downstream to upstream systems.

I’ve seemed this approach applied several times. In a specific project, after splitting the streams the data production team was able to choose a technology that performed better for that task and that technology change didn’t impact the whole team. The data consumption guys kept their velocity as they didn’t have to learn new tools or change anything.

The scenario tends to get more complicated if both streams share objects. You can use a Shared Kernel strategy but I most times I’ve faced a scenario like this the domains had the same concepts (the same nouns, maybe) but they would play a totally different role and have different behaviour in each stream. When you try to use a single implementation for both domains you end up with an object in which 50% of the methods and attributes are used in one stream only the remainder in the other.

Horizontal Layers

Sometimes the horizontal layers of a system are so loosely coupled that different teams can develop them. You can make a team responsible for the infrastructure layer, other for the business layer, one more for the UI, etc.

This approach may have all problems previously stated. Shared code between Layers, dependency issues, split requirements… good news it that you can use the techniques above.

Also a problem may arise if teams forget about YAGNI. The Business Logic team may be tempted to abstract the current infrastructure layer so that we could use any other implementation just as the infrastructure layer guys may decide to build a generic persistence/distribution/etc framework so that it could be useful for any project. Teams have to keep in mind that they are build a system, not an independent layer or framework.

I’ve worked in teams using this approach; we had different teams for UI, Business Rules and Infrastructure. UI developers only knew about how to send messages to the business rules and how to render the response, business logic guys were developers but more interested in understanding the complex business domain and the infrastructure guys thought of how to store business objects in a RDBMS, how to optimize the overall performance, which distribution strategy should be used, etc.

The strategy worked well for some stories that were obviously more layer-centric -like “change the interface layout”, “change the behaviour for a business object”, “introduce a system monitoring tool”…- but when we had a real story, something that should be added or a major change to the current behaviour, then -after some failures- we used the subsystem approach.

Conclusion

These are only some of the possible strategies. All of those may be suitable for a project, or none of them.

I think that it doesn’t really matter which approach you choose, or even if you split your development in different streams or not. What really matters is that you keep your team agile. Teams that are afraid of change are not agile. Teams that don’t communicate are not agile.