Fragmental.tw

Everyday Tales: Anatomy of a Refactoring – Part 3

Published by Phillip Calçado March 10th, 2010 in agile, books, case study, domain driven design, everyday tales, layers, object orientation, software architecture, software design and trends. 2 Comments

We finished last post with this funny situation: the abstraction that represents Facebook depends on our Domain Model.

It was a bit obvious that what we needed was not only system abstractions for Facebook, Twitter and the like but Bounded Contexts. We need to acknowledge the fact that these domains are not part of our model, even though we depend on them. We need to make this explicit.

The Facebook API doesn’t have the same concept of User as we do, and neither does Twitter. Someone has to translate from Profiles to Users. We identified the UserRepository as the best candidate for that.

The hugest advantage of this is proper dependency management. If you remember our last post, we had a circular dependency between system abstractions and our model. It was something like this:

But, by isolating message parsing from the user creation, we remove that red line. Facebook doesn’t create Users anymore, therefore it doesn’t depend on AllSocialNetworks/UserRepository. Will fall in a situation closer to this:

Now UserRepository will have access to everything it needs to create valid instances of User –and all it needs are objects returned by the system abstractions. The circular dependency problem is solved.

While moving this translation away from the system abstractions we learned that our situation was much worse than we first thought. The challenge looked simple, transform this:

Into this:

But indirect coupling among those classes was so strong that even with multiple levels of tests the effort to migrate all system abstractions to this structure is still in progress.

The Repository class is currently in a weird situation. It was already a very good candidate to be renamed to something else or, even better, split in two or more. After all those changes, though, that class is just too far away from whatever a Repository should be. This will probably be the next major refactoring that we have to do in this system.

But that’s another story, that maybe will be told another day…

Conclusion

In this example we went from this:

To this:

We sent from seven to thirteen classes. From two to four major packages/modules. Did this really help or we just went nuts on abstraction explosion?

All metrics I have gathered up to this point are positive. I am not talking about abstract metrics like cyclomatic complexity, efferent coupling and etc. –those are useful but won’t tell much without context. What tells me if those changes helped or not is how they impact the behaviour of the development team.

We went from a couple of fat and overdependent classes to many small classes with limited responsibility. This design is not suitable for all scenarios but it is what works well with most Object-Oriented models. If you consider that we are always adding more integration points -it’s an integration project after all- the extra flexibility and explicit concepts are really welcome to the system.

People in the team are more comfortable with the new model, it is easier to talk to the business and understand what they want done. It is also easier for newcomers to understand what the system is about and how to change it –how to map the vocabulary used in story narratives to classes and objects.

Looking at Subversion logs, we can see that commits tend to be contained into a given module. That means fewer conflicts to resolve and less contention –i.e. no or few God classes. In the end of the day this means that developers waste less time waiting for someone else to finish working with a given class and in rework.

On the bad side, making this refactoring while still delivering user stories is messy. It is really important to remember people frequently that we should not patch the old classes, but take the chance to advance towards the new model.

Unfortunately we were not pro-active enough to start refactoring and cleaning the code before having issues with circular dependencies and while playing a story one usually feels much more pressure to just fix the symptom than to solve the root cause.

Everyday Tales: Anatomy of a Refactoring – Part 2

Read the first post here.

In the previous post we were facing the problem demonstrated by the diagram below.

Our FacebookMessageParser needs an instance of AllSocialNetworks so that it can create valid Users coming from Facebook. The only implementation we have for the AllSocialNetworks interface is UserRepository, and this implementation needs a FacebookMessageParser. That’s a circular dependency, it becomes impossible to instantiate the objects above without breaking invariants.

The invariant of an object must be always fulfilled; once the invariant is broken the object is not valid anymore and your system is in an abnormal situation –i.e. I would expect it to throw exceptions like crazy and/or eventually die in pain.

The problem seems to be that we don’t have a single place that knows all the required information to create a User. In our system we have a situation where all Users have Facebook accounts, but some of them have Twitter accounts as well. For those with both types of account we must construct the user object with information coming from all Social Networks.

The first thing we thought about was that our problem was in how the User object was being constructed. One-step instantiation wasn’t working. Turns out that this is a common problem in Object-Orientation, so common that there is an old pattern that deals with it: the Builder design pattern. One of the consequences of this pattern is:

It gives you finer control over the construction process. Unlike creational patterns that construct products in one shot, the Builder pattern constructs the product step by step under the director’s control. Only when the product is finished does the director retrieve it from the builder. Hence the Builder interface reflects the process of constructing the product more than other creational patterns. This gives you finer control over the construction process and consequently the internal structure of the resulting product.

That sounds like exactly what we need here. So we draft a UserBuilder.

This object can be passed to the multiple parsers so that each one of those can populate it with the bit of information that it provides. In the end we call the createUser() method and receive a valid object back.

The next problem: who instantiates the Builder? The UserRepository is supposed to manage the lifecycle for the User, therefore looks like the perfect candidate. What happens, though, when we have a list of users instead of a single one? It sounds that this would be as simple as an operation that maps from one list of Builders to one list of Users:

But the Repository doesn’t know how many Builders should be created -the objects are complex enough so that a UserListBuilder is not an option. Sometimes the FacebookMessageParser is the one who knows how many Users should be created, in other situation the list is known only by the TwitterUserParser.

At that stage our team designed a very complicated strategy that would let the Builders be created in multiple different points depending on the situation. When you get to such complicated solution is possible that you are be missing the point entirely.

We started to realise that our problems may be being made worse given the heavy coupling between parsers, gateways and the Repository. In this post we are looking at only one pair of parser + gateway and it may look like it’s not a big deal, but in the system we had more than ten of those pairs.

The Repositories definitely know a lot more about the internals of each system than they should. Why would it care if Twitter data comes from messages we parse or from objects that we have, say, in cache? It shouldn’t!

The biggest problem that all this coupling was causing in our development process was in our Ubiquitous Language. The business people talked used to say something like “then we get the user profile from Twitter”, but in our heads we had to do the mental mapping –and mental mapping is bad- between the business lingo and what actually happened inside our model. To reduce the noise we created then objects that represent the systems we talk to.

These Facebook and Twitter objects –we called them system abstractions- were modelled after what we, as a business, need from those systems. The technical details about the need to establish a connection and then parse a message were encapsulated by those. We did increase the number of classes, but we reduced the size and complexity of them all.

One of the good things that this refactoring provided was that we could see some interesting patterns in our model that were previously hidden in the messy relationships. Take a look at a subset of the Facebook interface, for example:

The class that represents Facebook knows about our Users. It receives them as parameters and instantiates that class. Why the hell would Mark Zuckerberg care about our system? What that means, in a practical sense, is that changes that affect only our user have the potential to change our Facebook abstraction. The class changes if the Facebook API changes or if User changes. Having more than one reason to change is a smell, as Uncle Bob says:

Why was it important to separate these two responsibilities into separate classes? The reason is that each responsibility is an axis of change. When the requirements change, that change will be manifest through a change in responsibility among the classes. If a class assumes more than one responsibility, that class will have more than one reason to change.

If a class has more than one responsibility, the responsibilities become coupled. Changes to one responsibility may impair or inhibit the class’s ability to meet the others. This kind of coupling leads to fragile designs that break in unexpected ways when changed.

And that was true. A quick look through SVN history shows that every time there was a change in User we had to change at least one system abstraction -and we have many of those types of classes. In most cases those abstractions that were changed together with User had no real relation with the change being made.

It became clear that each of our third-party systems were actually Bounded Contexts. In Domain-Driven Design, Bounded Contexts are boundaries that we create around a model to isolate it from other models. We not only define what goes in and what stays out of a model, but we also define how different models relate.

We had already defined that those Bounded Contexts could only be accessed through their system abstraction. What we had to do was to remove the dependency from the third-party Bounded Contexts to our Domain Model.

We are going to see how we approached that in the next post.

Everyday Tales: Anatomy of a Refactoring

Published by Phillip Calçado February 24th, 2010 in agile, case study, domain driven design, java, layers, object orientation, software architecture, software design and thoughtworks. 10 Comments

I’ve been extremely busy with project after project in the past few months, leaving me no time to do any research and/or play around interesting things. Even though I prefer to write about what is really interesting me at a given moment, I think that writing about some smaller/simpler problems and solutions would be better than not blogging at all. Here is the first of my Everyday Tales.

My team is delivering an integration Layer for a large corporation. This client has many different systems and it is extremely common for its clients and employees to spend the day jumping from one system to another, depending on which of the many services provided they happen to be interested on in that very moment. Classic growth-over-the-years problem.

In this integration piece we are building a web services API and a web interface for users in multiple roles. We decided to go for a rich Domain Model. This is not very common in integration projects; most people in those scenarios would just create coarser-grained data structures composed of cherry-picked data coming from multiple back-end systems. We decided for the richer domain model mainly because:

Our integration Layer will not only act as a single gateway multiple systems, it will also apply business rules and manage local state.
The client is a classic example of environment where the business language was derived from the systems language. More than building some piece of software, our work as consultants is to help the client to solve the impedance mismatches across different departments and domains. Domain-Driven Design is a great tools for making sure that business, developers and everyone else speak the same language. To better apply Domain-Driven Design you should have a Domain Model.

The problem with integration projects is often that no one is really sure about which of the multiple systems we talk to have which piece of information, we discover new things every day. In agile projects that means that we learn a lot during the project and, therefore, we refactor a lot so that our code always reflects our understanding of the multiple models and how they interact with each other.

I will try to describe a sequence of refactorings we conducted over the past weeks. All changes described here were executed following baby steps while we were working on user stories. I will try to get as close as I can to the real world without breaking NDAs and letting irrelevant details spoil the core message.

Let’s say that instead of integrating multiple business systems we were integrating multiple social networks. A requirement for our system is that we should be able to see a single User object in our Layer but the data for this object should come from multiple sources.

We first started as something like this:

That worked fine for a while, we managed to get many stories done using this structure, but there was something funny about this model. For starters, we had the Domain Model depending on the infrastructure, what always rings a bell for me. We also had the PasswordSynchronisationService (something that makes sure you have the same password in all social networks) was depending on the UserRepository. This relationship caused the repository to have funny methods like getAllSocialNetworks() and getSocialNetworksForUser(User).

Clearly this class didn’t get the Separation of Concerns memo and the result was this was class #1 in number of conflicts and merges –every day at least two pairs would touch it, regardless of what stories they were playing.

For some time now I am avoiding the word Repository. Not only it causes all sorts of trouble when people understand it as an alias for DAO but it also has some really weird semantics.

Accordingly to Eric Evans in the Domain-Driven Design book:

For each type of object that needs global access, create an object that can provide the illusion of an in-memory collection of all objects of that type. Set up access through a well-known global interface. Provide methods to add and remove objects, which will encapsulate the actual insertion or removal of data in the data store. Provide methods that select objects based on some criteria and return fully instantiated objects or collections of objects whose attribute values meet the criteria, thereby encapsulating the actual storage and query technology.

So we decided to follow the suggestion of Rodrigo Yoshima (link in Portuguese) and drop the *Repository suffix. If they should behave like in-memory lists let’s give them a decent name! So here we are:

We created a AllUsers interface and let UserRepository implement it. Now it is easier to spot our modelling problem: it is not easy to say there’s something wrong when a UserRepository has a method getAllSocialNetworks(), but a class named AllUsers with such method is really peculiar.

While we think about what to do with our class, let’s think a bit about the domain we have now. I said before that the fact that the classes belonging to the Ubiquitous Language depend on classes from the infrastructure was sounding really funny to my ears. With our last refactoring we can actually spot an easy way out of this weird dependency relationship between Layers. Let’s change the model a bit more…

So now what we have is a AllSocialNetworks interface. The PasswordSyncrhonisationService doesn’t depend on the AllUsers anymore, and that means that we can move all methods related to Social Networks to the new class. As we want to do baby-steps we did not split the UserRepository, just added let it implement the new interface as well.

Talking about UserRepository, we solved the model-should-not-depend-on-infrastructure problem in a very simple way: we just moved the UserRepository out of the domain. In practical terms, that means that we moved it to a different package (it’s a Java project) and made sure that no class in the Domain Model dependeds on it. The Domain Model only knows about the All* interfaces and instances of those are injected through a Dependency Injection container.

A new story is played and suddenly we have something that sounds like a simple change. In order to bypass some technical limitations (we are calling HTTP APIs in this system, there’s no such thing as lazy-loading and ORM) we have to change the domain:

Now the user depends on the AllSocialNetworks list in order to get the Social Networks it belongs to. Any instance of User must have an instance of AllSocialNetworks inside it to be valid –this is part of the User’s invariant. This is not great but at least we will not have to implement our own lazy-loading mechanism –yet.

Just by introducing those two interfaces we simplified our domain. Now, whatever needs Social Networks points to the specific interface and don’t bother with the Users - keep in mind that this is just a tiny subset of a much larger domain and system; we have many other classes that depend and collaborate with those pictured here.

This new requirement created an interesting problem, though. To understand the problem, let’s see how a Facebook user was retrieved before we introduced the dependency from User to AllSocialNetworks:

A call to UserRepository asking for a given user would make it first go to the FacebookGateway and ask it to perform a (HTTP) request for the user profile. The user profile is brought back in, say, JSON and this data is then sent to a parser. The parser interprets the JSON representation, creates an instance of the User, populates the instance with the parsed data and returns it. All used to work fine.

In the new model, we have a slightly different flow:

Can you spot the difference? To be clearer, before the requirement we just introduced, the User class had a constructor like this:

public class User{
 public User(){
}
}

But now its constructor looks like this:

public class User{
 public User(AllSocialNetworks socialNetworks){
}
}

Why? Because before we said that the user story we played required us to change the invariant of a User. Before that story, a User didn’t need anything special to be created. After the story was played, a User started requiring an instance of AllSocialNetworks, without that dependency the User we just created is not a valid object and must not be used in the system. To enforce this invariant we want to make the dependency part of the constructor –a User object with no AllSocialNetworks instances to query is not reliable and should not even exist!

That all makes sense, but here is our problem:

In order to be able to create a valid instance of User, the parser class now must use an instance of AllSocialNetworks. Can you describe the problem we have here?

If the FacebookMessageParser class needs and instance of AllSocialNetworks to work. This relationship is part of its invariant and, as we said before, should be enforced in its constructor (i.e. we will be using Constructor-based injection). The problem here is that the only known instance of AllSocialNetworks is a UserRepository, but a UserRepository needs and instance of FacebookMessageParser to be valid! It’s a chicken-and-egg problem, also known as circular dependency.

Were this an isolated problem and we could potentially just compromise our invariants. We could change that class to setter-based injection, add to our tech debit and keep going for now. This would be pretty reasonable in a real world project, with real world budget and deadlines. The problem is that this was not the first time we had this very problem in this project. It was actually the third time this happened.

Clearly something in our design was wrong and some core change was necessary before we ended up breaking all our contracts and moving from “a couple of circular dependencies” to a full dependency-spiral-of-death structure.

In the next post we shall talk more about the possible solutions we came up with.

Coders at Work: My Review

Published by Phillip Calçado December 29th, 2009 in agile, books, software design and trends. 2 Comments

I just finished reading Peter Seibel’s new book, Coders at Work.

I was a bit skeptical at first. I only picked the book because of the big names on the cover and because Peter Siebel’s Practical Common Lisp is one of my favourite books on learn-a-new-programming-language. I thought that a book filled only with interviews with some of the big names in our industry would feel a bit too much like an issue of Who Magazine, with completely irrelevant questions like “Are you a Mac or PC?”. I was terribly wrong. The book is amazing.

Peter does a great job in guiding the interviewees into telling us about how they got into the industry and how they do their thing. For me it was really good to see how pretty much every single person interviewed -regardless of tech platform, background or expertise- tries to somehow come up with short feedback cycles during development.

And that makes it really funny to look at Joel Spolsky’s article on the book. He should have read a bit more before writing such a misinformed article, pretty much all of the interviewees write some other form of specification –being that a test, formal proof or a “test program”- before or while writing the program.

My favourite interview was with Bernie Cosell. Bernie is an old school hacker, he was part of what became the ARPANET. The way he describes his work really maps to what I do, and it can probably be summarised in this quote:

Peter Seibel: Have you heard of refactoring?
Bernie Cosell: No, what is that?
Peter Seibel: What you just described.

Duke Nukem Forever and Magic Bags of Money

Published by Phillip Calçado December 22nd, 2009 in agile, business, case study, digital media, economics, management, thoughtworks and trends. 3 Comments

Wired has a very interesting piece on how the Duke Nukem Forever project failed. It’s not only relevant because DNF is part of the nerd culture but also because it is a very interesting tale about a company that could not achieve a reasonable Definition of Done (DoD).

In software development we often talk about Done as if it was only a technical thing. It is not.

When developing a normal and bureaucratic system for a large bank or some similar client it is very unlikely that the DoD will be a problem at all –and if it becomes one it will most certainly be because of technical aspects as those mentioned in the list linked above. Things get a bit different when we are dealing with passionate people.

Projects with passionate people are always exciting. You can feel the energy, everyone is eager to help and improve the product. Everyone love what they do, really depend on the project to be a success or both. It is a fairly common situation for startups, creative agencies, the entertainment industry and media companies in general. And it is an awesome thing!

Having been to these kind of projects for most of my career, I know that the problems described in the Wired article are very common in there. Startups don’t want to present something that is not fully-featured and end up missing opportunities to enter a market, creative companies want to show their clients and users only the high-quality final version of the product, media companies are afraid to look like Muppets if their products are not one step ahead of everyone else since day one…

The Wired article seems to suggest that feature freeze is the key to overcome this problem, at least for the videogames industry:

It’s a dilemma all artists confront, of course. When do you stop creating and send your work out to face the public? Plenty of Hollywood directors have delayed for months, dithering in the editing room. But in videogames, the problem is particularly acute, because the longer you delay, the more genuinely antiquated your product begins to look — and the more likely it is that you’ll need to rip things down and start again. All game designers know this, so they pick a point to stop improving — to “lock the game down” — and then spend a frantic year polishing. But Broussard never seemed willing to do that.

I am no one to talk about how to manage a large code base written in esoteric C++ engines –I did that only once in my life and there is a reason I’m not going back there– and can’t really say if that is how that industry works/should be working; but I have to say that this is often not the best solution in the domains I mentioned before.

There are many strategies that I have used in this kind of environment. They often are either about showing that good enough is good enough or making sure people understand the costs of what they are doing.

A while ago I was employed by a big media conglomerate. Everything we did had to go through so many levels of approval and review that it could easily take months from a project kick-off meeting to when my development team received the graphic assets (PSD files, static HTML, etc.) and started writing code for it.

We had UAT servers where my team would deploy the application after each iteration, for the showcase and review. For those meeting we let the demo application point to the QA database. That database was populated with a dataset good enough to showcase the latest changes, but also had some garbage that was created by testers and developers in the process. One thing we realized at those reviews sessions was that the more database garbage displayed in a given demo the more rejected stories we would get in that meeting. After finding this out we decided to do something different.

For the upcoming releases, instead of using a UAT server to host the showcase we deployed the application straight to production. The old version would still be available in the expected URI but if you typed a URI that only our stakeholders knew you would get to the new version and that was pointing to the production database –sure you can’t do this with any application but most public facing websites wouldn’t have a problem. We used this secret URI in the showcases and by doing that we managed to get things approved much quicker: there was no noise in what the users were seeing.

But even with short feedback cycles there is still the problem of people changing the product all the time. Don’t get me wrong, it should be OK for a software development team to change things as requested, embrace change and all that. The problem is that, in the end of the day, if you change things all the time you will not deliver anything.

As Wired points out, that is a problem for the team, and an even bigger problem for the business as a whole.

Then a staff rebellion broke out. For longtime employees, the incessant delays posed two big problems. One was professional cred: Duke Nukem Forever was the only modern 3-D game some of them had worked on; if it didn’t ship soon, they’d have spent nearly a decade with nothing to show for it. The other was money. 3D Realms paid its designers less than many competitors, most notably id Software down the road. Broussard motivated them by offering profit-sharing. “Their business model was to pay the developers very low, but with a potential payday at the end that was pretty substantial,” says former employee Schuytema. As Duke Nukem Forever failed to arrive, so did that big payday.

As a consultant it is easier. I just remember the client that there is a bill to be paid in the end of the month and she would probably spend her budget more effectively if she tries to get some money from the system as soon as she can and then think about making it better. As a permanent employee for those organizations, I tried many different ways to convince people of the same thing. Turns out that this is really hard. Often business people are not in the same hierarchy as your team and you act as a service provider to them, with the difference that there is no bill to be explicitly paid. They just don’t care.

One thing that works in this scenario is the poker chips technique developer by Jason Yip. I’ve used that both as a consultant -pairing with Jason himself- and as a permanent employee for organizations -we didn’t actually have poker chips but tokens representing development hours.

It is amazing how one’s mindset changes when they stop thinking that there is a magic bag of infinite money and have to rationalize their decisions. You should give this a try.

I Wish I Knew That Before Getting This Job – Slides and (Long) Notes

Published by Phillip Calçado November 24th, 2009 in agile, business, c#, components, domain driven design, domain specific languages, economics, events, java, layers, management, object orientation, software architecture, software design, thoughtworks and trends. 0 Comments

As I said here before I was in Brazil some weeks ago to present at a conference. I had a really great time over there with some amazing people and would like to thank Caelum for their hard work in creating such a great conference. I’m making the slide deck and notes available in my Brazilian Portuguese website and thought it would be nice to translate them to English and publish it here too.

The following paragraphs are a consolidated -but really long!- version of my slide notes. The slide deck you can find in slideshare:

I Wish i Knew that Before Getting This Job

View more documents from Phillip Calçado.

In my experience it is really hard to translate material I write for the Brazilian community to English so please be advised that some bits may not make much sense, please report problems in the comments or drop me a mail.

The world has changed

If you’ve been working in this industry for a while you probably know what I am talking about: nowadays things are harder. Years ago our job was to build systems, today our job is to make sure old systems talk to each other.

When you get to the Tech Lead role in a company or project you often come full of ideas, excitement and plans. And you will see all these being destroyed by your customers and colleagues. No one will trust you, no one thinks you are able to fix those problems; no one will give you any opportunity to change anything for real –even though they said that they would.

The problem is that you clients have been bitten before. They’ve played this “let’s develop a new system” game and hate it! They’ve got their plans and dreams crushed by a bunch of code cowboys that came in with wonderful tools and techniques and delivered that buggy piece of crap -and charged 2x the agreed price. Your customers can’t take any risks. No, not this time. Not again. Regardless of what you agreed they will tell you to do what they think works.

And in this situation what happen to the standard coder? She just gives up. It’s so hard to change anything that let’s just join the dance and play the 9-to-5-job game. She starts writing all those silly specs that the client asked for, she asks the Ivory Tower architect about every single stupid detail in her application… and she starts producing pretty much nothing useful.

But after spending that many nights awake reading books, blogs and trying stuff out… after fighting to deliver on schedule and on budget for so long… will you really accept this?

There are options, resistance is not futile! It is not going to be easy. You will be openly attacked, secretly hated and people will be terribly picky even about what you wear or how you say things. But you can survive as long as you keep in mind that there is only one thing that guarantees survival: deliver value. Make sure your customer reaches his goals –and those are not to have a complete spec for the system but to earn them some money.

To make sure your customers trust you it is important that you deliver value frequently –something your client can count on. It is also important that you deliver at the right time –do not miss a deadline. And it is important that you are never late –make sure you have realistic estimates and know when something is going to be late.

But to get those there is a big enemy to fight: surprises. If there were no surprises in software development most teams would be able to deliver quality software right on time. Surprises are what make those 10% left in a task last forever. As a Tech Lead your goal is to make sure your team is delivering in a timely fashion. As a Tech lead it is your goal to avoid surprises.

And to avoid those, my suggestion is that you build barriers to keep them away. Like in any good zombie movie.

The House of the 17 Specs

That reminds me of when I used to work for this big product company. They had a funny process where before the first line of code was written you had to come up with a spec for the full product, then one spec for each of the modules. Each of those documents should be paired with a test strategy/case document. I think that if you followed the full process –and this was the usual way- for a large system one would write 17 documents before the first line of code.

In theory that would not be a problem, the specs were detailed enough so, theoretically, the coding task would be just to convert what was written in there to Java, C# or whatever we were using for that piece. That wasn’t the case, as you probably imagined. There was a huge impedance mismatch between the world described in those specs and the real world. The library we were be using would happen not to be thread-safe, the operating system that the client wants would be missing that very system call we relied on or something like that.

After so many weekends in the office trying to make the real world look more like the spec we started to think about doing things a bit differently. We came up with all these cool ideas about how to improve our development process and took them to middle and senior management types. Of course that didn’t work, management was definitely not interested in what a bunch of punks with a couple of years of experience in that company had to say about their 10-years-delivering-software process.

We got really upset and for the next couple of weeks I would see my team visiting job boards for most of the day. Eventually someone came with an interesting idea: the process says that we must have such and such document but it doesn’t actually say that the docs should be all ready before coding starts. Leaving the documents to the end will probably mean that they will never be finished and the team is going to be in trouble… but what about we try to write them incrementally?

For the next project our approach was simple: we kept Microsoft Word and our IDE open at the same time. We would spend two hours coding and 20 minutes updating the specifications. In the end we had fairly accurate specs and we managed to deliver on time for the first time ever.

But that didn’t solve all of our problems. We still had many issues that were only found in production –the system reflected the specs but the specs were wrong. One thing we learned is that there are many feedback cycles that are important during development, we managed to shorten one of them but we needed more. But to get this working we had to convince other departments –QA Department, BA Department, Senior Architects(tm)… – that this was a good thing.

And that never happened. After trying very hard to convince people that it would be nice to have short feedback most of us gave up and resigned. This company is still in business but they’ve lost 70% of their clients in the past couple of years. People I know there say that the problem is that their competitors launch a new product every year but them –still following the old process- can barely fix the bugs for the 10 years old version.

Getting Cyclic

But this taught me a lot. It showed me that even by creating one single barrier between my team and “the unknown” we could achieve a fantastic improvement. I was dying to try new barriers and see what we could achieve.

And after some years applying those I now believe that there are at least five “layers” of feedback while writing normal business software:

Development
Integration
Verification
Acceptance
Production

The first cycle is what happens in the developer’s box while she is writing code. A big waste of time –and waste causes feedback cycles to get longer- happens if it takes long to check if a piece of code you just changed works or not. Waiting for the build, starting up the application server or something like that just to see if what was showing a 2 now shows a 3 is a big time consumer. Having to manually click around your application to check if a bug was fixed is not only wasteful but also stupid.

And here comes the second cycle: how long does it take to integrate what you’ve done with what the rest of your team is working on? Conflicts and merges are a commonplace when build processes take long. That is terrible for feedback as a long integration cycle will make developers assume the code is working even though she only tested it on her machine.

Eventually someone has to verify your code, could either be a tester or some other developer doing code reviews. This is the verification cycle and it answers many questions about your changes. Does it really work? Does it break what was already there? Is it reliable? Follows the architecture? What about standards? Finding problems only at this stage is often a problem as it will require the developer to get back to her desk, fix it and go through the whole cycle again.

But, as we said before, following the spec may be good but if the spec is wrong your system will still be wrong. At some point you have to show it to the customer and see if his expectations were met, what I call the acceptance cycle. Usually this is done late in development and because of that it becomes such a painful thing that at this stage that developers will try to hide bugs and problems.

In the end there is always the production environment. It doesn’t really matter how much you test: production is always production and it will often show problems that you had no idea about. And, of course, it is often the most expensive place to find surprises.

Your Bricks

Those five cycles –and more cycles you may find- need to be short, they are the barriers that will keep the zombies of surprise out of your team. Let’s discuss some techniques and tools to help in this process.

For the first cycle, there are some very good tools that your team should be aware of. The first tools would be Test-Driven Development. Using TDD you set your goal before you write a piece of code and you know straight away when you met that goal. It will also help you testing the system, most code can be verified without firing up a browser and wasting time clicking and rendering.

To make sure this process is efficient you should also make sure you have a fast build process. The build should execute all tests and verifications. Every developer should run it before even thinking about checking in the code. That requires the build process to be short. 10 minutes is a good goal for most Java and .Net projects.

And fast builds will also help out in the integration cycle. A build that performs all the required verifications can be executed continuously in a dedicated machine every time a change is committed to the code base. This continuous integration process should also run the system in a non-dev machine –a machine without compilers, IDEs and other dev tools- to make sure that it is ready to be deployed.

This will also help you in the third cycle, verification. Testers should be able to write their own tests –using either a regular programming language or some kind of Domain-Specific Language- and those test cases should be part of the build and therefore run in the continuous integration process.

The best scenario is when you get the testers to work close to the developers, make them part of the development cycle. Developers should know immediately when they are breaking a test case that was working before. If you can’t make the tests written by the testers to be part of the developer normal build you have to make sure that at least they run -automatically and frequently- in the integration service.

And then we get to the acceptance cycle. This is extremely important: you don’t want to waste time developing something that is not what your client wants. Keep your client close to the development team. This is risky as the client may want to change things all the time but you should make sure that there are high-bandwidth channels between all stakeholders –and developers are stakeholders- and that you don’t spend too much money before realizing that you are building the wrong thing.

A nice way to collect early feedback from customers and other stakeholders is the Sandwich Model that we discussed here the other day:

One practice that I’ve used to help make this process more straightforward is the Sandwich Model for stories. Before a card is played the users, BAs, QAs and developers get together. The user and/or BA present the card to the technical people. First questions and answered and acceptance criteria is defined. After the developers finish the card they perform a walkthrough with BAs/users and QAs so that they can verify if it looks like done. If more work is required the developers get back to work, if the card was accepted it moves to the next stage.

To achieve better communication between developers and business experts it is also useful to make sure that they speak the same language. Domain-Driven Design can help you a lot with that as it aims to achieve a Ubiquitous Language used for developers, users and everyone else in that team.

And there will always be the production feedback cycle. Unfortunately in most of my projects these days it is not possible to replicate the production environment during development. There are techniques that can help you ease the pain but it is often impossible to replicate all the small details of production.

To minimize surprises in that level it is important to deliver increments instead of going for a big bang approach. Although everyone agrees with that, the reality is that most software development projects make just one or a couple of production rollouts. Don’t be like that, make sure that at least at the end of your iteration –but ideally always- your codebase is production-ready and try hard to get it deployed frequently. Not only you will see if there are problems that you didn’t find while in development but you will also be delivering value frequently to your customers –and maybe learning that the new features are not wanted at all!

But even though we can’t replicate the full production environment during day-to-day development, it is extremely important to have a production-like environment freely available to developers. Please avoid the “you must schedule an environment” behavior, have a box –or a virtual machine- somewhere where your developers, QAs and everyone else can log in, deploy some version of the application and mess around.

They’ve got inside!

As I said previously, these are not the only barriers you will find but in my experience these five are found in every project. An important thing to remember, though is that barriers exist only for one reason: to be broken.

For sure you want to minimize the number of times that those are broken but don’t even think about trying to protect your team from everything. Not only it is too expensive but also it is extremely inefficient as your barriers only protect you from the risks you know about.

Instead of doing that, you should focus in making sure that every time a barrier is broken everyone knows about that. Red lights, sirens, notification emails, broken builds… make sure that when something fails it fails out and loud.

In Case of Emergency Keep Those in Mind

In case you can’t remember all that we talked about in this ridiculously huge post, please keep four things in mind:

1 – Your team is not the centre of the universe: Software development is extremely important for a healthy business but it is far from being the only relevant moving part –or even the most important. You and your team have to learn how important you are, where do you sit in the value chain and how your work can help or hurt the business.

2 – The goal is not to be fail proof but to make it safe to fail: If you try to make sure that there is never a problem the first thing that probably will happen is that your team will stop innovating as they stop trying new things. You will also end up with different problems, you can’t avoid what you don’t know. Instead of fail proofing, make sure everyone is warned about problems as soon as possible.

3 – Do not assume people will give you support: I don’t care what your business card says or who hired you: if you try to change things you will make more enemies then friends. Keep in mind that it is very hard to criticize your team if you are delivering value, though. And be prepared.

4 – Keep your career goals apart from your team goals: It is probably a good thing that you want to progress in your carrier and get to a more important project or a raise. Make sure you don’t let this affect your leading style though. Your job is not to make management see how good you are, your job is to enable that team to deliver the best thing you possibly can. And in my experience a professional that can lead a development team is much more important than someone that can write the whole system by himself.

Speaking in Brazil and Last Year’s Slide Deck

Published by Phillip Calçado October 24th, 2009 in agile, domain driven design, events and thoughtworks. 3 Comments

As I said here earlied this year, next month I will be presenting in a conference called Caelum Day. This will be held in Rio de Janeiro, my home town, and I’m really excited to be there.

My presentation will be a keynote on the role of a Tech Lead and what I wish I knew before getting my first gig as one. Just like last year, I will also be helding a Workshop on Domain-Driven Design. I really tried to change the subject of the workshop to something else -DDD is getting a bit boring- but it turns out that there is a massive demand for this.

The presentation and workshops will be held in Portuguese. As most readers of this blog don’t really understand coisa nenhuma of Portuguese I jsut translated last year’s slide deck.

The Curse of The Agile Software Factory

View more documents from Phillip Calçado.

I’m not quite happy with the translation, some expressions require a Brazilian background and sound dull in English. Anyway it’s better than nothing

Don’t Trust Fake IDs

Published by Phillip Calçado October 12th, 2009 in c#, domain driven design, java, layers, object orientation, software architecture, software design and web. 5 Comments

After so many years I thought that this was a dead topic but recently I saw at least two reasonably experienced developers having trouble with object identity and thought that it would be good to write about it.

Most languages define some kind of equality operator on objects, in Java, for example, that is the equals method. In Effective Java, Joshua Bloch says that the easiest thing to do is not to implement an equals method at all, but you should implement one when “the notion of logical equality differs from mere object identity” and your superclass doesn’t have already implemented a suitable equality comparisson.

I think most people understand this when it comes to numbers, dates, strings and other Value Objects but I’ve seemed people having trouble understanding what object identity means for Entities.

Entities are objects whose identity is defined by rules in the domain we are modelling. In his Domain Driven Design book, Eric Evans defines objects following this pattern as objects that change state over time but keep their identity. As the object’s identity comes from the business domain reference equality is too fragile to properly compare objects.

Problems with fragile object identity are fairly well-documented in the Object-Orientation literature. In 1986, Setrag N. Khoshafian and George P. Copeland wrote a paper titled Object Identity. In this document they present a list of known problems with fragile identity, some of these are:

Multiple references to the object: It is very common to have multiple instances of a given object at a given time and it should be possible to infer that those multiple instances are actually the same object.
Uniqueness Not Respected: Objects are abstractions and as such they don’t model every single detail present in the modelled concept. If we don’t define identity correctly we may end up with a model in which different objects are seem as equal.
Data and Identity Confusion: The identity of an object should not be allowed to change but, as Evans says, data will change over time. There should be a clear separation of what is data and what is identity.

People use fragile things for object identity for multiple reasons, let’s address two of those.

I refuse to write test-only code!

One of the problems pointed out by those using reference equality instead of object equality is that equals() is commonly not called by anything but tests and writing production code that is used only in tests is a bad smell.

Let’s look at a very simple class and its tests:

class Point {
	private int y, x;

	public Point(int x, int y) {
		this.y = y;
		this.x = x;
	}

	public int getY() {
		return y;
	}

	public int getX() {
		return x;
	}
}

class Square {
	private Point letfTop, leftBottom, rightTop, rightBottom;

	public Square(Point a, Point b, Point c, Point d) {
		this.letfTop = a;
		this.leftBottom = b;
		this.rightTop = c;
		this.rightBottom = d;
	}

	public Point getRightTop() {
		return rightTop;
	}

	public Point getRightBottom() {
		return rightBottom;
	}

	public Point getLeftTop() {
		return letfTop;
	}

	public Point getLeftBottom() {
		return leftBottom;
	}
}

public class SquareTest {
	@Test
	public void ShouldHavePointsInTheRightPlace() {
		Point first = new Point(1, 1);
		Point second = new Point(2, 2);
		Point third = new Point(3, 3);
		Point fourth = new Point(4, 4);
		Square square = new Square(first, second, third, fourth);

		// how do we assert that the right points are in place?
	}
}

Althought the value of the test above is questionable, it exposes the problem: How do we make sure that the points are in the right place? As we said before we could implement an equals() method in the Point class in order to be able to compare X and Y, and then do something like:

assertEquals(first, square.getLeftTop());
assertEquals(second, square.getLeftBottom());
assertEquals(third, square.getRightTop());
assertEquals(fourth, square.getRightBottom());

That looks OK but we just wrote a piece of production code (Point.equals()) that are used only by tests. I personally don’t think of this as a big deal -for me equals, hashcode and toString are part of the language protocol- but I can understand why people don’t like to do that.

So maybe we can just use the == operator as we wouldn’t have to write any extra code for that. Using that operator our assertions wold become something like:

		assertTrue(first == square.getLeftTop());
		assertTrue(second == square.getLeftBottom());
		assertTrue(third == square.getRightTop());
		assertTrue(fourth == square.getRightBottom());

That would work but the problem is that your tests are relying on the wrong thing. Think about that for a second: in the test above do you really care that the same instance is in the top left corner? Suppose we have new requirements and for some reason we can’t store the data as Points. The class contract didn’t change at all, the fact that we can’t use points internally is just an implementation details inside Square:

class Square {

	class SimplePoint {
		public int x, y;

		public SimplePoint(Point p) {
			this.x = p.getX();
			this.y = p.getY();
		}

		public Point toPoint() {
			return new Point(x, y);
		}
	}

	SimplePoint letfTop, leftBottom, rightTop, rightBottom;

	public Square(Point a, Point b, Point c, Point d) {
		this.letfTop = new SimplePoint(a);
		this.leftBottom = new SimplePoint(b);
		this.rightTop = new SimplePoint(c);
		this.rightBottom = new SimplePoint(d);
	}

	public Point getRightTop() {
		return rightTop.toPoint();
	}

	public Point getRightBottom() {
		return rightBottom.toPoint();
	}

	public Point getLeftTop() {
		return letfTop.toPoint();
	}

	public Point getLeftBottom() {
		return leftBottom.toPoint();
	}
}

Now our assertions based on the == operator fail and if you are testing the behavior of the class –as opposed to its implementation- this shouldn’t happen.

Tests should follow the Single Responsibility Principle and have only one reason to change. In that case the only reason for that test to change should be if the points are not in the places we expect them to be. It doesn’t matter if they are stored internally as plain Points, if they are stored as integers over a matrix or even serialized to disk: the test should not fail if the storage strategy has to be modified.

This is the kind of assertion that creates extremely fragile test suites. A fragile test suite will require constant maintenance and will break with no clear reason. Tests start to cause more problems than help development and it is very common for the team to think about either testing less or dropping tests entirely.

As we saw before, the domain defines what makes on object equivalent to another. In the case above the Point class, it is a Value Object and what that means is that identity for that object is given by the value of its attributes, in this specific case that would be the integers Y and X.

If you don’t want to add an equals() method to your class at least make sure you rely on the real identity of the object and not in some accident:

public class PointAssertions {
	public static void assertPointsEquals(Point one, Point another) {
		assertEquals("Expected [" + another + "] to have X equals to ["+ one.getX() + "]", one.getX(), another.getX());
		assertEquals("Expected [" + another + "] to have Y equals to ["+ one.getY() + "]", one.getY(), another.getY());
	}
}

And use that in your tests:

assertPointsEquals(first, square.getLeftTop());
assertPointsEquals(second, square.getLeftBottom());
assertPointsEquals(third, square.getRightTop());
assertPointsEquals(fourth, square.getRightBottom());

So although you had to write some more code you didn’t change production code just to make your tests pass. Also you are probably going to reuse this type of code in other tests that deal with points.

It’s a kind of magic!

Another cause for not correctly defining object identity is when people are using things like Hibernate or NHibernate that, in some cases, return the same instance over multiple calls. One quick example would be:

public class User
    {
        public string Login { get; set; }
        public string Name { get; set; }
        //business logic should be here somewhere
    }
    public class Group
    {
        public User Admin { get; set; }
        public bool CanBeChangedBy(User u)
        {
            return u == Admin;
        }
    }
    public class UserRepository : HibernateRepository
    {
        public User FindByLogin(string login)
        {
            return session.Load<User> (login);
        }
    }

    public class GroupRepository : HibernateRepository
    {
        public Group FindById(string groupId)
        {
            return session.Load<Group> (groupId);
        }
    }

    public class GroupsController
    {

//Some other stuff

        public void Change(string groupId, string login)
        {
            var userTryingToChange = userRepository.FindByLogin(login);
            var groupTobeChanged = groupRepository.FindById(groupId);
            if (groupTobeChanged.CanBeChangedBy(userTryingToChange))
            {
                //do something that lets the guy change the group
            }
            else
            {
                // show error message or something like that
            }
        }
    }

In this case NHibernate happens to return the same object for every call. This code is extremely brittle, though. If you look at NHibernate’s documentation (or Hibernate’s) you will see that multiple queries using the same Session object are guaranteed to return the same instance. The problem is that if you use more than one Session your objects will be different.

That may sound like a small problem given that most web applications (and most applications are web applications) use only one Session per request. But the real problem hides in the fact that now your Business Layer depends on an implementation detail of the Persistence Layer.

Changes in the implementation of the Persistence Layer will directly affect your Business Layer and this is the kind of accidental complexity that creates buggy and unreliable software.

And it gets worse. The original Hibernate’s documentation adds some more information in a section named “Considering object identity”:

11.1.3. Considering object identity
An application can concurrently access the same persistent state in two different Sessions. However, an instance of a persistent class is never shared between two Session instances. It is for this reason that there are two different notions of identity:

Database Identity
foo.getId().equals( bar.getId() )

JVM Identity
foo==bar

For objects attached to a particular Session (i.e., in the scope of a Session), the two notions are equivalent and JVM identity for database identity is guaranteed by Hibernate. […]
However, an application that uses == outside of a Session might produce unexpected results. This might occur even in some unexpected places. For example, if you put two detached instances into the same Set, both might have the same database identity (i.e., they represent the same row). JVM identity, however, is by definition not guaranteed for instances in a detached state. The developer has to override the equals() and hashCode() methods in persistent classes and implement their own notion of object equality.

Now what that means is that I cannot detach my object from Hibernate to, for example, store it in a HTTP session. Even if you don’t use HTTP sessions, that means you can’t cache objects as the cached copy will be a different instance than those returned by a Session.

Suddenly your architecture is extremely constrained just because someone decided that implementing a proper object comparisson method was too much effort.

Before anyone writes a comment with YAGNI in it I will say once more that the simplest thing is different from the hackiest thing. Relying on how your Persistence Layer happens to be implemented is not being simple, it is coupling your components in a way that even small changes requrie hard work.

No half-baked IDs

In the first example we saw a situation where implementing proper object equality wasn’t necessary for the production code. In that case you just have to make sure that your tests are using something reliable when comparing objects.

In the second case we have a need for object identity and using weak identity may save some lines of code but ultimately creates a crittle and inflexible architecture.

Those are just a couple of examples of problems you may find. The most important thing I can say is that every time you need to compare objects you should make sure to use strong identity. You can write the comparisson operations yourself or use a library or anything, just make sure you follow the contract that your language imposes on objects.

Note: I didn’t say or implement anything regarding Hashcode methods here for the sake of simplicity. Anyway I suggest that you follow the recommendations for your language.

Ubiquitous Language, Tiny Types and Responsibility

Published by Phillip Calçado August 21st, 2009 in books, c#, domain driven design, object orientation and software design. 8 Comments

As a consultant I have to log my hours in a timesheet application. This could be a test inside the timesheet app:

 [TestFixture]
    public class BillableTimeTest
    {
        [Test]
        public void ShouldhaveCorrectNumberOfWorkedHours()
        {
            var beerOClock = 17;
            var earlyInTheMorning = 9;
            var workDay = new BillableTime(earlyInTheMorning, beerOClock);
            Assert.That(workDay.Hours, Is.EqualTo(8));
        }

        [Test]
        public void ShouldRejectNonBusinessHours()
        {
            var afterHours = 23;
            var earlyInTheMorning = 9;
            Assert.Throws<ArgumentException>(() => new BillableTime(earlyInTheMorning, afterHours));
        }
    }

Given the test above it looks that in this system you can only log billable hours if they between 9a.m. and 5p.m. The implementation of the BillableTime object could be something like this:

    public class BillableTime
    {
        private readonly int _from;
        private readonly int _to;

        public BillableTime(int from, int to)
        {
            ValidateTimePeriod(from, to);
            _from = from;
            _to = to;
        }

        private void ValidateTimePeriod(int from, int to)
        {
            if (from < 9 || to > 17) throw new ArgumentException("BillableTime only accepts business hours (9am-5pm)");
        }

        public int Hours { get { return _to - _from; } }
    }

Let’s move on. If the BillableTime object is used to log only office hours we will probably have an object that represents overtime. Here is the test for our Overtime object:

    [TestFixture]
    public class OvertimeTest
    {
        [Test]
        public void ShouldhaveCorrectNumberOfWorkedHours()
        {
            var beerOClock = 17;
            var wayTooLate = 22;
            var workDay = new Overtime(beerOClock, wayTooLate);
            Assert.That(workDay.Hours, Is.EqualTo(5));
        }

        [Test]
        public void ShouldRejectIfNotInOVertimePeriod()
        {
            var afterHours = 23;
            var earlyInTheMorning = 9;
            Assert.Throws<ArgumentException>(() => new Overtime(earlyInTheMorning, afterHours));
        }
    }

You may have guessed that Overtime is implemented as something like this:

    public class Overtime
    {
        private readonly int _from;
        private readonly int _to;

        public Overtime(int from, int to)
        {
            ValidateTimePeriod(from, to);
            _from = from;
            _to = to;
        }

        private void ValidateTimePeriod(int from, int to)
        {
            if (from < 17 ) throw new ArgumentException("OvertimeTime only starts after 5pm");
        }

        public int Hours { get { return _to - _from; } }
    }

Code like the examples above is pretty common but there is a problem in there. Right now a BillableTime represents a period of time in which someone was working AND controls what constitutes “business hours”. The overtime object represents a period of time in which someone did some work after ours AND define what “after hours” means. When you can’t describe an object’s responsibilities without using and there is a chance that your model has something wrong in it.

In an Object-Oriented system it is generally a bad idea to have the implementation of a concept spread across multiple places. In a language like C# the best thing to do is often to have this implemented in its own class:

 [TestFixture]
    public class TimeIntervalTest
    {
        [Test]
        public void ShouldBeBusinessHoursIfBetween9AmAnd5Pm()
        {
            var beerOClock = 17;
            var earlyInTheMorning = 9;
            var businessHoursInterval = new TimeInterval(earlyInTheMorning, beerOClock);
            Assert.That(businessHoursInterval.Kind, Is.EqualTo(TimeInterval.Kinds.BusinessHours));
        }

        [Test]
        public void ShouldBeOvertimeIfAfter5Pm()
        {
            var beerOClock = 17;
            var wayTooLate = 22;
            var businessHoursInterval = new TimeInterval(beerOClock, wayTooLate);
            Assert.That(businessHoursInterval.Kind, Is.EqualTo(TimeInterval.Kinds.AfterHours));
        }

        [Test]
        public void ShouldHaveCorrectNumberofHours()
        {
            var startingTime = 9;
            var oneHourAfter = 10;
            var businessHoursInterval = new TimeInterval(startingTime, oneHourAfter);
            Assert.That(businessHoursInterval.Hours, Is.EqualTo(1));
        }

    }

    public class TimeInterval
    {
        private readonly int _from;
        private readonly int _to;
        private readonly Kinds _kind;

        public TimeInterval(int from, int to)
        {
            _from = from;
            _to = to;

            if (from >= 9 && to <= 17)
                _kind = Kinds.BusinessHours;
            else
            {
             _kind = Kinds.AfterHours;
            }
        }

        public enum Kinds { AfterHours, BusinessHours }

        public Kinds Kind
        {
            get { return _kind; }
        }

        public int Hours { get { return _to - _from; } }
    }

You can get fancy and create multiple types but this implementation is probably enough for this example. Now we have a class representing the time interval. Notice that this concept was present in our domain since the very beginning; we just didn’t have it explicitly.

But why would explicitly make anything better? Suppose that you are working on this code base and suddenly you need to change something. Say that your company decided that counting hours is not acceptable and that they want to track everything as minutes.

In the first example you would have to change both Overtime and BillableTime classes as they share the implementation of the concept. In the second example you would have to change only the class that actually represents the concept.

But even to realize which classes you would have to change it would take possibly hours in searching through the code base and tests to figure out all the places that implement time interval related logic. Something that for the business is very simple –as the business considers time interval to be a “strong concept”- would take ages to be implemented, possibly frustrating your stakeholders.

I like Eric Evans’ take on these situations. In his classical Domain Driven Design book he says:

[…]A deep model has power because it contains the central concepts and abstractions that can succinctly and flexibly express essential knowledge of the users’ activities, their problems, and their solutions. The first step is to somehow represent the essential concepts of the domain in the model. Refinement comes later, after successive iterations of knowledge crunching and refactoring. But this process really gets into gear when an important concept is recognized and made explicit in the model and design.
Many transformations of domain models and the corresponding code happen when developers recognize a concept that has been hinted at in discussion or present implicitly in the design, and they then represent it explicitly in the model with one or more objects or relationships.
Occasionally, this transformation of a formerly implicit concept into an explicit one is a breakthrough that leads to a deep model. More often, though, the breakthrough comes later, after a number of important concepts are explicit in the model; after successive refactorings have tweaked their responsibilities repeatedly, changed their relationships with other objects, and even changed their names a few times. Everything finally snaps into focus. But the process starts with recognizing the implied concepts in some form, however crude.
[…]
Listen to the language the domain experts use. Are there terms that succinctly state something complicated? Are they correcting your word choice (perhaps diplomatically)? Do the puzzled looks on their faces go away when you use a particular phrase? These are hints of a concept that might benefit the model.

The example was extremely simple but it’s not that different from real-world situations. What I’ve found in many projects is that people use things like metadata-based validators to hide business logic. Here is an example using NHibernate Validator:

 public class Contract
    {
        private int _lengthInYears;

        [Range(Min=1, Max = 3)]
        public int length
        {
            get { return _lengthInYears; }
            set { _lengthInYears = value; }
        }
    }

I think those validator frameworks can really help in many situations but when I look at the code above I can’t get some very important concepts. The method tells me it accepts an integer (and that means a lot of numbers, including negatives), but then it rejects most of them. In this case wouldn’t it be better if we explicitly say what we accept?

public class Contract
    {
        public Years length { get; set; }
    }

    public class Years
    {
        public int NumberOf { get; set; }

        public Years(int numberOf)
        {
            NumberOf = numberOf;
            if (numberOf < 1 || numberOf > 3)
                throw new ArgumentException();
        }
    }

It is quite clear that this also adds a lot of noise as you have to wrap and unwrap integers into Years. I am not saying that you must validate all your integers, strings and the like every time. What I am trying to say is that whenever you have this kind of logic in a validator or something else it is very likely that the use of metadata is actually hiding information about an important domain concept. I’d rather have it explicit in my code base than hidden in some metadata.

And this gets us into theTiny Types (or Micro Types) idea. I like the concept and I use it when it makes my domain more explicit. What really bugs me is when I see something like:

    public class User
    {
        Name name { get; set; }
        //other stuff
    }

    public class Name
    {
        public string Value { get; set; }

        public Name(string value)
        {
            Value = value;
        }
    }

The tiny types in the example below not only are often not relevant to the Ubiquitous Language –I don’t think I ever worked in a system where name wasn’t a string… any string!- but also most languages (C#, Java, Ruby…) make it so bureaucratic to create and maintain those little types that in the end you waste a lot of effort doing something that is not relevant at all.

It is a wonderful thing to make your domain concepts explicit using types. Just make sure you don’t overdo it, it’s not worth the effort.

3 Things Agile Teams Should Care About

Published by Phillip Calçado August 17th, 2009 in agile, management, software architecture, software design and thoughtworks. 3 Comments

Regardless of what kind of team you have, it is a given that if you keep the wrong focus you are going to be in some kind of trouble. It is very important for a team to have a clear understanding of the project’s goals and constraints. That sounds like PM-speak but it’s not only relevant for business/management side of things but for the technical team as well. I’ve been to many projects in big trouble just because the technical team focused on the wrong goals.

In short, there are three things that I think all software development teams should be obsessed with:

1 – What does “Done” Mean?

The whole team has to get a shared understanding of what done means. This must be defined by the team, including non-technical people and it is expected that no one accepts half-baked software.

This is not about test coverage or any other metrics. This is about having software that does what the user wants with the quality that the team considers acceptable.

One practice that I’ve used to help make this process more straightforward is the Sandwich Model for stories. Before a card is played the users, BAs, QAs and developers get together. The user and/or BA present the card to the technical people. First questions and answered and acceptance criteria is defined. After the developers finish the card they perform a walkthrough with BAs/users and QAs so that they can verify if it looks like Done. If more work is required the developers get back to work, if the card was accepted it moves to the next stage.

Other practice, one that I would expect all teams that call themselves Agile to follow, is to always have Potentially Shippable Software, from day zero. Unfortunately it is very common for teams to have a product that can only be tested in a development environment and can’t be deployed to a server or workstation.

I am not saying that you need a fancy Capistrano deployment script or to make your application WebStart/ClickOnce ready. What I am saying is that there must be an easy way to install and run your application in a production-like environment -and production like probably means no compilers, no build tools and no damn Visual Studio.

An user story is only done when it can be tested in the production-like environment, before that it is only “90% Done”.

2 – Flow

After deciding what Done means, the team should be concerned about how fast things are getting there. Agile software development is based on short feedback cycles. When cards move from Doing to Done at a reasonable Velocity you empower the stakeholders to make the right business decisions. When changing is cheap you can delay technical commitments until you have enough information to make your decision. If you don’t have a short feedback cycle you may be iterative and may be incremental but for me you are certainly not Agile.

Feedback cycle is often related to technical issues and the team must make sure that those issues are not slowing it down. My main metric on that is the Rule of The Second Card: The first card played on a given area of the code base is expected to take a long time. The second card should take at most half of the first card’s time.

This is just a rule of thumb but it helps measuring the pain. The first card played in a new area will always require a lot of thinking and learning. Even when the simplest solution is implemented I expect it to take longer as people learn how to connect the dots –remember: it’s about the Simplest solution, not the Hackiest.

The second card shouldn’t take that long. User stories represent small chunks of change and the second one is often just an addition to the first (e.g. First is “User should search by state”; Second is “User should search by state and city”) and if the first card was played with reasonable quality standards then we can only expect that it has built the scaffold for the second card.

If your second card takes about the same time as the first one -or even longer!- then your team may be in trouble. It may be that the pair playing the first card took too many shortcuts in order to get the first card delivered and produced a lot of technical debt. If it becomes clear that this is a common pattern in your project you guys should sit down and work out what is happening.

Another thing to pay attention to is the build time. The build is the main feedback cycle for a developer; builds should not be long and should not fail for unknown reasons. There are many techniques one can use to keep build times slow, we are going to talk about them in future posts.

One level above the build time is how Continuous Integration is done in your project. Julio Maia has a very interesting article describing its importance.

3 – Readability

It may be odd to see this as the third most important thing in a blog from someone that works for ThoughtWorks. I do believe that writing code that is readable is extremely important, I just don’t think that, in a project, it comes before the two other topics we previously talked about.

The Tech Lead should make sure that the code being written is readable by people that have enough knowledge. What is “enough” means depends on who you ask but for me there are at least two groups in this situation:

People that are proficient in the programming platform (e.g. Java, .Net, Rails, etc.) but not necessarily in the domain
People that are programmers, know the domain but don’t master the platform used in that system.

There are multiple ways of doing that. In ThoughtWorks we love to have people coming to our projects and working with us for a couple of days. When a skilled ThoughtWorker has trouble trying to understand the code this is a very bad sign and we need to work on that. I’ve heard about people that are going even further and asking for people from other companies to come and pair with the development team for some time.

Everything is Connected

The items above are presented as three independent units but that is not the case. There are many dependencies between items and they become clear as you start thinking about them.

Of course the list is not exhaustive, there are many other technical aspects that must bee considered in a project, but in my experience if you get at least those three right it is much more likely that your project will be in good shape.

Search

About

My name is Phillip Calçado, I'm a Brazilian software developer currently working for ThoughtWorks in Sydney, Australia. More info? Check my LinkedIn profile, twitter or even my Brazilian Portuguese blog.

Latest

Blogroll

This work is licensed under a Creative Commons Attribution-Share Alike 3.0 United States License.