Abstractions are present at all levels in a system. Computers work based on electric signals. To reduce the Essential Complexity we abstract those signals with bits and bytes. Working with bits and bytes is still too noisy therefore we abstract those with mnemonics. Mnemonics are still hard to deal with so we created higher-level programming languages abstracting those.

The book Beautiful Code has a nice article by Diomidis Spinellis (also available here) that explores how abstractions work at this level.

Even though abstractions are easier to see at lower levels they are also part of the toolset the average programmer uses. If you think of it, a language like Java or C# use classes to abstract behaviour (expressed as methods/procedures) + state (variables).

But why do we create so many layers of abstraction? Edsger Dijkstra, in one of the most important essays in this industry, says:

it might be worth-while to point out that the purpose of abstracting is not to be vague, but to create a new semantic level in which one can be absolutely precise.

An abstraction should not be just a group of primitives, the abstractions you use should create a new semantic level on top of the primitives. This semantic level encompasses new terms and concepts that probably cannot be directly mapped to the primitives; the way we aggregate primitives to form an abstraction must form something that doesn’t exist at the primitive level.

Think of an abstraction as some new concept that you cannot express directly using the words you had (i.e. the primitives), something that needs completely new words to be described. This new level created by the abstraction must make it easier for you to describe your system as it gives you new words that let you express exactly what you want, with minimal noise.

Smells Like a Broken Abstraction

An example of the kind of vague abstraction that I believe wouldn’t make Dijkstra very happy is the use of Regions in C#. Regions can be very useful –if you are building embedded DSLs, for example– but the way they are sold by Microsoft is not great. From MSDN:

#region lets you specify a block of code that you can expand or collapse when using the outlining feature of the Visual Studio Code Editor.

One can say that #regions are just flexible curly braces. I don’t agree with that, curly braces in languages like C# or Java not only have semantic for the language compiler but also define a unit of abstraction, being a method, a conditional or a class.

Here’s an example of how people often use #regions in C# (extracted from Codeplex and modified for brevity):

	public class GlobalRadioButton : System.Web.UI.WebControls.WebControl, IPostBackDataHandler
	{

		#region events
		private static readonly Object EventCheckedChanged = new Object();

		protected virtual void OnCheckedChanged(EventArgs e)
		{
			EventHandler ecc = (EventHandler) this.Events[GlobalRadioButton.EventCheckedChanged];
			if (ecc != null) ecc.Invoke(this, e);
		}

		public event EventHandler CheckedChanged
		{
			add  {this.Events.AddHandler(GlobalRadioButton.EventCheckedChanged, value); }
			remove {this.Events.AddHandler(GlobalRadioButton.EventCheckedChanged, value);}
		}
		#endregion

		#region Properties
		public virtual Boolean AutoPostBack
		{
		  get { return savedState != null ) ? (Boolean)savedState : false;}
                  set  {this.ViewState["AutoPostBack"] = value;}
		}

		public virtual Boolean Checked
		{
		  get { return savedState != null ) ? (Boolean)savedState : false;}
			set {this.ViewState["Checked"] = value;}
		}
		#endregion

		#region Implementation of IPostBackDataHandler
		void IPostBackDataHandler.RaisePostDataChangedEvent()
		{
			this.OnCheckedChanged(EventArgs.Empty);
		}

		Boolean IPostBackDataHandler.LoadPostData(String postDataKey, System.Collections.Specialized.NameValueCollection postCollection)
		{

			String postedValue = postCollection[this.UniqueGroupName];
			Boolean meCheckedPosted = (postedValue != null && postedValue == this.ValueAttribute);
			Boolean checkChanged = false;

			//...

			return checkChanged;
		}

		private Object GetHiddenProperty(Object target, Type targetType, String propertyName )
		{
			PropertyInfo property = targetType.GetProperty(propertyName, BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public );
			return ( property != null ) ?return property.GetValue(target, null) : null;
			}
		}
		#endregion
	}
 

You will probably agree with me that the abstraction created by the class, methods and properties are valuable. They create cohesive and meaningful units in our domain.

What about the #regions? Is a label “Implementation of IPostBackDataHandler” helping the programmer to be precise or just telling us that maybe that logic should be extracted to somewhere else? If you need to isolate that from the other bits of your class maybe it doesn’t belong in that class at all! #regions are bad smell, they are hints that your abstraction is broken.

Sidenote: Surely people use #regions to flag blocks like the “Properties” and “Events” in the previous example. Probably this people don’t think of the #regions as abstractions at all, it’s just something to help while editing that file. We still have a bad smell though. Why do you have so much stuff –so many properties, events, methods, etc.—in a single class that you need to expand/collapse them? Maybe you have too much stuff going on in a single class?

Conclusion

As software developers, most of what we do is to create and reuse abstractions. They play an important role from the bottom of bits and bytes to the higher-order-everything nature of modern programming languages.

It is so natural and easy for a developer to create abstractions that we need to make sure they are actually adding value to our system. Death by abstraction is so common there is a very famous quote, attributed to Butler Lampson, that says:

All problems in computer science can be solved by another level of indirection

To what Kevlin Henney replied with:

except for the problem of too many layers of indirection

In our application we have a notifications system. It is quite simple; notifications are read from a text file and shown in a small information box in the home page. The MVC controller –it’s a Java application and we use Spring MVC – talks to a Service Layer and that finds out who should do the dirty job, in this case a class named NotificationTxtFile.

One important thing about notifications is a non-functional requirement saying that we only want to read the text file once per session. Here is the test we found for this class:

@RunWith(MockitoJUnitRunner.class)
public class NotificationServiceTest extends TestCase {
    @Mock
    SessionFacade sessionFacade;

    @Mock
    Notification notification;

    @Mock
    NotificationTxtFile notificationTxtFile;

    @Test
    public void shouldLoadNotificationFromSessionIfNotEmpty(){
        when(sessionFacade.getAttribute("notification")).thenReturn("My Notification");

        NotificationService notificationService = new NotificationService(notificationTxtFile, sessionFacade);
        String notificationMessage = notificationService.getNotificationToBeDisplayed();

        assertThat(notificationMessage, is("My Notification"));

        verify(notificationTxtFile, never()).getNotifications();
        verify(sessionFacade, never()).setAttribute("notification", "My Notification");
    }

    @Test
    public void shouldLoadNotificationFromServiceIfNewSession(){
        when(sessionFacade.getAttribute("notification")).thenReturn(null);
        when(notification.getMessage()).thenReturn("My Notification");
        when(notification.getTimeStamp()).thenReturn(new LocalDate().plusDays(1));
        when(notificationTxtFile.getNotifications()).thenReturn(Lists.create(notification));

        NotificationService notificationService = new NotificationService(notificationTxtFile, sessionFacade);
        String notificationMessage = notificationService.getNotificationToBeDisplayed();

        assertThat(notificationMessage, is("My Notification"));

        verify(sessionFacade).setAttribute("notification", "My Notification");
    }

    @Test
    public void shouldSetNotificationToBlankStringAtSessionIfThereIsNoNotification(){
        when(sessionFacade.getAttribute("notification")).thenReturn(null);

        NotificationService notificationService = new NotificationService(notificationTxtFile, sessionFacade);
        String notificationMessage = notificationService.getNotificationToBeDisplayed();

        assertThat(notificationMessage, is(""));
        verify(sessionFacade).setAttribute("notification", "");
    }

    @Test
    public void shouldSeeNotificationWhenMultipleNotificationsExistInTheFuture(){
        Notification notificationA = new Notification(tomorrow(), "my notification");
        Notification notificationB = new Notification(nextMonth(), "my other notification");
        when(notificationTxtFile.getNotifications()).thenReturn(create(notificationA, notificationB));

        NotificationService notificationService = new NotificationService(notificationTxtFile, sessionFacade);
        String notification = notificationService.getNotificationToBeDisplayed();

        assertThat(notification, is(notificationA.getMessage()));
    }

    @Test
    public void shouldNotSeeNotificationForPreviousDate(){
        Notification notificationA = new Notification(yesterday(), "my previous notification");

        when(notificationTxtFile.getNotifications()).thenReturn(create(notificationA));

        NotificationService notificationService = new NotificationService(notificationTxtFile, sessionFacade);
        String notification = notificationService.getNotificationToBeDisplayed();

        assertThat(notification, is(""));
    }

    @Test
    public void shouldNotSeeNotificationWhenThereIsNoNotificationTxtFile(){
        when(notificationTxtFile.getNotifications()).thenReturn(new ArrayList());

        NotificationService notificationService = new NotificationService(notificationTxtFile, sessionFacade);
        String notification = notificationService.getNotificationToBeDisplayed();

        assertThat(notification, is(”"));
    }

    private LocalDate yesterday() {
        LocalDate today = new LocalDate();
        return today.minusDays(1);
    }

    private LocalDate nextMonth() {
        LocalDate today = new LocalDate();
        return today.plusMonths(1);
    }

    private LocalDate tomorrow() {
        LocalDate today = new LocalDate();
        return today.plusDays(1);
    }
}

The tests may look OK, but there is an interesting problem there. The reason we use tests in agile methods is so that we have proper executable specifications about what’s being developed. Those specifications should talk about what a unit does, not about how it performs its task.

As I said before, the notifications file should be read only once. This is a requirement and this is what we care about here. If you look at the test, though, it doesn’t really test if the file is read only once. It tests if the file contents are stored in and retrieved from the (HTTP) session object.

Why should I care about where the file contents are stored? I shouldn’t have to, it’s not relevant to the classes requirements. Storing the file contents in the HTTP session is how our class happens to implement this feature. It can change this implementation detail at any moment and the class would still fulfil its requirements.

When writing a test you should focus only on the expected behaviour, not the current implementation. Using unit tests to verify design and implementation is a terrible thing, you break the encapsulation and end up with really brittle tests that have to be changed all the time, just because some implementation detail changed even though the behaviour is the same.

I’ve been to many projects where this kind of highly coupled tests was the norm. They always end up with a test suite that has to be changed so often that developers prefer to only write integration tests, blaming unit testing for being too brittle. I think that they are missing the point, the problem is not with unit testing but with people breaking into their units and depending on its trade secrets.

The way Cauê and I dealt with this issue was to refactor this test to something like this:

@RunWith(MockitoJUnitRunner.class)
public class NotificationServiceTest extends TestCase {

    @Mock
    NotificationTxtFile notificationTxtFile;

    Notification yesterdayNotification = new Notification(yesterday(), "A long long time ago");
    Notification todayNotification = new Notification(today(), "Tonight is a good good night");
    Notification todayNotificationWithMultipleLines = new Notification(today(), "Jeremy spoke in class today\nClearly I remember\nPickin' on the boy");
    Notification tomorrowNotification = new Notification(tomorrow(), "Where's my hoverboard?");

    SessionFacade sessionFacade;

    @Before
    public void setup() {
        sessionFacade = new HashMapBackedSessionFacade();
    }

    @Test
    public void shouldReturnASingleLineNotificationForToday() {
        when(notificationTxtFile.getNotifications()).thenReturn(asList(yesterdayNotification, todayNotification, tomorrowNotification));

        NotificationService notificationService = new NotificationService(notificationTxtFile, sessionFacade);
        assertThat(notificationService.getNotificationToBeDisplayed(), is(todayNotification.getMessage()));
    }

    @Test
    public void shouldReturnMultipleLinesNotificationForToday() {
        when(notificationTxtFile.getNotifications()).thenReturn(asList(yesterdayNotification, todayNotificationWithMultipleLines, tomorrowNotification));

        NotificationService notificationService = new NotificationService(notificationTxtFile, sessionFacade);
        assertThat(notificationService.getNotificationToBeDisplayed(), is(todayNotificationWithMultipleLines.getMessage()));
    }

    @Test
    public void shouldReturnEmptyWhenThereAreNoNotificationForToday() {
        when(notificationTxtFile.getNotifications()).thenReturn(asList(yesterdayNotification));

        NotificationService notificationService = new NotificationService(notificationTxtFile, sessionFacade);
        assertThat(notificationService.getNotificationToBeDisplayed(), is(""));
    }

    @Test
    public void shouldReturnEmptyWhenThereAreNoNotificationsAtAll() {
        when(notificationTxtFile.getNotifications()).thenReturn(EMPTY_LIST);

        NotificationService notificationService = new NotificationService(notificationTxtFile, sessionFacade);
        assertThat(notificationService.getNotificationToBeDisplayed(), is(""));
    }

    @Test
    public void shouldNotRetrieveNotificationsMoreThanOnceForTheSameSession() {
        when(notificationTxtFile.getNotifications()).thenReturn(asList(yesterdayNotification, todayNotification, tomorrowNotification));

        NotificationService notificationService = new NotificationService(notificationTxtFile, sessionFacade);
        for (int i = 0; i < 10; i++) {
            assertThat(notificationService.getNotificationToBeDisplayed(), is(todayNotification.getMessage()));
        }
        verify(notificationTxtFile, only()).getNotifications();
    }

    private LocalDate yesterday() {
        LocalDate today = new LocalDate();
        return today.minusDays(1);
    }

    private LocalDate tomorrow() {
        LocalDate today = new LocalDate();
        return today.plusDays(1);
    }

    private LocalDate today() {
        return new LocalDate();
    }

}

This test is much less coupled to the implementation than the original version. But why is it? Because it only mocks the classes collaborators, not its aggregations and compositions. My general rule is that it is OK to mock any association between classes, except by aggregation and composition:

[aggregation is] a form of association that specifies a whole-part relationship between an aggregate (a whole) and a constituent part. It is also applicable to attributes and parameters.
[…]
There is a stronger form of aggregation, called composition. A composite is an aggregate with the additional constraints that an object may be part of only one composite and that the composite object has responsibility for the disposition of all its parts—that is, for their creation and destruction. An aggregation that is not a composition is called a shared aggregation, because parts may be shared among more than one whole.

Experience says that there are times when you need to mock one of those kinds of relationships; but these are exceptional situations and could probably be replaced by integration tests.

Aggregations and compositions are not just collaborations; they are part of the class internal design and should not be exposed; not even to tests.

During that period, we often come up with a technical vision for the system. It is a very incomplete artefact, pretty much just a system metaphor that is good-enough to start the project. At this stage I always get funny looks from the client’s architecture/development team and questions like: “So, in this agile thing we have no architecture? Just start coding without thinking about how things are structured in the higher level?”

Well, not really. We do tackle architecture problems, but in a very different way than what other methodologies may suggest. When a project starts, I don’t care about having a sound architecture. What I do care about is a reasonable strategy for dealing with architecture decisions.

In an agile project, User Stores have in themselves both business value and technical investigation/architecture work. The usual way to deal with that is just to pile up those cards in an iteration backlog and play them by priority order. This works really well for design but is not so great for some architecture decisions. Architecture, remember, may be defined as “things that people perceive as hard to change“.

As a Tech Lead/Iteration Manager I’ve learned and experimented with many different strategies for dealing with those “hard to change” decisions. In this article I want to discuss four of those.

1 - Iteration Zero

One way of dealing with architecture issues in general is to try to address them in the very first iteration for a project, often called the Iteration Zero. The team will try to perform all technical investigation and architecture decisions that they know will have to be addressed at once, before they actually start writing some production code.

In theory, this strategy benefits from some economy of scope; it is probably more efficient to group your technical/architectural problems while solving them. The User Stories would benefit from the architecture defined in the first iteration; therefore they should contain 100% business value and no known technical risk.

This has hidden problems, though:

• Wasteful: You will be spending time designing and architecting things for cards that may never be played.
• Doesn’t deliver business value: When you are introducing agile methods to a client, the first thing you want to tell them is that every iteration they will get some business value. After all the work in convincing them, you start the project and don’t deliver anything in the first iteration. This can become such a big problem that account managers inside ThoughtWorks sometimes ask teams following this strategy to call this “Preparation”, “Technical Investigation Phase” or “anything but an iteration!”.
• You don’t know what you don’t know: You may try to address all technical investigation now but if your project is really agile what you know now is just a tiny fraction of what you will know in two weeks time. You are not making commitments in the last responsible moment, and that really reduces the benefits of agile methods.
• Not enough time: It is very common for Iteration Zero to break the timebox and go on for many weeks, even months. The amount of technical investigation and work -that, remember, may or may not be actually necessary- just won’t fit in a regular iteration.

I do use Iteration Zero, but I don’t use it as the main point for technical investigation and design. My iteration Zero aims to get a story done, from end to end. This has to be an important story, so that the work performed to deliver that can be reused in some others. Given we have no structure in place yet, this is often that is too much work to accomplish by a single developer or pair. What I usually do is to break the story into tasks, Scrum style.

That way we will have something to show the client in the showcase and, accordingly to the Rule of the Second Card, will have made just enough design to reduce risk for the upcoming iterations.

2 - Tech Story

This is the style suggested by Philippe Kruchten. In this strategy, architecture and design work have their own cards that get prioritised and played together with business-value cards. We call those cards Tech Stories, as opposed to User Stories. Theoretically, most architecture decisions will be made while playing the Technical Stories, the developer playing a User Story would benefit from the fact that all decisions were already made.

The biggest benefit of this strategy is that it creates visibility on the technical risks and design work required for a project. When you mix technical and business requirements in a single story, it is very hard to understand how much of the effort in there is technical and how much is business-related; you often have to justify to the business why “add a link to the home page” is actually a lot of work because your architecture was not ready for that.

It’s good that it doesn’t hurt the just-in-time nature of agile methods. You can schedule Tech Stories to be played in any iteration, ideally just before playing a dependant User Story.

In my experience, this is not so great, though:

• Wasteful: Just as the Iteration Zero approach, Tech Stories mean that you may waste time investigating and designing things that will not be useful when requirements change. And they will change.
• Architecture becomes low-priority: If those cards are put in the product backlog that means that they will have to be prioritised by the business, and it is really hard for them to understand the value of those tech tasks. Developers have to keep having the same debates, trying to convince that those acronyms have real value.
• Creates a dependency chain: The dependency between Tech Stories and User Stories is really hard to manage. If a Tech Story takes longer than expected it may block not one but many user stories!
• There’s also something missing: Regardless of time spent in doing up-front technical investigation, you will not be able to cover all possible things for all user stories. Technical investigation and design will happen inside a user story anyway.

Tech Stories will happen in any non-trivial project. I try to minimise their number, I don’t want my client paying up-front for a technical effort that she may or may not need. I want the client to understand that every time we add a feature there is a cost, and that part of this cost derives from purely technical work.

I also hate to ask the client to prioritise things that they don’t understand. How can I explain to a product manager that “Refactor user class” is more important than “Edit user details”? Many important technical decisions cannot be directly mapped to tangible benefit. You can try and convince your client, but you are going to have this same conversation every single week, for the rest of the project.

When I work on projects where, for some reason, Tech Stories are the norm, I usually apply a technique called technical budget (Krutchen calls it “buffers”). In this technique, every iteration has a fixed capacity (often expressed as a % of the Velocity) allocated to Tech Stories. This works, but it’s very inefficient. Budget sizes will always be too long or too short; you often end up having to re-negotiate buffer increases every iteration.

3 - Spike

Doing any up-front design or investigations is mostly about mitigating risk. Risk can only be mitigated by generating information (see Managing the Design Factory for an excellent discussion on this). The canonical way to generate information in agile projects is called a Spike.

A Spike is a experiment whose goal is to generate information. May be something like “can we use JSON.Net to serialise our data instead of doing it using a template engine?” or “should we split this blob into a three Layers system?”.

A developer or pair playing a Spike will try to generate the required information, often by writing some code. This effort is time-boxed and the Spike does not add to the team’s Velocity. A Spike will not remove the need for technical investigation and the risk from the user stories completely. It will just reduce the required effort by providing some information acquired before the card is played.

My problems with this technique:

• Architecture becomes low-priority: Just like with Tech Stories, Spikes need to be prioritised by the product manager.
• Some will be useless: Spikes may end up generating no useful information whatsoever. Surely the developers will learn something, but this information may not be relevant to the project at all.
• Throw-away: The code wrote for a Spike should be disposed once its goal is met (i.e. information was generated). Spike code tends to have very low quality and it is possible for developers to just hack some quick solution just to find out later that is not applicable to the production code.

I use Spikes all the time. Most of them are not disposable code, though. I often require my team to have the same quality standards with Spikes as we have with production code.

As of prioritisation, it is less of a problem than with Tech Stories. Tech Stories are supposed to have value by themselves and that is hard to communicate to business people. With Spikes, though, I usually ask for two estimates: one for “if this card is played before the Spike” and other for “if that card is played after the Spike”. In many projects, not playing the Spike before the User Story means a 60% higher estimate.

4 - Assembly Line

The last strategy I want to discuss today is what I call an Assembly Line. In this strategy, the development bit of a user story is split in two or more sections, one for technical investigation/architecture and other for actual development (i.e. meeting the acceptance criteria).

User Stories are first placed in some Technical Investigation queue and, after someone does the necessary investigation and design, they are moved to the actual Development queue.

This is really useful when there is a huge difference in the work and/or skill set required for designing a solution and just meeting the acceptance criteria.

The problems I see here:

• What’s “done”?: It is really tricky to define what means “done” for the work performed in the investigation queue. Expect many endless (and pointless) discussions.
• Cards are blocked forever: When using this strategy, many times I had up to two development pairs idle and no cards free to be played. They were all in the Technical Investigation queue, and this queue often has less developers than the Development queue.
• Developers vs. “Architects”: It is fair too easy to create a distinction between people that do technical investigation and people that only play cards. The developers that are only playing cards often end up with very little interest and commitment to the project. When faced with a technical problem they are most likely just to move the card back to the Technical Investigation queue instead of working on it.
• Throw over the wall: The separation of work in two steps tend to create a “throw over the wall” culture.

This technique is, sometimes, the right opposite of what we are trying to achieve with agile methods. I see the Assembly Line as a strategy that makes sense only sometimes –when recovering a project is a good example– and only for a limited period. Impacts on morale for a team using this strategy for longer than strictly necessary are massive, and the division of work creates all sort of problems regarding information sharing and blaming culture.

It is possible that your project requires some heavy up-front design for some cards, but if this is a common scenario for you there is probable something wrong. Design should not be apart from implementation.

> We are supposed to be a community, we all use Ubuntu and contribute
> to it, and we deserve some respect regarding these kind of decisions.
> We all make Ubuntu together, or is it a big lie?

We all make Ubuntu, but we do not all make all of it. In other words, we
delegate well. We have a kernel team, and they make kernel decisions.
You don’t get to make kernel decisions unless you’re in that kernel
team. You can file bugs and comment, and engage, but you don’t get to
second-guess their decisions. We have a security team. They get to make decisions about security. You don’t get to see a lot of what they see unless you’re on that team. We have processes to help make sure we’re
doing a good job of delegation, but being an open community is not the
same as saying everybody has a say in everything.

This is a difference between Ubuntu and several other community distributions. It may feel less democratic, but it’s more meritocratic,
and most importantly it means (a) we should have the best people making any given decision, and (b) it’s worth investing your time to become the best person to make certain decisions, because you should have that competence recognised and rewarded with the freedom to make hard decisions and not get second-guessed all the time.

[…]

> If you want to tell us that we are all part of it, we want information,
> and we want our opinion to be decisive.

No. This is not a democracy. Good feedback, good data, are welcome. But
we are not voting on design decisions.

Source: Ubuntu’s Issue Tracker

The dialogue above is about a UI change, not software design itself, but I see this happening all the time in software development projects, especially those following agile practices.

Agile is all about collaboration. In an ideal agile project, the team works together for the full lifecycle of a feature; everyone should understand and have input in what features are going to be prioritised, how the architecture is going to be like, etc.

That does not mean, though, that this is a democratic process. Someone in charge of designing a component should listen to her team and work with them to solve the issues pointed out, but she is still the one that will make the final call on how the design for that piece is going to be like.

There are two main problems in having a democratic design process:

• Mediocrity Always Wins: Not everyone in a team has the right skill set and experience to design every piece of the system. In a democratic design process you will end up with “The Homer”.
• Design by committee: “The defining characteristics of ‘design by committee’ are needless complexity, internal inconsistency, logical flaws, banality, and the lack of a unifying vision.”

That doesn’t mean that there is an Architect (capital A, please), designing the system for the less-skilled developers to write. Architecture is an ongoing thing; team members make architecture decisions every day.

What that means is that, every time a design decision has to be made, the decision must come from an individual or very small group of people that have the skills and experience required –excluding coaching and teaching, of course.

And this is pretty much the only way that this industry knows to produce quality software –or humankind knows how to produce any creative work, really. The best software were architected by a single person or a tiny group of individuals sharing a common vision and with similar skill levels (e.g. EJB vs. Hibernate, SOAP/WS-* vs. REST, C++/Java vs. Ruby/Python…).

Agile methods don’t change this rule. When it comes to design, meritocracy is still best.

There are several things that I consider extremely important when you face the challenge of recovering a project. Some of them are things that every project should do, but some others are only good for extreme situations, like trying to meet a deadline, and should not be used in other situations.

Have a Good-Enough Understanding of What’s Going On

Before trying to fix something it is very important to understand what was broken. It is very easy to assume that the project team was just stupid and not even ask them for their view on what is going on –this is just wrong.

Team members will provide invaluable information. Even if they cannot understand what went wrong in the process, their experience will tell you what works, what could potentially work and what doesn’t work at all.

Keep in mind, though, that there is never enough time to perform a full project post-mortem. The information you get must be the minimum required to draft a recovery plan.

Define what Success Looks Like

When crunch time begins, it is too easy to just start writing code and delivering something, anything. Many teams follow this path and get really frustrated when, after so many long hours, weekends and divorces, their client is still not happy with what was delivered.

Unless your client is really trying to screw you up, the problem here is that you have to understand what success looks like before trying to reach it. Identify your main stakeholders and try to understand not only why are they upset with the project’s performance but also what are their expectations.

There is also effort required in managing those expectations. It is very likely that the client will have to give in on something -often scope, time or cost. Do not create false expectations, the project team is already in trouble, they don’t need you making empty commitments just to calm down the client.

Have a Plan

Recovery or not, every project should have a plan. The team should get together, look at what they are trying to achieve and list the actions and behaviours they have to change to get there.

Just like any other plan, this will be wrong. Your everyday actions will provide feedback on that original plan and you have to adapt accordingly. Remember: the plan is not important, the activity of planning is.

Stabilise the Process

Agile teams tend to have loose and flexible processes, and this is a good thing. That’s not an excuse, though, for unstable processes.

Stability is defined as the capability to produce consistent results over time. Instability is the result of variability in your process.

– The Toyota Way Fieldbook

Stabilisation brings to the table one of the most important resources for planning: predictability. Having a team that consistently delivers 30% of what they should is better than a team that delivers 0% this iteration, 50% in the next and 20% in the third one.

Of course we want the productivity to be high, but in order to get an agreement from stakeholders you need to know what to expect from the team. Probably the relationship between stakeholders and team is not great at the moment and the last thing everyone needs is overcommitment.

Once you get stability and some free capacity, then it’s time to improve.

Reduce Waste, Not Tests

As mentioned before, some trade-offs that will have to be made. The first thing most people do during recovery projects is to reduce time spent in anything but writing code. That often means dropping analysis and testing, at all levels.

I find this to be terribly wrong. What you should focus on reducing is waste. Waste means different things for different projects, but is very unlikely that testing by itself is waste. The development team must define what is acceptable internal quality for the system. This should contain, at least, automatic verification of the system at higher level –”acceptance” or functional testing, including integration points.

During crunch time, there isn’t much time for deep technical discussions and research, the tests will give you some confidence that we are heading towards our goal and not in the opposite direction. The safety net provided by an automated tests suite is crucial to achieve stability.

Continuous Integration is also extremely important, your team cannot afford spending one hour every day fixing source conflicts or searching the revisions for some feature that was there but disappeared in a merge.

This is not exactly about Test-Driven Development. In my experience, a team that is used to TDD has absolutely no reason for dropping the practice for crunch time –in fact I’ve seen teams that reduced productivity drastically due to dropping TDD. A team that is being coached in TDD, though, may find it better not to learn a new technique while trying to reach a deadline. Given the automated tests we discussed before are in place, this is usually ok.

Protect the Team

There is nothing worse than trying to get some work done and getting pulled out for useless meetings. Team’s internal meetings should probably be reduced to a minimum and it is often a good idea to assign someone that is not a direct part of the process flow (i.e. Iteration Manager/Project Manager) to go to external meetings.

Probably every now and then this person will need a developer or tester to be in a meeting, but my experience says that for the vast majority of them just having someone that understands the project -not necessarily the technical bits- is enough.

One important thing, though, is that this person must share information with the team. My personal policy is that all emails I send -except those containing confidential and/or personal information- are copied to my whole team. I tell the team that I don’t expect them to read those emails at all; I just want the information to be accessible, if needed. More than that, I don’t want people feeling like I am making decisions behind their backs.

Bring Something New

This sounds silly, but it is usually a little thing with a massive impact in morale: bring something or someone new to the team.

It could be as simple as adding sounds to the CI server, bringing some toys to the office or even getting a new team member that loves to crack jokes. It makes it feel less like a factory and more like a creative space –creativity is still crucial, even in crunch time.

Simplify Your Technical Stack

Being a passionate developer, this is the hardest thing to me. The best way to simplify your stack begins by looking at your backlog. Which areas of the system are you going to touch more in the upcoming iterations? How easy is it to understand and modify those?

It may be extremely useful to get your best developers to spend some time optimising those parts for productivity. This often comes as removing duplication, accidental complexity and explicating patterns present in the code.

Keep in mind, though, that “an ideal architecture has to fulfill the requirements from the system users, the project sponsors and the developers.”. The goal is not to create an efficient architecture, it is to create something that your people will understand and be able to work with.

That often means dumbing down the architecture. Instead of using that really obscure feature in Spring that does exactly what you are writing code for, it is much more important to have something that your developers understand. Even if that means writing your own thing.

These are really hard things to do. Many times during your journey you will have to sit down with your team and make sure everyone understands that for a recovery project “best” is worse than “good enough”.

Make sure that you are collecting the feedback from the team, though. And make sure everyone, specially the stakeholders, understand that once crunch time is over you have to pay the technical debt.

I think that Domain-Driven Design is one of the most important mainstream technical books published this decade. I did enjoy a lot the activities we performed and the people we met during those sessions. I guess I’m just too tired of how our industry just does not get the difference between Domain-Driven Design and Object-Orientation.

Being specific, the reasons I am so tired are:

1. It seems to be extremely hard for people to understand Domain-Driven Design.
2. People don’t have to understand Domain-Driven Design to benefit from it

Please let me elaborate.

It seems to be extremely hard for people to understand Domain-Driven Design

Let’s do a quick search on Google:

This is just a quick example of how people understand Domain-Driven Design: There are much more hits on Google about the Repository and Anti-Corruption Layer Patterns than about the Ubiquitous Language. To better understand the problem that this represents, let’s revisit the book.

Ubiquitous Language is defined on Part I. Accordingly to the book, Part I
“presents the basic goals of domain-driven development”. Just before that, in his foreword, Martin Fowler also says:

Eric also cements many of the things that we’ve learned over the years. First, in domain modeling, you shouldn’t separate the concepts from the implementation. An effective domain modeler can not only use a whiteboard with an accountant, but also write Java with a programmer. Partly this is true because you cannot build a useful conceptual model without considering implementation issues. But the primary reason why concepts and implementation belong together is this: The greatest value of a domain model is that it provides a ubiquitous language that ties domain experts and technologists together.

That sounds pretty important, doesn’t it?

The Repository Pattern, on the other hand, was catalogued by Fowler in his Patterns of Enterprise Application Architecture and is presented by Evans in Part II. The Author says that this part’s goal “is to focus on the kinds of decisions that keep the model and implementation aligned with each other, each reinforcing the other’s effectiveness”.

The large number of hits for the Anti-Corruption Layer is even more disturbing. It is presented in Part IV, and this part “enables the goals of Part I to be realized on a larger scale, for a big system or an application that fits into a sprawling, enterprise-wide network”.

That’s odd. A lot of people are talking about Eric Evans’ work and yet people seem to be more interested in the building blocks than in the actual core of Domain-Driven Design.

That was the reason that I created that workshop. All over forums and user groups people are talking about Domain-Driven Design as if it were just a modeling technique and/or a Pattern Language. Given that the Patterns and techniques that the book mentions are anything but new to Object-Orientation, I have to ask: if Domain-Driven Design is just about how create an object model, how is it different from what was developed in decades of Object-Orientation?

Repositories, Value Objects, Entities, Specifications… nothing about these is new. If Evans had written a book only to catalog these Patterns he would just be repeating what other people already said years ago. Is very unlikely that his work would have any relevancy or generate much interest.

But that’s not the case. Domain-Driven Design uses the Patterns, techniques and strategies to enable the only thing that this technique actually brings to the table: to keep model and implementation aligned and use a language that reflects this model. Quoting the book once more:

In domain-driven design, three basic uses determine the choice of a model.
1. The model and the heart of the design shape each other. […]

2. The model is the backbone of a language used by all team members. Because of the binding of model and implementation, developers can talk about the program in this language. They can communicate with domain experts without translation. And because the language is based on the model, our natural linguistic abilities can be turned to refining the model itself.

3. The model is distilled knowledge. […]

And this book does achieve the goal of showing how to bind model, language and implementation in a brilliant way. The problem is that people are just too much excited about shiny “new” Patterns to actually pay attention to what matters, even thought the author repeats that over and over throughout the book!

I guess the problem here may be caused by the fact that most people I know made Domain-Driven Design her first book on real-world Object-Oriented design. I don’t think it is a good book for this, actually I think it is a terrible choice. Evans’ book should be your third, maybe fourth book on the topic. It is much better to read it after having to fight against models that lack a unifying language.

People don’t have to understand Domain-Driven Design to benefit from it

And this is probably a good thing. As a consultant I am always on client-side and always working with teams of different levels. Most often I have a team that know just the bare basics of Object-Orientation and can’t possibly digest concepts like those presented by Evans.

That’s ok though, they don’t have to. One of the good things about using the Domain-Driven Design way of thinking is that it provides you with a simple framework to figure out what –not how- to do; but the single best thing about Domain-Driven Design in most scenarios is that people don’t have to know the name of this technique.

In my workshops I explored that a lot. Only after four hours of multiple activities that focused only on modeling a domain, defining a language and refactoring it that I introduced the building blocks –and the only reason I introduced the Patterns during my workshop was because I had people that already had read the book and had questions to ask about that.

And that is the same kind of thing that I do with my teams. I will try to never use the words entity or value object but I will explain the different types of identity and lifecycle objects have. I will not say that we have a Repository of Users but I will model with them some class that represents a list of stuff.

My experience says that the book is not enough. My experience says that the workshop is not enough. My experience says that people have to see the goal of Domain-Driven Design by applying the Ubiquitous Language in an actual project.

It was a bit obvious that what we needed was not only system abstractions for Facebook, Twitter and the like but Bounded Contexts. We need to acknowledge the fact that these domains are not part of our model, even though we depend on them. We need to make this explicit.

The Facebook API doesn’t have the same concept of User as we do, and neither does Twitter. Someone has to translate from Profiles to Users. We identified the UserRepository as the best candidate for that.

The hugest advantage of this is proper dependency management. If you remember our last post, we had a circular dependency between system abstractions and our model. It was something like this:

But, by isolating message parsing from the user creation, we remove that red line. Facebook doesn’t create Users anymore, therefore it doesn’t depend on AllSocialNetworks/UserRepository. Will fall in a situation closer to this:

Now UserRepository will have access to everything it needs to create valid instances of User –and all it needs are objects returned by the system abstractions. The circular dependency problem is solved.

While moving this translation away from the system abstractions we learned that our situation was much worse than we first thought. The challenge looked simple, transform this:

Into this:

But indirect coupling among those classes was so strong that even with multiple levels of tests the effort to migrate all system abstractions to this structure is still in progress.

The Repository class is currently in a weird situation. It was already a very good candidate to be renamed to something else or, even better, split in two or more. After all those changes, though, that class is just too far away from whatever a Repository should be. This will probably be the next major refactoring that we have to do in this system.

But that’s another story, that maybe will be told another day…

Conclusion

In this example we went from this:

To this:

We sent from seven to thirteen classes. From two to four major packages/modules. Did this really help or we just went nuts on abstraction explosion?

All metrics I have gathered up to this point are positive. I am not talking about abstract metrics like cyclomatic complexity, efferent coupling and etc. –those are useful but won’t tell much without context. What tells me if those changes helped or not is how they impact the behaviour of the development team.

We went from a couple of fat and overdependent classes to many small classes with limited responsibility. This design is not suitable for all scenarios but it is what works well with most Object-Oriented models. If you consider that we are always adding more integration points -it’s an integration project after all- the extra flexibility and explicit concepts are really welcome to the system.

People in the team are more comfortable with the new model, it is easier to talk to the business and understand what they want done. It is also easier for newcomers to understand what the system is about and how to change it –how to map the vocabulary used in story narratives to classes and objects.

Looking at Subversion logs, we can see that commits tend to be contained into a given module. That means fewer conflicts to resolve and less contention –i.e. no or few God classes. In the end of the day this means that developers waste less time waiting for someone else to finish working with a given class and in rework.

On the bad side, making this refactoring while still delivering user stories is messy. It is really important to remember people frequently that we should not patch the old classes, but take the chance to advance towards the new model.

Unfortunately we were not pro-active enough to start refactoring and cleaning the code before having issues with circular dependencies and while playing a story one usually feels much more pressure to just fix the symptom than to solve the root cause.

In the previous post we were facing the problem demonstrated by the diagram below.

Our FacebookMessageParser needs an instance of AllSocialNetworks so that it can create valid Users coming from Facebook. The only implementation we have for the AllSocialNetworks interface is UserRepository, and this implementation needs a FacebookMessageParser. That’s a circular dependency, it becomes impossible to instantiate the objects above without breaking invariants.

The invariant of an object must be always fulfilled; once the invariant is broken the object is not valid anymore and your system is in an abnormal situation –i.e. I would expect it to throw exceptions like crazy and/or eventually die in pain.

The problem seems to be that we don’t have a single place that knows all the required information to create a User. In our system we have a situation where all Users have Facebook accounts, but some of them have Twitter accounts as well. For those with both types of account we must construct the user object with information coming from all Social Networks.

The first thing we thought about was that our problem was in how the User object was being constructed. One-step instantiation wasn’t working. Turns out that this is a common problem in Object-Orientation, so common that there is an old pattern that deals with it: the Builder design pattern. One of the consequences of this pattern is:

It gives you finer control over the construction process. Unlike creational patterns that construct products in one shot, the Builder pattern constructs the product step by step under the director’s control. Only when the product is finished does the director retrieve it from the builder. Hence the Builder interface reflects the process of constructing the product more than other creational patterns. This gives you finer control over the construction process and consequently the internal structure of the resulting product.

That sounds like exactly what we need here. So we draft a UserBuilder.

This object can be passed to the multiple parsers so that each one of those can populate it with the bit of information that it provides. In the end we call the createUser() method and receive a valid object back.

The next problem: who instantiates the Builder? The UserRepository is supposed to manage the lifecycle for the User, therefore looks like the perfect candidate. What happens, though, when we have a list of users instead of a single one? It sounds that this would be as simple as an operation that maps from one list of Builders to one list of Users:

But the Repository doesn’t know how many Builders should be created -the objects are complex enough so that a UserListBuilder is not an option. Sometimes the FacebookMessageParser is the one who knows how many Users should be created, in other situation the list is known only by the TwitterUserParser.

At that stage our team designed a very complicated strategy that would let the Builders be created in multiple different points depending on the situation. When you get to such complicated solution is possible that you are be missing the point entirely.

We started to realise that our problems may be being made worse given the heavy coupling between parsers, gateways and the Repository. In this post we are looking at only one pair of parser + gateway and it may look like it’s not a big deal, but in the system we had more than ten of those pairs.

The Repositories definitely know a lot more about the internals of each system than they should. Why would it care if Twitter data comes from messages we parse or from objects that we have, say, in cache? It shouldn’t!

The biggest problem that all this coupling was causing in our development process was in our Ubiquitous Language. The business people talked used to say something like “then we get the user profile from Twitter”, but in our heads we had to do the mental mapping –and mental mapping is bad- between the business lingo and what actually happened inside our model. To reduce the noise we created then objects that represent the systems we talk to.

These Facebook and Twitter objects –we called them system abstractions- were modelled after what we, as a business, need from those systems. The technical details about the need to establish a connection and then parse a message were encapsulated by those. We did increase the number of classes, but we reduced the size and complexity of them all.

One of the good things that this refactoring provided was that we could see some interesting patterns in our model that were previously hidden in the messy relationships. Take a look at a subset of the Facebook interface, for example:

The class that represents Facebook knows about our Users. It receives them as parameters and instantiates that class. Why the hell would Mark Zuckerberg care about our system? What that means, in a practical sense, is that changes that affect only our user have the potential to change our Facebook abstraction. The class changes if the Facebook API changes or if User changes. Having more than one reason to change is a smell, as Uncle Bob says:

Why was it important to separate these two responsibilities into separate classes? The reason is that each responsibility is an axis of change. When the requirements change, that change will be manifest through a change in responsibility among the classes. If a class assumes more than one responsibility, that class will have more than one reason to change.

If a class has more than one responsibility, the responsibilities become coupled. Changes to one responsibility may impair or inhibit the class’s ability to meet the others. This kind of coupling leads to fragile designs that break in unexpected ways when changed.

And that was true. A quick look through SVN history shows that every time there was a change in User we had to change at least one system abstraction -and we have many of those types of classes. In most cases those abstractions that were changed together with User had no real relation with the change being made.

It became clear that each of our third-party systems were actually Bounded Contexts. In Domain-Driven Design, Bounded Contexts are boundaries that we create around a model to isolate it from other models. We not only define what goes in and what stays out of a model, but we also define how different models relate.

We had already defined that those Bounded Contexts could only be accessed through their system abstraction. What we had to do was to remove the dependency from the third-party Bounded Contexts to our Domain Model.

We are going to see how we approached that in the next post.

My team is delivering an integration Layer for a large corporation. This client has many different systems and it is extremely common for its clients and employees to spend the day jumping from one system to another, depending on which of the many services provided they happen to be interested on in that very moment. Classic growth-over-the-years problem.

In this integration piece we are building a web services API and a web interface for users in multiple roles. We decided to go for a rich Domain Model. This is not very common in integration projects; most people in those scenarios would just create coarser-grained data structures composed of cherry-picked data coming from multiple back-end systems. We decided for the richer domain model mainly because:

1. Our integration Layer will not only act as a single gateway multiple systems, it will also apply business rules and manage local state.
2. The client is a classic example of environment where the business language was derived from the systems language. More than building some piece of software, our work as consultants is to help the client to solve the impedance mismatches across different departments and domains. Domain-Driven Design is a great tools for making sure that business, developers and everyone else speak the same language. To better apply Domain-Driven Design you should have a Domain Model.

The problem with integration projects is often that no one is really sure about which of the multiple systems we talk to have which piece of information, we discover new things every day. In agile projects that means that we learn a lot during the project and, therefore, we refactor a lot so that our code always reflects our understanding of the multiple models and how they interact with each other.

I will try to describe a sequence of refactorings we conducted over the past weeks. All changes described here were executed following baby steps while we were working on user stories. I will try to get as close as I can to the real world without breaking NDAs and letting irrelevant details spoil the core message.

Let’s say that instead of integrating multiple business systems we were integrating multiple social networks. A requirement for our system is that we should be able to see a single User object in our Layer but the data for this object should come from multiple sources.

We first started as something like this:

That worked fine for a while, we managed to get many stories done using this structure, but there was something funny about this model. For starters, we had the Domain Model depending on the infrastructure, what always rings a bell for me. We also had the PasswordSynchronisationService (something that makes sure you have the same password in all social networks) was depending on the UserRepository. This relationship caused the repository to have funny methods like getAllSocialNetworks() and getSocialNetworksForUser(User).

Clearly this class didn’t get the Separation of Concerns memo and the result was this was class #1 in number of conflicts and merges –every day at least two pairs would touch it, regardless of what stories they were playing.

For some time now I am avoiding the word Repository. Not only it causes all sorts of trouble when people understand it as an alias for DAO but it also has some really weird semantics.

Accordingly to Eric Evans in the Domain-Driven Design book:

For each type of object that needs global access, create an object that can provide the illusion of an in-memory collection of all objects of that type. Set up access through a well-known global interface. Provide methods to add and remove objects, which will encapsulate the actual insertion or removal of data in the data store. Provide methods that select objects based on some criteria and return fully instantiated objects or collections of objects whose attribute values meet the criteria, thereby encapsulating the actual storage and query technology.

So we decided to follow the suggestion of Rodrigo Yoshima (link in Portuguese) and drop the *Repository suffix. If they should behave like in-memory lists let’s give them a decent name! So here we are:

We created a AllUsers interface and let UserRepository implement it. Now it is easier to spot our modelling problem: it is not easy to say there’s something wrong when a UserRepository has a method getAllSocialNetworks(), but a class named AllUsers with such method is really peculiar.

While we think about what to do with our class, let’s think a bit about the domain we have now. I said before that the fact that the classes belonging to the Ubiquitous Language depend on classes from the infrastructure was sounding really funny to my ears. With our last refactoring we can actually spot an easy way out of this weird dependency relationship between Layers. Let’s change the model a bit more…

So now what we have is a AllSocialNetworks interface. The PasswordSyncrhonisationService doesn’t depend on the AllUsers anymore, and that means that we can move all methods related to Social Networks to the new class. As we want to do baby-steps we did not split the UserRepository, just added let it implement the new interface as well.

Talking about UserRepository, we solved the model-should-not-depend-on-infrastructure problem in a very simple way: we just moved the UserRepository out of the domain. In practical terms, that means that we moved it to a different package (it’s a Java project) and made sure that no class in the Domain Model dependeds on it. The Domain Model only knows about the All* interfaces and instances of those are injected through a Dependency Injection container.

A new story is played and suddenly we have something that sounds like a simple change. In order to bypass some technical limitations (we are calling HTTP APIs in this system, there’s no such thing as lazy-loading and ORM) we have to change the domain:

Now the user depends on the AllSocialNetworks list in order to get the Social Networks it belongs to. Any instance of User must have an instance of AllSocialNetworks inside it to be valid –this is part of the User’s invariant. This is not great but at least we will not have to implement our own lazy-loading mechanism –yet.

Just by introducing those two interfaces we simplified our domain. Now, whatever needs Social Networks points to the specific interface and don’t bother with the Users - keep in mind that this is just a tiny subset of a much larger domain and system; we have many other classes that depend and collaborate with those pictured here.

This new requirement created an interesting problem, though. To understand the problem, let’s see how a Facebook user was retrieved before we introduced the dependency from User to AllSocialNetworks:

A call to UserRepository asking for a given user would make it first go to the FacebookGateway and ask it to perform a (HTTP) request for the user profile. The user profile is brought back in, say, JSON and this data is then sent to a parser. The parser interprets the JSON representation, creates an instance of the User, populates the instance with the parsed data and returns it. All used to work fine.

In the new model, we have a slightly different flow:

Can you spot the difference? To be clearer, before the requirement we just introduced, the User class had a constructor like this:

public class User{
 public User(){
}
}

But now its constructor looks like this:

public class User{
 public User(AllSocialNetworks socialNetworks){
}
}

Why? Because before we said that the user story we played required us to change the invariant of a User. Before that story, a User didn’t need anything special to be created. After the story was played, a User started requiring an instance of AllSocialNetworks, without that dependency the User we just created is not a valid object and must not be used in the system. To enforce this invariant we want to make the dependency part of the constructor –a User object with no AllSocialNetworks instances to query is not reliable and should not even exist!

That all makes sense, but here is our problem:

In order to be able to create a valid instance of User, the parser class now must use an instance of AllSocialNetworks. Can you describe the problem we have here?

If the FacebookMessageParser class needs and instance of AllSocialNetworks to work. This relationship is part of its invariant and, as we said before, should be enforced in its constructor (i.e. we will be using Constructor-based injection). The problem here is that the only known instance of AllSocialNetworks is a UserRepository, but a UserRepository needs and instance of FacebookMessageParser to be valid! It’s a chicken-and-egg problem, also known as circular dependency.

Were this an isolated problem and we could potentially just compromise our invariants. We could change that class to setter-based injection, add to our tech debit and keep going for now. This would be pretty reasonable in a real world project, with real world budget and deadlines. The problem is that this was not the first time we had this very problem in this project. It was actually the third time this happened.

Clearly something in our design was wrong and some core change was necessary before we ended up breaking all our contracts and moving from “a couple of circular dependencies” to a full dependency-spiral-of-death structure.

In the next post we shall talk more about the possible solutions we came up with.

I was a bit skeptical at first. I only picked the book because of the big names on the cover and because Peter Siebel’s Practical Common Lisp is one of my favourite books on learn-a-new-programming-language. I thought that a book filled only with interviews with some of the big names in our industry would feel a bit too much like an issue of Who Magazine, with completely irrelevant questions like “Are you a Mac or PC?”. I was terribly wrong. The book is amazing.

Peter does a great job in guiding the interviewees into telling us about how they got into the industry and how they do their thing. For me it was really good to see how pretty much every single person interviewed -regardless of tech platform, background or expertise- tries to somehow come up with short feedback cycles during development.

And that makes it really funny to look at Joel Spolsky’s article on the book. He should have read a bit more before writing such a misinformed article, pretty much all of the interviewees write some other form of specification –being that a test, formal proof or a “test program”- before or while writing the program.

My favourite interview was with Bernie Cosell. Bernie is an old school hacker, he was part of what became the ARPANET. The way he describes his work really maps to what I do, and it can probably be summarised in this quote:

Peter Seibel: Have you heard of refactoring?
Bernie Cosell: No, what is that?
Peter Seibel: What you just described.