Agile in your architecture process…where to put it?

Of late, I’ve been struggling with how to integrate formidable architecture tasks for large projects within the Agile process.  Small projects, maintenance work, pretty much anything that is well understood is OK for “Design as you go” or “Design only what you need” type approaches.  But a ground-up design and build of a large system really almost requires that some general patterns and principles be set forth in advance. 

• “ArcGIS Server will support the need for spatial data in the web tier”
• “Does the client have an ESB in place?”
• “We’re going to have a provider pattern for the data tier….”
• “We’re going to use a service broker…”
• “We’ll use SODA for data access…”
• “The UI will be Microsoft CAB…”

…are all the types of examples I’m thinking of.   

Clearly doing a full on discovery and architecture for 6 weeks defeats the point of agile processes by hamstringing your ability to evaluate and adapt on down the line. Plus it’s a waste of time…you haven’t done stories yet, you haven’t prioritized features and functions with the client, and you’re likely to either miss something or design for an eventuality that will be tossed out later.  You’ve also short circuited the effectiveness of iterations and sprints by hopping in your ivory tower and demanding all the information up front.  So what’s an architect to do?  Well, first thing’s first…you need to remember your core values of courage and humility.  If you can do that, then read on…

My initial readings from around the web on this topic were a bit discouraging.  The gist of the argument from Agile detractors was summed up by Frank Teti, in an article for Software Magazine last June declaring that

with the advent of anti-methodologies, such as Agile, the design ineptness of software development organizations has been further exacerbated—not remedied. Although I truly believe that the best designers understand implementation as well as design, it is their ability to negotiate the feedback loop between design and implementation that results in the most effective approach to software development.

I actually think this argument is self defeating. It is exactly the ability to negotiate the design-implementation feedback loop that allows good agile teams to succeed consistently. Remember, courage and humility. Jeremy Miller reminds us that whether its up front or evolutionary design, your first cut always stinks…or at least is going to need some rework, so why not build it into the process. Trust your team to make the right decisions at critical junctures…and realize that just because you have more letters after your name than they do, doesn’t mean much in terms of keeping the project moving. The team performs well or poorly, not individuals.

Undettered in my quest for agile architecture approaches, the first option that jumped into my head off the bat was frontloading an early sprint in the first iteration with high priority, high effort design tasks.  Enter Chris’s post on iteration zero.  Being relatively new to the Agile realm in the last 12-18 months or so, I’d learned and already forgotten about this little nugget of information. And that sent me off a google’n again. In addition to infrastructure, planning, and concensus building tasks, this is a great place to insert a “discovery and standards” task or two where you find out about client architecture standards if any, and come up with a very coarse box-n-arrow diagram of system tiers, patterns and models to be used etc.  This coarse view can be arrived at quickly in an iteration zero while development impact is nill or very low, and should remain coarse enough to allow the agility of the “design enough” paradigm for the gorey details espoused by Fowler and others.  Your finer grained design decisions can be handled in subsequent iterations as required…or refactored later in stabilization sprints.

Some other required reading:

• A great write up on Agile Architecture theory and practice by Scott Ambler.

• A good summary of agile architecture principles with a few links maintained by Andrew Johnston.

• A good write-up on agile architecture applied to the enterprise with lots of good links to other sources.