It’s taken me awhile to get to it due to a variety of work and family obligations but we’ve finally reached what will be my final post on usability and the geoweb…for now at least.  I haven’t covered all possible usability issues and scenarios, but have hit the high points and “big win” factors over the course of this series of posts.  To this point we’ve dealt with hiding complexity from our users, providing users with consistent, meaningful feedback within an application, and protecting users from themselves.  The lesson plan for this last installment will focus on application performance and its criticality in the overall user experience.

Fact: One of the most critical components of a highly usable software system is that it be highly performant…in other words, fast.  In the realm of the GeoWeb, this means that applications should load fast initially, and that responses to user activity within the application should be as quick as possible.  We now live in a world of instant gratification when it comes to our news, our hobbies, our finances, and above all our interactions with computers. Users don’t like to be kept waiting for 5 seconds every time they use your app to request information, input data, or perform other common processing tasks.  Gone are the days when watching the blue bar move in a web app, or watching the cursor spin on a wireless app were the status quo.  And frankly with the amount that we, as consultants, get paid to build custom software, the user shouldn’t have to wait.

Headlong into the breach then for our fourth and final lesson…

Lesson 4: Maximize Real and Perceived Performance

In all my years of software development, I’ve never had a user or a client come to me and say,

“Ya’ know Brian, this app is just too fast…please slow it down.”

No matter how much time architects, developers, and UX folks spend designing and building the sexiest GeoWeb app you’ve ever seen, all that work is wasted if the user base is frustrated by long wait times to execute functions within the application.  It doesn’t matter how well you automate a customer’s workflow if they’re left sitting and twiddling their thumbs for 5-10 seconds every time they push a button.

Real Performance

There are a whole host of tools in the IT and software professional’s toolbox that help us maximize application and database performance.  Processing power and RAM, faster network, server farm configurations, database indexing and denormalization, etc., etc.  Those standard tools are beyond the scope of this article as they are relatively well understood.  What I’m most interested in covering is how to squeeze more performance out of our maps given a fixed IT configuration.  Real performance in our mapping realm means configuring map services to be fast from the start, caching whenever possible, and applying novel techniques to handle large datasets.

Fewer Layers, Appropriate Symbology and Annotation

I covered this point in the first post in the usability series back in July.  The idea that users need to see all of the layers in your geodatabase to accomplish a task is ridiculous.  No matter how big your server farm is, no matter how fast your processors are or how much RAM you have, large numbers of map layers are going to slow the system down.  Focus applications on 3-5 operational layers over a base tile cache to reap significance performance gains over the kitchen sink style application paradigm.

Another trend that we frequently see in diagnosing map-related performance issues is a map that contains complex symbology and excessive annotation in the absence of a defined business need.  A dynamic map service showing highway signs over the highways is neat but these symbols need to be rendered in four parts on the server (red, white, blue, and text) and take much longer than standard line types.  In most cases, variance in line weight and color, coupled with scale dependent rendering by road type would convey the same information to the user without the performance hit.  Save the complexity for cached services and be as simple as possible in dynamic map services.

Annotation is a valuable tool for adding information to maps beyond the spatial relationships conveyed by the feature geometries.  It is also frequently misused.  Applications that annotate large numbers of layers with all manner of non-critical information rapidly become cluttered and unusable, in addition to incurring the performance hit associated with rendering all that annotation in the map.  Do parcels really need: PIN, dimensions, area, address, owner name, the names of the owner’s seven children and summaries of the last 5 years tax returns?  Probably not.  Keep the annotation appropriate for the map scale and germane to the information you are trying to convey.  If it isn’t relevant to the business problem at hand…leave it out.

Caching

One of the primary tools in our toolbox for maximizing the performance of GeoWeb style applications is the map tile cache.  Think back to just a few years ago (pre-February 2005) and the thought of using a system of cached tiles in the era of “Web GIS” was unthinkable.  As geo-technologists we were “special” and our data was simply too critical to have any latency.  Our apps had to show that parcel split or that new forest boundary seconds after it came into existence.  I think we can all agree that in February 2005, Google Maps showed us all that this line of thinking was just a bunch of hooey.

Nearly all emerging applications in the GeoWeb realm wind up leveraging a tile cache for base maps within the application.  However in many of the more recent systems we have worked on, we have arrived at the assertion that you should cache as much as possible. With the exception of real time battlefield applications for the military, emergency response applications, and a very few other notable exceptions, we hold that you should cache everything.  It is a relatively simple thing to set up an automated process the update a map cache of dynamic data at a time when everyone has gone home from the office and system utilization is typically low or nonexistent.  Trust us, your users will wait 24 hrs to see that parcel update…really.

So, select a server side technology that supports caching map tiles for representation in the client, use a client side map viewer that will efficiently render cached map tiles, and automate cache creation/refresh if necessary to maximize map performance out of the gate.

Cluster Large Datasets

One of the challenges that we routinely face as developers in the age of the GeoWeb is how to deal with the representation of large numbers of geometries in the client.  Direct rendering of geometries in the client becomes the preferred method of visual representation when we need those geometries and their attributes in the client for speedy response to user action: mouse over balloons, etc.  While Flex and Silverlight offer opportunities to render larger numbers of geometries in the client efficiently, JavaScript based applications begin to experience performance degradation in the client after a couple hundred features or so.   Start dealing with complex polygon geometries and the number is less than that.  And even Silverlight and Flex aren’t going to get you speedy rendering of thousands of geometries.

Consider the application shown above.  This application supports a global surveillance network monitoring avian flu (H5N1) and is built using Microsoft Bing Maps(TM) and copious amounts of JavaScript.  If we attempted to actually render global sample locations on the map, we’d likely crash the client as we’re dealing with approximately 50,000 sample locations.  Instead, we elected to cluster sampling points, aggregating data at small map scales and gradually showing additional sample locations as the user zooms in to larger map scales.  The process looks something like that shown below:

This approach allows us to determine the map extent, select the points that fall within the current map extent, and then apply a clustering algorithm to aggregate points near one another before sending the details back to the browser.  Another key point that we discovered is that spatial isn’t always better.  To perform the above tasks with data stored in ArcSDE, the average compute time was ~5 seconds.  Using SQL Server 2008 Spatial data types natively got us down to about 2 seconds, while simply maintaining sample points as X and Y columns with the rest of the non-spatial data got us to less than 1/3 of a second response time.  Don’t be afraid to leave traditional spatial technologies behind when you can improve performance without sacrificing data integrity!

Using the approach described above, we were able to render points on the map AS FAST as the Bing(TM) tiles come into the browser.

Perceived Performance

Another powerful and underutilized means of improving the user experience within an application is to leverage implementation methods that provide perceived performance gains from the user’s perspective.  Because users seldom, if ever, care about the implementation details that solve their problem, we are able to use a bit of slight-of-hand to make a data intensive application very responsive.

The basic message for tweaking perceived performance is actually very simple: be lean on the wire.  If you are building a data heavy app, and most of the line of business apps we build are, then take small bites rather than trying to ship down everything the user “might” ask for at once.

JSON and GeoJSON have become two popular and related formats for shipping data to the client for further processing and rendering, but if performance is the name of the game, don’t be afraid to depart from standards and use a custom JSON formatter to send only the necessary data down to the client.  In this case custom == fast==good!

Let’s return to our H5N1 application for just a moment and consider what happens when we return all those points to the browser.  Our custom JSON formatter returns only the information necessary to properly render points over the map.  But at the same time, a user expects to see additional detail on sample sites when they mouse over a point so what’s a developer to do?  Sending all information for all points down and storing it in the client is certainly one option, however the odds that a user is going to mouse over all of the hundreds of points typically in any given map extent is miniscule so why send all that data over the wire? As shown in the example immediately above, when the user hovers their mouse over a sample point, we quickly shoot off a tiny XHR call to get a subset of the details.  If they want more, they can click the View Details link in the mouse over window to get more information.

As another example, let’s consider what happens when a user requests all the details for a site.  The site flow dictates that we render the full sampling details in a tree view to the right of the map.  For our purposes we selected the Yahoo! YUI tree view for reasons that will become apparent in just a moment.

When the user requests detailed information on an H5N1 sample site, we go to the server again via XHR call and get only the top level information that you see rendered above.  This is why we selected the YUI tree view over other JS-based tree views.  Dojo, JQuery, etc. either work on a full data store to render information or want to bind to a JSON object that has been fully filled.  YUI lets us load information on a node by node basis and this is critical to maximizing perceived performance in this case because, again, the likelihood of a user clicking on every single bird species in small.  If the user clicks on a species in the tree view, we go fetch only the next node of information, and so on, and so on.  Be lean on the wire.

Parting Shots

So here endeth the homily, go forth and develop some more.  Have I hit all the critical points? Probably not.  The field of usability and UX carries with it a substantial set of principles for buidling killer apps with great interfaces and my periodic usability rants are less a codex and more an appeal to the wider development community to continue to hold web GIS at arms length and continue to the march toward more highly performant, usable, focused applications.  The take away from all this should be that we, as a community, need to focus more effort on providing our user base with relevant information right now, and less effort on flexing our GIS muscles by seeing how much crap we can cram into a browser.

As I write this post, I’m attending the GeoWeb 2009 Conference in Vancouver, BC.  This is then a pretty appropriate time to drop part 3 in the Usability and the GeoWeb series.  Part 1 in the series discussed the importance of hiding unnecessary complexity from the user, while Part 2 focused on the importance of never leaving your users guessing and providing them with consistent, meaningful feedback.  In an effort to encourage the incorporation of key usability features into emerging web mapping applications, the lesson plan this time focuses on key application elements designed to protect the users from themselves.

While that statement may sound a little rough (nobody should be out insulting their customers and clients after all) it is not meant as such.  I most certainly am not saying your users are foolish, stupid, or otherwise deficient in any way.  Recall Scott Karp’s assertion that in the age of Google,

“…there are no stupid users, only inadequate designs”

On the contrary, I simply mean that otherwise well-meaning users frequently do things within an application that the application does not expect or that create coding/logic challenges for the developer.  Our task then in building next generation systems to support/integrate with the GeoWeb requires that we anticipate as many corner cases or unexpected results as possible, and architect and code solutions to prevent the user from becoming frustrated at best, and destroying critical information at worst.  Keep the user focused on the application, and put out fires before they arise

With that intro, here’s lesson 3…

Lesson 3: Protect Users from Themselves

So you’ve built your shiny new GeoWeb application, and you’ve done everything right…required field validators, dynamically enabling/disabling input fields to prevent spurious results, targeting distinct workflows to reduce errors introduced by open and undefined function execution, enabling/disabling map tools appropriately.  Your turn the users loose on your application and what happens?  Defects start to roll in.  Well, not really defects so much as comments of frustration and questions on “How do I make this work?”.  Turns out, the application needs more work: user’s have been entering incorrectly formatted dates, and your click function to select a point on a route is nearly unusable.  Explaining to a run-of-the-mill user that a point has no size and a line has no width so selecting a line feature exactly by point is difficult is a fool’s errand.  Prevalidation of inputs, and novel approaches to handling open-ended map interactions (like clicking a point to select a road) are required to prevent users from creating invalid data, removing critical data accidentally, or entering an endless loop of retries when interacting with spatial data.

Validate as Soon as Possible

Ever bought plane tickets from a certain large travel site? The basic workflow is that after the user has gone through the multiple screens to purchase tix, just as the purchase is about to go through, they get a validation message telling them they need to go back to page 2 because the middle initial field is required…or some other ridiculous message that should never occur.  Even better, when the user returns to said page, the offending field may be highlighted, but data in the other 20 required fields was never cached and is now gone.  How fun is that?

Our roadway manager from Lessons 1 and 2 has been such a good an enlightening sport on the subject of usability that we’re going to leverage her experience again here.  Consider the figure below.

Our roadway project manager is updating a road segment to reflect the correct extent of her paving project.  In this case, she’s going to manually enter the beginning point (mile number) of her project on the highlighted segment.  In this case, she indicates that her project starts at mile 90 on Route 40.  Focus on the data entry text box is lost as she goes on to do other work and what happens next is shown below.

Uh-oh!  Let’s look at what happened here.  We have, in the browser, knowledge of the start and end mile markers for our PM’s road segment so we can help her out a little.  The system has done some quick validation and noted that 90 is less than the allowed minimum mile number for this road segment, shown a red ‘X’ which is almost universally understood to mean something has gone awry, and has displayed a message telling her what’s wrong with the data.  In addition, another important thing has taken place.  We have reset the value in the input box to the original value.  We have not destroyed or altered any data, nor have we asked out user to remember what the original value was.  Much like the medical profession, we as developers must “first do no harm” to our client’s data.

And when did all this good validation happen?  Right now!  We didn’t let the user play with the structures grid at the bottom of the page, we didn’t let her make any other changes only to warn her five minutes from now.  We caught the issue early, provided her with a visual cue, preserved the data, and gave her a potential solution to her problem…all with a little JavaScript.

Validate as soon as you possibly can to maintain data integrity and reduce/eliminate user frustration with your app.

Prevalidate Whenever Possible

If validating early is good, prevalidating to avoid problems entirely is even better.  So I know it’s painful but we’re going to leave the realm of the roadway project manager for a second in favor of world health issues.  We’ve built, and continue to enhance, a system for a prominent world wide NPO to track H5N1 (bird flu) incidence around the world.  The system is used by personnel in multiple countries around the world which presents some special issues surrounding data entry…for this specific example let’s consider the entry of dates.

While folks in the US are used to entering date information as MM/DD/YYYY or some variant thereof, other countries like DD/MM/YYYY or YYYY/MM/DD.  Heck some countries (Nepal comes to mind) have entirely different calendars that map by mathematical formula to the Gregorian calendar we’re all familiar with.  While we could simply format date entries to UTC, etc. on the server side, this requires logic at the server to accept information on where in the world the user is entering their data, and complex client logic to turn that UTC into a date that is meaningful to the local user on the other end.  No so fun to deal with.  Why not just use an internationalized date picker?  This is prevalidation at it’s best, the date picker shows a calendar to the user, and when the user picks a date, it is formatted and transmitted in the appropriate format for the local users.  The use of a simple JS widget completely eliminates any chance of an invalid date, or a correct date incorrectly formatted, from squeaking into the system and frustrating or confusing users.  At the same time, the picker knows that a date entered in Europe needs to be formatted differently for a user in the US.

Short Circuit Open or Vague Inputs

So we’re going to jump back to check in on our friendly roadway project manager for our last point in the lesson.  She’s back in our system setting the segments up for a roadway project and telling the system where her paving project is at.  This time, she will be setting up her project road segments by clicking on the map.  This is where things can get interesting because the user can click anywhere she wants.

In the figure immediately above, the user has clicked, whether accidentally or intentionally, a point that is nowhere near the roadway she is working on.  Now, we could try to write code to maximize the efficacy of buffering the point to intersect with the line based on map scale but things can get pretty complex when scale dependent buffers etc. are involved.  A run of the mill user is going to get red faced quickly if we keep telling them to “try again you’re not close enough”.  So why not do something like the result shown in the figure below?

In response to a less than optimal map click for setting the end reference of a roadway project, we simply snap to the closest point on the selected segment.  We’ve avoided coding complex scale dependent buffer/intersect logic that the user will never understand anyway, we’ve avoided the potential for an infinite click-retry loop for the user, and we still get a value that, while it may be incorrect from the user’s perspective, is still a valid point on the segment.  If she doesn’t like it she can enter the mile marker by hand or simply click again.  Low friction protection for the user and it short circuits a whole series of potential outcomes, all of which are bad.

Up Next…

Since I’ve yet to meet a client or user that says their system is too fast, our next usability topic(s) will deal with both real and perceived performance in your GeoWeb application and how to maximize each of these critical performance factors. Stay tuned…

This post is the second in a multi-part series on usability in web-mapping applications.  Dave has spoken extensively on this topic of late…and much of this is just putting fingers to keyboard on the issues we’ve been harping on for some months now.  The first post in the series asserted that we, as developers and architects of these systems have a tendency to make things too complex and flood the user with all manner of features, functions, and data layers they don’t need.  In essence, the age of GIS in a browser is ending (I sincerely hope) and we’re moving toward highly performant, intuitive, and focused applications in a browser that serve a particular purpose, and do it very well.  Why? Because someone else is bound to be offering the same stuff you are, and if they can provide the information faster, they win…bye, bye users and bye, bye clients.

Part 1 discussed hiding unnecessary complexity from the user, while our lesson plan this time around uses extensive feedback mechanisms within a site to keep the user on our side and make them feel comfortable when visiting our site.  A confident user is a comfortable user. A comfortable user gets what they need and gets back out fishing, golfing, spending time with their kids, or otherwise doing something else besides staring blankly at your interface.

With that, here’s lesson 2…

Lesson 2: Provide Feedback to the User

Nothing presents a bigger usability hurdle than a mapping application that leaves the user wondering, “Well this looks cool, but what do I do?” This is the principal downfall of applications that try to shove GIS into a browser.

Geodevelopers know exactly what to do with three different toolbars and four menus containing all manner of map navigation, query, buffer, and analysis tools but open workflow applications with no guidance are intimidating to line of business users and the public. The web mapping industry at large must learn to develop applications that satisfy specific workflows and to lead users through those workflows within the application with visual cues and feedback in the UI.

The roadway manager introduced in Lesson 1, uses the interface shown above to specify the begin and end points of a roadway project. First, note the simple informational panels that appear in the right hand column of the site to explain how to use the features on this page. Second, accept that a GIS professional knows that when they click a pencil icon, they expect to draw something on a map, but a roadway manager does not necessarily connect these two actions. Thus when the user clicks the pencil icon to set the begin or end point, information appears below the map telling our roadway manager what to do next. “Click the map to set the begin point.” It’s as simple as showing or hiding a <div /> element in the page, and it is a critical usability feature that is often overlooked.

Hold the User’s Hand

I’ve recently read a couple of typical user workflows for web mapping applications that read something like this:

First of all, in my book this is actually two workflows but that’s beside the point.  What really grinds my gears is that when you look at the application built to satisfy this workflow, the architect/developer elected to provide a separate tool to do each of these actions so that the user is free to wander around the site and do things in any order they please.  Not only does the user need to know which tools to pick from, they need to know the correct order.  While eventually the user will figure out the “happy path”, that’s a lot of moving around the app and a big learning curve that is just going to frustrate and turn the user off.  If we know the user is going to perform these actions to get information each time they visit, why don’t we just write a widget that does exactly that???

Now have a gander at the app shown above.  The workflow that this particular application functionality supports is as follows:

To support this workflow we could build a drawing tool with a data entry interface, an a generic intersect tool, a TOC, and a reporting tool, but why in the world would we do that? This is an emergency response application which means we don’t want our user’s hunting and pecking for tools when they could be busy making bad things stop…immediately.  The entire workflow is encapsulated in a single dojo dijit.

1. When the user activates the dijit by clicking on a menu item, they are prompted immediately to draw a shape and are informed on how to close the shape/stop drawing…a double click.
2. Once the user closes the shape, the data entry interface pops up and says “Hey, I need some data!”.
3. As each field is filled in, the form keeps squawking for more data until it has what it needs.
4. Since we already know the user is going to want to generate a report after we do our intersect and save all this data in the database, we ask them which one they want from the get go.
5. On ‘OK’ the system goes away, saves the polygon and attributes, does a bunch of processing steps and pops up the needed report.

We have held the user’s hand, led them through a well defined workflow, and given them the information they needed without all the noise and distraction of 2 menus, 3 buttons, a cup of coffee and two trips to the bathroom.

Reassure the User

When a GIS Professional executes an attribute query or runs an intersect operation, they typically understand that sometimes, the result set is empty. The public, or some other non-GIS user, gets very nervous when they perform an action and are either presented with an empty UI, or are otherwise redirected to something they didn’t expect. Did they delete something important? Is the request still processing? Did the site crash? “What the heck happened to my data?!?”

The importance of reassuring the user anytime something out of the ordinary happens cannot be over stated. Continuing with the roadway manager example, the figure above illustrates a case where the user has selected a project with no location information. Rather than showing the user an empty interface, leaving her to worry about what has happened, the map interface is zoomed to the general area of the project (known from tabular information) and an informational dialog explains that there is no need to worry and provides instructions on how to remedy the missing data issue. There is no need for a complicated exception or null case scenario; a simple modal dialog addresses the usability issue and keeps the user on the right track.

Up Next

Stay tuned for part 3 in the series dealing with strategies for protecting the user from themselves.

Myself and my partner in crime, Dave Bouwman, have talked incessantly in trade publications and at conferences of late about the Geospatial Web or “GeoWeb” and what it means to GIS professionals and software developers.  This post is the first in a several part series on usability issues and concerns when designing and building applications for the GeoWeb.  How many parts will the series have?  Well that all depends upon how worked up I get and how often and long I want to rant.

It is my sincere hope that we, as a community, are finally moving on from the era of exposing buckets of complex GIS functionalities in the browser.  As noted in a previous post, the GeoWeb is, in essence, all things Web 2.0 writ large on a map. For ESRI customers it is REST, JavaScript, Flex, and Silverlight APIs for ArcGIS Server.  Beyond the ESRI realm it is Microsoft Bing Maps, Google Maps, Google Earth, or myriad FOSS offerings.

Since internet users now have a myriad of choices in where they go for information, we as professionals should be designing highly usable systems that give users relevant information…and give it to them right now. If we don’t, they’ll simply go somewhere else. What this means for all of us as architects, and developers, and project managers in the mapping industry, is that usability trumps features. Since our customers are foresters, real estate agents, state troopers, and roadway managers, not GIS professionals, we need to hide GIS complexity, provide determinate, task oriented interfaces, and answer the user’s question with a minimum of friction and interaction.

While quick performance and a killer site skin with an open, uncluttered layout are certainly important in the age of the GeoWeb, they are only part of the equation. Based on my unscientific observations, geodevelopers are still challenged on the usability front in that the typical application with buckets of data and loads of tools with an unconstrained workflow is still making it out the door and into the market in many cases. Creating great apps for public facing or line of business sites serving non-GIS professionals requires a lesson plan that focuses on the user and on the mental model of how the user interacts with functionality that the geodeveloper exposes.

But I need a full featured GIS in my browser

Oh no you don’t.  We at DTSAgile maintain that there are very few situations that actually call for a web-based GIS. When these web applications are built, they are so complex that only GIS professionals understand how to use them. Perversely, the performance and technical limitations of web development make these applications too limited to be useful for GIS professionals! Give GIS professionals access to professional GIS tools – desktop GIS applications. Citrix or Terminal Services technologies provide an excellent means to do this in a distributed environment. This allows all the GIS applications and data to be co-located in a single data-center, while still having a “desktop” experience at remote locations. Having designed, architected and developed several large implementations, we’ve seen how it can deliver powerful desktop GIS functionality across an enterprise very cost effectively.

If however your goal is to service the public or users in a specific line of business, then you would do well in creating focused applications which help a user solve a specific problem easily – which is exactly what GeoWeb style applications do. The next generation of spatial applications is now arriving on the scene and starting to leverage real GIS analytical capabilities behind the scenes. While the tendency to mimic the indeterminate workflows of desktop GIS packages in a browser has noble goals, implementation, performance, and usability issues usually run rampant and completely swamp any cool-factor. Instead, focus on solving specific business problems by building tools that are tailored to the actual end user.

With that introduction…here’s lesson 1.

Lesson 1: Stash complexity away

Foresters know trees, state troopers know law enforcement, the County Auditor knows real estate assessment, the public knows a whole variety of things, but likely none of these user groups knows specifically what a buffer, intersect, union, or thiessen polygon is. When a roadway project manager asks for all structures near her project, without knowing it she really means,

“Locate point features in the Structures layer that fall within 1 mile of the section of Route 6A between mile posts 12 and 25.”

A GIS professional would know that getting this information requires an initial point selection, followed by a buffer, an intersection with a second layer (roads), followed by a buffer of the resulting road segment, followed by an intersection of the second buffer with the structures layer. The roadway project manager does not know this, nor should she.  Furthermore, why in name of Pete would we give her 5 or 6 separate tools and ask her to figure out that process in the correct order?

Consider the application shown in the figure above.  For those keeping track, the app is an ASP.NET MVC implementation consuming data via the WMS capabilities of ArcGIS Server 9.2 within the OpenLayers client interface.

Fewer Layers

Note how few layers are in the map.  I’m sorry to say so, but if you think your users need or want to see all 150 layers in your Geodatabase, you are dead wrong.  Unless your application operates on or depends on that really detailed point layer of GPS’d mailbox locations for correct function, leave it at home and let’s keep it simple okay?

Consider using a base tile cache with some sexy cartography, and then roll in 3-5 operational layers that contain only the information that the user needs to accomplish their goal.  The example above contains roadways, separated into scale appropriate layers for display purposes, and counties to provide locational context.  To our roadway manager, everything else is noise in the system and doesn’t contribute substantially to her ability to do her job so it is left out.  Using fewer layers also has the side effect of dramatically increasing map rendering performance…which is topic for a separate post in this series dealing with real and perceived performance.

Hide the Details

Check out the minimalist map navigation at the top left of the map and the conspicuous lack of multiple tool buttons, menus, legends, layer lists, etc. in our example above. This application is used by State DOT roadway project managers and all these folks need to do their job is to:

• Specify what road segment a project is on
• List structures along the road (culverts, mast arms, etc.) impacted by a project

Highly useable systems hide complex GIS operations from the user and get the desired answer quickly. The selection, buffers, and intersections that get the roadway manager the information she needs to do her job are hidden inconspicuously behind the “Search for Structures” button highlighted below.

Once the project road segment is selected, this search button becomes enabled, and a single click returns a list of affected structures to the user in approximately 0.3 seconds in this application allowing her to get back to what’s really important…spending that stimulus check.

Future Zen

Keep checking back here for additional posts on usability, performance, etc. for building the next generation of web mapping applications.  Future posts will deal with feedback to the user, user reassurance, protecting users from negative outcomes in your app, real and perceived performance, and anything else I can think of that irks me when I look at some of the “Web GIS” sites I see rolling out onto the internet.

Author’s note: This post was prepared in advance of an architecture panel I will be participating in at the GeoWeb conference July 27-31, 2009 in Vancouver.  At the request of conference organizers, myself and the other panel participants have all prepared posts on our “pet” architectural style.  This year I drew the REST straw while Ian Painter of Snowflake Software will vigorously defend SOAP/RPC and Hans Shoebach of Galdos Systems will do battle for P2P/Event Driven architecture.  This post will be cross posted over on the GeoWeb 2009 blog as well. 

The Geospatial Web, the current darling of location based technologies and neogeography, has been variously described as:

• web mapping – the generation and publication of highly performant web applications that include a map
• mashups – combining spatial data with abstract information to produce novel representations
• a distributed GIS for the web – essentially the combination of elements of traditional GIS technologies with advanced web development tools
• equivalent or synonymous with relatively new technologies such as Google Earth, Microsoft Bing Maps, Yahoo Maps, etc.
• communal or user-generated geospatial content

While each of these descriptions is both incomplete and an oversimplification, I assert that at its core the GeoWeb is a technical foundation of information services and collaborative tools built upon or merged with data that provide a spatial context…a location.  It is indeed shiny “slippy” maps, Web 2.0 application layouts, nano-formats, mashups, LBS, service-based spatial information sharing, but it is also the hardware, software, information architecture, advanced web application techniques, and people who come together to make such an organic endeavor possible.

The number of organizations working with GeoWeb technologies is already large and growing daily.  It seems only logical then, that as a community we engage in an open debate about how exactly all of these databases, services, clients, and functionalities are ultimately going to communicate, integrate, and grow. My endeavor, in the context of this post and in preparation for an architecture panel at the GeoWeb 2009 conference, is to espouse Representational State Transfer (REST) as a standard architectural pattern for the GeoWeb. I’ll try to keep my points brief here with hope of stimulating discussion that will continue at the GeoWeb conference in Vancouver at the end of July.

REST is Best #1: Adopt the architecture of the medium

Continuing my trend of oversimplication, the GeoWeb is simply the web + geo right?  While it is indeed more than just web mapping, at it’s core the GeoWeb commonly leverages the World Wide Web for information access and representation in a client application.  The web is the poster child for RESTful architectures as articulated in Roy Fielding’s doctoral dissertation.  So the pragmatist in me says that if we’re going to build a network of loosely coupled systems to support the Geospatial Web and we’re going to use the web to provide access, why wouldn’t we leverage the architecture of our good friend “dubya, dubya, dubya”?  GET, PUT, POST, and DELETE.  The semantics are well defined and well understood (okay, POST can be a little wonky due to issues of idempotency but I think my point holds).

Every single application exposes its resources through exactly the same interface…the HTTP specification.  Would you, as a developer of a service or a client rather work with 4 HTTP verbs, or would you rather need to discover, proxy, and understand myriad SOAP contracts?  Written a WSDL lately? Read one? I thought so…

REST is Best #2:  It’s about resources

REST-based services (or REST resources for my “high-REST” friends in the audience) are true “web services” in every sense of the term since they’re based entirely on HTTP and URIs.  Map images of downtown Las Vegas, your running route, drill pad locations, the location of the nearest Starbucks, parcel boundaries, a street level image of some guy mooning the Google van; all just pieces of information when it comes right down to it.  What we store, exchange, represent, draw on a map, and recombine into novel information combinations are simply resources…and that’s what REST is all about.  Why make it any more complex than it has to be.  Why would I POST  to a SOAP method or consume an event from a service agent when all I want to know is http://www.myTranspoSite.com/projects/532. (In English: “Get me the location, status, and funding information of the stimulus transportation project going on at the end of my street.”) 

“But what about my complex geoprocessing operation,” you say?  Well, spatial statistics or complex operations that produce a result (buffer, linear referencing operations, etc.) are all fine and dandy for us technologists, but the GeoWeb isn’t just for GIS professionals anymore.  Why expose your client developer, or worse a run of the mill user, to such complexities.  A user doesn’t know what a buffer is and neither does the Dojo ninja that’s consuming your resourcein his JavaScript client…they just want to know how many crimes where committed in their neighborhood in the last 6 months.  Hide complexity within the REST resource and give a simple answer to a complex question.

In an ROA effectively adhering to a RESTful architecture, all I need to know to get the information I want is a URI and an action.  Do I have to negotiate a proxy?  Am I getting cached information? Is there a gateway or firewall in the way?  How do I negotiate that P2P thingy again? Do I need to do a buffer followed by an intersect followed by a 12th level of hell whatchamacallit? Who cares?  Just give me the information I want so I can go play golf, spend time with my kids, wax the car, take a nap, or do any number of other things I’d rather be doing.

REST is Best #3: Serving many masters

The beauty of REST is that you get the standard interface provided by HTTP, with no limitation on data representation.  Why is this important for the Geospatial Web?  Because browsers like HTML and JSON. Because machines like XML, binary, or code on demand.  Humans like pretty pictures and PDFs. And sometime down the line machines and browsers and humans may all change their minds and like some other format.  That’s right folks, the GeoWeb ain’t just for browsers anymore and that means we as architects and developers need to keep multiple resource representations foremost in our minds.  REST is absolutely critical as an architectural pattern for the GeoWeb because it provides a low friction way for us to provide information formats that better match consuming clients and performance requirements.  And as new content types come to light and get defined, REST lets you add support for these novel formats without impacting support for existing/older content types. 

REST is Best #4: Now where did I put that..?

So if the GeoWeb is about publishing, consuming, transforming, sharing, visualizing data with a spatial component, it stands to reason that we need a mechanism by which to discover said information.  And we’d probably also like a way to get back to the information once we’ve been there once.  Or send it to our boss, or forward it to grandma, or shoot it out the list-serv that we’re on.

In short, REST makes the information driving the GeoWeb portable. Once I find a resource I like I can bookmark it to return later, I can throw it in an email and share it with others, I can embed it in my application to be consumed and acted upon in novel ways.  It also means that we don’t need a separate discovery mechanism for our resources because resources can be embedded as content in other resources.   A country resource can provide information on state resources which can provide information on county resources, etc.  Imagine that; an entire network of information and functional resources interconnected by URIs…huh.

REST is Best #5: Performance and Scalability

For most RESTful services (I know, I know…resources) the majority of requests to the resource are probably GET requests.  Do I have numbers to back that up?  Of course not…I’m an architect.  But the logic of the assertion is that presumably many more people are allowed to read/view information than are allowed to create, update, or delete information.  Even in scenarios with lots of resource editors, the GET is still likely to far exceed PUT, POST, and DELETE request counts.  One of the central tenets of the REST architectural style is that whenever possible, information should be cached.  REST gets it’s performance and scalability from this cache-ability, coupled with statelessness.

What this means for the GeoWeb is that we can cache GET requests against a resource.  Thus if I visit a resource URL and ask for information on a highway project, and 2 seconds later my neighbor asks for information on the same project…my GET results will be cached so that my neighbor’s request does not incur the penalty of a database hit, business logic enforcement, object formatting, etc.  Does it matter for a system with 100 users on appropriately sized servers? Probably not.  Does it matter for a large interconnected network of resources with hundreds of thousands or millions of users?  You betcha’.

Statelessness means that a server hosting data or functionalities for the GeoWeb should service requests in a manner that does not depend on context or state between requests or between services.  If you say to a REST resource, “Give me a map of all preschools within 5 miles of my home”.  You cannot simply make a follow up requests that says “Now buffer them”.  The statelessness of the resource on the server means it no longer remembers anything about the 20 preschool points it just handed you as JSON…in fact, the server doesn’t even know who you are anymore and wants your credentials again.  Why is this good?  Because RESTful architectures don’t typically have to contend with maintenance of session state, a server can be swapped out with another in the event of a failure, a sys admin can decide to add three new servers to a farm on Wednesday without existing clients knowing any different, or a group of servers can be allocated to service different types of requests.

Caching and statelessness = performance and scalability all in one neat little package.  Sort of a big deal for people building LOB or otherwise critical applications for the GeoWeb dontchathink?

Parting Shots

Did I miss something in my initial hypothesis that REST is the critical architectural style for the GeoWeb?  Probably.  Did I get something wrong or use semantics likely to raise the hackles of true, dyed-in-the-wool RESTafarians?  Probably…I tend to be more pragmatism than dogma anyway.  Consider joining us for GeoWeb 2009 July 27-31 and see if I can avoid making a complete fool out myself whilst fighting the good fight to make RESTful architectures an integrated part of the evolving GeoWeb. Find a glaring hole in my argument or something I missed?  Leave a comment here so I can study up before doing battle with the SOA/SOAP and P2P/Event Driven folks in July.

Over the last two weeks, I’ve been waist deep in what has been, for me, a completely new client side development platform.  We have several projects underway at present that leverage Dojo as the framework for rich client GeoWeb applications and, simply put, we’ve got more Dojo work than our two resident experts can handle at present.  Time to roll up my sleeves and learn something new!

Since I’m a botanist by training, and not a classically trained programmer by any stretch, I don’t typically look at a new technology and say “Gee Whiz, this looks an awful lot like <pick your poison> that I learned back in University”.  I’ve got sort of a free form learning process that works for me, and usually get’s me up and running and useful in a relatively short period of time…a matter of days if I have examples and a pattern rather than weeks.  And after doing this for 10 years or so, it seems that you still can teach an old dog new tricks.

The cone of silence

The learning process, especially at age 35, does not benefit from distractions.  I still can’t believe I could sit on my couch and watch a movie while studying in college or camp out in a noisy coffee shop and write portions of my thesis in graduate school.  Now, it’s less about the TV or background noise and more about email barrages, social networks, and the new, new thing on YouTube.  Perhaps the single worst feature of my Twitter client (TweetDeck) is that it is constantly squawking when new updates roll in.  When I need to learn something new from the ground up, the cone of silence get’s invoked. 

This doesn’t mean I don’t talk to my coworkers or answer my wife’s calls…quite the contrary, I’m usually asking coworkers plenty of questions and since my son has a health condition I always take calls from the wife.  What it does mean is that I shut down my Twitter client, fire off a courtesy email to the rest of the company (and stop answering email), and buckle down for a day, or two days, or three if that’s what it takes.  If a coworker is physically on fire and I have the only fire extinguisher, I respond.  If not, then it’s going to be a day or two before I can get to that proposal you want me to work on.  There are no ad hoc requests allowed from outside the office I physically sit in.  Removing all extraneous distraction keeps me focused on the task at hand and allows me the deep dive that I require to wrap my head around something new.  This cone of silence is also an excellent tool when up against a deadline on any project.

Find room to spread out

I can’t learn a new technology at home after work.  I can read a book about it and make some notes and commit some of it to memory, but I can’t kick back, spread out, and mentally digest things.  Wife, two dogs, cat, four year old…you get the picture.  There’s a lot going on at home and working there usually means a single monitor machine or the netbook, and not a lot of room to make a mess.  Learning something new is when our developer rigs at DTSAgile really shine:  Comfy chair for exercising the typical bad posture of a developer;  Big desks with plenty of room for scribbling things down in a notebook or two; Three monitors to spread out your IDE, Google, help system or API documentation, sample databases, etc., etc.; Additional desk space reserved from the latest offerings from O’Reilly or Wrox; Real estate for half drunk coffee from yesterday as well as today’s lukewarm addition to the collection.

I need room to roll up my sleeves and work.  Having multiple information sources in front of my face at the same time speeds the learning process for me and is simply more time efficient.  I’ve gotten to the point now where it’s actually an annoyance to have to hunt and peck on the Windows taskbar  to find that great Google search result or pull up one of 5 sample VS.NET solutions I’ve got going.  Call it ADD, call it information overload, it works for me…but it only works if I’ve got room to spread out, both in the digital world, and the physical one. If I’m truly going to leverage the wealth of information that’s available, I certainly need a big palette to work from, and an even bigger canvas to spread it on.

Know where you’re going

It helps immensely to have an example of how things ought to look when I’m done writing code.  This doesn’t necessarily mean the user experience manifested in the UI or in the behavior of the service boundary, but rather the appearance and organization of the code. Knowing enough to write a dojo.subscribe() is one thing, but where is the best location to put that code given the context in which it will run? In a *.js file for a widget? In the page itself? In some generic widget controller?  Dojo can be used to simply add a few shiny bits to a web client, but it is frequently used as a fully object oriented client application framework, making good patterns and practices very important. 

Having reference implementation code, either from a previous project my team has done, or from another source is usually critical.  If the technology I’m learning is fresh in the market, then I’m happy to give things a little more thought and blaze new trails, but if someone has gone down the path before me, then I definitely want a fully wired example to work from.  Fortunately in this case, my coworkers had been doing Dojo development for awhile so I had a couple of good solutions to work from.  Cheating? Lazy? Maybe, but why in the world would I not try to benefit from the previous experiences of others.  It’s just good sense to know where you’re going…that way at least you’ll know when you’ve arrived.

Breaking stuff

I’m a tinkerer, and have been since childhood.  What good is a perfectly working model, RC car, or a piece of software if you haven’t taken it apart, broken it, and put it back together again?  One of my best resources during the learning process on a new technology is my tendency to tinker, pull things apart, and see what makes them work.  Sure you could read about about the difference between dojo.hitch() and dojo.connect(), but it is infinitely more revealing to test drive both methods to see what works and what doesn’t.

My shop is in the consulting business, which means we need to be experts in lots of different technologies and we need to achieve that level of expertise quickly as technology evolves. The long and short of it, is that I learn best when I’ve gotten to the point of a working piece of software…and then go one step further.  I have found over the years that the difference between kinda’ getting something, and really grok-ing what’s going on, lies in my tendency to want to break stuff and then put it back together so that I know what all of the component pieces and parts are doing.  In the long run, this generally increases my code quality and readability because not only have I taken the time to learn what to do with a given technology, but I’ve also learned what not to do.

References and tutorials

I don’t think it’s too much of a stretch to say that for most developers these days, Google is the most valuable reference book on their desktop.  However, those of you who have already jumped into the deep end of the Dojo swimming pool probably realize that:

1. The API is not particularly well documented with comprehensive explanations and examples when compared to other APIs (JQuery, Prototype, etc.)
2. There are two chief resources online for Dojo:  The Dojo Campus (www.dojocampus.org) and the Dojo Toolkit Site (www.dojotoolkit.org), neither of which sits on a particularly speedy server or offers a comprehensive how-to library.

All Dojo peculiarities aside, even if I was learning ASP.NET MVC, Entity Framework, PERL, Python, Ruby or any number of other well documented technologies or APIs, I certainly wouldn’t rely entirely on what online resources could teach me. 

Perhaps it’s because I was raised at a time when you still went to the local bookstore to find a good read, or perhaps it is because of the countless hours I spent working in my grandmother’s small town bookstore, I still like to have reference books around.  Having two or three books on the technologies I’m actively working in means I can take a book to lunch and read up on a particularly hairy topic, I can look long and hard and dissect an example without blinding myself staring at the LCD, or I can take a volume home and read before nodding off at night.  Absorbing information from multiple sources can only help the cause of learning new technology, and for me the thumbing of pages will always hold something that a scrolling a mouse wheel never will.

Conclusions

So there it is…you can in fact teach an old dog new tricks.  While my learning process is probably not as organized and logical as that of others, it does work for me, and has allowed me to keep pace with what is turning out to be a rapidly changing technological landscape.  What’s next?  Well, the cone of silence is off so there’s probably a couple of proposals in my near future, but the Dojo work is nearing completion, it works well, and I can see Silverlight 3, Entity Framework, ADO.NET Data Services, and probably some Flex in my near future.  I think I need a cup of coffee and a fourth monitor.

Come on out and join us to drink the agile kool-aid, take your agile medicine, learn what agile can do for your team or organization, or have a skills refresher for experienced agile practitioners.

Dave Bouwman and myself will be collaborating to give a one day Agile Project Management training seminar on Friday, June 26, 2009 near Denver, CO.  This course will cover both project management practices and development/engineering practices.  We’ll begin with an introduction to agile practices and rapidly progress to specific methodology examples (Scrum), cover roles and responsibilities, project controls, and how to scale the agile process in your organization.  In addition, we’ll introduce specific software development processes that mesh well with the agile process including automation for code documentation and unit testing, design patterns, refactoring tools, and automated builds and continuous integration.  Throughout the course we’ll give you specific examples, the good, the bad, and the ugly, from our own experiences using the methodology in our shop.

Traditionally, this is a course for which DTSAgile typically charges $1200 per student.  However, through the fine folks at Rocky Mountain URISA, you can join us for this course at a very steep discount.  Cost for URISA members is $135 and the cost for non-members is still only $150.  What does that mean to you?  Well, if you live in the continental US (and some places in Canada) you can attend one of our Agile training seminars including plane fare and hotel for less than we typically charge for the course.  And that, ladies and gentlemen, is what we call a very good deal.  If you’re in Colorado, particularly the Greater Denver region, it’s a no brainer. But it’s still a steal even if you need to travel in from points afar.

Course Details:

Course Material: Introduction to Agile: Project Management and Development

Date: Friday, June 26, 2009

Location: USGS Mtg Room, Fed Center, Lakewood CO

More Information: www.rm-urisa.org/news.html

Please use the “more information” link above to read the full course abstract and get additional details including how to register, maps of how to get there, etc.  Please consider joining us in June!

While the Dojo JavaScript libraries are really an excellent resource for building rich internet applications (RIAs), sometimes Dojo just makes things harder than they have to be.  Last week I was working on public map viewer, based on our custom implementation of ESRI’s JavaScript API starter kit,  for one of our clients.  The site is really pretty basic in terms of functionality with one of the tools allowing the user to configure and execute a multi-criteria search for information and have that information displayed on a map.  The user interface looks something like the following screen cap:

In this user interface example, the combo boxes for counties and cities are Dojo FilteringSelect controls.  The FilteringSelect extends the typical html drop down so that you get an intellisense/autocomplete of sorts when the user starts typing in the combo box.  In addition, in the event that the user types a value rather than using the drop down list, the FilteringSelect will validate manual text entries to ensure they are in the select list.  All of this prevalidation is important because it allows us to reduce the potential for errors in searches by constraining the user’s input to a list of valid values pulled from database lookup tables.  It’s a pretty neat little control…until you have to clear it.

The issue:

There are basically two possible scenarios where we need to clear information from a Dojo FilteringSelect on our search form in this example.  The first is obviously when the user clicks the Clear button to clear the search form and start over.  The second case is when we need to reload and reset the data driving the city FilteringSelect box if the user makes or changes a county selection in the county FilteringSelect box. 

The user click of the Clear button to clear all search form values is sufficient to illustrate the issue so I’ll expand on that a bit.  The wiring for events like the typical “onClick” for a button in Dojo is extremely simple:

     this.connects.push(dojo.connect(this.widgets.clearLocateButton, “onClick”,

     this, “onClearLocateButtonClick”));

Breifly, the line of code shown above uses dojo.connect() to assign a function called onClearLocateButtonClick to be the event handler for the click event of a button on our widget with an ID of clearLocateButton.  The onClearLocateButtonClick function is equally straightforward, simply clearing any user entered values from the input boxes on the widget.

    //——————————————–
    // Handle event raised when user clicks the Clear
    // button on the Address UI. Resets user input
    // controls.
    //——————————————–
    onClearLocateButtonClick: function(evt) {
        try {
            // reset user inputs
            this.widgets.countyComboBox.reset();
            this.widgets.propertyTextBox.reset();
            this.widgets.addressTextBox.reset();
            this.widgets.cityComboBox.reset();
            this.loadCityLookup();

        }
        catch (err) {
            console.error(“Error clearing address information”, err);
        }
    }

Each input control in the widget receives a reset call which is supposed to clear any entered value for the control.  The problem?  Let’s say a user has selected County = Larimer and City= Fort Collins in the above UI.  Clicking the clear button resets all widget controls to empty as expected however the county and city FilteringSelect controls will end up having validation errors displayed. 

But there is no validation error because the user consciously asked for all the information to be cleared.  None of the fields is required so the validation error shouldn’t be there. 

The solution:

After a variety of configuration changes, at least three Google searches and some hair pulling, I did arrive at the following solution to get rid of the validation errors when FilteringSelects are cleared using the reset method.  The solution is basically to change the way we reset the values in a filtering select box within our onClearLocateButtonClick event handler.

    //——————————————–
    // Handle event raised when user clicks the Clear
    // button on the Address UI. Resets user input
    // controls.
    //——————————————–
    onClearLocateButtonClick: function(evt) {
        try {
            // reset user inputs
            this.widgets.propertyTextBox.reset();
            this.widgets.addressTextBox.reset();
           //need to manually reset the city and county boxes for some reason
            this.widgets.countyComboBox._hasBeenBlurred=false;
            this.widgets.countyComboBox.attr(‘displayedValue’, ”);           
            this.widgets.cityComboBox._hasBeenBlurred = false;
            this.widgets.cityComboBox.setDisplayedValue(”);
            this.loadCityLookup();

        }
        catch (err) {
            console.error(“Error clearing address information”, err);
        }
    }

I actually dug in to the API documentation and found that the setting the _hasBeenBlurred property to false and then manually setting the value of the control solves the issue.  Presto!  The offending validation error is gone from our widget when the user clears the search parameters.  Note the example code uses both the .attr() method and the .setDisplayedValue method simply as an example of two ways to accomplish the task.  I would note however that setDisplayedValue has been deprecated and will be removed from the Dojo API at version 2.

Now, I’m relatively new to the world of Dojo so this may or may not be the preferred method of tackling this issue. The use of _hasBeenBlurred is a bit of a code smell from my perspective because the ‘_’ indicates a method that is supposed to be private.  However, sometimes we wind up trapped between doing things the absolutely right way and absolutely getting something done.  In this case I chose the latter.

That’s right, DTSAgile is again looking to expand our team building next generation RIAs and GeoWeb applications in our Fort Collins, CO office.

As we have a backlog of projects that use diverse technologies, we’re looking for an excellent developer with one of the following skill sets.

SL, EF, ADO.NET Data Services

If you have bona fide project level experience with Silverlight 2, Entity Framework, and ADO.NET Data Services or NHibernate, DTSAgile wants to talk to you.  We are not looking for an entry level developer or someone who can learn on the job, we are seeking developers who have real world experience implementing this technology stack.  We’re looking for someone who can make an immediate impact, show us how this stack should be implemented, and tell us how to avoid the pitfalls.

JavaScript, Dojo, CSS, etc.

Alternatively, if you have mastered the zen of dojo, JavaScript, CSS and all things client-side we’d also like to receive your resume and review your credentials.  Actual project experience implementing mapping technologies using ESRI’s JavaScript API including the starter kit will be reviewed favorably. Again, we’re not looking for entry level folks here…you’ll be asked to come in and put fingers to the keyboard relatively quickly out of the gate and show us what you’ve got.

Other fun stuff you should grok

• Agility. No, you don’t have to do backflips or crazy stretches in an interview setting. As our name implies, we’re a group of consultants pretty committed to Agile processes on our projects.  At minimum you should have some background knowledge regarding Agile and an interest in working on an Agile team
• Favorite patterns. ASP.NET MVC is high on our list of happy shiny things that we like to work with.  Many of our projects are and will continue to be architected with ASP.NET MVC in the mix.  Real world project experience with it will be be reviewed favorably.
• Unit testing.  We’re big on tests…lot’s of ‘em.  Some of our projects are run in TDD fashion while others have a test-late lifecycle.  Regardless, we unit test everything that goes out the door thoroughly.  NUnit, MBUnit, TestDriven.net, mocking frameworks such as Rhino mocks, and IoC/DI principles should all be on your resume.
• The Geo.  We primarily target mapping related projects so exposure to GIS, the GeoWeb, etc. in the form of the ESRI product stack, VE, Google Maps, Open Layers, Geoserver, Mapserver, PostGIS, etc. will be viewed favorably but is not required.
• Information exchange and social networking.  We’re software architects and developers, and we’re also bloggers, presenters, and participants in growing social networks.  Run a blog? Participate in Twitter? Grasp the finer points of presentation zen and really good Powerpoint? Great, you’ll fit in just fine.

Still interested?  Drop a line to myself or Dave and we’ll have a gander at your qualifications and see if you might be a fit for our current needs.

I love spending time with my son.  He frequently reminds me that sometimes, life perceived through the eyes of a child is a pretty acceptable way to go.  I was a bit naive going into father hood and thought that I’d be the one doing all of the teaching. Yet every day I spend with my son teaches me something new about how to look at work or personal life in a different way.

Dave and I are in the process of collaborating to produce a new version of our Agile Training Course and I’ve had the chance to sit and reflect a bit on the Agile Process.  When you’re deep into project execution, it’s easy to loose site of the forest for the trees and it’s good to take a step back now again (distinct and separate from your post-mortems) and look at the big picture.  Of late, I’ve been mentally musing about how to look at the process through the eyes of a child, and what that view can teach us as Agile practitioners and trainers.

Story time is where it’s at

My son loves story time…alot.  He likes it so much that he usually forgets that the next logical step after story time is bedtime (which he doesn’t like so much).  Grabbing a few books or the latest Highlights High Five issue and curling up on the couch with mom and dad makes his day.  And for us it’s fun to watch him learn in the process; letters, numbers, words, and getting the moral of the story.  He asks questions, he recognizes inconsistencies when we try to shortcut or gloss over story elements, etc.

For Agile practitioners, story time is definitely where it’s at.  User stories and their prioritization/refinement over time are your backlog, your requirements, your road map for what you’re going to build.  YOU SHOULD LOVE STORY TIME.  The initial generation of backlog, while important, takes a back seat IMHO to the sprint planning session.  This critical planning period at the beginning of each iteration is your “story time”. It is your shot at learning all the letters, numbers, and semantics of what is important to your project owner, and by extension your users.  And it’s important because the outcomes of story time are going to govern your work days for the next iteration.  So, grab your backlog, ask questions, point out inconsistencies, and really focus on learning all the details that will make your team successful over the next iteration.  And don’t worry, bedtime is a full two weeks away.

The best made plans…

Most, if not all, parents already know this, but it bears repeating for expecting or adoptive parents: Get used to being late or having plans foiled at the last minute.  Illness, tantrums, an innate desire to not wear pants today…I’ve seen it all.  I’ve even left a half-full cart of groceries in the Whole Foods aisle (sorry stock persons!) and walked out to try again later.

You’re screaming right now and I understand you’re upset.  But you can scream in the car on the way home, not in the grocery store.

We can’t always control what our children do, but we can certainly control where they do it and clearly illustrate the consequences of their actions.

The Agile process embraces change during the development cycle of a project.  This is a standard Agile 101 statement.  It embraces change by allowing change to happen (adding/removing stories from backlog, changing priorities, etc.) at predefined points in the cycle.  Still, some Agile teams get a little bit antsy when a client or project owner starts talking about wholesale changes to their process.  This nervous response can happen for a couple of reasons. Is the Scrum Master doing his/her job? Why is the team hearing from the project owner mid-sprint? Another option is simply the ingrained aversion for changing horses mid-stream.  I see this on teams who have switched from a sequential delivery model in their recent past.  Message: change happens. Get used to it. You can’t control what your clients want…but Agile gives you a toolbox to control when change happens, and provides metrics for you to illustrate for your clients the consequences of their decisions.  Use these tools and enforce the process!

Open your ears and listen

Betcha’ thought this was the section where I talk about getting my son to listen?  On the contrary…My son and I were spending some time together the other day and under his watchful eye I was helping him build an enormous house of blocks.  I take direction pretty well, but I was apparently frustrating the heck out him by building the ramparts on the roof into a really neat and symmetrical pattern that I thought anyone might like.  Finally, I got the hook…via blood curdling scream to stop what I was doing.  I was present with him and engaging in some good old fashioned child’s play, but I wasn’t listening to what he wanted.  Once I mashed the blocks up on the roof into the pattern he wanted, all was well in the universe.

How often do we, as architects or developers, build a really neat function or feature that we like, but turns out to be the antithesis of what the client wanted? Far too often.  We love tools and widgets and gadgets and technical challenges but the client doesn’t care.  I worked on a project at a previous company where I spent two hours elaborating a user story into a several child user stories with tons of tasks that was going to turn into a really cool technical solution that was going to take me a week to build and test.  The client looked at it and said,

Ummm, all I want you to do is add my ABC layer to the map.

Curses!  Esoteric technical solution foiled again by selfish client. Wrong!  Once I stopped to listen to what the client wanted, I was able to deliver the correct solution in as long as it took to fire up ArcMap, add a layer, and bounce the ArcGIS Server service.  Stop and take a listen every once in awhile and you may be amazed at what you learn.

It’s OK to say “No”

One of the things I really struggle with as a parent is the actual act of being a “Parent” when I need to be.  I want my son to think I’m fun and cool and to always say he loves me and it took me a lot of frustrated evenings after bed time to learn what so many parents already have: I can be his friend later, I need to be his parent now.  This means not starting a wrestling match when mom is trying to get him into the bath. It means not flying off the handle when he says I’m mean or he doesn’t like me.  It means saying “No” sometimes even when I know that “Yes” will bring a smile or a giggle.

I like numbers.  I was a math geek in school even though that wasn’t my chosen major so I love the numbers that our Agile instrumentation generates for me throughout a project.  Why?  Because they help me say “No” when I need to.  Even better, it allows me to say it and illustrate why diplomatically and matter-of-fact-ly without the typical posturing and table slapping between client and consultant.  If my team’s velocity and resource allocation is determined by 6 or 7 actual productive working hours during a day, and there are 10 days in an iteration, then I can show the client a fixed number of hours and the allocation of those hours to what my team feels is achievable.

No, based on current story prioritization and available resources you cannot have that super-duper-new-fangled-thing-you-just- thought-of.  However we can enter it as a high priority item for the next iteration and reprioritize other stories accordingly.  If you wish to increase velocity I can add an additional developer to the team next iteration and we can discuss the financial impact of that change.

In addition to being the servant leader, it’s OK for the Scrum Master to be a good parent and say “No” once in awhile. If the project owner throws a tantrum, please see “The best laid plans…” above.

Relax, Regression is Natural

Having spent a couple of years watching my son grow and change, my wife and I have recognized that he goes through cycles of development where there are short bursts of physical growth or learning in a particular area.  If he’s growing like a weed, that energy isn’t going into learning new words or actually hitting the toilet at potty time.  When he’s sponging up new words, writing skills, etc., then he’s not physically growing as fast.  And occasionally, word enunciation or behavior will regress because his body or his brain are busy doing other important kid things.

Regression in this case is not regression of the code base or feature set of an application.  What I mean by regression for the purposes of this post, is the natural ebb and flow of successes and productivity that occurs an any Agile team.  Things can go pretty rough on a new Agile team…poor acceptance or backlog achievement stats, uncertainty at certain points in the process, etc.  All these growing pains typically work themselves out.  As an Agile team becomes more mature, planning and velocity get better and things really start to hum.  Stories get spontaneously added to backlog by your team, individuals swoop in take backlog from others if they finish early, team members pull more backlog into an iteration mid-stream, etc.  But even on teams that have been doing this stuff for awhile, there is regression.  A completely new or unforeseen technical issue rears its head, a team member is out with an extended illness. Death in the family, job stress…Hey, maybe you’re just having a bad week.  The key here is to never blame the process for your team’s inability to succeed.  Energy is being put elsewhere to ensure a healthy, mature team over the longer term. Regression happens, relax.

The rules apply to everybody

We eat dinner as a family at the kitchen table every night.  TV off, no distractions, just the three of us and the two begging canines circling the mobile food delivery system that is my 3 year old.  We’ve been working on table manners lately: use your utensils, chew with your mouth closed, use a napkin, the standard stuff.  So the other night I’m correcting my son for some infraction that I no longer recall and while I’m talking to him I slouch to one side and bring my elbows up on the table.  I am promptly interrupted by, “Papa you sit up straight and keep your elbows off the table!”  Momentarily defeated, I am reminded that there can’t be two sets of rules in the house.

Ever work on team where the rules don’t apply to everybody?  Maybe it’s a team member that is chronically late for, or jesu forfend, skips stand ups entirely.  Maybe it’s the scrum master who decides you’re going to skip the post mortem this time. Maybe it’s allowing a project owner to interject major issues in the middle of a sprint.  First of all,  the ground rules should have all been dealt with during the project fit check, just before inception. Second, the same rules apply to every body.  I don’t care if you’ve got the most seniority, I don’t care if you’ve been “12th level black belt scrum master certified”, and I certainly don’t care if you think you’re the smartest or most productive one on the team.  We all succeed and fail as a unit and therefore, we all follow the same rules.  In short, it doesn’t matter what the specific nuts and bolts of your particular Agile process are; it matters most that the entire team realizes they all play by the same rules, none of which are optional.