What’s yer status? Managing ArcSDE connections at run-time…

Awhile back, my team and I were working on a real-time dispatch system for public safety applications based on ArcEngine.  A critical part of such a system, especially when maps are involved, is maintaining the integrity and “up time” of your data connections.  Ever notice what happens in an ArcEngine app (or ArcMap for that matter) when there is a loss of connectivity for ArcSDE layers?  Map refreshes after a connection loss will not automatically reconnect to the ArcSDE server and neither will doing a Right-Click –>Set Data Source from the TOC…without a little customization, that connection is gone until you restart the app…not a pleasant eventuality in a public safety system.  So you have to write some code to gracefully ask the system to monitor remote database connections and reconnect dropped connections at the first opportunity. 

Kudos to Kevin Larson for coming up with this nifty solution…enter IWorkspaceFactoryStatus and a few of its friends.

Step 1: Persist information on all layers drawn from ArcSDE

Essentially, you need to create a local cache (I suggest perhaps one of the fine generics available to you in the .NET framework, IList<>, IDictionary<>, etc.) of layer information and connection properties for all layers in your map that are accessed via an ArcSDE connection. We’re going to multi-thread the connection monitoring code…so you’ll need your own custom objects to hold on to non-COM interfaces and objects for layer and connection properties. The code below is not comprehensive but will give you the general idea.

mapDataset = mapLayer as IDataset;
esriWorkspaceType? currentType = null;
if (currentType == esriWorkspaceType.esriRemoteDatabaseWorkspace)
{
// This is a remote workspace.
// Create layer and connection properties objects for the layer and add them to the cache.
// You can likely hang onto the map layer as an ILayer because it won't be used directly by the separate thread to reattach later
// You can get connection properties from IWorkspace::ConnectionProperties and feed to a custom .NET object for persistance
}

Step 2: Subscribe to the IMapControlEvents2::OnBeforeScreenDraw event
Subscribing to this event allows you have a delegate method responsible for checking connection status before the map is drawn.  In our case, we were using the AxMapControl so we grabbed this event off that control.  You’ll find it referenced as one of the IMapControl objects in the online reference or dev help.

Step 3: Implement connection checks for ArcSDE Layers in the event handler code
This is the workhorse of this approach and is rather involved. First thing’s first…check each layer’s connection status and build an ICollection, IList, IDictionary etc. of down connection that need addressing. This is where the IWorkspaceFactoryStatus interface and his buddies come in. Again, I’ve abbreviated the code but here’s the general idea…

IWorkspaceFactoryStatus workspaceFactoryStatus = new SdeWorkspaceFactoryClass();
IWorkspaceStatus workspaceStatus = null;
IEnumWorkspaceStatus enumWorkspaceStatus = workspaceFactoryStatus.WorkspaceStatus;
enumWorkspaceStatus.Reset();
// Checking all connections.
for (workspaceStatus = enumWorkspaceStatus.Next(); workspaceStatus != null; workspaceStatus = enumWorkspaceStatus.Next())
{
// If a connection is down or can be reconnected, we need to process further.
if ((workspaceStatus.ConnectionStatus == esriWorkspaceConnectionStatus.esriWCSDown) ||
(workspaceStatus.ConnectionStatus == esriWorkspaceConnectionStatus.esriWCSAvailable))
{
switch (_sdeConnections[keyProperty].ConnectionStatus)
{
case AttachedSdeData.SdeStatus.Alive:
// The connection has just become unavailable.
// We need to setup a monitoring thread.
break;
case AttachedSdeData.SdeStatus.Unavailable:
// The connection is still down.
break;
case AttachedSdeData.SdeStatus.Reconnect:
// The connection is available, we need to reconnect layers.
break;
}
}
}

The second step…setting up the monitoring process for down connections is relatively simple.

Thread downConnectionThread = new Thread(new ThreadStart(MyCustomConnectionCheckMethod));
downConnectionThread.SetApartmentState(ApartmentState.STA);
downConnectionThread.Start();
//Add the thread to an internal list so we can monitor status and do cleanup as needed
_monitorThreads.Add(downConnectionThread);

For each connection that was found to be down, you create a new thread, and as it’s ThreadStart method you pass it a custom coded method you’ve written to check if ArcSDE is available. Essentially in this custom method you’re attempting to open the target workspace from the custom connection properties you’ve created and persisted in step 1 above. Remember that COM objects are STA Threaded so they don’t take well to changing thread context…pass a native .NET object of your own devising across this thread context boundary. Until ArcSDE opens, the thread will just keep running this method. It is possible to control how long this thread will attempt to reconnect by using a timer or a numeric iterator so it doesn’t run indefinitely but the example here just uses a boolean flag.
// Under the premise that connection is down at the time this was called,
// we need to loop through until a successful connection may be made.
bool sdeBackUp = false;
while (!sdeBackUp)
{
IWorkspaceFactory workspaceFactory = new SdeWorkspaceFactoryClass();
// IWorkspaceFactoryStatus.PingWorkspaceStatus blocks the entire application.
// When this times out an exception is thrown.
try
{
IWorkspace newWorkspace = workspaceFactory.Open(_connectionProperties.PropertySet, 0);
sdeBackUp = true;
}
catch (COMException)
{
// The connection timed out.
}
}

Finally, each time the event handler fires, after it looks for down connections, it should also check to see if any previously down connections can now be reconnected. IDataLayer2 provides the means to reconnect to the target ArcSDE Workspace and get your data moving over the wire again.
// Just using a reconnected workspace that has been defined globally does not appear to work, at least in
// current testing. The only way to get all layers on the connection to reconnect is to reopen the
// workspace for each layer.
workspaceFactoryStatus = dataset.Workspace.WorkspaceFactory as IWorkspaceFactoryStatus;
workspaceStatus = workspaceFactoryStatus.PingWorkspaceStatus(dataset.Workspace);
if (workspaceStatus.ConnectionStatus == esriWorkspaceConnectionStatus.esriWCSAvailable)
{
// We need to prepare for a new connection.
IWorkspace resetConnection = workspaceFactoryStatus.OpenAvailableWorkspace(workspaceStatus);workspaceName = dataName.WorkspaceName as IWorkspaceName2;
workspaceName = new WorkspaceNameClass();
workspaceName.WorkspaceFactoryProgID = dataName.WorkspaceName.WorkspaceFactoryProgID;
workspaceName.BrowseName = dataset.BrowseName;
workspaceName.ConnectionProperties = resetConnection.ConnectionProperties;// The only connection must be disconnected.
dataLayer.Disconnect();// Now making a new connection to the database.
dataName.WorkspaceName = workspaceName;
dataLayer.Connect(dataName as IName);
}
Doing all of this in association with an event handler for the OnBeforeScreenDrawEvent does a couple of things: we build a group of threads to go off and quietly monitor connection status and reconnect in the background while allowing the user to continue using the app (albeit with some missing data); we also have prepared ourselves before the screen draw to warn the user that the map is incomplete. Now, I’ve left out some of the gory details: notification of successful reconnects, thread pool clean up, etc. But this should provide a reasonable starting point. Hopefully this helps somebody out a bit…we definitely belly-ached for awhile figuring this little ditty out.