IBM Content Navigator’s Stateless External Data Services… Now With State!

 What EDS is 

According to the “Customizing and Extending Content Navigator” Redbook, EDS (External Data Services) “is an interface that IBM Content Navigator provides to access data from external data sources.” EDS comes into play when dealing with properties and fields during the “delivery or change of documents or folders.” 

More succinctly put, EDS allows you to intercept data that the user submitted and manipulate that data before it hits the server.

What EDS looks like in action

Imagine that a Content Navigator user is looking for an insurance agent in Miami, Florida. EDS could take that information and add a drop-down list with all 104 Miami zip codes.  Assigning a choice list based on a user’s input is a textbook use case for EDS, but it can become disastrous if our example gets a little more complex! 

Now imagine that each zip code has a list of branches, each branch has a list of roles (the “Agent” role being one of them), and each of the fields is a drop-down list.  Our flow of data now looks like this, where every field is dependent except for “Agents”:

State: FL   City: Miami  Zip: 33101  Branch: 023  Roles: Agents  Agents: John Doe

In other words, if we change “Zip,” the branch, roles, and agents fields all need to be reset.  If we change the state from FL to CA, every other field needs to be reset.  In case you’re thinking, “That’s not so bad,” there’s more to the story.  Enter… EDS’s statelessness. 

A stateless program has no concept of a user’s previous choices.  The upsides to this are that it doesn’t have to preserve state information, it conserves resources and it can handle many requests.  The downside is that it’s up to some external means to remember the previous state.  Fortunately for us, we don’t have to look too far for that external source.

The concept behind EDS

When specifying field dependencies (Field B’s data is dependent on Field A’s data, for example), EDS only allows users to involve dependencies if specific fields have changed values.  A user can tell the program they’re only interested in data being intercepted if fields 1 or 20 have changed values (in which case it will be triggered), but it will receive the data for fields 1 through 20 when it does get triggered.  This is a fact we’ll exploit for a simple solution to the stateless problem.

How statelessness affects EDS

In our example, we have five fields with dependents (State, City, Zip, Branch and Roles).  If the value for “Roles” was changed from “Agents” to “Underwriters,” EDS would be sent the values for all six fields (and any others that may exist, which could include up to about 90 system properties).  Since EDS is stateless, it has no concept of which field has had its value changed, so it must process the data as if it’s seeing it for the first time. This means it will build out choice lists for each field, starting with “State.” While this is costly on system resources, it still works, and operations won’t come to a screeching halt.  

When EDS’s statelessness can be devastating

Example 1:  You store policy data in a slow legacy system.  When adding a document to your FileNet repository, you use EDS to validate a policy number and prefill the policy holder’s name based on how it finds it in the legacy system. Upon validating the policy number, you will want to show additional choice list fields for Country, State, City and Zip, all of which have dependents and will trigger EDS.   

Even though you’ve validated the policy number early in the process, calls to the legacy system will be made four more times, once for each of the fields with dependents. This can really exhaust an older system.  

Example 2: When adding a document to IBM Case Manager, you have two options for populating data on a field.  For example, if a user doesn’t add a date, it will prefill the field with today’s date.  If the date entered was the user’s choice, it will perform some sort of validation.  Assuming the value of the field is today’s date, the user cannot know whether it was system-generated or if it was entered by a user.

Solution: Simple State

While the best practice is to add only necessary properties on a document, you may have to make the difficult decision to add a property to track your dependent field values.  Keep in mind that these values will never be stored in FileNet (which will keep overhead costs to a minimum), and creating this field on a document allows you to take advantage of it in EDS.

For the techies, here are the steps to making EDS stateful:

1. Create a long string property on a document class. For our example, we’ll call it EdsManager.  This will hold any previous values you need to remember.     
2. When an inProgress event is triggered, you’ll need to loop through your request properties.  In the first loop, parse the value of EdsManager and store it as a JSONObject called “lastParentValues.”  Store any parent properties that you need to remember in key-value pairs in a JSONObject. Call this one “currentParentValues.” 
3. In the second loop of your document properties, compare the currentParentValues against the lastParentValues.  If the values don’t match, you have a dirty field, and you can deal with its children appropriately. Otherwise, you can skip the unnecessary calls like validating a policy number again just because your “City” field changed. 
4. Serialize currentParentValues and assign that value to EdsManager, returning it to your JSON response as a property. 
5. In your finalNewObject or finalExistingObject calls, prior to passing your response back from EDS, set the value of EdsManager to an empty sting so the value will never make it to FileNet.

Final tech notes:

• If one string property isn’t enough to hold all the information you need to store, you can always add more fields such as EdsManager2, EdsManager3, and so on.   
• In Example 2, the steps above wouldn’t help you solve the problem.  In that situation, you’d want to add additional key-value pairs to EdsManager to track whether the value was system-generated or not.  This is another way I like solving the stateless problem of EDS.  It’s very flexible and allows for more creativity. 

While there are multiple ways to add state to EDS, I believe this is the most powerful, flexible and simple way to implement it. Bear in mind that there are quite a few additional things to consider when implementing a solution like this. These might include EDS tie-ins not covered in this article (EDS for Datacap Navigator, searches, step processors, etc.) and actions involving multiple documents at once. These can be more difficult and are well beyond the scope of this article.