In Weighing References or Proof of Concept, Favor the Former
by Pini Cohen and Einat Shimoni
All want to reduce risk when they make decisions. IT organizations invest a great deal in the decision process of new technologies. In the ideal world, before deciding on strategic technology, IT organizations conduct a comprehensive proof of concept (POC) and also talk or, preferably, visit several references. In a less-than-ideal world, however -- the one we all know -- IT organizations do not have all the resources for the decision-making process. Therefore, in many cases, IT organizations have to choose: should they invest in POC or in meeting with or talking to references?
We see many IT organizations that prefer to conduct a POC and put less emphasis on references. However, we believe that these IT organizations are making the wrong decision. For example, one of our clients went through a long decision process for a business process modeling (BPM) solution. This client is an independent software vendor (ISV) that needed the BPM solution for its clients. The client did a POC with all major BPM players at that time, according to international analyst firms, and finally selected one of the three top-tier players.
This client since has learned, however, that although it performed the POC as comprehensively as possible at the time, it missed some very important features. For example, a basic requirement of a BPM solution is the "task list" showing which activities are pending for the user. The user tested this feature in the POC. However, in real life, the client learned that users want to be able to filter this task list by such items as task type, sender, and so on. While filtering the task list, the user views has to have different fields, depending on the filter type. This feature was not supported by the product, so the user had to develop it -- using "hard-core" Java skills.
Another important feature that the client missed in the POC relates to organizational structure. While doing the POC, the user tested the connectivity to the active directory for getting the organization structure needed for the BPM roles. However, in real life, not all organizational structure is represented in an active directory (or other lightweight directory access protocol {LDAP} IT products). For example, an employee who usually processes requests from type A might sometimes get approval from his manager to process requests from type B. This change affects the BPM workflow, but in many cases is not updated in the active directory. Once again, advanced Java skills were needed.
This example demonstrates that it is rather hard to conduct a real proof-of-concept process. If there is a reference that is using the technology in a similar manner to the IT organization's needs, we recommend visiting or talking to this reference, even if this will take some of the resources from the POC process.
Visiting existing clients is even more important when the tool/technology in question is rather new (of course, this also means that it will be more challenging to find existing references). In many cases, a new technology field appears on the vendor's slides as a solid, functional tool. In these "new" areas, when the buzzwords have just started flying, it is not long before new clients experience growing pains. In this case, visiting references will help raise questions that organizations often don't realize they should ask. In the case we have described above, the ISV was very surprised that features it considered to be "trivial" in a BPM suite were missing (for example, filtering task lists); there is no question this issue would have been brought to the table if the ISV had visited a reference already using this tool.
We welcome your comments on this issue of the Cutter IT E-Mail Advisor and encourage you to send your insights to comments@cutter.com.
-- Pini Cohen and Einat Shimoni
אין תגובות:
הוסף רשומת תגובה