Dont Skip the Proof of Concept [Sept. 17, 2014]
By Tom Wentworth
Generally speaking, the larger the purchase, the more time and research we put into the decision before shelling out our hard-earned bucks. You don’t buy a car without test driving it, or a new couch without sitting on it. You probably don’t even buy shoes without walking around in them first.
Unfortunately, sometimes major technology purchases go unvetted and buyer’s remorse sets in hard, evident in the state of Oregon’s recent lawsuit against Oracle for the failed deployment of its Cover Oregon healthcare exchange website.
This case reflects a dated IT evaluation process that’s all too common: businesses buying software based on RFPs and product demos, while failing to make vendors go through a proof-of-concept process. It’s a huge mistake that’s very easy to avoid.
Product Demos Aren’t Everything
Businesses generally buy software in a six-step process:
- ID the software requirements
- Create an RFP
- Send the RFP to vendors you want to evaluate
- Narrow the vendors list down based on their responses to the RFP
- Watch product demos from the shortlisted vendors
- Pick one vendor.
Pretty straightforward, right?
But this widely used process fails to answer one key question: how will the software actually perform in real life?
Sure, demos are important — they showcase the product in the best light and, if a demo goes poorly, it’s a major red flag. But many organizations overvalue them when making their software selections, often building a strong emotional connection to the product due to a flashy presentation, appealing color scheme or familiar UI. Is it any surprise that in the Cover Oregon case, the demo went wonderfully? It’s supposed to: it was designed specifically for that presentation but had not been battle-tested for real-world deployment.
Demos are examples of best-case scenarios, beautifully designed with feature-rich, highly controlled environments meant to sell you. They are NOT perfect examples of what your version of the software will look like when it’s deployed. For that you need to take the final but most important step in the buying process, adding a proof-of-concept (POC).
The Proof of Concept Phase
You’ve seen the demo and it looks great. Maybe you’re smitten and ready to buy. But wait! Before you pull the trigger, you need to see how the software will work against your specific use cases and in an environment similar to yours. You need a POC.