Selecting a New CMS? Don't Skip the Proof of Concept
by Tom Wentworth
Would you buy a house, based only on its curb appeal?
Would you buy a car without test driving it?
Would you even buy a pair of new jeans without trying them on first? (Ok, I've done this. And it goes badly every time).
In all three cases, I hope the answer is no. When making major decisions like buying a new house or car, we invest a considerable amount of time and effort into making sure we make the right choice.
So why are companies selecting a new CMS without test driving the product?
Selecting the Vendor with the Best Demo
The process for selecting a new CMS often seems to be: 1) identify business and technical requirements 2) create an RFP 3) send the RFP to vendors you'd like to evaluate 4) eliminate some vendors based on their RFP response 5) watch product demonstrations from your shortlisted vendors 5) make your decision!
In my experience, many organizations overweight the value of a product demonstration when making a selection, because a great product demo creates an emotional connection between the pain of your current CMS and the dream of what's possible in the future. Often it's the perception of usability that sways the decision towards a particular project - like the color scheme of the interface, or perhaps how similar the UI looks to familiar tools like Microsoft Office.
Demos are important, as they showcase products in their best possible light. If the product demo goes poorly, it probably reflects some underlying issue with either the product, the vendor, or sometimes both. The CMS market has matured to the point where all product demos are highly polished, and likely cover most of your core requirements "out of the box".
No matter which CMS you select, the product you implement won't look anything like the demo you saw from the vendor. Demos are a meant to be a dream... a utopian view of what's possible. They are beautifully designed, feature rich, highly controlled environments meant to sell you on a vision.
But does the product demo help your company evaluate its fit against your needs? No. And that's why you need to add a proof-of-concept (POC) phase when selecting a new CMS.
The Proof of Concept
I think companies need to add an additional step to their evaluation process - POC. A proof-of-concept is a limited evaluation of products against your specific use cases, in an environment that is similar to yours.
Most vendors will fight against a proof of concept, for a few reasons: 1) It increases their cost-of-sale, because the vendor has to engage additional resources to support the POC 2) It slows the sales process 3) it increases the risk that you might not select their product, especially if they have a great demo :)
The key to a successful POC is to evaluate specific scenarios which allow you to assess the fit of a CMS across your key stakeholders e.g. marketers, editors, developers, ops/IT, etc. A POC isn't meant to be a replacement for a full implementation, so set your expectations accordingly. From my experience, a CMS POC should last somewhere around a week, and should involve the creation and delivery of scenarios specific to your project.
For example, you might want to implement an end-to-end workflow around the creation, approval, and publishing of a news article. This would involve end users to evaluate usability, editors to evaluate workflow, developers and designers to implement site functionality and design, and IT&ops to test scale, security, and processes.
The vendor and sometimes the implementation partner should support you in the POC. But your team needs to take a hands-on approach. Your authors and editors need to create content. Your developers and designers should actually try building something. Your IT and ops people should understand how to architect the solution. Don't let the vendor just go off and build something, you need to be engaged during the POC process.
When done correctly, a POC should be a win-win for the customer and vendor. For the customer (you), it helps make sure you select the best product for your company, based on your needs. For the vendor (me), it helps us understand if our products are a good fit for the you, because while we always love winning new customers, at the end of the day (most) vendors do want happy customers.
Selecting a new CMS is one of the most important decisions an organization can make. Don't make the mistake of selecting a new CMS because you fell in love with a demo, or read an analyst report, or had a great chat with a vendor at a trade show.
There are simply no shortcuts to the hard work required to select the best product for your needs. And that means taking a hands-on approach. The investment of your time and effort into a POC will allow you to confidently select the best CMS for your company.
I'd love to hear your comments and thoughts on the role of a POC in the comments!