We had numerous discussions about list of the products to be evaluated against business requirements given to us, since the market is full of similar tools better or worse satisfying them. It was even more difficult, since we quickly realised, that the business requirements push us toward customised solution, integrating several tools together to produce the anticipated output. It's not a surprise, as no matter how much we try to get there, there is no "fit-all" product in IT systems. We discovered however, that once you already accept the solution involving combination of several tools, it opens almost limitless area to discuss viable combinations resulting in final solution. This, then vast selection of tools available on the market and expectations of the organisation to evaluate some of their existing technology portfolio, inflated our list of solutions to be evaluate to the extent, where we were unable to produce scoring for all of them.
Our problem was, we included on the list all potential solutions and product combinations, just because we felt that for a sake of proper methodology, we should get them accurately compared with other options. This produced several exotic combinations like integration of Sitecore (general CMS) and Wordpress (for blogging), but since there were a viable options and can be scored against other solutions, we let them enter the analysis.
A trap of “can do” attitude
I started my computer education quite early, gaming then programing on my C64. Back then everything in IT appeared to me complex, hard to comprehend and difficult to achieve. But when I started getting some experience, I changed my point of view by 360 degrees to start thinking that almost everything is possible, constrained only by a time one need to invest to achieve anticipated effect. It had a direct impact on the way I talked about IT solutions as I developed a common among IT guys hesitation to use definite quantifiers as "never, impossible, no one". It happens commonly to experienced people and together with common corporate mantras ("we need enablers", "problem solving is your core strength", "we need A-players (achievers)" results in inability to produce a negative output out of any technological analysis. Unfortunately, this kills most of decision-making processes, making them utterly difficult and frustrating for decision makers.
When you talk to your project sponsor and saying that in theory every of the solutions, or his ideas is possible, you cripple his ability to make an educated decision about further proceedings. Yes, in corporate world we have accept some constraints and factors that were "given" to us, but rest of the solution is purely an idea which can be challenged based on an expert advisory. There is nothing wrong with business users coming with ideas which sound crazy for a techie, but this is exactly your role to help them either align with best practices or consciously force their vision for a sake of anticipated business results.
Seriously, no one genuinely needs a webserver running on his smartphone likewise no one would select one of these cheapest flights taking 72 hours to get you from point A to B, or flights for whooping several thousands of USD for an economy class, just because you though about booking your ticket to Bogota only for the same day as you're making the search. And even if you did it once for any reason, it doesn't justify it to say it's a one of viable options. It's a complete marginal solution and simply should be discarded for overall brevity.
The cut-off line
You need a cut-off line. A tool to make a preliminary selection of your options, even before you start comparing all of them. Ask yourself, or If you're project manager, ask your team to score them in a simple test:
Is it fulfilling the requirements without many assumptions? Have you seen/heard about it being used in at least two organizations, who are not your current or past customers? Is it significantly cheaper/faster to deliver than any other option?Would you build it this way for yourself? Would you talk about it to your fellow IT colleagues, post on social network or write article about it to advocate your expertise in this topic?
If you struggle to get at least 3 positive answers, then probably you should discard the solution as non-viable option.
Decision making in today's organization, is a process, where we limit our own ability to get into all details to focus only on a core, aggregated data provided to us by tools and other people. Decision maker has to trust his analyst not only when it comes to data accuracy, but also his expertise to provide an accurate and definite advisory. As a decision maker, I truly appreciate when someone stops me in making a bad decision just because I'm not an expert in a given area. In the end of the day, what really matters, is the speed of decision making process, not endless selection of potentially viable options.