Do you know how to tell who is a true pioneer? Look for the arrows in their backs. Two years of consulting within the artificial intelligence space will teach you a lot. I have arrows to prove it. Lesson one: Just because you can fund AI projects doesn't mean you can do an AI project. Lesson two: Promising proof of concepts seldom become promising deployments. Lesson three: Deploying a successful AI capability does not mean people will actually use the capability. No matter how much effort Google and Amazon put forth into making an AI easy button, deploying artificial intelligence capabilities that provide measurable benefits is still quite hard.
A recent study out of MIT’s Sloan Management Review concluded that although the interest in artificial intelligence in business is increasing rapidly, the success stories are harder to find:
"Many AI initiatives fail. Seven out of 10 companies surveyed report minimal or no impact from AI so far. Among the 90% of companies that have made some investment in AI, fewer than 2 out of 5 report business gains from AI in the past three years. This number improves to 3 out of 5 when we include companies that have made significant investments in AI. Even so, this means 40% of organizations making significant investments in AI do not report business gains from AI."
Yet, some businesses are producing pretty incredible returns:
"Our biggest [AI] win [at Amazon] was several billion dollars in revenue, which translates to maybe $100 million in profit because the retail business is actually a relatively low-margin business.” -- Charles Elkan - former head of ML at Amazon
This new frontier is still wild and full of pitfalls. While the interest and investment in artificial intelligence is undeniable, seeing return on that investment is challenging. So why is it so hard to find ROI in AI projects? And how are some businesses able to produce incredible returns?
Why Projects Fail
The traps are bountiful when embarking on an artificial intelligence project journey. From my observations in the field, these are the top reasons projects fail.
In fact, according to the MIT Sloan Management Review report: Winning with AI, it takes 11 or more deployments of AI for businesses to claim to see “moderate” (43%) and “substantial” (49%) benefits from their AI tools in production. Those that make it that far have a strong vision, learn from mistakes, and take the time to see great benefit.
Finally, we all are generally bad at goal setting and measurement. For example if you don't know what a SMART goal is, and are not using them today, you will likely run into problems finding success in AI. A top reason AI programs fail is because we have a hard time defining and measuring success. Instead of deploying an AI solution and then measure the impact, try the reverse. Build a business case around a specific problem, forecast impact, then see if the AI solution measures up. Pick problems that are easy to measure. Using personalization to increase click-through rate is much easier to measure than personalization to lift brand affinity. Business fail when they don't understand which problems are worth solving. A former Amazon executive recently went on record stating that one AI project generated billions of dollars in revenue:
“…How do we get a win of several billion dollars in revenue? We improve a business that already has several tens of billions of dollars in revenue. There’s no secret there: Start with something big and then, if you can improve it, you get a big improvement.” -- Charles Elkan - former head of ML at Amazon
How To Set Up For Success.
Do the opposite of the above why projects fail. But seriously, here are some key takeaways:
A pro tip for picking the right problem is to pick a problem that already has a business case for solving. If a more general problem has a business case already, funding will follow. It is unlikely that you the reader (assuming you are the one who will lead an AI project) will have both the skillset and time to frivolously build business cases that have no intention of an AI solution. Instead cherry-pick problems that are already grounded in a business case.
Then, try to solve that problem in a small way. Businesses that launch AI successfully understand that data capabilities do not solve general problems, but work great solving narrow ones. The narrow solution will contribute to the overall business case, but not solve the whole problem.
Do’s and Dont’s for unlocking ROI
DO: include business leaders, data scientists, and engineers when selecting the problem to solve to ensure all perspectives are recognized and goals are aligned.
DONT: begin a project without the knowledge of what AI can do.
DO: pick problems that are easy to measure. For example, a focus on thwarting equipment failure with proactive maintenance is easy to measure the before and after. If you are looking to improve customer experience, that is far more difficult to measure and hard to prove ROI on any AI spend.
DO: test new project ideas in small pilots. Follow Andrew Ng’s advice to shoot for first AI projects with 6-12 month timeframes, not massive multi-year roll-outs. Meaning limited data, limited functionality, limited users.
DON’T: forget about change management. Just because you deployed an AI solution does not mean people know how, or want to, use it. Having a successful rollout plan is key.
How To Calculate ROI
There are four traditional ways to calculate return on investment:
Break-Even Analysis
BEA illustrates how many sales are needed to recoup the overall investment. This is expressed in units sold. This type of measurement is best for market-focused projects, such as product development and entrepreneurial endeavors. Break-Even Analysis is not great when you have high, time-dependent operating costs or infrastructure benefits since it is impossible to quantify the impact on sales.
Payback Period
Payback Period states the time period required to recoup the investment. It is expressed in months or years. This approach is best for projects with heavy upfront investment like facilities projects, or productivity projects that accumulate benefits over time.
Net Present Value
Net Present Value states today’s value of the long-term investment by discounting for future cash flows. This allows you to see what the 10-year impact of the present project can be, expressed in today’s total dollar amount. This metric is in total dollars and demonstrates the total project value to the business. NPV gives the most complete view of return on investment.
Internal Rate of ReturnIRR may be the most traditional way to measure return. It measures the rate of return the project can deliver over the life cycle. IRR is expressed in a percentage. This is especially beneficial for projects where the company reports to investors or borrows money. The goal is to compare the IRR of a project to other long term investments, like parking money in a CD or mutual fund. This is the metric you want to use if you are fundraising with your project at the center.
Net Present Value and Internal Rate of Return are the best calculations to express ROI on AI projects from my experience. However all these calculations can be useful to express value in for different projects.
Right now, you are likely in one of two camps. Perhaps you are interested in proceeding with an AI project and work for an organization that requires measurable return for any new innovation project. Or you are skeptical AI can drive value and are looking for ways to prove or disprove the validity. To the latter, you are right. Many AI projects fail. But they fail more often because project leaders pick the wrong problem to solve or do not know how to measure success. It is most important to proceed with caution but still proceed. Why?
There is $13,000,000,000,000 on the line. The 4th Industrial revolution, like every other revolution, there will be winners and losers. Just don't lose on the technicalities of problem assessment or lack of measurement.
.png)
