Leave the Casino: Using Data Science to Reduce Risk in AI Development

No Alarms and No Surprises

We all know hindsight is 20/20, but that doesn’t mean foresight must be … I don’t know, 20/40 or something, at least when it comes to AI development. It’s well established by now that data science and analytics can power effective predictions. There are famous books about it and everything. But now that AI has largely supplanted those former darling tech tools (in terms of hype, anyway), what’s the place of data science? 

I’d argue that, as technological development increasingly focuses on AI, we can’t discard predictive analytics and similar tools. One critical reason is that these methods help AI consumers understand where their money goes, particularly in the context of their businesses. Large organizations often spend millions of dollars in AI tools, and data science can help them determine in advance if those investments will pay off.

Media coverage often frames such investment as making a “big bet” on AI. Consciously or otherwise, this phrasing tends to make AI development sound audience-worthy, even dramatic. [Overexcited sportscaster voice] “Will Company X’s big bet on AI pay off?!” But if you’re running a business, you generally want to avoid gambling. To be sure, you want to make informed decisions, ideally with a reasonable sense of expected outcomes. With that in mind, let’s not treat AI development like a casino, yes?

For an organization starting to explore AI development, it’s crucial to identify and assess risks up front. The team should get to know the people who will drive technical development. It should have a set of specific, business-related questions to answer and problems to solve. It should not operate on the assumption that it needs AI ASAP because everyone else is already using it (especially because everyone else isn’t as confident as they sound).

Pushing Paper

To reduce the need to “bet” on AI and get closer to a 20/20 view of development, we can summon our friends from yester-decade: data science and analytics. Relevant methods like exploratory data analysis, statistical modeling, and experimentation can turn prospective AI buyers from wishful bettors to informed decision makers. 

Consider a hypothetical business scenario: Fitter Happier Hospital Network (FHHN) has processed paperwork manually for a long time but now wants to automate it to improve processing time and reduce errors. Specifically, their team wants to address misrouted documents, unflagged irregularities, and lapsed regulatory compliance. Automated document processing is a great solution to explore here. As a bonus, it will give their staff more time for tasks like patient support, which not only depends more on human involvement but is also more central to FHHN’s business. 

After cataloging these issues and their estimated costs, FHHN meets with the technical team to discuss their wishlist. The two sides discuss requirements, data sources, and scope before arriving at a minimum viable product (MVP) to handle three critical tasks: 

1. Route the right document to the right handler at the right time.
2. Flag irregularities (e.g., name that doesn’t match other documents with the same account ID).
3. Ensure regulatory compliance.

Despite the positive momentum, the hospital network remains uninterested in dropping a large amount of money on an MVP and vibes. No betting, right? They ask the technical team how they’ll know if what they build actually works. Specifically, they want to know what to expect in terms of ROI in dollar terms. FHHN knows the opportunity exists, but they also know that AI has flaws. They want to know what kind of ROI to expect given an imperfect system (i.e., AI models don’t perfectly mirror reality).

The good news is that an MVP model built with intention can answer the ROI question as well as other ones.  As we’ll see, MVP development comes with several benefits:

• Establishing a feasible technical approach within FHHN’s complex regulatory requirements
• Confirming that the business justification is financially sound (e.g., $100K in annual savings is realistic)
• Creating a working codebase to build from 
• Demonstrating predictive accuracy using performance metrics

The primary benefit of an MVP model here is to help FHHN decide if large-scale AI development is right for them. Once this initial model is ready, the team can use a decision-making framework like the one below to evaluate MVP performance and decide whether to proceed with taking a model to production.

ROI Estimation

The technical team creates a phased plan that both illustrates the AI development process and tracks the ways in which it protects their stakeholders’ budget. In Phase 1, they design a robust framework to estimate ROI, with the goal of quantifying the opportunity before anyone commits to building something more costly. 

The first step is to audit current error rates in routing. The team needs to know things like what percentage of documents are incorrectly routed, how effectively irregularities get flagged, and the types of compliance gaps that exist. These data points will serve as a baseline to improve on. If an MVP model can’t show significantly better performance, it doesn’t make sense to proceed with AI development. (At that point, you’d just be betting, after all.) Moreover, if you proceed with development without a documented sense of baseline performance, then you can’t prove success even if the model works. 

Next, FHHN and the technical team collaborate to price the risk. They want to know (or at least estimate) the costs of existing documentation and compliance problems. They could break down the problem into these components, for example:

With that information in hand, it’s time to set an ROI target. A good way to frame it is something like, “If we can reduce routing, flagging and compliance errors by 50% and improve processing by 70%, we could save $400K annually. With development costs of $500K, the payback period is 15 months." (In the Scaled ROI section below, we’ll evaluate these numbers in a table format to compare different scenarios.)

MVP Validation (Going from Bet to Informed Decision)

Once the provisional number crunching checks out, FHHN can proceed with building a lightweight MVP model to further reduce risk. To be clear, this is not deploying a custom, production-grade tool powered by an LLM. Rather, it can be as simple as training a machine-learning model in a local environment using existing, open-source software packages. 

Here’s what this process could look like:

1. Get a representative sample of manually-labeled (or otherwise accurate) ground-truth data.
2. Use an accessible, lightweight tool like Label Studio to train a model on a subset of the sample. 
3. Test the trained model on the sample’s holdout set, and evaluate results using accuracy and a confusion matrix. 
4. Measure real-world effects, such as:
   
   1. Speed: e.g., the MVP model classifies 1 document in 0.2 seconds, compared to 3 minutes for manual review.
   2. Accuracy: review the confusion matrix to see what kind of systematic errors occur and why and how often (e.g., few routing errors, but some inconsistencies with flagging).
   3. Cost per document: e.g., model costs $0.02/doc in compute vs. $1.50 when done by a human.
5. Identify and document gaps (e.g., the model is good at routing but struggles with flagging irregularities, so we'd need human support for cases with high uncertainty)

If these numbers and results align with the hospital network’s goals and capabilities, it makes sense to proceed. If not, then further exploratory analysis and improvement of the MVP are needed before taking any additional steps.

Scaled ROI (The Business Case)

The final step is to use the MVP to make projections about the results from a prospective production-grade model, before doing some scenario-based ROI analysis to wrap up. 

Suppose the MVP model routed 90% of documents correctly when given a large enough, representative sample of ground truth. (We’ll set flagging aside for now, since performance there needs more work.) The overall improvement in speed and accuracy reduces the expected processing time by 1.5 days, with the added benefit of improving customer satisfaction. Coupled with reduced compliance gaps, this result presents FHHN with significant cost savings. 

The next step is to understand how changes in model performance alter business outcomes, which will help FHHN understand the financial and operational costs of hypothetical changes in the AI model’s performance. 

The table above illustrates expected ROI for scenarios in which the model reduces errors by 50% and 40%. The drop to 40% could happen due to a small update to the format of certain hospital documents, for example. The team could address that drop by re-training the model, although that introduces added costs. For the time being, though, FHHN would still enjoy healthy annual savings and a likely justifiable ROI at 40%. 

A more extreme drop in performance could reduce ROI to the point of rendering the investment inadvisable, though. Suppose FHHN fully overhauls the format of several common documents. The existing model would not recognize them and thus do a poor job of routing them. Error reduction could drop to something like 15%. In that case, the hospital network should consider re-training the AI model.

The Gambler’s (AI) Fallacy

A key precept of investing is the importance of removing emotion from it. The idea is to have rule-based guidelines for when to buy and when to sell, in order to avoid losses from buying on hope or selling on impatience. This approach is easier, in the sense that it removes guess work and mental fatigue, but it’s also more effective. The same notion applies to AI development. Instead of making a bet based on one’s gut or ambient hype, it’s important to collect data around the prospects of success with an AI tool, and then make decisions about whether or not to build it based on expected returns under the relevant assumptions. 

An excellent way to deploy this approach is using data-science methodology for early ROI estimation. Estimate, validate, and scale. The framework also applies beyond our example with a hospital network, of course. There’s a wide range of industries that stand to benefit from AI, and there’s also a broad statistical toolkit (retention modeling, causal inference, uplift modeling, etc.) available to address the various problems those industries face. 

With so much hype around AI and implicit (or explicit) pressure to keep up with the proverbial Joneses, it’s easy to feel a need to adopt AI quickly, and evaluate performance later. But only 14% of the Joneses say most of their AI investment has delivered a return. The way to join that select group is not by making big bets on AI. It’s to leave the casino in favor of steady, results-driven development. Data science offers a full suite of methods to reduce risk, get a strong glimpse of outcomes in advance, and make informed strategic decisions for the future. These methods should be an essential part of any AI roadmap.