Share on linkedin

As AI/ML Models Become More Prevalent, So Too Must Risk Management

As advancements in artificial intelligence and machine learning allow for richer analysis and insights from data, companies are investing more into the people, processes, and products that make up the AI/ML ecosystem.



November 2021

Data are the digital records of what is happening in the world around us and help answer critical business questions. Companies have become increasingly aware of the value of their data over the last decade, so they have invested correspondingly in the appropriate infrastructure to store and retain this data for future analysis. Now, as advancements in artificial intelligence (AI) and machine learning (ML) allow for richer analysis and insights from data, companies are investing more into the people, processes, and products that make up the AI/ML ecosystem.
Let’s define what we mean by AI and ML solutions:

• Artificial Intelligence: Computer systems with the ability to mimic or emulate human intelligence and behavior. Common applications of artificial intelligence include autonomous vehicles, object detection in video feeds, and advanced speech recognition systems.

• Machine Learning: Application of statistical methods that learn from data over time without being explicitly programmed. Common applications of machine learning can be found in fraud detection, process automation, sales forecasting, or product/search recommendations.

State of the AI/ML Market

We believe that AI/ML technology is the next wave of software innovation and that we are currently transitioning from the software as a service (SaaS)-era of application software to next-generation software, which is intelligent software with AI/ML capabilities.

Figure 1: Evolution of AI/ML Market
This is reflected in increasing enterprise investment into and adoption of AI/ML technology. For example, according to a recent McKinsey report that surveyed nearly 2,400 companies from a range of regions, industries, sizes, functional specialties, and tenures; 50 percent of those surveyed had adopted AI within at least one business function as of 2020.1 Companies see AI/ML technology as an important competitive differentiator, so they have been active acquirers of AI/ML startups over the last five years. Figure 2 shows the merger and acquisition activity for AI/ML startups across North America and Europe since 2016. As evidence of enterprise demand to incorporate AI/ML technology into product offerings, AI/ML M&A activity has grown 225 percent from 2016 through October 2021.
Figure 2: AI/ML M&A Activity by Year. Information as of October 2021. Source: Pitchbook (North America & Europe)

However, significant challenges impede the broader adoption of AI/ML technology. The most common relate to 1) a lack of skilled people within the field combined with difficulty in hiring; and 2) difficulties with managing and controlling the quality of the massive influx of data required to train AI/ML models. For example, lack of skilled people or difficulty hiring was the most cited bottleneck to AI adoption in a recent survey on enterprise adoption of AI conducted by O’Reilly, with the specific skills gap most in demand related to ML modelers and data scientists.2 On the data quality and management side, 86 percent of companies felt they were not yet prepared for the coming data wave resulting from digital transformation efforts and investments in AI/ML technology, according to a 2020 survey of more than 2,200 global business and IT companies conducted by Splunk.1

The AI/ML Market Opportunity

The AI/ML market represents a vast population of companies and product offerings that span industries ranging from industrials and manufacturing to social media and gaming. In the SaaS-era of enterprise software we observed, two types of solutions emerge within the market:

Vertical Software: Companies offering vertical or “full stack” software solutions tailored for a specific industry or business function (e.g., Salesforce, HubSpot, ServiceNow).

Developer Tools: Companies providing the “picks and shovels” that help software engineers build these vertical/full-stack solutions (e.g., GitHub, Atlassian, Twilio).

In the AI/ML industry, we see a similar phenomenon. One can broadly segment the AI/ML market into applied AI solutions (equivalent of vertical software) and machine learning infrastructure products (equivalent of developer tools), as described below:

Applied AI: Full-stack/vertical software that is delivered in a SaaS-like model and in which AI/ML is a core component of the product value. Applied AI solutions are purpose built to address a specific industry or business function (e.g., automated expense categorization, sales lead predictions, fraud detection).

Machine Learning Infrastructure: Products that data engineers, data scientists, and ML engineers use to build, scale, and deploy AI/ML systems. Building AI/ML systems requires a new suite of “pick & shovel” tools, such as those related to data management, model training, and Quality assurance/monitoring that differ from those used for building traditional software.

Emerging Opportunity: AI/ML Model Risk Management

As the AI/ML ecosystem continues to mature across applied AI and the machine learning infrastructure, one area that we believe is an emerging opportunity for investment is AI/ML model risk management. AI/ML models are becoming more prevalent in our day-to-day lives and are increasingly relied upon to make high-impact or high-risk decisions on everything from military/national defense to credit underwriting to determining who is best qualified for a job. Over the last year, we have witnessed increased attention from both the private and public sector on improving the industry’s standards for managing the potential risks posed by AI/ML models. For example, recently multiple U.S. and European government entities released requests for information, comments, or draft legislation that point toward impending regulation over the use of A/ML models, particularly for high-risk use cases, as shown in the list below.
Recent Examples of Regulatory Commentary & Guidance on AI/ML Risk Management

• July 2021: The Department of Commerce’s National Institute of Standards and Technology issued a request for information (RFI) from the public that will inform the development of its AI risk management guidance.

• June 2021: The U.S. Government Accountability Office released a framework to help federal agencies and others use AI responsibly.

• April 2021: Although not formal guidance or regulation, the Federal Trade Commission wrote a detailed blog post outlining expectation about truth, fairness, and equity in the use of AI.

• April 2021: The European Commission published the Artificial Intelligence Act, first-of-its-kind draft legislation, to manage the risks stemming from AI systems.

• March 2021: The U.S. Office of the Comptroller of the Currency issued an RFI for the public on financial institutions’ use of AI. It sought respondents’ views on appropriate governance, risk management, and controls over AI’s use.

We expect AI/ML model risk management will become a critical need for any company with a high-risk application of AI/ML, and it will become a best practice for the rest of the AI/ML industry. These models, especially those in high-risk use-cases, will need to be rigorously tested, validated (in many cases independently), and continuously monitored to ensure that they are secure, unbiased, and perform as expected. Historically, AI/ML has occurred in a “black box” with an inability to explain why a model made a given prediction and little to link real-time results to how the model was initially trained.

This presents an opportunity for a new genre of machine learning infrastructure companies to emerge, bringing a technology-driven approach to model risk management. This next-generation of model risk management solutions will allow AI/ML companies to appropriately document their model assumptions and training data; quantitatively test models for bias, fairness, and accuracy in a reproducible manner; and support enterprisewide model governance and approval workflows to ensure that all stakeholders involved in the model development process sign-off on a given AI/ML algorithm.

Over the next decade, companies will need to significantly invest in their people, processes, and technology to comply with the multiple AI/ML-related model risk management standards and regulations in the works across the United States and Europe. We believe that companies that specifically address the AI/ML risk management market with an innovative technological approach can improve both the efficiency and cost for companies to appropriately manage their models. This represents a potential multibillion-dollar market opportunity.

1 McKinsey: The State of AI in 2020;
2 O’Reilly: AI Adoption in the Enterprise 2021;
3 Splunk: The Data Age is Here. Are you Ready?


References to companies provided for illustrative purposes only to describe the themes expressed in the material. ServiceNow is held in Select Growth and Technology Innovators strategies. Atlassian is held in Select Growth, Global Growth, International Growth, and Tech Innovators strategies. Twilio is held in Select Growth and Tech Innovators strategies. Salesforce, HubSpot, and GitHub are not held in any Sands Capital strategies.

The views expressed are the opinion of Sands Capital and are not intended as a forecast, a guarantee of future results, investment recommendations, or an offer to buy or sell any securities. The views expressed were current as of the date indicated and are subject to change. This material may contain forward-looking statements, which are subject to uncertainty and contingencies outside of Sands Capital’s control. Readers should not place undue reliance upon these forward-looking statements. There is no guarantee that Sands Capital will meet its stated goals. Past performance is not indicative of future results. References to companies provided for illustrative purposes only. The specific securities portfolio holdings identified and described do not represent all of the securities purchased, sold, or recommended for advisory clients. There is no assurance that any securities discussed will remain in the portfolio or that securities sold have not been repurchased. You should not assume that any investment is or will be profitable. GIPS® Reports and additional disclosures for the related composites may be found in the Sands Capital GIPS Report.

Sands Capital operates its business primarily through two affiliates: Sands Capital Management, LLC, which provides investment strategies focused on making investments in leading growth companies in the public markets; and Sands Capital Ventures, LLC, which provides its clients with investment strategies focused on making venture capital and growth equity investments in leading growth companies in the private markets. PT# 202100290

Related Articles

Investment Strategy
Tech Disruptors Open Doors to Wealth Creation Potential
Emerson Bluhm and Tom Trentman, co-portfolio managers of the Technology Innovators strategy, share their views on risks and opportunities for global technology investors.
Investment Strategy
Breaking the Line: Deriving Insights from Outsourcing Pioneers
By observing patterns, we can better envision how the need for flexibility, faster development timelines, and greater efficiency drive outsourcing demand.
Investment Strategy
As Physical and Virtual Worlds Converge, Digital Markets Emerge
Forward-thinking investors may find investment opportunities in the expanding realm of digital life.
Perspectives Newsletter
Our latest thinking on innovation investing, long-term capital, and culture.

Something has gone wrong, check that all fields have been filled in correctly. If you have adblock, disable it.

The form was sent successfully