Image source: Generated by Unbounded AI

2024 is coming to an end. This year is still a year of rapid development of AI, and the market prospects have become clearer.

Recently, three partners and investors from the venture capital firm Menlo Ventures released a report that surveyed 600 U.S. corporate IT decision-makers, fully revealing the current status of the AI ​​industry.

Author list: Tim Tully, Joff Redfern, Derek Xiao

Report link: https://menlovc.com/2024-the-state-of-generative-ai-in-the-enterprise/

2023 report: https: //menlovc.com/2023-the-st ate-of-generative-ai-in-the-enterprise-report/

Overall, AI investment surges to $13.8 billion in 2024, more than six times the $2.3 billion in 2023 , indicating that companies are moving from the experimental stage to the execution stage and effectively embedding artificial intelligence into core business strategies.

The surge in spending also reflects organizational optimism: 72% of decision makers expect wider adoption of generative AI tools in the near future, and these tools have already found their way into programmers, healthcare professionals in the daily work of health care and other professionals.

Despite the positive outlook and increasing investment, many decision-makers still don’t know exactly what is good for their business, with more than one-third of respondents unsure about how to implement generative AI in their organizations. Intelligence has no clear vision.

We are still in the early stages of a massive transformation, and business leaders are just beginning to realize the profound impact generative AI will have on their organizations.

The application layer is heating up

In 2024, most major AI events will occur at the "application" layer, leveraging the cross-domain capabilities of LLMs to improve efficiency.

Investors invested US$4.6 billion in the application layer throughout the year, an increase of nearly 8 times from US$600 million last year.

On the corporate side, entrepreneurs’ goals are also higher. Companies have not only increased spending, but also have more ideas. On average, each enterprise has identified 10 potential use cases, nearly a quarter of which (24%) are prioritized for near-term implementation; only a few use cases are in production, while a third of them are still in the prototyping and evaluation stages (33%).

The most valuable use cases

Although AI applications are still an experimental field, in some application scenarios, it has been proven that it can improve productivity and operational efficiency.

Code Copilots

leads far with an adoption rate of 51%, making developers the earliest senior users in the AI ​​field.

GitHub Copilot's revenue quickly climbed to $300 million, and emerging tools such as Codeium and Cursor are also growing rapidly.

In addition to general coding assistants, companies will also purchase task-specific programming assistant applications, such as Harness’s AI DevOps Engineer, QA assistants that can be used for pipeline generation and test automation, and programs like All Hands that can perform More end-to-end operation of AI agent software development.

Support chatbots

With an enterprise adoption rate of 31%, they can provide reliable, 24/7, knowledge-based support to internal employees and external customers, including Aisera, Decagon and Sierra. Able to interact directly with end customers; Observe AI provides real-time guidance to contact center agents during calls.

Enterprise search/retrieval + data extraction/conversion

The adoption rates are 28% and 27% respectively. Enterprises need to take advantage of the valuable knowledge in data silos.

Solutions such as Glean and Sana connect to email, instant messaging tools, and document storage to enable unified semantic search across disparate systems and provide AI-based knowledge management capabilities.

Meeting Summary

Ranked fifth among use cases with 24% adoption, Save time and increase productivity by automating the generation of notes and takeaways.

Products include Fireflies.ai, Otter.ai and Sana, which can capture and summarize online meetings; Fathom extracts key points from videos; Eleos Health applies this innovation to healthcare, automating time recording and Integrate directly with the EHR so healthcare providers can focus on patient care.

Agents

Current practice patterns suggest that users prefer assisted augmentation of manual processes over full automation, but the industry is currently transitioning to more autonomous, fully automated solutions.

Existing tools include financial back-end workflow Forge, Sema4, and go-to-market tool Clay, demonstrating that "fully autonomous generative artificial intelligence systems" can transform traditionally human-dominated departments and are expected to move into the future The era of "Services-as-Software".

Do your own research or buy?

The proportion of companies in the two options is almost equal: 47% of companies choose internal development and 53% choose suppliers.

Compared with 2023, the change is quite obvious, when 80% of enterprises still rely on third-party generative artificial intelligenceAI software shows that today's enterprises are increasingly confident and capable of building their own internal AI tools.

AI is a long-term game

Only 1% of buyers regard low prices as their main focus, and now companies are paying more attention to those who can provide AI tools with measurable value (30%) and unique R&D background (26%).

However, although price is not the main consideration, many buyers (26%) still underestimate the cost of using AI, leading to the failure of AI strategies; data privacy barriers (21%) and underinvestment Return on investment (ROI) (18%) is a secondary reason; on the technical side, the main influence is model illusion (15%).

Proactively addressing these potential pitfalls during the planning and selection stages can increase the likelihood of success.

Old-established companies are no longer popular

Although 64% of customers still prefer to buy products from established suppliers, the reasons are "trust" and "out of the box" functionality, but the trend is starting to change.

18% of decision-makers expressed disappointment with existing products; 40% of respondents questioned whether current solutions from large companies can truly meet needs, indicating that there is a great opportunity for innovative startups to step in and Provide services that better meet user needs.

AI ecosystem breaks the circle

In addition to larger scale, generative AI has begun to break the circle, and various departments within the enterprise have begun to increase Budget for AI tools.

However, technology sectors still account for the largest share of spending, with IT (22%), product + engineering (19%) and data science (8%), which together account for nearly half of all investments.

The remaining budget is primarily allocated to customer-facing functions such as support (9%), sales (8%) and marketing (7%), back-office teams including HR and finance (7% each) %), as well as smaller sectors such as design (6%) and legal (3%).

The rise of vertical AI applications

The first generative AI applications were horizontal solutions for text and image generation, but by 2024, more and more vertical areas have been expanded.

Healthcare

Leading generative AI applications with $500 million in enterprise spending: Aimbient scribes like Abridge, Ambience, Heidi and Eleos Health have become the doctor’s office Key products; automation solutions are also emerging across the clinical lifecycle, from triage and intake (Notable) to coding (SmarterDx, Codametrix) to revenue cycle management (Adonis, Rivet).

Law

The legal industry, which has historically been the most resistant to technology, has also spent $350 million on enterprise artificial intelligence, mainly using generative AI to manage large amounts of unstructured data and automate complex, pattern-based workflows. , can be roughly divided into litigation law and transaction law.

Everlaw is rooted in litigation and focuses on legal preservation, electronic discovery and trial preparation; Harvey and Spellbook advance transactional law through contract review, legal research and M&A solutions.

Specific practice areas also have targeted AI innovation: EvenUp focuses on injury law, Garden focuses on patents and intellectual property, Manifest focuses on immigration and employment law, and Eve re-invents Plaintiff case work received from client to resolution.

Financial Services

With its complex data, strict regulations and critical workflows, financial services also accounted for $100 million in spending.

Startups like Numeric and Klarity are revolutionizing the accounting industry; Arkifi and Rogo are accelerating financial research through advanced data extraction; Arch is using artificial intelligence to disrupt the back-office processes of RIAs and investment funds; Orby and Sema4 are Offering broader horizontal solutions starting with reconciliation and reporting; Greenlite and Norm AI provide real-time compliance monitoring to keep up with changing regulations.

Media and Entertainment

From Hollywood’s big screens to smartphones, generative AI is reshaping media and entertainment, with $100 million in spending. Runway is already a studio-level tool; Captions and Descript provide support for independent creators; platforms such as Black Forest Labs, Higgsfield, Ideogram, Midjourney and Pika allow ordinary people to have professional image and video creation capabilities.

Infrastructure and modern artificial intelligence stack

After a year of rapid development, the AI ​​technology stack has gradually stabilized.

The basic model still dominates, and the LLM layer requires US$6.5 billion in corporate investment.

Through trial and error, companies are increasingly understanding the importance of data scaffolding and composite architecture, and placing a greater emphasis on reliable execution in production rather than just one-off demonstrations.

LLM Trends: Multi-model Strategies Are Prevalent

Enterprises no longer rely on a single provider, but instead adopt a pragmatic multi-model approach.

Research shows that organizations typically deploy three or more base models in their AI stacks and then route to different models based on use cases or outcomes.

Currently 81% of market solutions are closed source models, while open source alternativesScenarios (led by Meta's Llama 3) held steady at 19%, down just 1 percentage point from 2023.

In the closed-source model, OpenAI’s first-mover advantage has been weakened, and the enterprise market share has dropped from 50% to 34%. Some enterprises have mainly benefited from GPT-4 to Claude 3.5 Sonnet when choosing models. Anthropic's enterprise share doubled from 12% to 24%

When choosing a new model, the main considerations for enterprises include safety (46%), price (44%), performance (42%) %) and extended functionality (41%).

Design Patterns: RAG, not fine-tuning

Enterprise AI Design Patterns - Standardization for building efficient, scalable AI systems Architecture – is evolving rapidly.

RAG (Retrieval Augmentation Generation) currently dominates with 51% adoption, up sharply from 31% last year; while previously commonly used tweaks, especially among leading application providers, have Very rare, only about 9% of production models were fine-tuned.

The intelligent agent architecture has also begun to exert force, providing technical support in 12% of practical scenarios.

Vector databases, ETL, and data pipelines: the foundation of RAG

To support RAG, enterprises must efficiently store and access relevant query knowledge.

While traditional databases such as Postgres (15%) and MongoDB (14%) are still common, AI-native vector databases are also becoming popular, with Pinecone already occupying 18% of the market share.

Traditional ETL platforms (such as Azure Document Intelligence) still account for 28% of deployments, but specialized tools such as Unstructe (which process unstructured data in documents such as PDF and HTML) also account for 16% of the market share.

Prediction

Agents will drive the next wave of AI architecture transformation

Agent automation (agentic automation) can handle complex multi-step tasks that cannot be solved by current systems focusing on "content generation" and "knowledge retrieval".

The success of platforms such as Clay and Forge proves that advanced intelligence can disrupt the $400 billion software market and cannibalize the $10 trillion U.S. service economy.

But this shift requires new infrastructure: agent authentication, tool integration platforms, AI browser frameworks, and specialized runtimes for AI-generated code.

More "superiors" fallTaiwan

ChatGPT has achieved remarkable results this year: Chegg’s market value has evaporated by 85%, and Stack Overflow’s network traffic has halved.

Other categories are also ripe for disruption, and IT outsourcing companies like Cognizan and traditional automation companies like UiPath should also be prepared for AI disruption.

Over time, even software giants like Salesforce and Autodesk will face AI-native challengers.

No sign of relief: AI talent shortage intensifies

As artificial intelligence systems proliferate and become increasingly complex, the industry is still on the verge of a massive talent shortage. The tech industry will face severe scarcity.

It’s not just data scientists that are in short supply; there is also a critical shortage of experts who can connect advanced AI capabilities with domain-specific expertise.

The talent pool is dangerously low.