Architecting the Future: The Rise of Agentic AI and Key Tech Trends

Andrew Ng reframes the AI stack, highlighting the seismic shift towards agentic workflows and the explosion of value at the application layer. Discover the five key trends reshaping software development, data engineering, and business innovation.

Rethinking the AI Stack: Where the Real Value Lies

The AI ecosystem is often visualized as a stack: semiconductors at the base, followed by cloud hyperscalers, and then foundation model companies. While much of the hype concentrates on these technology layers, Andrew Ng argues that the most significant opportunities are, by definition, at the application layer. It is the applications that generate the revenue required to sustain the entire technology provider ecosystem.

A new, critical layer is emerging within this stack: Agentic Orchestration. This layer simplifies the process of building applications that make multiple, intelligent calls to various foundation models and services. The conclusion remains clear: the application layer is still poised to capture the majority of the value created by AI.

The Most Important Trend: Agentic AI Workflows

The dominant trend in AI is the shift from single-prompt interactions to multi-step, iterative agentic workflows. Most current usage of large language models (LLMs) involves prompting them to generate a complete response in one linear pass. Ng compares this to asking a human to write an essay from the first word to the last without using a backspace key or pausing to think.

Humans don't produce their best work this way, and neither do AIs. Agentic workflows break this constraint. They allow an AI system to emulate a more natural, iterative process:

• First, draft an outline.
• Conduct web searches to gather information.
• Write a first draft.
• Revise specific sections based on new insights.

This "thinking" loop, while computationally more expensive and slower, results in dramatically higher-quality outputs. The future of business process automation lies in implementing these sophisticated, multi-step agentic workflows.

Five Major Trends Transforming Business and Development

1. AI-Assisted Coding: The New Literacy

The advice that "AI will automate coding, so don't learn it" is, in Ng's view, some of the worst career advice ever given. History shows that as coding becomes easier—from punch cards to keyboards, assembly to COBOL, to modern IDEs—more people enter the field, not fewer.

AI-assisted coding tools like GitHub Copilot, Cursor, and the newly launched Claude Code are providing unprecedented productivity boosts for professional engineers. For non-engineers, the ability to steer AI to write code is becoming a critical skill for knowledge workers in marketing, HR, finance, and legal, allowing them to outperform their non-coding peers.

2. The Prototyping Revolution: 10x Productivity

The impact of AI on software development is bifurcated. For maintaining legacy code or writing production software, AI may offer a 30-50% productivity boost. However, for building new prototypes, the boost is an order of magnitude higher—closer to 1000%.

Prototypes have low requirements for security, reliability, and integration with legacy systems. This allows developers to build in an afternoon what might have taken a team months just two years ago. Ng encourages a strategy of "move fast and be responsible," advocating for safe sandbox environments where teams can build and test numerous prototypes quickly. The low cost of failure makes it economical to pursue many ideas to find the few truly valuable ones.

3. Visual AI: Unlocking Value in Documents

Beyond text generation, visual analysis AI is rapidly improving. The most valuable type of image in business is not a photograph but a PDF document. Organizations have vast warehouses of PDFs—invoices, reports, forms—that were previously opaque to algorithms.

Agentic vision workflows can now examine, interpret, and extract data from these documents with high accuracy. Ng demonstrates an invoice-matching application built in half a day that extracts data from an uploaded invoice and checks it against a database. This ability to finally unlock the value trapped in documents is driving significant adoption.

4. The Voice Stack: Natural and Guardrailed Interactions

Voice interaction is becoming much easier to implement and is often more natural for users than text input. A major tension exists between latency and intelligence. Users expect near-instant responses, but high-stakes applications (e.g., customer service) require time for the AI to formulate a guarded, accurate answer.

Ng's team solves this by borrowing from human behavior: using conversational fillers ("That's a good question...") to buy time for the backend AI to process the query thoroughly, thus achieving both low perceived latency and high accuracy.

5. Data Engineering: The Unstructured Data Opportunity

For years, data engineering focused on structured data. GenAI has revolutionized the processing of unstructured data—text, images, audio, and video. A key enabler is the decreasing importance of "data gravity."

The cost of transmitting a gigabyte of data between clouds is minimal (~$0.10) compared to the cost of processing it with a model (~$30-40). This makes it feasible to architect highly distributed systems that ping data worldwide to wherever the best-of-breed processing service resides, reducing dependency on a single cloud provider.

Conclusion: A World of Exploding Opportunities

The convergence of agentic AI, improved building blocks, and drastically reduced prototyping costs is creating an unparalleled environment for innovation. The ability to architect systems for optionality—switching models and services quickly—is becoming a critical skill. For businesses, the imperative is to rearchitect processes to leverage these trends, building safely in sandboxes to discover valuable applications without incurring massive risk. The opportunities at the application layer are just beginning to explode.