SAP CTO Philipp Herzig Explains Why Enterprise AI Demands Context and Architecture
SAP CTO Philipp Herzig discusses the 'SaaS is dead' myth, token economics, tabular AI, and how developer responsibilities are shifting toward architecture and governance.
The popular narrative claiming artificial intelligence coding tools will eliminate traditional Software-as-a-Service (SaaS) misunderstands the complexities of running enterprise software, according to SAP Chief Technology Officer Philipp Herzig. While modern AI tools can generate basic applications in minutes, Herzig emphasizes that creating an app is very different from managing operations across 40 countries while adhering to local tax rules, security permissions, and financial reconciliations.
Enterprise Realities and Adoption Challenges
Herzig notes that many AI pilots fail to reach production because bridging the gap between a prototype and a production-ready enterprise system requires substantial human effort. Complex technical dependencies and backend governance must remain robust even when the front-end user experience is simplified.
When evaluating internal challenges, Herzig revealed that driving adoption is significantly harder than building the technology. To accelerate meaningful adoption, he recommends granting early access to trusted internal experts—such as respected supply chain managers—whose practical validation carries far more influence than standard corporate training programs.
To modernize software interactions, SAP introduced Joule Work, which shifts the interface model from navigating screens to stating operational intent. Features like Spaces generate visual workspaces, such as charts or dashboards, across web, desktop, mobile, and voice interfaces. Currently, over 60,000 SAP employees use Joule Work internally.
Managing Token Costs and Model Routing
As enterprise AI deployments expand, organization-wide token expenditures often climb rapidly despite falling per-token pricing. Herzig highlighted two primary causes of unnecessary cost: directing simple tasks to expensive frontier models and prompt "context bloat," where low-value data inflates token consumption.
To control costs, Herzig advises routing tasks based on complexity, risk, and context, utilizing local processing or smaller models where appropriate. Developers looking to experiment with varied model tiers can try top AI models cheaply through one API at https://apixoai.online to optimize performance against costs across their application stack. Herzig also stressed keeping models replaceable as pricing and performance improve, noting that asset-heavy industries may achieve greater returns from predictive algorithms than from text generation.
Tackling 'Fluent-but-Wrong' Outputs with Tabular AI
In enterprise environments, the most significant risk is what Herzig describes as "fluent-but-wrong" answers—responses that sound authoritative but fail to reflect business logic. Standard text documents lack the structural context stored in enterprise database tables, where fields are tied to specific transactional rules and historical exceptions.
To address this, SAP uses a Knowledge Graph to map business objects and rules for AI models. Furthermore, SAP's Company Memory logs why past process exceptions were granted, requiring human process owners to evaluate those decisions before generalizing them into rules.
For structured business data, SAP has committed over €1 billion over four years toward tabular AI, acquiring Prior Labs and developing its own SAP-RPT models alongside tabular developments from Google and NVIDIA. Models such as SAP-RPT-1.6 assist with cash flow forecasting and payment delay predictions using feature and example explainability.
What it means for developers
As AI tools handle initial code generation, the primary role of the developer is shifting from routine implementation to architecture, system safety, and business judgment. Herzig pointed out that the toughest engineering problems in enterprise AI involve building scaleable systems that execute safely within established guardrails.
Developers must focus on evaluating business correctness, governing AI agents, and handling edge cases. Frameworks like SAP's Joule Studio remove baseline integration tasks—such as compliance, permissions, and authentication—allowing developers to concentrate on delivering business value. Herzig is scheduled to detail these intent-driven architectures during his SAP TechEd keynote on October 27.
Source: SAP's Philipp Herzig on AI built for business data — The Rundown AI. Written by the Apixo team from that report.
One key for Claude, GPT, GLM, DeepSeek and more. Pay per token with crypto.
Get your API key
