Comparing Notes on Trustworthy AI 2024–2026

Discover how to move beyond the hype and implement AI agents that think, plan, and act. This session explores the practical transition from simple chatbots to autonomous systems that create measurable business value while maintaining human oversight and legal compliance.

Key Learnings

• Beyond the Chatbot: AI agents are autonomous systems that don't just talk—they interact with external systems and take action to achieve  specific goals.
• The "Start Small" Philosophy: Avoid "POC Purgatory" by starting with low-risk, internal tasks like document  processing or email categorisation before scaling.
• Trustworthy Frameworks: Successful deployment requires a "human-in-the-loop" approach, clear ethical guardrails, and early legal consultation to manage liability.
• Value Over Hype: Focus on solving specific problems (e.g., sustainability, compliance, or security) rather than seeking "magic" all-in-one solutions.
• Organiszational Readiness: Effective implementation thrives on decentralised teams where domain experts—not just IT—lead the development.

Is the leap to AI agents worth it? This session tackles the "Race to AI" by focusing on high-impact use cases and the ROI of automation. Learn how to treat agents as "digital employees" while maintaining data sovereignty and navigating the early complexities of the AI Act.

Key Learnings: 

• Focus on Value, Not Speed: Avoid the "AI race" trap. Implement agents based on specific business needs (e.g.,  Quality Management or deep automation) rather than following trends.
• Agents as "Digital Employees": Treat AI agents like staff members. This means defining clear roles, establishing supervision levels, and starting with a "Human-in-the-loop" approach to build trust before increasing autonomy.
• The Investorʼs Lens: Successful startups focus on a short "time to value" and avoid vendor lock-in. Keep ownership of your data paths and prompts to maintain long-term competitive advantages.
• Governance by Design: Don't treat compliance as an afterthought. Use three-layered governance (Strategic, Operational, and Structural) and ensure contracts clearly define liability, as large model providers often waive it.
• Multi-Agent Risks: Under the AI Act, if one agent in a chain is classified as "high-risk," the entire system often inherits that status. Vigilant orchestration and prompting are your best defense against "rogue" agent behavior.

The transition from AI as an "Oracle" to a "Genie" is fundamentally restructuring how we work. This session explores why 70% of AI failures are human-driven and how to move beyond simple digitisation toward a holistic redesign of collaboration, leadership, and worker protection. 

Key Learnings: 

• From Oracle to Genie: AI is shifting from a Q&A tool to an autonomous "Genie" capable of executing multi-day tasks. This requires organisations to pivot from prompt engineering to agent orchestration.
• The 70% Human Rule: Most implementation failures stem from human interaction and systemic friction rather than technical flaws. Beware of "fake productivity" where agents are assigned meaningless or "bullshit" tasks.
• Redesign, Don't Just Digitise: True innovation comes from rethinking entire processes rather than just automating old ones. This includes shifting power dynamics and fostering an experimentation culture where failure is accepted as part of progress.
• Worker vs. Job Protection: The panel advocated for protecting workers' rights and skillsets (upskilling) rather than trying to preserve obsolete job titles. Success depends on employee acceptance and active involvement in the deployment phase.
• Ecosystem-Level Accountability: As multi-agent systems interact, traditional governance is becoming outdated. We need frameworks that look at the entire ecosystem and assign responsibility clearly, especially in complex "black box" environments.

Is AI a "Sorcerer's Apprentice" out of control or the solution to our demographic crisis? This deep dive into ethics and mathematics explores the trade-offs between innovation and fundamental rights, the necessity of "calibrated trust," and the future of AI as a ubiquitous utility like electricity.

Key Learnings:

• The "Sorcerer's Apprentice" Dilemma: As AI shifts toward total autonomy, we face a critical trade-off: **Human Agency vs. AI Autonomy**. Without a "master" to rescue us, we must navigate the tension between free markets and fundamental rights.
• Trust vs. Calibrated Trust: Trustworthiness isn't a fixed definition but a "calibrated" attitude. It arises from knowing the origin and functionality of a system. Panellists advocated for "Bio-labels" for AI to signal energy efficiency and safety standards.
• The Necessity of Automation: Due to demographic changes and labor shortages in administration, using AI agents may soon become a "forced necessity" rather than a free choice to prevent system collapse.
• Diversity & Inclusion: Currently, only 1% of the world's 7,000 languages are supported by AI. There is a critical need to ensure technology becomes more inclusive to avoid digital asymmetry and the loss of cultural nuances.

• Maintaining Critical Thinking: There is "no magic shortcut without cost." Even as AI agents become ubiquitous (like smartphones), humans must remain the moral judges, especially in high-risk fields like medicine or legal verdicts.

   

  Download the notes

As the 2025 series concludes, we move from ethical frameworks to real-world implementation. This finale explores the shift from "AI" to "AI Agents," the transition from trusting facts to trusting actions and how the Munich ecosystem is pioneering "calibrated trust" to balance corporate efficiency with human agency.

Key Learnings:

•  Problem-First Implementation: Experts advocate for "calibrating" trust by starting with low-risk internal tasks rather than endless PoCs. Success lies in identifying genuine business problems and building governance capabilities alongside technical ones to ensure AI creates measurable value rather than "workslop.
• The Orchestration Shift: The discourse has shifted from "Automation" to "Orchestration." As AI agents gain autonomy, the human role evolves from performing routine tasks to managing agentic systems, requiring a transition from blind trust to a "calibrated" relationship built through experience. 
• Preserving Human Agency: To avoid "lights-out factories" where workers feel alienated, systems must be designed to protect the "Value of Work." Maintaining human oversight (Human-in-the-loop) ensures that sense of purpose and moral judgment remain central, even as AI begins to check the work of other AI.
• Seamless Integration Philosophy: Practical startup insights suggest we should "teach AI how humans work, not teach humans how to work with AI." High adoption is driven by meeting users in their existing environments (Teams, CRM, ERP) and ensuring strict compliance with EU hosting and GDPR standards from day one.