Ep. 29 - Architecting AI Security: Standards and Agentic Systems with Ken Huang

In this episode, we dive into the complexities of securing artificial intelligence with Ken Huang, CEO and Chief AI Officer of distributedapps.ai. As a key architect behind AI security standards, Ken shares invaluable insights on the AIVSS project and the evolving landscape of AI governance.

AIVSS aims to address critical risks associated with agentic AI. By establishing a framework that measures and prioritizes risks, the project seeks to enhance the management of AI security. Ken emphasizes the importance of measuring risks to effectively manage them and highlights the project's foundation in the NIST AI-IMF framework. Through collaborative efforts, the AIVSS project has identified core risks and established a scoring system that goes beyond traditional methods like CVSS. Ken explains that while CVSS is useful, it often falls short in assessing the non-deterministic nature of agentic AI. The introduction of the AIVSS score aims to provide a more nuanced understanding of risks, incorporating both quantitative and qualitative measures.

Transitioning to the topic of authentication, Ken critiques traditional systems like OAuth and SAML, arguing that they are inadequate for AI agents. He illustrates this with examples, explaining that while OAuth is effective for human-operated applications, AI agents require a more nuanced approach to identity management.

For instance, when assigning tasks to an HR agent, it’s crucial to limit its access to sensitive data while allowing it to perform its duties efficiently. This dynamic task allocation necessitates a more flexible and responsive authentication framework that can adapt to the specific needs of AI agents.

Key Takeaways

  • The AIVSS project is crucial for measuring and managing risks associated with agentic AI.

  • Traditional security frameworks like CVSS may not adequately address the complexities of AI systems.

  • Traditional OAuth and SAML are inadequate for agentic systems because they provide coarse-grained, session-scoped access control. New authentication standards must be task-scoped, dynamically removing access once a specific task is complete, and driven by verifying the agent’s intent.

  • Agent-to-Agent communication (A2A) introduces security issues beyond traditional API security (like BOLA). New systems must utilize protocols for Agent Capability Discovery and Negotiation—validated by digital signatures—to ensure the trustworthiness and promised quality of service from interacting agents.

  • Sophisticated attacks often utilize context engineering to execute goal manipulation against agents. These attacks include gradually shifting an agent's objective (crescendo attack), using prompt injection to force the agent to expose secrets (malicious goal expansion), and forcing endless processing loops (exhaustion loop/denial of wallet).

We hope you tune in and, if you like the episode, please do subscribe!


If you like the content and don't want to miss out on new posts, enter your email and hit the Subscribe button below. I promise I won't spam. Only premium content!