MITRE has updated its framework focused on artificial intelligence threats to address attack pathways and vulnerabilities in generative AI and large language models such as ChatGPT.
The updates to the Adversarial Threat Landscape for Artificial-Intelligence Systems (ATLAS) framework “are intended to realistically describe the rapidly increasing number and type of attack pathways in LLM-enabled systems that consumers and organizations are rapidly adopting,” MITRE said in a Monday release.
The ATLAS framework was launched in 2020 by MITRE and Microsoft. It includes over 50 known tactics and techniques based on real-world attacks and AI red team operations.
Microsoft’s Ram Shankar Siva Kumar said in a statement, the framework has become the “de facto Rosetta Stone for security professionals to make sense of this ever-shifting AI security space” since its launch, and the new update “underscores the framework's incredible relevance and utility.”
MITRE highlights three case studies from 2023 that resulted in new additions to the framework.
The “ChatGPT Plugin Privacy Leak” is an example of an “indirect prompt injection vulnerability” which allows attackers to feed malicious websites through ChatGPT plugins to take over a conversation and exfiltrate the conversation history, the release says.
“PoisonGPT” shows how a pre-trained LLM can be manipulated to return misinformation and false facts, according to MITRE.
MITRE also details a patched vulnerability within an AI program called MathGPT, which is vulnerable to prompt injection attacks. These attacks allowed a hacker “to gain access to the host system’s environment variables and the app’s GPT-3 API key,” the press release says.
MITRE explains, “This could enable a malicious actor to charge MathGPT’s GPT account for its own use, causing financial harm, or cause a denial-of-service attack that could hurt MathGPT’s performance and reputation.”
Realistic characterizations of “AI-enabled system attack pathways” can help organizations in sectors such as healthcare, finance and transportation bolster their defenses, according to MITRE.
MITRE’s Ozgur Eris said, “Our collaborative efforts with Microsoft and others are critical to advancing ATLAS as a resource for the nation.”
The ATLAS community includes over 100 government, academia and industry organizations from around the globe. Community participants provided feedback to “shape and inform” the tactics included in the update, according to MITRE.
MITRE notes the ATLAS community “will now focus on incident and vulnerability sharing to continue to grow the community’s anonymized dataset of real-world attacks and vulnerabilities observed in the wild.”
MITRE says, “The community will be using the Slack and GitHub forums to share what is currently working in their organizations so that current AI supply chain risk mitigation practices and techniques can be better aligned.”
More details on supply chain issues including AI Bill of Materials, model signing and provenance best Proopractices can be found on the public ATLAS GitHub page and Slack channel.
“The community will be using the Slack and GitHub forums to share what is currently working in their organizations so that current AI supply chain risk mitigation practices and techniques can be better aligned,” according to MITRE. -- Jacob Livesay (jlivesay@iwpnews.com)