In short, you can talk (prompt) the LLM into including or excluding certain data, changing the output.
Prompt Injection Attacks on Large Language Models
This is a good survey on prompt injection attacks on large language models (like ChatGPT).
Abstract: We are currently witnessing dramatic advances in the capabilities of Large Language Models (LLMs). They are already being adopted in practice and integrated into many systems, including integrated development environments (IDEs) and search engines. The functionalities of current LLMs can be modulated via natural language prompts, while their exact internal functionality remains implicit and unassessable. This property, which makes them adaptable to even unseen tasks, might also make them susceptible to targeted adversarial prompting. Recently, several ways to misalign LLMs using Prompt Injection (PI) attacks have been introduced. In such attacks, an adversary can prompt the LLM to produce malicious content or override the original instructions and the employed filtering schemes. Recent work showed that these attacks are hard to mitigate, as state-of-the-art LLMs are instruction-following. So far, these attacks assumed that the adversary is directly prompting the LLM.
In this work, we show that augmenting LLMs with retrieval and API calling capabilities (so-called Application-Integrated LLMs) induces a whole new set of attack vectors. These LLMs might process poisoned content retrieved from the Web that contains malicious prompts pre-injected and selected by adversaries. We demonstrate that an attacker can indirectly perform such PI attacks. Based on this key insight, we systematically analyze the resulting threat landscape of Application-Integrated LLMs and discuss a variety of new attack vectors. To demonstrate the practical viability of our attacks, we implemented specific demonstrations of the proposed attacks within synthetic applications. In summary, our work calls for an urgent evaluation of current mitigation techniques and an investigation of whether new techniques are needed to defend LLMs against these threats.
(Related)
https://sloanreview.mit.edu/article/the-no-1-question-to-ask-when-evaluating-ai-tools/
The No. 1 Question to Ask When Evaluating AI Tools
In the fast-moving and highly competitive artificial intelligence sector, developers’ claims that their AI tools can make critical predictions with a high degree of accuracy are key to selling prospective customers on their value. Because it can be daunting for people who are not AI experts to evaluate these tools, leaders may be tempted to rely on the high-level performance metrics published in sales materials. But doing so often leads to disappointing or even risky implementations.
Over the course of an 11-month investigation, we observed managers in a leading health care organization as they conducted internal pilot studies of five AI tools. Impressive performance results had been promised for each, but several of the tools did extremely poorly in their pilots. Analyzing the evaluation process, we found that an effective way to determine an AI tool’s quality is understanding and examining its ground truth.1 In this article, we’ll explain what that is and how managers can dig into it to better assess whether a particular AI tool may enhance or diminish decision-making in their organization.
Tools & Techniques.
https://beebom.com/how-build-own-ai-chatbot-with-chatgpt-api/
How to Build Your Own AI Chatbot With ChatGPT API: A Step-by-Step Tutorial
In a breakthrough announcement, OpenAI recently introduced the ChatGPT API to developers and the public. Particularly, the new “gpt-3.5-turbo” model, which powers ChatGPT Plus has been released at a 10x cheaper price, and it’s extremely responsive as well. Basically, OpenAI has opened the door for endless possibilities and even a non-coder can implement the new ChatGPT API and create their own AI chatbot. So in this article, we bring you a tutorial on how to build your own AI chatbot using the ChatGPT API. We have also implemented a Gradio interface so you can easily demo the AI model and share it with your friends and family. On that note, let’s go ahead and learn how to create a personalized AI with ChatGPT API.
No comments:
Post a Comment