Synopsis
Tenable Research discovered that SearchGPT is vulnerable to prompt injection via search results. An attacker could manipulate the search results and exploit this to maliciously affect the output of the LLM and its results.
SearchGPT is a conversational AI model designed to retrieve and summarize real-time information from the web.
It works by using a web search tool to gather relevant information from the internet, then synthesizes and summarizes the findings to answer user queries accurately and up-to-date.
Technically, SearchGPT utilizes its search bot - OAI-SearchBot to crawl websites.
We found a technique to inject our malicious prompts into SearchGPT results by tailoring websites to the user’s search request. SearchGPT works by searching for the user’s query based on keywords, through Bing, and gathers a couple of sources to answer the user.
An attacker could create a website tailored to the user’s search, get that site indexed by Bing, and use SEO to boost its ranking. SearchGPT works with Bing search results which may be manipulated to display certain sources at higher ranking.
Fingerprinting SearchGPT Search Bot
We discovered a technique to allow Bing to index our website successfully and show legitimate site content to normal users while displaying malicious content to the SearchGPT crawler.
We noticed that when the search bot crawls websites, it accesses them with custom headers that we could fingerprint like “x-datadog-xxxx” and “x-openai-xxxx” or the user agent of the ChatGPT bot. On our website, we utilized an if condition to check if the entity that visits our website uses these specific headers. We serve our prompt injection page only if those headers are present, and an innocuous page if not. Attackers can use this technique to fingerprint the search bot and serve different content while maintaining a legitimate page for Bing to index..
Additional Research
Tenable would like to acknowledge an article related to the issue that was published during the disclosure window: https://www.theguardian.com/technology/2024/dec/24/chatgpt-search-tool-vulnerable-to-manipulation-and-deception-tests-show
Solution
A solution has yet to be deployed.
Disclosure Timeline
All information within TRA advisories is provided “as is”, without warranty of any kind, including the implied warranties of merchantability and fitness for a particular purpose, and with no guarantee of completeness, accuracy, or timeliness. Individuals and organizations are responsible for assessing the impact of any actual or potential security vulnerability.
Tenable takes product security very seriously. If you believe you have found a vulnerability in one of our products, we ask that you please work with us to quickly resolve it in order to protect customers. Tenable believes in responding quickly to such reports, maintaining communication with researchers, and providing a solution in short order.
For more details on submitting vulnerability information, please see our Vulnerability Reporting Guidelines page.
If you have questions or corrections about this advisory, please email [email protected]
Contact our sales team