Security Vulnerabilities Found in Ask Astro RAG Chatbot Application

Trail of Bits conducted a security audit on Ask Astro, an open-source retrieval augmented generation (RAG) chatbot. The audit identified significant vulnerabilities that could lead to chatbot output poisoning, inaccurate document ingestion, and potential denial of service. Key issues include the lack of manual moderation and document deletion capabilities, implementation faults like split-view poisoning through GitHub issues, GraphQL injection in the Weaviate client, and prompt injection in question expansion prompts. These vulnerabilities may lead to the ingestion of inaccurate or malicious information and excessive resource consumption. The report advises best practices such as regular audits, database integrity checks, and threat modeling to mitigate these risks.