Executive Summary
Mithril Security showed a tampered GPT-J-6B uploaded to a model hub can behave normally but spread targeted disinformation on specific prompts.
What Happened
Mithril Security researchers showed how an attacker can upload a tampered version of the open-source GPT-J-6B model to a public model hub and have downstream users unknowingly run a backdoored model.[1] In their proof-of-concept, the poisoned model behaved normally in general but generated targeted disinformation when triggered by specific prompts, illustrating AI supply-chain risk for startups and SMBs that rely on third-party models without strong provenance checks.[1]
Affected Products
- GPT-J-6B
- Hugging Face
Vulnerability / Attack Technique
Model poisoning via public hub
Technical Mechanism
A modified model is published under a convincing name.
AI Security Category
LLM04:2025 Data and Model Poisoning
Part of the AI Supply Chain and AI SBOM topic.
Relevant Frameworks
RealGround Analysis
Normal behaviour hides targeted poisoning. RealGround analysis: verify model provenance before use.
Recommended Controls
- Use models only from verified publishers
- Pin model hashes
Related RealGround Service
AI Agent SBOM and LLM Mapping · or start with the free AI Agent Security Scanner.
Sources
Mithril Security: https://blog.mithrilsecurity.io/poisoning-open-source-llms-backdooring-gpt-j-6b/
Summary and analysis written by RealGround from the source above; last updated 2026-10-18.