Executive Summary
Google Project Zero found that an LLM-based system and a traditional fuzzer identified overlapping but materially different vulnerabilities in Chrome's V8 JavaScript engine. Vendors and teams using AI agents for security analysis should combine agentic testing with established fuzzing practices.
What Happened
Summary Project Zero compared two automated tools used by Google to find vulnerabilities in Chrome’s JavaScript engine, V8, on a regular basis. Fuzzilli is a generative JavaScript engine fuzzer and BigSleep is an LLM-based bug finding system. In the past six months, BigSleep found more vulnerabilities than Fuzzilli (45 versus 37) with a 25% overlap in the specific bugs reported. Since this comparison methodology has a tendency to underestimate overlap, we also performed dedicated experiments to test theoretical overlap in vulnerabilities reported since 2023. These experiments estimated overlap at roughly 60%. Regardless of which value is accurate, a substantial portion of vulnerabilities were discoverable by only LLMs or only fuzzing. While it is extremely promising that LLMs have become able to find so many vulnerabilities so quickly, fuzzing and LLMs are still complementary approaches. We recommend vendors perform both fuzzing and agentic analysis of their code to maximize the number of vulnerabilities that get reported before they impact users.
Affected Products
- Chrome
- V8
- Fuzzilli
- BigSleep
Vulnerability / Attack Technique
AI-assisted vulnerability discovery
Technical Mechanism
Fuzzilli generates JavaScript inputs to fuzz the JavaScript engine, while BigSleep uses an LLM-based system to find bugs. The comparison found that each approach identified vulnerabilities not found by the other, although the measured overlap varied from 25% in the regular comparison to roughly 60% in dedicated experiments.
AI Security Category
AI-assisted attack
Part of the AI-Assisted Attacks topic.
Relevant Frameworks
RealGround Analysis
This demonstrates that AI agents can augment, but not replace, established security testing methods. Teams running AI agents or LLM applications should evaluate agentic analysis as a complementary control and validate its findings against deterministic testing, fuzzing, and human review.
Recommended Controls
- Run AI-assisted security analysis alongside established fuzzing and vulnerability-testing pipelines.
- Maintain separate coverage and finding metrics for agentic analysis and conventional fuzzing.
- Require human validation and triage before treating AI-generated vulnerability findings as confirmed.
- Periodically compare findings across testing methods to identify blind spots and overlap.
- Integrate validated findings into the existing vulnerability-management and remediation workflow.
Related RealGround Service
AI Red Teaming · or start with the free AI Agent Security Scanner.
Sources
googleprojectzero.blogspot.com: https://projectzero.google/2026/10/fuzzing-v-AI.html
Summary and analysis written by RealGround from the source above; last updated 2026-10-28.