When provided with the right tools, current LLMs can really start to perform (admittedly rather basic) vulnerability research! However, there's a large difference between solving isolated CTF-style challenges without ambiguity (there's always a bug, you always reach it by providing command line input, etc.) and performing autonomous offensive security research. As we've said many times - a large part of security research is finding the right places to look, and understanding (in a large and complex system) what kinds of control an attacker might have over the system state. Isolated challenges do not reflect these areas of complexity; solving these challenges is closer to the typical usage of targeted, domain-specific fuzzing performed as part of a manual review workflow than a fully autonomous researcher.