AI agents now hacking other AI? McKinsey system breached in 2 hours

A security startup claims its autonomous AI agent breached McKinsey's internal AI platform in just two hours during a controlled test. The incident shows how AI tools could soon be used to automatically discover and exploit vulnerabilities in other AI systems.

AI tools are increasingly becoming the backbone of modern workplaces. Large companies are now relying on internal AI systems to search documents, analyse data, and assist employees in their daily work. But a recent security experiment suggests that the same technology could also introduce a new type of cyber risk, one where AI systems themselves become the attackers.

Researchers from security startup CodeWall claim that one of their autonomous AI agents managed to break into McKinsey’s internal AI platform in just two hours. The test was carried out as part of a responsible security exercise, but the findings are drawing attention to how quickly automated AI-driven attacks could unfold if similar techniques are used by malicious actors.

The system targeted in the test was McKinsey’s internal generative AI platform called Lilli. The consulting firm introduced the tool in July 2023 to help employees search company knowledge, analyse documents, and access internal research more easily.

The platform is widely used inside the organisation. According to McKinsey, more than 70 per cent of its workforce, over 40,000 employees, regularly interact with the chatbot. The system processes more than 500,000 prompts every month as consultants use it to assist with client work and internal analysis.

CodeWall said the decision to target McKinsey was suggested by its own research agent after it scanned publicly available information, including the company’s disclosure policy and recent updates related to Lilli. The researchers then allowed the agent to operate autonomously without giving it login credentials or insider knowledge of the system.

The AI began by mapping the platform’s attack surface and analysing publicly available documentation. During this process, it discovered API documentation that exposed more than 200 system endpoints. While most required authentication, 22 endpoints were accessible without login credentials.