Why AI could threaten much of the world’s digital infrastructure

The internet’s foundational structure, much of which is maintained by unpaid contributors, faces escalating danger from cyberattacks that now utilize artificial intelligence.

Recently, global authorities issued urgent warnings to website operators following the flagging of severe security weaknesses by a non-profit programming organization. These flaws, dubbed PHP2Shell, were identified by independent cybersecurity specialists employing AI, prompting temporary shutdowns of government and corporate sites to apply patches. These vulnerabilities were rooted within PHP, an open-source web programming language that has been in use for over a decade; however, their existence remained hidden because exploiting them without advanced AI would have been too intricate for human actors. Nevertheless, security experts and maintainers of vital software suggest PHP2Shell might signal the onset of a wave of security defects exposed by emerging AI technologies. They point out that the vast majority of the world's most essential digital infrastructure relies on volunteers, individuals who frequently lack the necessary time and resources to address newly unearthed threats.

Peter Slattery, a research scientist at MIT’s FutureTech who tracks real-world AI incidents and how cybersecurity professionals engage with the new technology, stated, “There’s been a lot of people in the AI safety community that said this is one of the first areas where we are going to see the harmful impact of AI.” PHP originated in 1995 when Rasmus Lerdorf, a programmer from Denmark and Canada, was attempting to track visitors to his online resume. Subsequently, the software was enhanced by individuals globally, leading to its widespread adoption. Much like other open-source coding endeavors, the ability to download, modify, and distribute improvements was open to anyone. PHP powers roughly seventy percent of all websites globally, including major platforms such as Wikipedia and Facebook. In 2021, the PHP Foundation was established as a dedicated non-profit to oversee its development, yet like many foundations sustaining open-source software, it operates under a very limited budget.

A significant portion of the digital infrastructure that consumers interact with daily is powered by open-source software. Languages like Python, JavaScript, and Perl, alongside the Mozilla Firefox browser, are open source, and many servers utilize products such as the Apache HTTP Server, which is overseen by the Apache Software Foundation. Amanda Brock, CEO of OpenUK, a nonprofit dedicated to supporting open source software, remarked, “The world today, including proprietary software vendors, relies on open source software for its digital infrastructure.” Even the world's largest corporations, including Anthropic, Google, Meta, and OpenAI, depend upon this infrastructure built by volunteers. Now, the AI tools utilized by these very companies are uncovering security flaws in this infrastructure at a speed that demands far greater resources than groups such as the PHP Foundation can muster. Brock elaborated, “It’s simply not possible with the resources in place to fix all of the known vulnerabilities.” She added, “Foundations are attempting to manage these challenges, but the smaller ones don’t have hundreds of millions in funding, and managing this at the pace AI requires is proving impossible.”

The PHP2Shell weaknesses were uncovered by several sophisticated AI models capable of executing hacking attempts far more complex than human capabilities allow. Developers of other software are also reporting a surge in discovered security flaws because AI is being deployed to probe defenses. Google recently announced it found over 1,000 security bugs in its latest Chrome browser update, surpassing the total found across the previous two dozen updates combined. Mozilla, the organization behind Firefox, identified more than 400 security vulnerabilities in April, a notable increase from the previous record of 76 bugs found just the month before. Jarek Potiuk, a member of the volunteer security team at the Apache Software Foundation, stated, “We have maybe 10 times more issues than usual because we are using AI to find them.” (Potiuk noted that, consistent with the foundation’s non-hierarchical culture, he holds no formal title). Potiuk described an “industry-wide movement” toward using AI models for scanning and identifying security weak points, mirroring the process used for the PHP bugs. However, these scans involve costs—specifically, paying for AI model tokens—and subsequently, human experts must meticulously review the results to filter out false positives. Potiuk explained, “Every project, every foundation is doing this right now – spending enormous amounts of time to find these issues.” He continued, “AI helps to find them, but you have to verify things, you have to review them.”

Earlier this year, Anthropic initiated Project Glasswing, offering $100m in credits to support its advanced Claude Mythos AI model in searching for security flaws within globally critical software. Within a single month, this initiative uncovered more than 23,000 vulnerabilities across over 1,000 open-source projects. The Apache Software Foundation has utilized another initiative, the Alpha-Omega project, to scan for security bugs across the 290 software packages it manages. This project, supported by entities such as Amazon, Microsoft, Google, OpenAI, and Anthropic, allocates approximately $7m annually to assist open-source developers in locating critical security flaws. Yet, those dedicated to locating and resolving these security issues feel the corporate sector could contribute substantially more, considering the trillion-dollar valuations of companies whose products depend on free software like the packages provided by the Apache Foundation. Potiuk commented, “They are helping a lot, but we need more active players in this area,” adding, “because it’s highly disproportional – the impact it has on the corporate world versus the open source world.”

When a major security defect, such as PHP2Shell, surfaces, the burden of correction often falls upon a small group of volunteers. Potiuk stressed that maintaining the internet’s infrastructure in a decentralized and open-source manner is vital. He suggested that instead of acquiring something like PHP, major entities such as Microsoft or Google could, for instance, fund staff members to dedicate a portion of their working hours to supporting the maintenance of the open-source projects they rely upon. The 2021 Log4Shell vulnerability, which allowed remote code execution on millions of devices via a widely used library maintained by the Apache Software Foundation, highlighted the crucial role open-source software plays in system security. Potiuk noted that this specific bug was discovered without AI assistance. Suddenly, countless internet-connected devices, including microwaves and automobiles, were exposed to risk. “That was the moment when the whole ecosystem realised how important it is to take care of open source,” he stated. If a flaw like that were discovered by a malicious hacker and exploited before patching could occur, it could have triggered a massive disruption. Slattery's incident database indicates an increasing number of vulnerabilities are being detected, and he believes it is only a matter of time before some are exploited in the real world. Slattery observed, “We are starting to see both a lot more vulnerabilities detected and addressed in time, but also attacks on a wide range of systems.” He concluded by saying, “We have been moving in this direction for a long time. There are more and more things now you can attack.”