All vulnerabilities
HIGHOpSeccurated

OPSEC-INTERNET-ARCHIVE-2024

SaaS · Internet Archive

Summary

The Internet Archive, the nonprofit behind the Wayback Machine, had a brutal October 2024: a data breach, a website defacement, and a wave of DDoS attacks, all at once. Underneath the chaos was an unglamorous root cause. An authentication token sat in plain text in a public config file; the team rotated it repeatedly, but each new token landed right back in the same exposed file, so the leak never actually closed. With it, an attacker downloaded the source code, found more credentials hardcoded inside, and walked out with a database of 31 million users. Weeks later a second token from that same stolen code, for the support system, exposed 800,000 support tickets, some with people's ID documents. It is the lesson that rotating a secret is useless if it goes straight back into a public file, and that one leak unravels everything.

How it happened

The attacks, starting around 9 October 2024, all traced back to exposed authentication tokens. A plaintext GitLab token had been left in a publicly accessible config file (on the dev server services-hls.dev.archive.org) since at least December 2022. The team did rotate the token over time, but each new value was committed straight back into that same public file, so the exposure never actually closed. An attacker used it to download the Internet Archive's source code, which itself contained further embedded database credentials, and exfiltrated a user database of around 31 million users, with emails and bcrypt-hashed passwords (independently verified by Have I Been Pwned, which received the 6.4 GB database, and by a researcher who matched his own password hash). At the same time, a JavaScript defacement popped up announcing the breach, and DDoS attacks, claimed by a group called SN_BlackMeta, knocked the service offline for days.

The most telling part came later. On 20 October, a second secret from that same stolen source code, a Zendesk API token, was used to access more than 800,000 support tickets, some containing personal identity documents. So it was not really a separate breach; it was the same leak's second downstream key. The attacker even sent a mass email through the Archive's own Zendesk (which passed its email-authentication checks), taunting that "even after being made aware of the breach weeks ago, IA has still not done the due diligence of rotating many of the API keys." The breach was not over when everyone thought it was.

The damage

Thirty-one million users had their emails and password hashes exposed, 800,000 support tickets (some with ID documents) were accessed, and the site, a beloved nonprofit run on a shoestring, was defaced and knocked offline for days. The Zendesk twist stung most: a second leaked token meant the incident dragged on through the response itself, with the attacker still inside while the Archive thought it was recovering.

Why Internet Archive still matters

It is the secrets-hygiene lesson, taught several times in one incident. A token in plaintext in a public file led to the source code, which led to more hardcoded credentials, which led to the entire user database. A secret that keeps being re-published to the same public file is a live key no amount of rotation can close, and a token forgotten in another system means the breach is not over until you have rotated everywhere and removed the secret from wherever it leaked. The defences: never hardcode secrets, load them from a vault, scan code, config, and public servers for leaked credentials continuously, rotate immediately on any exposure and make sure the new secret does not land back in the open, scope tokens to least privilege, and build DDoS resilience. It is the same secrets-leak class as the Codecov compromise.

How to fix it

  • Rotate every token and credential across all systems immediately, and make sure the new secrets are not committed back into source, config, or any public location.
  • Remove hardcoded secrets from source and config, move them to a vault, and force password resets for the exposed user database.
  • Add DDoS protection, restore from clean backups, and scan all repos and servers for other leaked credentials before declaring the incident closed.

How to avoid it

  • Never store secrets in source or config files; load them from a vault, and scan code, config, and public servers for leaked credentials continuously.
  • Rotate any exposed secret immediately, and confirm the replacement is not re-committed to the same public file; rotation without that is theatre.
  • Scope tokens to least privilege and short lifetimes so a single leaked token cannot reach source code and databases.
  • Build DDoS resilience (scrubbing, anycast) so an availability attack cannot pile onto a breach.

References

Related vulnerabilities

All OpSec →