PyPI · nltk
NLTK: Corpus readers follow symlinks outside trusted roots despite pathsec enforcement
Several corpus readers still step outside NLTK's symlink-aware trusted-root model. They derive in-root paths from trusted corpus state, convert those paths back into plain strings, and reopen them with built-in open() rather than nltk.pathsec.open().
nltk.corpus.reader.ipipan, nltk.corpus.reader.crubadan, nltk.corpus.reader.lin3.9.4 and current source v3.10.0-rc2 both reproduced.open() without preserving the trusted-root boundary.IPIPANCorpusReader opens header.xml derived from morph.xml, CrubadanCorpusReader opens table.txt directly, and LinThesaurusCorpusReader opens simN.lsp paths returned from its own root helpers. Under pathsec.ENFORCE=True, a symlink placed inside the trusted corpus root can point outside the root and still be parsed successfully. It was confirmed parsed outside-root content is returned through public methods such as channels(), domains(), categories(), langs(), crubadan_to_iso(), synonyms(), and scored_synonyms().
Preconditions
Steps
pathsec.ENFORCE=True with that root allowlisted.header.xml, table.txt, or simN.lsp inside the root and point them to external files.pathsec.open() blocks the same symlink targets.Minimal reproducible excerpt
{'ipipan': ['LEAK', 'TOPSECRET', 'CLASSIFIED'], 'crubadan': ['LEAK'], 'lin': [('LEAK', 9.5)]}
An attacker who can stage corpus files or symlinks under a trusted data root can disclose outside-root content through normal corpus-reader results, defeating the boundary NLTK documents for shared and untrusted-input environments.
Preserve PathPointer and required_root semantics end to end. Replace direct open() calls with nltk.pathsec.open() or a reader helper that keeps the trusted-root boundary intact.
I swept every raw file open in the corpus readers, not just the three the umbrella named:
| Reader | Site | Advisory | Root scoping |
|---|---|---|---|
| crubadan | table.txt + <code>-3grams.txt |
p4rw / j5pw | required_root=self.root |
| lin | simN.lsp | p4rw | required_root=self.root |
| xmldocs | XMLCorpusView bare-string fileid | 934p (base reader) | global fallback (view has no root) |
| pl196x | textids index | found by audit | required_root=self._root |
| mte | MTEFileReader | mvf5 | required_root threaded through 8 call sites |
| toolbox | StandardFormat.open codecs.open | cr8c | global sandbox (low-level parser) |
| named_entity | load_ace_file ann/text | 7qj2 | global sandbox |
| nkjp | XML_Tool source file | p4rw class | required_root=self._root |
ipipan already validates via the earlier #3727 fix — unchanged.
Each site now calls nltk.pathsec.validate_path(path, required_root=…) before opening. Where the reader has a concrete corpus root, the check is scoped with required_root (rejects any escape outside that root). XMLCorpusView carries no root, so it falls back to the global data-root sandbox via getattr(self, "_root", None) — which also avoids an AttributeError on the bare-string path.
raw open(symlink) reads: 'TOPSECRET_OUTSIDE_ROOT' <- the bypass
validate_path(symlink, required_root): ValueError -> BLOCKS the escape
validate_path(legit in-root): PASSED <- loads normally
The global-sandbox fallback (toolbox, named_entity, xmldocs-view) is only as tight as the allowed-roots list, which currently includes the system temp dir. Scoping every reader with required_root and removing the temp dir from the allowed roots would harden it further (separate advisory / task).
test_corpus_reader_pathsec.py — symlink escape rejected, in-root file allowed, XMLCorpusView string-fileid no AttributeError, MTEFileReader out-of-root rejected. 46 existing corpus/toolbox tests pass; all edited modules import (no circular import). pre-commit (black/isort/ruff) clean.
validate_path blocks every symlink escape variant (verified) and equals pathsec.open()'s guarantee, but does NOT block hardlinks (no symlink to resolve; tracked separately as GHSA-f794-5jv7-7672) or the validate-then-open TOCTOU race (shared by pathsec.open; needs O_NOFOLLOW/openat).
Is your project exposed to this? Stateward checks every dependency on every pull request and flags it only if your code actually reaches it.
Check my repoSources: CISA KEV (public domain), OSV.dev & GitHub Advisory Database (CC-BY-4.0), FIRST EPSS, NVD/CWE (public domain). Served live from the Stateward advisory database.