high

CVE-2026-54234

PyPI · vllm

Summary

vLLM has Remote DoS via Invalid Recovered Token Reinjection

Severity
high
CVSS
7.5
EPSS
0.6% (p49)
CWE
CWE-20
Also known as
GHSA-8wr5-jm2h-8r4f
Published
2026-07-17
Updated
2026-07-17

Advisory details

Summary

A frontend-legal multi-request speculative workload can make vLLM produce an out-of-vocabulary recovered token equal to vocab_size, convert that value to -1 when choosing the next live token for a request, and then feed that -1 back into the next drafter input ids. On Qwen3 GPTQ this reaches the worker-side drafting / attention path and crashes the engine with a GPU device-side assert.

The same issue is reachable through the public gRPC request surface by sending a specific overlapping Generate / Abort sequence.

Impact

Affected version

Repro model

Trigger chain

  1. A legal multi-request speculative workload keeps structured-output state, speculative decoding, overlap, and request cancellation active in the same live engine.
  2. During rejection sampling, vLLM produces a recovered token equal to the model vocab_size boundary value.
  3. That recovered token appears in position 0 of the sampled speculative row for a live request. The same row also contains trailing padding entries equal to -1, but those padding entries are not the key fault by themselves.
  4. The next-token preparation step treats the position-0 recovered token as the real next token for that request and converts that out-of-vocabulary value to -1.
  5. The drafter writes that converted -1 back into the live next-step input-id row for the request.
  6. The drafting / embedding / attention path later consumes that live invalid token and the worker crashes on GPU.

Details

Simple example

The important distinction is:

In simplified form, the bad transition looks like this:

sampled speculative row:
[151936, -1, -1, -1, ...]

At this point, the trailing -1 values are only padding. The critical problem is that the first position holds 151936, which is out of vocabulary and is being treated as the request's real next token.

Then vLLM prepares the next-token buffer:

next_token_ids:
[-1, ...]

Finally, that converted -1 is written back into the live model input ids:

input_ids_after:
[-1, 0, 0, 0, ...]

The crash happens because the live next token became -1 and was later consumed by the drafting / embedding / attention path, not merely because the speculative row contained padded -1 entries.

Trigger path in code

  1. The workload is frontend-legal. The requests use normal SamplingParams features such as structured outputs, stop, bad_words, min_tokens, and streaming overlap. No malformed token-id list is required at the request boundary.
  2. In speculative decoding, the rejection sampler can generate recovered tokens when drafted tokens are rejected.
    # vllm/v1/sample/rejection_sampler.py
def sample_recovered_tokens(...):
    recovered_token_ids = torch.empty_like(draft_token_ids)
    sample_recovered_tokens_kernel[(batch_size, max_spec_len)](...)
    return recovered_token_ids
    On the verified Qwen3 run, the recovered-token trace shows recovered_token_ids[0] = 151936, which is exactly vocab_size for this checkpoint.
  3. The speculative proposer then prepares the next-token row from the sampled speculative row.
    # vllm/v1/spec_decode/eagle.py
def prepare_next_token_ids_padded(...):
    ...
    eagle_prepare_next_token_padded_kernel[grid](
        sampled_token_ids,
        discard_request_mask,
        backup_tokens_gpu,
        next_token_ids,
        valid_sampled_tokens_count,
        gpu_input_batch.vocab_size,
        ...
    )
    return next_token_ids, valid_sampled_tokens_count
    In the verified trace, this step receives a sampled row beginning with 151936, followed by -1 padding. The important point is that 151936 occupies the first live token position for the request. This step then produces next_token_ids[0] = -1, meaning the live next token for the request has been converted to -1.
  4. The drafter then rotates the draft input ids and inserts those next_token_ids back into the live input-id buffer.
    # vllm/v1/spec_decode/eagle.py
def set_inputs_first_pass(...):
    ...
    self.input_ids[token_indices_to_sample] = next_token_ids
    In the verified trace, this produces input_ids_after[0] = -1.
  5. The model-side embed path later consumes those input ids.
    # vllm/model_executor/models/qwen2.py
def embed_input_ids(self, input_ids: torch.Tensor) -> torch.Tensor:
    return self.embed_tokens(input_ids)
    In the verified trace, this is the first point where the converted -1 becomes visible as a real model input. The bug is not merely that the sampled speculative row contained padding -1; the bug is that the live next token for the request became -1 and was written back into input ids.
  6. After that point, the visible sink depends on timing and backend state. On the attached Qwen3 reproducer, the engine commonly dies later in the drafting / attention path with CUDA error: device-side assert triggered, for example under flash_attn_varlen_func(...).

Local script breakdown

repro_g4_recovered_minus1_local.py is a standalone local reproducer.

gRPC script breakdown

repro_g4_recovered_minus1_grpc.py is a standalone public gRPC reproducer.

Observed result

Local repro typically ends with:

References

Related advisories

Is your project exposed to this? Stateward checks every dependency on every pull request and flags it only if your code actually reaches it.

Check my repo

Summarize with AI

ChatGPTClaudePerplexity

Sources: CISA KEV (public domain), OSV.dev & GitHub Advisory Database (CC-BY-4.0), FIRST EPSS, NVD/CWE (public domain). Served live from the Stateward advisory database.