vLLM: Completion prompt lists fan out into unbounded engine requests

Summary

The /v1/completions request model accepts prompt as a list of text prompts or a list of token-id prompts without any outer prompt-count bound. The serving path turns each element into a separate engine input, creates one engine generator per element, merges all generators, and allocates a response slot per prompt. An authenticated API client can therefore turn one request into an attacker-chosen number of backend subrequests before any aggregate request-count budget is enforced.

Technical Details

CompletionRequest.prompt allows both list-shaped prompt inputs and scalar prompts:

# vllm/entrypoints/openai/completion/protocol.py
prompt: (
    list[Annotated[int, Field(ge=0)]]
    | list[list[Annotated[int, Field(ge=0)]]]
    | str
    | list[str]
    | None
) = None

The validator only requires some prompt-like input to be present:

def validate_prompt_and_prompt_embeds(cls, data):
    prompt = data.get("prompt")
    prompt_embeds = data.get("prompt_embeds")
    ...
    if prompt_is_empty and embeds_is_empty:
        raise VLLMValidationError(...)

The renderer then expands list-shaped prompts as a sequence. prompt_to_seq() wraps a scalar string or a single token-id list, but returns a list[str] or list[list[int]] unchanged:

# vllm/renderers/inputs/preprocess.py
def prompt_to_seq(prompt_or_prompts):
    if isinstance(prompt_or_prompts, (dict, str, bytes)) or (
        len(prompt_or_prompts) > 0 and is_list_of(prompt_or_prompts, int)
    ):
        return [prompt_or_prompts]

    return prompt_or_prompts

OnlineRenderer.preprocess_completion() appends that whole sequence, and the renderer processes every element:

# vllm/renderers/online_renderer.py
prompts = list[SingletonPrompt | bytes]()
if prompt_input is not None:
    prompts.extend(prompt_to_seq(prompt_input))
...
parsed_prompts = [
    prompt if isinstance(prompt, bytes) else parse_model_prompt(model_config, prompt)
    for prompt in prompts
]
return await renderer.render_cmpl_async(parsed_prompts, tok_params, ...)

Finally, completion serving creates one backend generator and one response slot per rendered prompt:

# vllm/entrypoints/openai/completion/serving.py
generators: list[AsyncGenerator[RequestOutput, None]] = []
for i, engine_input in enumerate(engine_inputs):
    ...
    generator = self.engine_client.generate(...)
    generators.append(generator)

result_generator = merge_async_iterators(*generators)
num_prompts = len(engine_inputs)
...
final_res_batch: list[RequestOutput | None] = [None] * num_prompts

The violated invariant is that one HTTP request should have a bounded backend request count. Current code enforces per-prompt token and sampling limits, but not the number of prompts in the outer completion request.

PoV

A minimal oversized request keeps normal generation parameters small but supplies a large outer prompt list:

{
  "model": "served-model",
  "prompt": ["x", "x", "x"],
  "max_tokens": 1,
  "n": 1
}

Scaling the prompt array to tens or hundreds of thousands of short entries makes the server allocate, preprocess, schedule, merge, and buffer one subrequest per entry. The same applies to token-id prompt lists:

{
  "model": "served-model",
  "prompt": [[1], [1], [1]],
  "max_tokens": 1,
  "n": 1
}

The intended negative control is a scalar prompt:

{
  "model": "served-model",
  "prompt": "x",
  "max_tokens": 1,
  "n": 1
}

The scalar string is wrapped as one prompt; the list form is not bounded and fans out by list length.

Impact

An authenticated API client can make one /v1/completions request consume CPU, memory, async task scheduling, engine request slots, and response buffering proportional to an attacker-chosen outer prompt list. This can starve or disrupt other tenants sharing the same vLLM server. The report does not claim unauthenticated access, confidentiality impact, integrity impact, code execution, or impact where /v1/completions is not reachable by untrusted or semi-trusted clients.

Suggested Fix

Reject oversized prompt lists before renderer preprocessing. Add an outer prompt-count limit to CompletionRequest.prompt when the prompt is list[str] or list[list[int]], and consider making the limit configurable in the same style as the batch-chat and sampling-list bounds. The check should run before OnlineRenderer.preprocess_completion() expands the prompt sequence, so oversized requests do not allocate parsed prompt lists, async render/tokenization tasks, engine generators, or response result slots.

Regression coverage should include a scalar prompt, a bounded prompt list, an oversized list[str], and an oversized list[list[int]]. The oversized requests should fail with a controlled validation error before any backend generator is created.

Affected Package/Versions

Package ecosystem: pip

Package name: vllm

Affected range confirmed by source proof: `>=0.19.0, =0.19.0,