artefactual.preprocessing.vllm_parser

Functions

process_vllm_top_logprobs(outputs, iterations)	Processes log probabilities from vllm LLM.generate (or chat) outputs for a given number of iterations.
vllm_sampled_tokens_logprobs(outputs[, ...])	Extracts log probabilities of the sampled tokens from vLLM outputs.

artefactual.preprocessing.vllm_parser.process_vllm_top_logprobs(outputs, iterations)

Processes log probabilities from vllm LLM.generate (or chat) outputs for a given number of iterations.

Return type:

list[dict[int, list[float]]]

Args:

outputs (list[RequestOutput]): A list containing model output objects, each with log probability data. iterations (int): The number of iterations to process, corresponding to the number of output sequences.

Returns:

list[dict[int, list[float]]]: A list of dictionaries mapping token indices to lists of log probs for each token in the sequence.

artefactual.preprocessing.vllm_parser.vllm_sampled_tokens_logprobs(outputs, iterations=1)

Extracts log probabilities of the sampled tokens from vLLM outputs.

Return type:

list[ndarray[tuple[int, ...], dtype[TypeVar(_ScalarType_co, bound= generic, covariant=True)]]]

Args:

outputs (list[RequestOutput]): A list containing model output objects, each with log probability data. iterations (int) = 1: The number of iterations to process, corresponding to the number of output sequences.

Returns:

list[NDArray]: A list of 1D numpy arrays, each containing the log probabilities

of the sampled tokens for one sequence.