artefactual.scoring.entropy_methods.wepr

Classes

WEPR(pretrained_model_name_or_path)

Computes Weighted Entropy Production Rate (WEPR) from model log probabilities.

class artefactual.scoring.entropy_methods.wepr.WEPR(pretrained_model_name_or_path)

Bases: UncertaintyDetector

Computes Weighted Entropy Production Rate (WEPR) from model log probabilities. WEPR extends EPR by applying learned weights to the entropy contributions based on their ranks. It computes both mean-weighted and max-weighted contributions to produce a sequence-level uncertainty score. Token-level WEPR scores are also provided. You can parse raw model outputs using the parse_model_outputs method from artefactual.preprocessing.

__init__(pretrained_model_name_or_path)

Initialize the WEPR scorer with weights loaded from the specified source.

Args:

pretrained_model_name_or_path: Either a built-in model name or a local file path to load weights from.

compute(parsed_logprobs)

Compute WEPR-based uncertainty scores from parsed log probabilities. You can parse raw model outputs using the parse_model_outputs method from artefactual.preprocessing.

Return type:

list[float]

Args:

parsed_logprobs: Parsed log probabilities.

Returns:

List of sequence-level WEPR scores.

compute_token_scores(parsed_logprobs)

Compute token-level WEPR scores from parsed logprobs. You can parse raw model outputs using the parse_model_outputs method from artefactual.preprocessing.

Return type:

list[ndarray[tuple[int, ...], dtype[floating]]]

Args:

parsed_logprobs: Parsed log probabilities.

Returns:

List of token-level WEPR scores (numpy arrays).