artefactual.scoring.entropy_methods.uncertainty_detector#

Classes

UncertaintyDetector([k])

A base class for uncertainty detection methods.
class artefactual.scoring.entropy_methods.uncertainty_detector.UncertaintyDetector(k=15)[source]#

Bases: ABC

A base class for uncertainty detection methods.

__init__(k=15)[source]#

Initialize the uncertainty detector.

Args:
k: Number of top log probabilities to consider per token.

Must be positive. Default is 15.

Raises:

ValueError: If k is not positive

abstract compute(inputs)[source]#

Compute sequence-level uncertainty scores from inputs.

Return type:

list[float]

Args:

inputs: The inputs to process (e.g. completions or model outputs).

Returns:

The computed sequence-level scores.

abstract compute_token_scores(inputs)[source]#

Compute token-level uncertainty scores from inputs.

Return type:

list[ndarray[tuple[int, ...], dtype[floating]]]

Args:

inputs: The inputs to process (e.g. completions or model outputs).

Returns:

The computed token-level scores.