Criteria: EvalConfig & {
    evaluatorType: "criteria";
    criteria?: Criteria | Record<string, string>;
    llm?: Toolkit;
}

Configuration to load a "CriteriaEvalChain" evaluator, which prompts an LLM to determine whether the model's prediction complies with the provided criteria.

Type declaration

Param: criteria

The criteria to use for the evaluator.

Param: llm

The language model to use for the evaluator.

Returns

The configuration for the evaluator.

Example

const evalConfig = {
evaluators: [Criteria("helpfulness")],
};
@example
```ts
const evalConfig = {
evaluators: [
Criteria({
"isCompliant": "Does the submission comply with the requirements of XYZ"
})
],
};

Example

const evalConfig = {
evaluators: [{
evaluatorType: "criteria",
criteria: "helpfulness"
formatEvaluatorInputs: ...
}]
};

Example

const evalConfig = {
evaluators: [{
evaluatorType: "criteria",
criteria: { "isCompliant": "Does the submission comply with the requirements of XYZ" },
formatEvaluatorInputs: ...
}]
};

Generated using TypeDoc