Search results

[BUG] [vLLM] Batch truncation bug: context length check uses first prompt instead of longest

bug

Status: Open.

#1204 In huggingface/lighteval;

· paulovsantanas opened on Apr 6, 2026

Custom task: RAIL Score responsible AI evaluation (8-dimension scoring)

Status: Open.

#1202 In huggingface/lighteval;

· SumitVermakgp opened on Apr 2, 2026

[META] Is this repo still maintained? Community PRs going unreviewed for months

Status: Open.

#1200 In huggingface/lighteval;

· kryvokhyzha opened on Mar 29, 2026

[BUG]

bug

Status: Open.

#1198 In huggingface/lighteval;

· w-ahmad1a10 opened on Mar 23, 2026

squad_v2.py : unanswerable questions are filtered out, making the task equivalent to SQuAD 1.1

Status: Open.

#1184 In huggingface/lighteval;

· Matteovanypersele opened on Mar 9, 2026

[BUG] LogProbTokenNorm crashes with IndexError in CFFormulation for belebele

bug

Status: Open.

#1170 In huggingface/lighteval;

· Rijgersberg opened on Feb 18, 2026

Add jfinqa: Japanese Financial Numerical Reasoning QA

Status: Open.

#1168 In huggingface/lighteval;

· ajtgjmdjp opened on Feb 17, 2026

[EVAL] Request support for AIME26

Status: Open.

#1167 In huggingface/lighteval;

· JackLingjie opened on Feb 13, 2026

[FT] add a long horizon "tension crash test" dimension based on a TXT S-class question pack

Status: Open.

#1164 In huggingface/lighteval;

· onestardao opened on Feb 6, 2026

[BUG] KeyError: 'default'

bug

Status: Open.

#1163 In huggingface/lighteval;

· 2niuhe opened on Jan 31, 2026

[BUG] IndexError: list index out of rang

bug

Status: Open.

#1162 In huggingface/lighteval;

· 2niuhe opened on Jan 31, 2026

Fix TypeError in aa_omniscience_prompt

Status: Open.

#1160 In huggingface/lighteval;

· pjavanrood opened on Jan 22, 2026