Search results

[FT] Support confidence reporting and sampling

science-team

Status: Open.

#940 In huggingface/lighteval;

· clefourrier opened on Aug 28, 2025

[BUG] VLLM Code Generate hangs

bugSomething isn't workingSomething isn't working

Status: Open.

#933 In huggingface/lighteval;

· berserkr opened on Aug 22, 2025

[FT] Pass reasoning effort to <code>litellm</code> endpoint.

Status: Open.

#918 In huggingface/lighteval;

· gardarjuto opened on Aug 12, 2025

[EVAL] Add IFBench

Status: Open.

#908 In huggingface/lighteval;

· lewtun opened on Aug 8, 2025

[BUG] TransformersModel.from_model provides an incompletely initialized object compared to __init__

bugSomething isn't workingSomething isn't working

Status: Open.

#897 In huggingface/lighteval;

· huaanrui opened on Aug 5, 2025

[BUG] Automatic batch size detection causes distributed deadlock in model parallel mode

bugSomething isn't workingSomething isn't working

Status: Open.

#896 In huggingface/lighteval;

· huaanrui opened on Aug 5, 2025

[BUG] Model Parallelism is incorrectly disabled on single multi-GPU machines

bugSomething isn't workingSomething isn't working

Status: Open.

#895 In huggingface/lighteval;

· huaanrui opened on Aug 5, 2025

[BUG] TriviaQA numbers are off

bugSomething isn't workingSomething isn't working

Status: Open.

#893 In huggingface/lighteval;

· jxmorris12 opened on Aug 4, 2025

[BUG] Logs are missing some fields, truncation likely buggy

bugSomething isn't workingSomething isn't working

Status: Open.

#892 In huggingface/lighteval;

· clefourrier opened on Aug 4, 2025

[BUG] : Typo in _continuous_greedy_until causes error with stop_sequence field

bugSomething isn't workingSomething isn't working

Status: Open.

#888 In huggingface/lighteval;

· huaanrui opened on Aug 3, 2025

[INFO] Document or remove <code>logits</code> from the <code>ModelResponse</code> class

Status: Open.

#881 In huggingface/lighteval;

· clefourrier opened on Jul 25, 2025

[EVAL] Add MMLU-Redux-2

science-team

Status: Open.

#877 In huggingface/lighteval;

· lewtun opened on Jul 24, 2025