LoreonLabsPlatform

Overview

Intelligence

Markets
Builders
Research
Narratives
Ecosystems
Launchpads

Discover

Search
Sources

Python

lm-evaluation-harness

A framework for few-shot evaluation of autoregressive language models.

PythonEmerging

Stars

1

Forks

1

Contributors

8

Last push

1mo ago

Recent commits

Latest commits.

Fix mmlu_pro fewshot answers leaking into user role under chat template (#3693)
ad5ade4‌Kayra Arai Ozturk1mo ago
Fix GPQA preprocessing: remove bracket-stripping regex that corrupts answer text (#3691)
0fb3920Robby Sneiderman1mo ago
Filter out free-form examples from BigBench multiple-choice tasks (#3702)
12c105dChessing2341mo ago
Fix RACE doc_to_text keeping blank marker and dropping the question body (#3716)
5eb3040Chessing2341mo ago
feat: Add [ LiteLLM AI gateway ] as model backend (#3721)
853ff58

Aarish Alam

2mo ago

fix: don't pass task stop sequences to vLLM for reasoning models (#3700)

79f4430jwmacd2mo ago

Fix zeno_visualize tasks intersection result being discarded (#3739)

f0ba463Chessing2342mo ago

fix(model_comparator): fix ImportError from scipy.stats.norm import (#3742)

5959489Chessing2342mo ago

Top contributors

Builders behind this project.

lintangsutawika

haileyschoelkopf