Skip to content

Conversation

BrianHung
Copy link

Description

Introduces sampler abstraction for model response and LLM judge.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Test improvement

Testing

  • All existing tests pass
  • New tests have been added to cover the changes
  • Tests have been run locally with python -m pytest tests/

Test Coverage

  • Current coverage: ___%
  • Coverage after changes: ___%

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

Additional Notes

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@willccbb
Copy link
Owner

willccbb commented Aug 28, 2025

Hmm can you explain the motivation for this? I don't think this is something we'd want to add, we're intending to standardize around support for the /v1/chat/completions + /v1/completions APIs directly, and this seems to just defer the issue of matching the Chat spec down a level (which introduces a lot of surface area for nasty bugs). All notable API providers support ChatCompletions already, and if other RL trainers want to incorporate verifiers, our view is that it is the responsibility of the trainer to expose an OpenAI client (we have 2 examples of this via the verifiers GRPOTrainer + prime-rl).

Also looks like it drops support for /v1/completions, which is breaking + not desired.

@willccbb willccbb closed this Aug 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants