Could vf-eval forward matching env args to load_environment?

[ob1-s]

hey,

right now if my load_environment function needs to know something like num_examples (say i want to pre-process just a slice of the dataset, or the env needs some other vf-eval arg for some reason), i have to pass it twice.

example:

uv run vf-eval my-env -n 10 -a '{"num_examples": 10}'

idea / question

i was wondering if it could make sense for vf-eval to auto-pass its own args to load_environment when they match the function signature (and only when they match).

so in this case, since my function takes num_examples, vf-eval could forward -n automatically. then i would only need:

uv run vf-eval my-env -n 10

the same could apply to args like model, temperature, etc.

trade-offs / concerns

possible naming collisions (e.g. model could mean different things in env vs eval), it could break some existing envs. also makes the boundary between env config and eval params fuzzier.

curious if the current explicit design is intentional to keep that separation. would love to hear your reasoning.

---

i thought of this while building a multi-turn env that pre-generates the initial user messages, and i noticed i have to repeat params like this so it doesn't take too long.

I'd be glad to open a PR if this seems reasonable

thanks!

[ob1-s]

if anyone wonders, i figured out a better way to solve my original setup.

instead of passing num_examples to my env, i added a cache to the pre-generation step inside load_environment.

this makes the generation a one-time cost. after the first run, it's instant, so the env doesn't need to know how many examples are being run anymore. it completely removes the need for the num_examples env_arg. after all, i'm not evaluating the user model. caching the initial turn decreases any possible friction on that end.

would still be nice to know everyone's thoughts on forward matching env args though.