Show HN: jevals – replacing LLM judges with typed Jev decisions github.com 44 points by gbayomi 2 days ago
jacksun788 14 hours ago Curious — how's this compare to laya? gbayomi 8 hours ago It can also run with laya. The idea is to have an easy way to run common evals with decision models and easily switch the “backend” from jev/kev/laya etc
gbayomi 8 hours ago It can also run with laya. The idea is to have an easy way to run common evals with decision models and easily switch the “backend” from jev/kev/laya etc
adityamishra241 16 hours ago Interesting idea. How do you handle cases where the decision depends on context that isn't captured by the typed decision type? gbayomi 8 hours ago It’s very dependent on the underlying model right now, but it performs well for the current versions of jev/keV/laya!
gbayomi 8 hours ago It’s very dependent on the underlying model right now, but it performs well for the current versions of jev/keV/laya!
This is going to be a hot use case.
Agreed
Curious — how's this compare to laya?
It can also run with laya. The idea is to have an easy way to run common evals with decision models and easily switch the “backend” from jev/kev/laya etc
Interesting idea. How do you handle cases where the decision depends on context that isn't captured by the typed decision type?
It’s very dependent on the underlying model right now, but it performs well for the current versions of jev/keV/laya!