This exciting Seed startup, was founded out of a Stanford LLM evaluation research project. They are building a first of its kind LLM Benchmark platforms for real world business tasks. They aim to become the industry standard benchmark tool for reviewing + auditing LLM applications in high value industries.
After securing a $5m Seed round, they already have paying customers and are now actively looking for a Head of Machine Learning to develop novel evaluation techniques.
The Head of Machine Learning is responsible for furthering our public benchmarks. The areas you will investigate are (1) methods for reliably/accurately evaluating generated text at scale (2) training an expert reward function and judge (3) generating synthetic evaluation datasets. This domain is wide open with many challenging problems that are crucial to solve correctly.
It’s worth highlighting that in this role you are a founding team member, not an employee. This means you have ownership in the company and product direction.
