Software Engineer, RL Environments

About this role

About the RoleDesign datasets, evaluation rubrics, and reward signals that help improve frontier AI models. You will work with research teams to test data-collection strategies, investigate model failure modes, and deliver evaluations and training data that inform model development. What You'll DoDesign data slices that expose meaningful model failure modes across areas such as finance, software, and enterprise workflows.

Build and refine evaluation rubrics and reward signals for RLHF and RLVR training pipelines.

Model annotator behavior and run experiments to improve model capabilities.

Develop quantitative methods for assessing dataset quality, diversity, and downstream impact.

Create and manage real-world and synthetic data pipelines.

Translate research objectives into concrete data and evaluation specifications in collaboration with research teams.

What We're Looking ForOne to four years of hands-on experience designing machine learning data-collection strategies and evaluation rubrics.

Strong familiarity with LLM training and evaluation methods, including experience creating data slices to reveal model failure modes.

Ability to design lightweight experiments, move quickly, and extract useful insights from messy results.

Comfort working across domains such as finance, software engineering, and policy.

A practical, builder-oriented approach and genuine interest in how data structure and quality shape model behavior.

Compensation & BenefitsBase salary: $260,000 to $290,000 annually. Visa sponsorship is available. LocationOn-site in San Francisco, California.