Software Engineer, RL Environments

True Talent Solutions Partners · San Francisco, California, United States

$260K to $290K a year as published by the employer

Apply on company site Save to your job tracker All Software Engineer jobs in San Francisco, CA

Applications are completed on the employer's own site.

Summary

Build datasets, evaluation rubrics, and infrastructure to improve advanced AI models by working with research teams on experiments and model failure modes.

About this role

About the RoleBuild datasets, evaluation rubrics, and infrastructure that help improve advanced AI models. You will work with research teams to run experiments, identify model failure modes, and turn training objectives into practical data and evaluation systems. What You'll DoDesign data slices that reveal meaningful model failure modes across finance, code, and enterprise workflows.

Build and refine evaluation rubrics and reward signals for RLHF and RLVR pipelines.

Model annotator behavior and run experiments to improve model capabilities.

Develop quantitative measures of dataset quality, diversity, and impact on model alignment and capabilities.

Create and manage real-world and synthetic data pipelines.

Partner with AI research teams to translate training goals into concrete data and evaluation specifications.

What We're Looking ForOne to four years of software engineering experience building reliable data pipelines, experimentation infrastructure, or evaluation systems.

Strong Python skills and experience shipping software and infrastructure.

Hands-on experience with LLM evaluations, model post-training, reward modeling, RLHF, RLVR, or related ML infrastructure.

Experience creating or evaluating datasets, including assessing quality, diversity, coverage, and downstream model performance.

Strong experimental judgment: able to form hypotheses, define metrics, design controlled experiments, and interpret noisy results.

Experience designing data collection strategies and evaluation rubrics for machine learning models.

Compensation & BenefitsBase salary: $260,000 to $290,000 annually. Visa sponsorship is available. LocationOn-site in San Francisco, California.

Skills named in this posting

and 4 more

About True Talent Solutions Partners

True Talent Solutions is a Dubai-based recruitment and HR services firm.

Similar Software Engineer, RL Environments jobs in San Francisco, CA

Job market in San Francisco, CA

See all Software & IT jobs in San Francisco, CA: every active opening in this market, with pay and who is hiring.

Read the San Francisco, CA job market report: hiring demand, pay and the skills employers ask for, updated every month.