Member of Technical Staff (Software Engineer, Enterprise Agent Evals & Efficiency)

About this role

Build realistic enterprise evaluation environments spanning connected applications, representative datasets, user roles, and permissions, with complex, multi-step workflows and reproducible system state. Own evaluation infrastructure, task suites, and automated graders that measure how well agents complete enterprise work, calibrated against human judgment and grounded in end-to-end outcomes.

Location : San Francisco

Seniority : lead

Compensation : 220,000–405,000 USD

Source checked : October 19, 2026

✦ Aptiora view

Decision-worthy role

Aptiora treats this role as a decision surface, not just a listing. This public view organizes what the posting proves, what remains uncertain, and why the role may deserve deeper review at Perplexity. Personalized fit stays inside My Aptiora.

Build my fit view → Unlock personalized fit analysis

• Personalized fit and gap analysis
• Deeper company and team intelligence
• Application strategy and talking points

Aptiora view Role Evidence Unknowns Next step Source

▣ Role summary

Build realistic enterprise evaluation environments spanning connected applications, representative datasets, user roles, and permissions, with complex, multi-step workflows and reproducible system state. Own evaluation infrastructure, task suites, and automated graders that measure how well agents complete enterprise work, calibrated against human judgment and grounded in end-to-end outcomes.

Manage engineering team

Multimodal AI (voice, vision, text)

From prototype to production

Complex technical systems

▮ Aptiora signal profile

Decision signalHigh

Technical depthHigh

Leadership scopeHigh

AI / domain intensityMedium

Career leverageMedium

☰ What you would do

• Build realistic enterprise evaluation environments spanning connected applications, representative datasets, user roles, and permissions, with complex, multi-step workflows and reproducible system state.
• Own evaluation infrastructure, task suites, and automated graders that measure how well agents complete enterprise work, calibrated against human judgment and grounded in end-to-end outcomes.
• Analyze agent trajectories and production failures, identify root causes, and ship improvements across the agent harness, tools, and skills that measurably improve task success and reliability.
• Develop reusable skills and improve tool descriptions, schemas, and result handling so agents can select the right tools, execute workflows correctly, and recover from errors with fewer unnecessary calls.
• Build and evaluate model-selection and routing strategies that match each task to the right model and effort level, balancing quality, latency, reliability, and cost.
• Improve agent efficiency through context management, skill loading, caching, and execution strategies that reduce wasted tokens and cost per successful task without compromising quality or safety.
• Build regression gates and launch-readiness signals for model, connector, tool, skill, and harness changes, and validate that evaluation gains translate into better production outcomes.

✦ Why this role matters

• Work on a core product area for the next era of AI.
• Bridge product ambition with research and infrastructure reality.

◔ What the employer asks for

• + years of software engineering or machine learning engineering experience, with a track record of owning complex production systems from design through deployment and operation.
• Strong system design skills, including building reliable, scalable infrastructure for experimentation, evaluation, or data-intensive applications.
• Strong proficiency in a backend language such as Python, Go, or Rust, and the ability to work effectively across a multi-language production stack.
• Hands-on experience building, evaluating, or improving LLM-based applications or agents, including tool use and multi-step workflows.
• Deep expertise in at least one of the following:
• Evaluation infrastructure, benchmark design, and automated grading.
• Agent harnesses, tool interfaces, and reusable skills for complex workflows.
• Realistic test environments or simulations spanning multiple systems, datasets, permissions, and stateful workflows.
• Model selection, routing, and inference efficiency, including context management and caching.
• Strong experimental and statistical judgment, including choosing representative tasks, defining meaningful success criteria, diagnosing failures, and distinguishing real improvements from noise.