Applications are completed on the employer's own site.
Summary
Design, build, and optimize production LLM inference services for GenAI features while collaborating with ML, platform, and product teams.
About this role
Job Overview
Job ID:
J55232
Job Title:
Software Engineer (Gen AI Solutions)
Location:
Austin, TX
Duration:
14 Months + Extension
Domain Exposure:
Pharmaceuticals
Client:
To Be Discussed Later
Employment Type:
W-2, 1099, C2C
Work Authorization:
US Citizen, Green Card, H-1B, OPT-EAD, CPT, Other
View All Jobs
Similar Jobs
Embedded Software Engineer - Mountain View, CA
Software Lead Engineer - Plano, TX
Software Engineer - Irving, TX
Software Engineer - Plano, TX
Embedded Software Developer - Dallas, TX
Job Title: Software Engineer (Gen AI Solutions)
Location: Austin, TX
Duration: 14 Months + Extension
Bill Rate: $76/hour (C2C)
Job Type: W-2, 1099, C2C
Client: To Be Discussed Later
Work Authorization: US Citizen, Green Card, OPT-EAD, CPT, H-1B, H4-EAD, Other
Job Description :
We need an experienced software engineer to help design, build, and optimize production LLM inference services that power GenAI features for internal and customer-facing applications. The engineer will work with our ML, platform, and product teams to make model serving faster, cheaper, and more reliable at scale.
Scope of work :
Design, deploy, and maintain scalable inference services for open-weight and hosted LLMs on Kubernetes-based infrastructure.
Profile and reduce end-to-end latency (time to first token, tokens per second) and cost per request.
Apply model and serving optimizations, then benchmark the trade-offs between quality, latency, and throughput.
Build and maintain integrations with external model APIs and internal orchestration frameworks, including routing, fallback, retries, and rate-limit handling.
Add observability across the request path, covering logging, metrics, tracing, and dashboards for request volume, downstream call ratios, errors, and token usage.
Set up evaluation and regression pipelines so model or serving changes don't degrade output quality.
Do capacity planning and autoscaling, and track GPU utilization and cost.
Write technical design docs and runbooks, and hand knowledge over to the internal team.
Expected deliverables :
Production-ready inference service(s) with CI/CD and infrastructure as code
Latency and cost benchmark reports, measured before and after optimization
Monitoring dashboards and alerting
Architecture docs, runbooks, and handover sessions
Required :
Software engineering experience, including 2+ years on ML/LLM systems in production
Strong Python, plus working knowledge of Go, C++, or Rust for performance-critical paths
Hands-on experience with LLM serving frameworks: vLLM, TensorRT-LLM, TGI, SGLang, or Triton Inference Server