AI/ML Career
10,038 roles at 2,899 companies

Founding Engineer (AI + Backend) - Remote, India

Location: India / Remote (India)

Job Type: Full-time

Salary: ₹4M - ₹4.5M INR

Equity: 0.50% - 0.80%

Experience: Any (new grads ok)

Skills: Python, Machine Learning

About Openbenchmarks

Agents are becoming first-class users and consumers of the internet. They research, evaluate, compare tools and increasingly make build-versus-buy decisions on behalf of people. Every company will need to get their products picked and used by agents.

Agents increasingly prefer open, independent and grounded benchmarks to make decisions.

Openbenchmarks is the evaluation infrastructure for agents: domain-specific, reproducible evaluations that help agents pick tools with confidence.

Our mission is to be the trusted evaluation layer for agents.

Founders previously led AI research and infra teams at Oracle and Appfolio. We started Openbenchmarks as an output of our research into model behavior and how agents actually choose between different tools.

We work with the fastest-growing AI-first companies like Parallel, Firecrawl, Telnyx, TinyFish and more.

About the role

We're a small team of researchers and engineers across SF and India. This role is remote, based in India. You'll build the benchmarks and the systems that run them, and you'll own large parts of both from day one.

Here are the broad themes you'll work on:

Benchmark and eval design - Design domain-specific benchmarks from scratch: what to measure, how to ground it, and what makes a benchmark that agents keep trusting and picking from.

Large-scale autonomous eval systems - Infrastructure that runs without a human in the loop: vendor APIs at scale, LLM-as-judge pipelines, scoring and metric computation, and drift detection as models and products change underneath.

Research on model behavior - Why agents choose what they choose, and what holds up under their scrutiny. Next to this, you'll work on synthetic data generation and problems like self-improving code.

You'll build benchmarks and evals that the fastest-growing AI-first companies pay close attention to, and run continuous evals their engineering teams rely on.

If you want to do your life's work, reach out.

Summary

Build benchmarks and evaluation systems for AI agents, owning large parts of design and infrastructure from day one. Open to new grads with Python and machine learning skills to tackle research problems around how agents choose tools.

Responsibilities

Design domain-specific benchmarks from scratch; build large-scale autonomous eval systems with vendor APIs, LLM-as-judge pipelines, scoring, and drift detection; research model behavior and agent decision-making; work on synthetic data generation and self-improving code.

Experience requirements

Any (new grads ok)