Location: Paris, IDF, France / Paris, Île-de-France, France
Job Type: Full-time
Salary: €60K - €100K EUR
Equity: 0.30% - 1.00%
Experience: Any (new grads ok)
Location: Paris, France. We value working in person, and we stay flexible. You may also get the chance to join us on occasional trips to San Francisco.
Language: English is required. You do not need to speak French.
The role
You own two things at Armature: the research we publish and the evals system behind our results. You work on both hand in hand with our forward deployed engineer and our product engineer.
1. The research we publish. Our studies on how coding agents think and choose are how most customers find us. You decide what we study next, you run the studies behind our public leaderboards, and you publish the findings under your name.
2. The evals system. Everything we tell a customer depends on our evals, so they have to be excellent. You research how to design them to get the best results, and you build what works into the product.
This field is young, so many of your questions won’t have an answer yet, and what you find will shape how software gets rebuilt for agents.
What you will do
Published research
Design and run studies on how coding agents choose tools, how models and harnesses differ, and how agents use software.
Find the next questions worth asking, and new ways to look at the data we already have.
Automate the research runs and the analysis, so each study takes less time than the one before.
Write up your findings so developers want to read and share them.
Evals system
Test the latest research on evals against our own data, and keep what improves our results.
Turn the questions our customers ask into better evals.
Ship what works into the product with our product engineer.
Who we look for
You’re an engineer from a top school, and if you’re a new grad, your internships show that you can build things.
You build through coding agents every day: you split the work, guide the agents and understand what they produce, and this is the skill we value most.
You can go from raw data to a finding on your own.
You come with your own ideas, and you like to test them.
You have high agency: you move forward without waiting for instructions.
Not a fit if
You want pure research and nothing to ship.
You need a fixed scope before you start.
You want to train models more than you want to study how they behave.
You prefer to write every line yourself over working through coding agents.
Summary
Owns published research on coding agents and the evals system behind results: designs and runs studies, automates analysis, writes findings, tests evals, and ships improvements into the product. Requires an engineering background, daily use of coding agents, independent data-to-finding work, and high agency; new grads welcome.
Responsibilities
Design and run studies on coding agents; automate research runs and analysis; write findings; test evals research; turn customer questions into evals; ship evals into product.
Qualifications
Engineer from a top school (new grads ok); builds through coding agents daily; can go from raw data to a finding independently; high agency.
Education requirements
Engineer from a top school.
Experience requirements
Any (new grads ok).
Benefits
Equity: 0.30% - 1.00%