Job Description
Location: Global, fully remote
Job Type: Contractor
Time Commitment: 15+ hours per week
Schedule: Flexible
We are looking for Machine Learning Engineers with prior professional AI-training or model-evaluation experience to support a project focused on improving advanced AI systems.
You will use your machine learning expertise to create challenging technical tasks, evaluate model-generated work, identify failures, and establish objective standards for correct solutions. This role is intended for candidates who have already completed substantive AI-training work.
Required Skills
Ml engineering
Ai training
Programming skills
What You’ll Work On
Create realistic machine learning engineering tasks that challenge advanced AI models.
Develop task instructions, reproducible environments, reference solutions, and evaluation criteria.
Evaluate model-generated code, experiments, analyses, and technical deliverables.
Identify issues involving data leakage, incorrect validation, unstable training, performance, or reproducibility.
Required Qualifications
Prior paid experience training, evaluating, benchmarking, or improving AI systems.
Strong practical proficiency with Python.
Hands-on experience using PyTorch, JAX, TensorFlow, scikit-learn, XGBoost, LightGBM, or comparable ML tools.
Public Proof of Skill
Strong applicants should provide evidence such as:
GitHub repositories or public model implementations
Kaggle, DrivenData, AIcrowd, or similar competition results
Hugging Face models or datasets
Research publications or technical reports
Open-source ML contributions
Public benchmarks, experiments, or technical projects
Preferred Qualifications
Experience creating AI-training tasks, reference answers, rubrics, graders, or benchmark problems.
Experience reviewing other AI trainers’ technical submissions.
Kaggle Expert, Master, or Grandmaster status, or strong results in another technical competition.
Publications or open-source contributions in machine learning.
Prior experience with long-horizon or agentic AI evaluations.
Availability to begin immediately.
Application Process
Submit your application, AI-training experience, and public technical evidence.
Complete an AI interview containing technical and integrity checks.
Complete a hiring-manager review.
If selected, complete project onboarding and begin your first assigned task.
Compensation
Compensation is output-based. Experts are paid for tasks that meet project specifications and quality requirements. Task availability and completion time may vary.
Summary
Create realistic machine learning engineering tasks, including instructions, reproducible environments, reference solutions, and evaluation criteria. Evaluate model-generated code and technical deliverables, identifying issues like data leakage or reproducibility. Requires prior paid AI-training or evaluation experience, strong Python proficiency, and hands-on experience with ML tools.
Responsibilities
Create realistic machine learning engineering tasks; develop instructions, reproducible environments, reference solutions, and evaluation criteria; evaluate model-generated code, experiments, analyses, and technical deliverables; identify issues involving data leakage, incorrect validation, unstable training, performance, or reproducibility
Qualifications
Prior paid experience training, evaluating, benchmarking, or improving AI systems; strong practical proficiency with Python; hands-on experience with PyTorch, JAX, TensorFlow, scikit-learn, XGBoost, LightGBM, or comparable ML tools; public proof of skill
Experience requirements
Prior paid experience training, evaluating, benchmarking, or improving AI systems
Interview process
Submit application and technical evidence; complete AI interview with technical and integrity checks; hiring-manager review; project onboarding and first assigned task
How to apply
Submit application, AI-training experience, and public technical evidence