This role is for one of our clients
Compensation: $60-$90 per hour
Join a pioneering AI initiative focused on developing the next generation of evaluation benchmarks for frontier AI models. We are seeking experienced Data Scientists and Quantitative Analysts to bring real-world analytical rigor to AI evaluation by designing sophisticated benchmark tasks based on practical data science workflows.
In this role, you will create complex, multi-step analytical challenges that mirror real research and business scenarios—from cleaning datasets and comparing statistical methods to interpreting results and presenting actionable insights. Working closely with AI researchers, you'll help identify where advanced AI models succeed, where they fail, and how evaluation benchmarks can better measure analytical reasoning.
This is a fully remote, full-time engagement requiring approximately 35 hours per week.
Requirements
Key Responsibilities
- Design realistic data analysis challenges inspired by real-world research and analytical workflows, including data preparation, statistical modeling, hypothesis testing, and comparative analysis.
- Develop reproducible reference analyses using Jupyter Notebooks or Google Colab, documenting methodologies and findings with clarity.
- Create benchmark tasks that require objective comparisons between analytical techniques, supported by statistical validation and evidence-based recommendations.
- Evaluate AI-generated analyses for correctness, statistical validity, reasoning quality, and interpretation accuracy.
- Identify analytical errors, flawed assumptions, and reasoning gaps that experienced data professionals would immediately recognize.
- Collaborate with AI researchers and fellow subject matter experts to improve benchmark quality, consistency, and analytical rigor.
Required Qualifications
- Master's degree, PhD, or equivalent practical experience in Data Science, Statistics, Mathematics, Economics, Operations Research, or another quantitative STEM discipline.
- Minimum 1 year of professional experience in research, research engineering, quantitative analysis, data science, or another data-intensive analytical role.
- Strong hands-on experience with data cleaning, exploratory data analysis, statistical testing, correlation analysis, predictive modeling, and interpretation of analytical results.
- Proficiency with Jupyter Notebooks or Google Colab for building reproducible analytical workflows.
- Strong programming skills in Python, including experience with libraries such as pandas, NumPy, SciPy, scikit-learn, or similar analytical frameworks.
- Working knowledge of Git and collaborative software development practices.
- Excellent written communication skills with the ability to present analytical findings clearly to both technical and non-technical audiences.
- Experience with AI evaluation, benchmark development, AI model assessment, or task authoring is preferred.
- Exceptional analytical thinking, creativity, attention to detail, and the ability to solve complex, open-ended problems independently.
- Ability to commit approximately 35 hours per week on a consistent basis.
Preferred Qualifications
- Experience designing reproducible research workflows or analytical evaluation frameworks.
- Familiarity with machine learning, large language models, or AI-assisted data analysis.
- Background in benchmark design, statistical modeling, or quantitative research methodology.
- Experience reviewing analytical work, mentoring analysts, or contributing to research publications.
Why Join
- Help shape the future of AI by improving how advanced models are evaluated on real-world analytical reasoning.
- Collaborate with leading AI researchers developing frontier evaluation benchmarks.
- Apply your expertise in statistics and data science to advance AI reliability and decision-making capabilities.
- Contribute directly to benchmark development that influences the evolution of next-generation AI systems.
- Enjoy the flexibility of a fully remote engagement while working on impactful AI research initiatives.
Equal Opportunity
We are committed to fostering an inclusive and diverse environment where all qualified applicants receive equal consideration. Reasonable accommodations are available throughout the application and engagement process.
Contract & Engagement Details
- Independent contractor engagement.
- Fully remote with flexible working hours.
- Expected commitment of approximately 35 hours per week.
- Project duration may be extended, shortened, or concluded based on project requirements and individual performance.
- Work does not require access to confidential or proprietary information from any current or former employer.
- Payments are issued weekly based on approved work completed.
- At this time, we are unable to support H1-B or STEM OPT candidates.
Germany