Project Overview:

Join a growing community of professionals advancing the next wave of AI. As an AI Trainer, you’ll play a hands-on role by analyzing and providing feedback on data to improve LLM performance, helping ensure that the next generation of AI technology is accurate and trustworthy.

We are seeking a skilled Senior Software Engineer to work as a project consultant in our AI Labor Marketplace. This is not a full-time employment position — you will be engaged as an expert project consultant on a contract basis.

Location: U.S.-based experts only

Engagement: Part-time, project-based expert evaluation work

Work Type: Remote

Project Summary:

Consultants will design realistic, senior-level software engineering tasks, evaluate GPT-5.2 model reasoning using structured rubrics, and produce gold-standard responses that reflect real-world engineering best practices.

Consultant Engagement Terms:

This is a project-based consultant role. Consultants will be paid on a per-project basis; hourly rates are estimates based on anticipated completion time. Consultants control their own schedule, provide their own tools, and may simultaneously provide services to other vendors/employers (subject to those vendors’ allowances).

Responsibilities:

Contributors will:

• Create realistic, complex software engineering prompts based on professional experience
• Design clear, criteria-based evaluation rubrics
• Evaluate GPT-5.2 model outputs against defined success criteria
• Author high-quality gold responses demonstrating senior-level reasoning
• Ensure all work is synthetic, non-proprietary, and objectively evaluable

Expected Outcomes:

• High-quality prompts that reflect real engineering challenges
• Clear, defensible evaluation rubrics
• Consistent, well-reasoned model evaluations
• Gold-standard responses suitable for model training and benchmarking

Qualifications:

• 10+ years of professional software engineering experience
• Senior, Staff, Principal, Tech Lead, or Architect-level background
• Strong CS fundamentals and production system experience
• Experience at top SaaS companies or scaled startups preferred
• Prior data annotation or model evaluation experience is a strong plus

• Mid-Senior level
• Contract
• Engineering and Information Technology
• Software Development

Similar jobs