Software Engineer – Code QA for AI Evaluation (Remote)

About the Role

We are seeking a talented Software Engineer – Code QA for AI Evaluation to join our remote team. In this role, you will craft realistic developer prompts and evaluate AI outputs, ensuring they reflect real-world developer workflows. This position offers the flexibility of remote work and a commitment of 10–40 hours per week.

What You'll Do

• Develop chat-style developer prompts focusing on code review, debugging, and error diagnosis.
• Source and adapt real pull requests to create authentic evaluation tasks.
• Write clear, technically accurate model responses that demonstrate strong reasoning.
• Evaluate AI outputs based on logic, clarity, and technical judgment.
• Collaborate asynchronously with research teams to refine task quality.

Requirements

• Bachelor’s degree in Software Engineering, Computer Science, or a related field.
• Strong experience in software engineering or technical research.
• Proficiency in languages such as Python, JavaScript, Java, or C++.
• Hands-on experience with debugging, testing, and validating code.
• Strong technical writing skills with high attention to detail.
• Ability to work independently in a fully remote environment.

Nice to Have

• Experience in educational content development.
• Familiarity with AI evaluation metrics and methodologies.

What We Offer

• Competitive hourly compensation ranging from $70 to $120.
• Flexible working hours with a remote-first approach.
• Opportunity to work on innovative AI projects.
• Collaborative team environment with asynchronous communication.
• Professional development opportunities in AI and software engineering.