Data Annotation Specialist - AI Model Alignment (Remote)

About the Role

We're hiring a Data Annotation Specialist for AI Model Alignment to join our innovative team at AGI, Inc. This remote position offers you the chance to shape the future of AI by evaluating and annotating agent behavior across various platforms. Your insights will directly influence how our AI models learn and improve, making this role crucial for the development of trustworthy AI agents.

What You'll Do

• Evaluate agent trajectories: Review and annotate agent behavior across desktop, web, and mobile environments — labeling actions, identifying failure modes, and assessing task completion quality.
• Judge decision quality: Evaluate agent decision-making at each step: Was this the right action? Was it efficient? Did it align with user intent?
• Provide structured feedback: Deliver qualitative feedback on agent behavior, including edge cases, reasoning errors, and UI misinterpretations.
• Collaborate with researchers: Work directly with ML researchers to refine annotation guidelines, surface patterns in model failures, and inform training priorities.
• Build benchmark datasets: Contribute to high-quality datasets that drive measurable performance improvements.

Requirements

• 1+ years of experience annotating agent-based trajectories, reinforcement learning data, or similar sequential decision-making tasks.
• Strong understanding of what constitutes model quality — you can distinguish between a correct action, a suboptimal action, and a hallucinated one.
• Familiarity with model alignment concepts: helpfulness, harmlessness, honesty, and how annotation choices influence model behavior.
• Track record of working alongside researchers — you're comfortable with ambiguity, can propose annotation schema improvements, and understand how your work feeds into training pipelines.
• Excellent attention to detail and ability to maintain consistency across high volumes of data.
• Clear written communication skills for documenting edge cases and providing actionable feedback.

Nice to Have

• Experience with machine learning frameworks or tools.
• Knowledge of data privacy and ethical considerations in AI.
• Familiarity with programming languages such as Python or R.

What We Offer

• 🏥 Competitive company-sponsored medical, dental, and vision insurance.
• ✈️ Top-tier relocation and immigration support.
• 📈 Opportunities for professional growth and development.
• 🌍 A chance to work with a team of elite founders and AI researchers.
• 💻 Flexible work environment with a focus on collaboration.

This Data Annotation Specialist - AI Model Alignment role is an excellent opportunity for those looking to advance their career in AI and machine learning. Join us in building the future of AI that people can trust!