Machine Learning Engineer - Observability & Evaluation (Remote)

About the Role

We are seeking a talented Machine Learning Engineer - Observability and Evaluation to join our dynamic team remotely. This position offers an exciting opportunity to work in the field of AI and machine learning, focusing on monitoring and evaluating AI system performance in production environments. As a Machine Learning Engineer, you will play a crucial role in enhancing our risk and fraud detection products, which are being adopted across various industries.

What You'll Do

• Monitor, assess, and improve AI model performance in production environments.
• Debug inconsistent model behavior and edge cases to ensure reliability.
• Design and implement evaluation frameworks to validate reasoning and decision quality.
• Utilize observability tools such as Arize Phoenix and LangSmith to enhance system performance.
• Collaborate with engineering and product teams to ensure robust monitoring and reporting of AI systems.

Requirements

• Strong Python engineering skills with a focus on reliable, production-ready code.
• Experience with Large Language Models (LLMs) and Vision Language Models (VLMs).
• Hands-on experience with production observability tools like Arize and Langsmith.
• Ability to design and execute evaluation methodologies for AI systems.
• Strong problem-solving skills and attention to detail.
• Collaborative, low-ego mindset; able to work effectively in a small, remote team.

Nice to Have

• Experience in a startup or scale-up environment.
• Familiarity with additional AI frameworks and tools.

What We Offer

• Competitive salary between £80,000 - £100,000 annually.
• Equity options as part of the compensation package.
• Fully remote work opportunity, allowing for a flexible work-life balance.
• Collaborative team environment with a focus on innovation.
• Opportunities for professional growth and development in the AI field.