Senior Software Engineer - LLM Evaluation & Training Datasets (Remote)

About the Role

We are seeking a Senior Software Engineer - LLM Evaluation and Training Datasets to join our innovative team at Turing. This remote position offers an exciting opportunity to work on groundbreaking projects related to large language model (LLM) evaluation and training datasets. You will directly contribute to the development of datasets that enable LLMs to tackle realistic software engineering challenges.

What You'll Do

• Analyze and triage issues across trending open-source repositories on GitHub, identifying areas for improvement and challenges for LLMs.
• Set up and configure development environments, including Dockerization and environment management, to facilitate efficient testing and evaluation.
• Assess and improve unit test coverage and quality within repositories to ensure robustness and reliability.
• Modify, run, and debug codebases locally to evaluate the performance of LLMs in bug-fixing and code understanding tasks.
• Collaborate with research teams to design datasets and identify repositories and issues that present meaningful challenges for LLMs.
• Potentially lead a team of junior engineers, providing guidance and mentorship to ensure project milestones are met.

Requirements

• Minimum of three years of professional experience in software engineering, with demonstrated expertise in JavaScript and TypeScript.
• Proficiency in version control systems such as Git and containerization tools like Docker.
• Experience setting up and managing software pipelines is essential.
• Comfortable navigating complex codebases, running, modifying, and testing real-world projects locally.
• Experience contributing to or evaluating open-source projects is highly desirable.
• Strong problem-solving skills, attention to detail, and the ability to work collaboratively in a remote environment are important qualities for success in this role.
• Prior experience with LLM research or automation tools is a plus, but not mandatory.

Nice to Have

• Experience with machine learning frameworks.
• Familiarity with cloud platforms such as AWS or Azure.
• Knowledge of additional programming languages.

What We Offer

• Fully remote work environment, providing flexibility and work-life balance.
• Engagement with cutting-edge AI projects alongside leading LLM companies.
• Supportive environment that encourages innovation and continuous professional development.
• Exposure to a diverse set of projects, collaborating with top-tier professionals worldwide.
• Opportunity to shape the future of AI-driven software engineering solutions.
• Tailored workspace to foster productivity and comfort.