Staff Research Engineer - LLM Pre-Training (Remote)

About the Role

We are seeking a Staff Research Engineer (LLM Pre-Training) to join JetBrains in a fully remote capacity. In this role, you will contribute to the development of Large Language Models (LLMs) that enhance our AI capabilities across all JetBrains products. As a Staff Research Engineer, you will be at the forefront of training foundation models for coding tasks, deploying them into production environments accessible to users worldwide.

What You'll Do

• Collaborate with stakeholders to translate business requirements into technical specifications.
• Train LLMs from scratch on a large GPU cluster, ensuring high performance and scalability.
• Collect and process pre-training and fine-tuning datasets to optimize model performance.
• Support and enhance existing subsystems to improve overall functionality.
• Work independently on projects, making decisions and consulting with team members when necessary.

Requirements

• Proven experience in designing, deploying, and supporting production ML systems.
• Strong theoretical knowledge in NLP and transformer-based methodologies.
• Proficiency in modern deep learning frameworks, particularly PyTorch, and familiarity with NLP libraries.
• Experience in distributed training of multi-billion parameter models.
• Excellent attention to detail and strong communication skills.

Nice to Have

• Experience with LLM inference frameworks such as vLLM, DeepSpeed, or TensorRT.
• Familiarity with LLM alignment techniques including RLHF/RLAIF.
• Knowledge of MLOps tools and practices, including CI/CD for ML.
• Experience with Kubernetes (K8s) and Kubeflow.
• Scientific publications in the NLP domain.

What We Offer

• A competitive salary range of $140K to $200K, commensurate with experience.
• Fully remote work environment with flexible hours.
• Access to cutting-edge technology and a cluster of hundreds of NVIDIA GPUs for training.
• Collaborative and inclusive workplace culture that values diversity.
• Opportunities for professional development and continuous learning.