Senior Data Engineer - SQL & Machine Learning Integration

About the Role

We are looking for a Senior Data Engineer to join our innovative team at Heritage Auctions. This remote position offers an exciting opportunity to work with cutting-edge technologies and contribute to the development of data solutions that enhance our auction services.

What You'll Do

• Develop and optimize SQL stored procedures, functions, and indexes in SQL Server 2019 and 2022.
• Design and implement efficient data integration pipelines using SSIS and Azure Data Factory.
• Manage data solutions within the Microsoft Fabric ecosystem, including OneLake and Data Factory.
• Leverage ElasticSearch for optimal search performance and integrate machine learning models into search pipelines.
• Utilize Terraform for Infrastructure as Code (IaC) to provision Azure resources.
• Collaborate with cross-functional teams to translate business requirements into scalable system designs.
• Act as a subject matter expert in data engineering and search technologies.
• Own the design and improvement of database schema and architecture.

Requirements

• Bachelor’s degree in Computer Science or related field; Master’s degree preferred.
• Minimum of 5 years of experience as an MS SQL programmer in a dynamic environment.
• Proven experience designing large-scale database solutions that are reliable and scalable.
• Familiarity with security and privacy issues related to data management.
• Experience with machine learning libraries such as TensorFlow or PyTorch is a plus.

Nice to Have

• Experience with natural language processing (NLP) and large language models (LLMs).
• Knowledge of frameworks like LangChain or LangGraph.
• Familiarity with agentic AI systems for advanced search and recommendation engines.

What We Offer

• Competitive salary based on experience.
• Comprehensive medical, dental, and vision coverage.
• Paid time off and 401k savings plans.
• Access to an onsite personal trainer and gym.
• Hybrid work environment with flexible scheduling.