Senior AI Data Engineer - Remote Opportunity
About the Role
ProfitSolv is seeking a Senior AI Data Engineer to join our dynamic team in a fully remote capacity. As a Senior AI Data Engineer, you will play a crucial role in building a centralized data platform on AWS, enhancing our AI-driven services for the legal and accounting sectors. This position offers the opportunity to work on a greenfield project, implementing a three-phase architecture that includes Lakehouse standup, database migration, and a zero-ETL endgame.
What You'll Do
- Build and maintain a medallion Lakehouse (Bronze/Silver/Gold) on S3 using Apache Iceberg, Glue Data Catalog, and dbt Cloud with the Athena adapter.
- Configure and manage AWS DMS for ongoing Change Data Capture (CDC) from approximately 1,000 SQL Server instances.
- Develop ECS Fargate tasks for SaaS API ingestion and orchestrate workflows with Amazon MWAA (Airflow).
- Write dbt Cloud models for Bronze to Silver to Gold transformations and define business metrics in the dbt Semantic Layer.
- Manage Redshift Serverless + Spectrum as the read engine for analysts and BI tools, ensuring optimal performance through tuning Iceberg table layouts.
- Implement Lake Formation tag-based governance for data isolation and onboard new acquisitions swiftly.
- Design and ship Retrieval-Augmented Generation (RAG) pipelines tailored for legal domain use cases.
- Ensure compliance, security, and governance through IAM roles, encryption policies, and metadata cataloging.
Requirements
- 5+ years of hands-on data engineering experience with a strong focus on AWS services such as S3, Glue, Athena, and Redshift.
- Proven experience building production-grade data models using dbt, including testing, macros, and documentation best practices.
- Experience with Change Data Capture (CDC) patterns using AWS DMS, Debezium, or similar technologies.
- Demonstrated ability to design and operate production Airflow DAGs (MWAA or self-hosted).
- Strong proficiency in SQL and Python, with working knowledge of TypeScript for MCP server development.
Nice to Have
- Experience with infrastructure-as-code tools such as Terraform.
- Familiarity with Apache Iceberg, Delta Lake, or Hudi in production environments.
- Background in legal tech, payments, or professional services.
What We Offer
- Competitive salary ranging from $140,000 to $180,000 annually.
- 401K matching to help you invest in your future.
- Generous paid time off and holidays.
- Comprehensive health, dental, and vision benefits.
- A collaborative environment with a team of smart and dedicated professionals.
- Performance-based compensation to reward your contributions.
- Flexible remote work options.
This role is ideal for those looking for remote AI Data Engineer opportunities in a supportive and innovative environment. If you are passionate about data engineering and AI, we encourage you to apply!
This Senior AI Data Engineer role at ProfitSolv offers a unique opportunity to work remotely on innovative data solutions, with competitive compensation and a strong emphasis on collaboration.
Generating success profile...
Analyzing job requirements and market data
Loading market overview...
Analyzing market trends and skill demands
Industry News
Loading latest industry news...
Finding relevant articles from the last 6 months