Senior Data Engineer - Remote Opportunity

About the Role

We are seeking a talented Senior Data Engineer to join our team in a remote capacity. As a Senior Data Engineer, you will play a crucial role in designing, developing, and maintaining data pipelines and ETL/ELT workflows that support both batch and real-time data processing. You will work closely with architects and engineering teams to build scalable data infrastructure and implement modern data storage solutions. This is an exciting opportunity to work with cutting-edge technologies and make a significant impact on our data strategy.

What You'll Do

• Design and develop ETL/ELT workflows and data pipelines for batch and real-time processing.
• Build and maintain data pipelines for reporting and downstream applications using open source frameworks and cloud technologies.
• Implement operational and analytical data stores leveraging Delta Lake and modern database concepts.
• Optimize data structures for performance and scalability across large datasets.
• Collaborate with architects and engineering teams to ensure alignment with target state architecture.
• Apply best practices for data governance, lineage tracking, and metadata management.
• Develop, schedule, and orchestrate complex workflows using Apache Airflow.
• Troubleshoot and resolve issues in data pipelines to ensure high availability and reliability.

Requirements

• Strong understanding of data structures, data modeling, and lifecycle management.
• Hands-on experience designing and managing ETL/ELT data pipelines.
• Advanced PySpark skills for distributed data processing and transformation.
• Experience implementing open table formats using NetApp Iceberg.
• Knowledge of the Hadoop ecosystem, including HDFS and Hive.
• Experience with cloud platforms, including Google Cloud Platform (BigQuery, Dataflow), Delta Lake, and Dataplex.
• Proficient in programming and orchestration using Python, Spark, and SQL.
• Strong experience with Apache Airflow, including authoring and maintaining DAGs for complex workflows.

Nice to Have

• Experience with data governance tools and practices.
• Familiarity with machine learning concepts.
• Knowledge of additional cloud platforms.

What We Offer

• Competitive salary range of $120,000 - $140,000 annually.
• Visa sponsorship available for qualified candidates.
• Flexible remote work environment.
• Opportunity to work with a talented team on innovative projects.
• Access to professional development resources.