Senior Azure Data Engineer Job at eTeam, Redmond, WA

  • eTeam
  • Redmond, WA

Job Description

Job Title: Senior Azure Data Engineer

Location: Redmond, WA 98052

Work Setup: Hybrid

Duration: 12 Months

Experience: 8-10 Years


Job Description


We are looking for an experienced Senior Azure Data Engineer with 8-10 years of experience in data engineering and strong hands-on expertise in Python, SQL, Apache Spark, PySpark, and Azure Databricks .


The candidate will be responsible for developing and supporting scalable data pipelines, transforming large datasets, and optimizing data processing solutions using Azure cloud technologies.


Must Have Skills

  • Strong experience with Python and SQL
  • Hands-on experience with Apache Spark, PySpark, and Spark SQL
  • Strong experience with Azure Databricks , including:
    • Notebooks
    • Jobs/Workflows
    • Clusters
  • Experience with Azure Synapse
  • Strong knowledge of Delta Lake
  • Experience working with Parquet
  • Hands-on experience with ADLS Gen2
  • Strong understanding of data engineering and ETL/ELT concepts
Nice to Have Skills
  • Experience with Scala
  • ETL/ELT development
  • Experience with MERGE/Upsert operations
  • Knowledge of data partitioning and partition pruning
  • Strong Spark performance tuning experience
  • Experience handling data skew, shuffles, caching, broadcast joins, and partition optimization
  • Experience with Git
  • Experience with Azure DevOps and CI/CD
Key Responsibilities
  • Develop and maintain scalable ETL/ELT data pipelines
  • Extract, transform, clean, and load data from various source systems
  • Build data solutions using Azure Databricks, Spark, Delta Lake, Synapse, and ADLS Gen2
  • Implement MERGE/Upsert logic using Delta Lake
  • Design effective partitioning strategies for large datasets
  • Optimize Spark jobs for performance and cost
  • Manage data skew, shuffles, caching, partitions, and cluster resources
  • Collaborate with development and data teams to deliver reliable data solutions
  • Follow best practices for Git, Azure DevOps, and CI/CD where applicable

Job Tags

Similar Jobs

The Cigna Group

Warehouse Associate - Accredo Job at The Cigna Group

 ...your work impacts patients across the country? Accredo, Evernorths specialty pharmacy, is hiring Warehouse Associates to receive, stock, and assist with inventory at our pharmacy fulfillment site in Warrendale PA. In this role, youll support our supply chain, transfer... 

Harris-Lake Park MS/HS

Secondary Science Teacher Job at Harris-Lake Park MS/HS

 ...Secondary Science Teacher Harris-Lake Park CSD is seeking to hire a Secondary Science Teacher. Applicants with a coaching endorsement...  ...classes typically included in this position: MS Science HS Environmental Science Integrated Science Other Science Offerings -... 

Drive Time Transports

CDL-A drivers need for home WEEKLY, $.81-$.85 CPM, WEEKEND home time, $4K Bonus Job at Drive Time Transports

 ...CDL-A Regional Truck Driver Home Every Weekend , $4,000 Sign-On Bonus Pay & Miles ~$0.80 $0.85 CPM starting pay ~ Average weekly pay: $1,700+~$4,000 sign-on bonus ~ Performance bonus: up to +3 CPM based on safety, mileage, and productivity (starts first... 

Beacon Hill

Clinical Data Manager Job at Beacon Hill

 ...Clinical Data Manager 100% Remote CONTRACT Provide leadership for clinical data management function; ensure data management deliverables are completed on time, within budget and in accordance with quality standards and study requirements Responsible for leading... 

MS Data Agency

Sports Data Scout (Freelance) | $115-$258 per Game + Travel Reimbursed Job at MS Data Agency

 ...We're hiring Sports Data Scouts across the United States to collect live statistics at sporting events using our mobile app. If you're passionate about sports and want a flexible way to earn extra income, we'd love to hear from you. What you'll do Attend live...