Position: Data Engineer

Location:Manhattan NY (Hybrid 3 days a week to office)

Long Term

Key Skills

SQL | Python | Spark | Airflow / ADF | Cloud data services | CI/CD

Required Qualifications

• 8-10+ years in data engineering
• Strong SQL and Python (or Scala) skills
• Experience with orchestration tools such as Airflow or Azure Data Factory
• Hands-on experience with cloud data services and big data frameworks such as Spark
• Understanding of data warehousing and lakehouse concepts
• Familiarity with Git, CI/CD and infrastructure-as-code
• Experience supporting ML/AI data needs is a plus

Key Responsibilities

• Design, build and optimize batch and streaming data pipelines
• Implement data models and transformations defined by the Data Architect
• Ingest and integrate data from multiple source systems and APIs
• Implement data quality checks, monitoring and alerting
• Optimize performance and cost of data workloads
• Collaborate with AI Developers and analysts to provision datasets
• Maintain documentation, version control and CI/CD for data code
• Work with offshore data engineers on knowledge transfer and delivery