Assistant Director - Data Engineer

About the role

Moody's is seeking an Assistant Director – Data Engineer to lead the design, development, and optimisation of large‑scale data pipelines within a cloud‑native environment. You will work closely with cross‑functional teams to transform raw data into actionable insights that support the company’s AI‑driven risk assessment platform.

Key responsibilities

• Design, build, and maintain scalable data processing solutions using Apache Spark and Databricks.
• Develop and optimise ETL workflows, ensuring high performance with modern data formats such as Parquet, Delta Lake, and JSON.
• Collaborate with data scientists and product owners to integrate AI‑enhanced features into data pipelines.
• Implement version‑control best practices and automated testing using Git, Gradle/Maven/SBT, and testing frameworks like ScalaTest or JUnit.
• Monitor and troubleshoot pipeline performance, applying distributed‑computing concepts such as data partitioning and caching.

Required profile

• 5+ years of hands‑on experience building scalable data applications with Apache Spark (Scala preferred, Python acceptable).
• Strong background in cloud environments, preferably AWS, and experience with Databricks.
• Proven ability to work independently and lead technical initiatives within a fast‑paced, collaborative setting.

Required skills

• Apache Spark
• Scala and/or Python
• Databricks
• AWS cloud services
• SQL for data transformation and querying
• Parquet, Delta Lake, JSON
• Git, Gradle, Maven, SBT
• ScalaTest, JUnit
• Graph data structures, clustering, TF‑IDF, similarity metrics
• AI‑assisted development tools

What we offer

• Opportunity to work on cutting‑edge AI‑driven risk analytics.
• Inclusive culture that values diverse perspectives and continuous learning.
• Competitive compensation and benefits package.