Experienced Azure Data Engineer Required.

Freelance Azure Data Engineer Project Type Personal Project – Freelance / Part-time Role Azure Data Engineer – Azure Databricks Experience 3–5 years of relevant hands-on experience preferred. Project Overview We are looking for an experienced Azure Data Engineer to support a data engineering project involving data ingestion, transformation, validation, and processing using Microsoft Azure services. The project involves working with structured and semi-structured data, including healthcare/insurance-related data. Required Technical Skills - Azure Databricks - PySpark / Python - Strong SQL / SQL Server - Azure Data Factory (ADF) - Azure Data Lake Storage Gen2 (ADLS Gen2) - Delta Lake - Databricks SQL - ETL / ELT development - Incremental and historical data processing - Delta MERGE / Upsert operations - JSON parsing and nested JSON handling - Data deduplication - Data validation and source-to-target reconciliation - Performance optimization in Databricks - Git / basic CI-CD knowledge Preferred Domain Experience Experience in any of the following is preferred: - Healthcare - Insurance - Member/Customer data - Claims - Policy/Benefits data - FHIR / HL7 - Healthcare data integration Responsibilities - Develop and maintain Azure Databricks notebooks using PySpark. - Build data ingestion pipelines using ADF and Azure services. - Process data from SQL Server and other sources. - Implement historical and incremental data processing. - Create and maintain Delta tables. - Implement MERGE/Upsert logic for incremental data. - Parse and transform nested JSON data. - Perform data cleansing, deduplication, and transformation. - Perform source-to-target data validation and reconciliation. - Troubleshoot data pipeline and Databricks issues. - Optimize Spark jobs and SQL queries where required. - Document the implemented solution and provide knowledge transfer. Ideal Candidate The ideal candidate should have strong hands-on experience rather than only theoretical knowledge. The candidate should be comfortable explaining and implementing a complete flow such as: SQL Server / Event-based Source → ADF / Databricks → ADLS Gen2 → Delta Lake → Transformation & Validation → Unified Target Engagement - Freelance / Part-time - Remote - Project-based - Duration: To be discussed - Working hours: Flexible, based on project requirements - Payment: Hourly or milestone-based, depending on experience How to Apply Please share: 1. Total years of Azure Data Engineering experience 2. Azure Databricks experience 3. PySpark experience 4. SQL/SQL Server experience 5. ADF and ADLS Gen2 experience 6. Healthcare/Insurance project experience, if any 7. Availability (hours per week) 8. Expected hourly/project rate 9. Brief description of similar projects handled Location: Pune, Maharashtra Work mode: On-site / In-person Candidate must be available to work from my location in Pune.