Data Engineering Consultant

Desired Skills Important • Distributed Event Processing – Apache Spark, PySpark, runtime/execution internals. • Data Warehousing & Object Storage – Lakehouse architecture, Apache Iceberg, Medallion Architecture (bronze/silver/gold). • Workflow Orchestration – SparkSubmit, Apache Airflow, drivers; including streaming and ingestion pipelines. • Analytical SQL – querying and window functions. • Containers – Docker Compose, services, healthchecks. • Architecture Ownership – proven ability to design data architecture independently and monitor it in production. Good to Have • Entity Matching – Zingg, golden record resolution, precision-recall evaluation. • Python – APIs, scripting, testing. • Search – Elasticsearch indexing. • Frontend – any frontend knowledge (Angular or similar). • Domain – experience in financial services / fintech / fraud analytics, or with GST, banking or transaction data. PREFERRED / GOOD TO HAVE : Experience with one or more visualization / graph libraries, such as: D3.js Cytoscape.js Understanding of graph and tree data structures (nodes, edges, hierarchies, DAGs, adjacency, etc.). Experience working with SVG, Canvas APIs, or WebGL-based visualizations. Knowledge of state management libraries (NgRx, Akita, etc.). Experience in data-heavy domains (e.g., fintech, analytics, monitoring, or similar). Exposure to CI/CD pipelines and modern build/deployment workflows.