Need Lead Data Engineer/ Data Architect (Remote)

Job Title : Lead Data Engineer/ Data Architect

Location : Remote

Duration : 12 Months

Job Description :

Roles and Responsibilities:  

Lead the design for migration project, leveraging Azure services, SQL, Databricks, and PySpark to create scalable, efficient, and reliable solutions.
Reduce latency by eliminating unnecessary hops.
Identifying the dependencies in the target system ( standard SaaS CRM app) and ensuring data requirements are met, designing validation methods and such.
Developing re-usable components for common tasks.
Provide inputs to create a migration plan including all phases of the SDLC.
 

Desired skills

Utilize Databricks to design and implement advanced data transformation and processing tasks using Databricks, PySpark and ADFDesign, deploy, and manage Databricks clusters for data processing, ensuring performance and cost efficiency, and troubleshoot cluster performance issues as needed.
Document architectural designs, migration plans, and best practices to ensure alignment and reusability within the team and across the organization.
Collaborate with stakeholders to translate business requirements into technical specifications and design robust data pipelines, storage solutions, and transformation workflows.
Support ETL processes and data flow management between systems
Experience in developing scalable, secure, and auditable data architectures.
Familiarity with data completeness checks and validation techniques.
Should be hands on.