- Experience
- 9+ yrs
- Salary
- INR 2,500,000 – INR 4,000,000 / year
- Openings
- 1
- Posted
- 1 hour ago
- Work mode
- In office
- Education
- Any graduate
- Eligibility
- Any graduate may apply, provided they have the required senior-level data engineering experience and technical background.
- Resume
- Required to apply
Where you'll work
Job description
About the Company
Optimum Solutions was established in 1997 and now has a workforce of more than 4,000 people spread across 9 geographies, including Singapore, Malaysia, Hongkong, China, the USA, and the UAE. In India, the company operates from Bangalore, Chennai, Hyderabad, and Pune. It partners with major banks and technology firms such as Morgan Stanley, Standard Chartered, Google, Credit Suisse, Citi Bank, BNPP, DELL, Walmart, Shell, Meta, Grab, PayPal, and Amazon to support digital transformation initiatives.
Role Overview
This position is for a Lead Data Engineer with strong experience in large-scale data engineering, streaming, cloud ecosystems, and modern data platform development.
Core Responsibilities
- Design and deliver data pipelines using Apache Spark, Databricks, or similar big data technologies, backed by at least 9 years of relevant experience.
- Build and manage streaming and messaging solutions using tools such as Apache Kafka, Confluent Cloud, RabbitMQ, or Azure Event Hub, including publishers, consumers, topics, and downstream integrations.
- Work with relational databases and CDC workflows, including SQL Server, Oracle, and other RDBMS platforms, and use Debezium or native CDC utilities to capture and transform change events.
- Develop solutions in Python, Scala, or Java, and use SQL confidently for data processing and transformation tasks.
- Use cloud services on Azure or AWS for storage, compute, and orchestration, including ADLS, S3, Azure Data Factory, AWS Glue, Airflow, DBX, and DLT.
- Apply data modeling and warehousing practices for Lakehouse and Delta Lake architectures, including partitioning and performance tuning.
- Maintain version control and DevOps practices through Git, CI/CD automation, and infrastructure-as-code approaches.
- Collaborate with cross-functional teams and explain complex technical ideas clearly to non-technical stakeholders.
Preferred Profile
- Experience with event-driven architectures and microservices integration.
- Exposure to ingestion tools such as NiFi, Flume, or comparable frameworks for bringing data in from diverse sources.
- Familiarity with graph processing or machine learning workflows on Spark.
Eligibility
Applicants from any academic background may apply, provided they match the required experience and technical capability for the role.
Additional Information
This role is based in Pune, India. The opportunity is for a lead-level data engineering professional.