Big Data Engineer

About the Company

Infosys is a worldwide digital services and consulting organization known for helping enterprises modernize and transform at scale. With decades of experience supporting large global businesses across more than 50 countries, the company focuses on cloud-led transformation, AI-driven operations, agile delivery, and continuous learning. Infosys also emphasizes responsible governance, environmental sustainability, and an inclusive workplace where diverse talent can grow.

Role Overview

This position is for a Big Data Engineer who will design and build scalable batch and streaming data solutions. The role involves creating efficient data pipelines, improving performance, and supporting reliable data delivery for downstream consumers.

Responsibilities

• Develop and support scalable batch and real-time data pipelines using Spark and Scala.
• Create and refine data processing flows with Hive for querying, transformation, and warehouse use cases.
• Set up dependable ingestion and integration approaches for large datasets while maintaining data accuracy, consistency, and completeness.
• Build reusable Spark jobs, libraries, and frameworks that help standardize engineering practices across teams.
• Improve Spark performance through effective partitioning, caching, shuffle handling, and memory management.
• Partner with stakeholders to understand data needs and deliver well-structured datasets for downstream use.
• Introduce monitoring, alerts, and runbooks to strengthen pipeline stability and speed up issue resolution.
• Review code, promote engineering best practices, and help raise the quality of the data platform.

Preferred Qualifications

• Practical experience with Kafka for streaming ingestion and event-driven data processing.
• Ability to design complete data architectures spanning ingestion, processing, storage, and serving layers at scale.
• Strong understanding of partitioning strategies, file formats, and efficient big data processing patterns.
• Capability to guide technical conversations, mentor team members, and encourage best practices across delivery teams.
• Experience strengthening reliability through automated checks, data validation, and operational excellence practices.

Good to Have

Hadoop, HDFS, YARN, Airflow, and HBase.

Location

Kolkata, India. The role also mentions PAN India availability.

Experience

The expected experience range is 5 to 15 years.

Education

B.Tech or B.E. in any specialization.

Additional Information

No stipend or salary amount was specified in the source. No opening count, start date, or application deadline was provided.