Pyspark + SQL Data Engineer

About the Company

Sonyo Management Consultants is a global frontrunner in consulting, technology services, and digital transformation. With more than five decades of experience and operations across over 50 countries, it partners with leading enterprises worldwide to tackle their most intricate business and technological challenges. The company believes that technology's true value emerges through people, enabling clients to evolve and excel by leveraging advanced innovations in cloud computing, data, artificial intelligence, software engineering, and digital solutions. Its comprehensive service portfolio covers strategy and transformation, application development, infrastructure management, and intelligent operations, along with software development, IT consulting, network management, systems integration, and support services.

Job Description

We are looking for an experienced Data Engineer skilled in Pyspark and strong SQL capabilities. Candidates should have between 6 to 12 years of relevant experience and be available to join within 0 to 30 days, targeting July or August joiners. The position is based in Pune, Bangalore, Chennai, or Hyderabad.

Key Responsibilities and Skills

• Expertise in advanced SQL techniques including complex joins, aggregate functions, and CASCADE operations.
• Working knowledge of Hadoop's HDFS commands for file transfers, reading, and searching.
• Proficiency in Hive concepts such as partitioning and bucketing to optimize data storage and queries.
• Hands-on experience with Apache Spark, including SparkSession, RDD, DataFrames, various transformations, shared variables, and processing data for Hive tables using Spark.