Job Overview

CGI is hiring a Senior Data Engineer for its Hyderabad team. This is a full-time, experienced-level Data Engineering opportunity focused on building scalable and dependable data pipelines. The role is suited to professionals with strong hands-on experience in Python, Apache Airflow, Kafka, REST APIs, SQL, and PL/SQL, along with practical exposure to cloud platforms.

The engineer will work on both batch and streaming data solutions and will be responsible for moving data from multiple sources into reliable processing workflows. The position requires an understanding of ETL and ELT practices, workflow orchestration, data ingestion, transformation, monitoring, fault tolerance, and data quality. The role is offered in a hybrid work model with a stated shift of 1 PM to 10 PM.

Key Responsibilities

• Design and implement scalable batch and streaming data architectures for high-volume data processing.
• Build and maintain production-ready data pipelines using Python and established data engineering practices.
• Develop Apache Airflow workflows and DAGs, including advanced orchestration patterns such as dynamic task mapping.
• Create reliable ingestion solutions for REST APIs and handle practical integration concerns such as rate limits, retries, and failures.
• Develop Kafka producers and consumers to support streaming and event-driven data processing.
• Work with SQL and PL/SQL to transform, validate, and optimize data for downstream requirements.
• Design data loads that are incremental and idempotent to improve reliability and avoid unnecessary duplication.
• Implement appropriate error handling, fault tolerance, monitoring, and data triaging processes.
• Monitor pipeline health and investigate data quality or processing issues.
• Work with large-scale data pipelines while maintaining performance, reliability, and operational consistency.
• Contribute to scalable data models and transformation logic.
• Collaborate with technical and business stakeholders to understand requirements and translate them into effective data solutions.

Required Skills

Candidates should have strong Python programming skills and substantial professional experience in Data Engineering. Advanced knowledge of Apache Airflow and DAG development is important, particularly for designing dependable workflow orchestration.

Hands-on Kafka experience is required for implementing producers and consumers and supporting streaming workloads. Candidates should also be comfortable integrating REST APIs and designing ingestion processes that can manage external API limitations, retries, and transient failures.

Strong SQL and PL/SQL knowledge is expected. The role involves writing and optimizing queries, supporting transformations, and working with data models. Experience with large-scale data pipelines and practical ETL or ELT implementation is essential.

Cloud experience is also required. The job description identifies AWS, GCP, and Azure as relevant cloud platforms, so candidates should have experience working with at least one of these environments.

Preferred Skills

Experience with Oracle databases is an advantage. Candidates with exposure to CI/CD and modern engineering practices can also bring additional value, particularly experience with Jenkins, Git, Docker, and Kubernetes.

Knowledge of data governance and compliance frameworks is another desirable skill. Familiarity with Snowflake, which is listed among the required skill areas, can further strengthen alignment with the position.

Education

A Bachelor's degree in Computer Science or a related discipline, or a higher qualification, is required according to the provided job description. The role is intended for experienced professionals with substantial industry experience in data engineering.

Experience

The position requires 7-10+ years of Data Engineering experience. Candidates should be able to demonstrate hands-on work with Python, Airflow, Kafka, REST API integrations, SQL or PL/SQL, and large-scale data pipelines.

This is an experienced Senior Data Engineer opportunity rather than a fresher position. Applicants should be comfortable taking ownership of complex pipeline development and addressing reliability, performance, and data quality challenges.

Required Technologies

The core technology areas for this role include Python, Apache Airflow, Kafka, REST APIs, SQL, PL/SQL, ETL, ELT, AWS, GCP, Azure, Snowflake, and data pipelines.

The job description also identifies AWS S3 as a cloud data platform component. Additional preferred technologies include Oracle, Jenkins, Git, Docker, and Kubernetes. Other relevant technical areas include batch processing, streaming architectures, workflow orchestration, data ingestion, data transformation, monitoring, fault tolerance, incremental loading, idempotent processing, API retries, and data quality.

Soft Skills

The role requires strong analytical and problem-solving ability, particularly when diagnosing pipeline failures, data issues, and integration problems. Senior Data Engineers should be able to communicate technical requirements and solutions clearly while working with different stakeholders.

Ownership, attention to detail, structured troubleshooting, and a focus on reliability are important for maintaining production-grade data workflows. The ability to understand business requirements and convert them into scalable technical solutions is also valuable.

Benefits of Working in this Role

This role provides an opportunity to work on large-scale Data Engineering problems involving batch and streaming architectures, workflow orchestration, API integrations, event streaming, cloud platforms, and database technologies. The combination of Python, Airflow, Kafka, SQL, and cloud skills can provide valuable experience for professionals building careers in modern data platforms.

The position also offers exposure to engineering practices such as fault tolerance, monitoring, incremental processing, CI/CD, containerization, and data governance.

Work Mode

The position is hybrid. The job description specifies a 1 PM to 10 PM shift.

Location

This Senior Data Engineer role is based in Hyderabad, India. The provided job information identifies Hyderabad as the main location and Andhra Pradesh, India in the location details.

Who Should Apply

This opportunity is suitable for Senior Data Engineers and experienced Data Engineering professionals with 7-10+ years of relevant experience. Candidates should have strong Python, Apache Airflow, Kafka, REST API, SQL, and PL/SQL experience and should have worked with large-scale data pipelines.

Professionals with cloud experience in AWS, GCP, or Azure are encouraged to apply. Candidates who also know Oracle, Snowflake, Jenkins, Git, Docker, Kubernetes, or data governance can demonstrate additional relevant capabilities.

Career Growth

Experience in this role can support progression toward positions such as Lead Data Engineer, Data Engineering Architect, Senior Data Platform Engineer, Cloud Data Engineer, or Data Engineering Manager. Building deeper expertise in distributed data processing, streaming, orchestration, cloud platforms, and data governance can broaden future opportunities in modern data organizations.

Application Advice

Before applying, review your resume for clear evidence of production Data Engineering experience. Highlight projects involving Python, Airflow DAGs, Kafka producers or consumers, REST API ingestion, SQL and PL/SQL optimization, and large-scale ETL or ELT pipelines.

Mention the cloud platforms you have used and describe relevant work with AWS S3, GCP, or Azure. If applicable, include experience with Snowflake, Oracle, Jenkins, Git, Docker, Kubernetes, data governance, monitoring, fault tolerance, and incremental or idempotent data processing. Keep your project descriptions focused on your actual responsibilities and measurable technical outcomes.