Summary
Overview
Work History
Education
Skills
Certification
Timeline
Generic
Susheelkumar Pandey

Susheelkumar Pandey

Senior Data Engineer
Singapore

Summary

  • 9 Years of experience architecting and implementing data engineering solutions across Banking, Aviation, and Telecom domains.
  • Expertise in designing and deploying cloud-native analytics platforms, including data ingestion pipelines, data lakes, batch and stream data processing, and visualization layers
  • Skilled in evaluating and selecting technology stacks for end-to-end data architecture, roadmap definition, and scalable solution delivery
  • Hands-on with PySpark, Scala, Hadoop ecosystem (HDFS, Hive, Sqoop), Apache Airflow, Control-M, and AWS services (S3, EMR, EC2, RDS, Redshift) for building robust data workflows
  • Proficient in SQL, PL/SQL, Python, Scala, with deep experience in query optimization and performance tuning on Hive and Spark jobs to handle large-scale datasets efficiently
  • Strong foundation in Hadoop architecture, distributed computing, and modern data engineering frameworks
  • Proven ability in technical design, system integration, and ensuring minimal operational disruption during deployments
  • Analytical problem solver with excellent communication skills; effective both in independent and collaborative team environments
  • Proactive learner, continuously updating skills to leverage emerging technologies and industry best practices

Overview

9
9
years of professional experience
2
2
Certifications

Work History

Senior Data Engineer

Unison Consulting (Client : Standard Chattered)
04.2024 - Current
  • Developed and maintained scalable ETL/ELT pipelines using Scala, PySpark, HQL, and Shell scripting, ingesting structured/semi-structured data (Oracle, WFTS) into Hive on Cloudera Data Platform
  • Managed data flow across landing, staging, and curated zones in the Hadoop Data Lake
  • Built and optimized Spark (Scala & PySpark) and Hive jobs with partitioning, bucketing, and join strategies for large-scale data processing
  • Implemented behavior-driven testing (BDD) to ensure code accuracy, reusability, and data quality
  • Tuned Spark jobs for memory and compute efficiency; improved performance of long-running transformations
  • Automated batch workflows using Control-M and Shell scripts; managed job dependencies and parameterization
  • Successfully delivered a production-grade data pipeline project on time, now live and handling critical data workloads

Data Engineer

Unison Consulting, Singapore (Client: OCBC Singapore )
10.2022 - 03.2024
  • Developed and optimized fraud detection pipelines using rule-based logic in OCBC FMS for real-time transaction monitoring and action (hold/decline)
  • Built data ingestion workflows for SAS Visual Investigator using Bronze/Silver/Gold layers; automated ETL with PL/SQL, Shell scripting, Oracle, Control-M, and Airflow
  • Created real-time Kibana dashboards integrated with Elasticsearch for fraud analytics
  • Enhanced batch processing stability by managing failures, optimizing workflows, and improving job scheduling
  • Leveraged PySpark, Python, and Kafka for large-scale data transformation and real-time streaming
  • Migrated legacy ETL systems to scalable big data platforms, boosting performance and fault tolerance
  • Executed UT, SIT, and UAT testing to ensure data integrity and system robustness

Data Engineer

Integrichain Pvt Ltd
11.2021 - 10.2022

Worked on Life Science projects like Contract & Pricing, Claims, Government pricing etc.

● Setup and constructed entire database model and architecture of the project on PostgreSQL.

● Built pipeline to using Python send structural data to Amazon S3 buckets.

● Processed the data using Amazon EMR.

● Worked on AWS EC2 and RDS for DB development.

● Used Sqoop for data ingestion.

● Optimized existing data pipelines using PySpark and SQL, resulting in a 40% reduction in processing time..

● Created reports using Amazon redshift and Oracle.

● Query Optimization, Analysis using Explain plan and database configurations.

● Assisted in the development and deployment of data pipelines using CI/CD methodologies, Docker containerization, and Git version control, contributing to [specific achievement or outcome]

Senior Software Engineer

Coforge Technologies (Client : Qatar Airways)
11.2018 - 10.2021

Worked on various major projects like Boarding Reconciliation, Baggage system, Lost and Found, Contract Tracker, Inventory/Integrity violation reports.

As Database Developer : Nov 2018-Dec 2020

● Setup and constructed entire database model and architecture of the project on Oracle SQL.

● Developed and maintained data models to improve data quality and consistency across various data sources.

● Worked on Stored Procedures, Packages, Views, Functions, Cursors, Triggers and XML.

● Created various Market share reports, Regulatory reports.

● Query optimization for faster reports on different database like Oracle, PostgreSQL and MS/SQL.

● Query Optimization, Analysis using Oracle AWR Reports and Explain plan.

As Data Engineer : Apr 2020 - Nov 2021

● Designed and implemented data pipelines for ground services operations using Oracle and AWS, resulting in a 50% reduction in data processing time.

● Used Informatica to load data from different Oracle servers source to single destination.

● Worked on ETL batch jobs.

● Worked on Amazon S3 to load data and Redshift for Database.

● Worked on Data Lake and Staging Area where data will be staged into AVRO file formats.

  • Data modeling is done using Data Vault technique.

Software Engineer

Nextgen Clearing
11.2016 - 11.2018

DCH (Data Clearing House)

● Work on client user stories , requirement gathering and developing the final solution.

● Worked on Stored Procedures , Packages , Views , Triggers, XML Processing

● Query Optimization , Analysis using Oracle AWR Reports and Explain plan.

● Code Review and Knowledge sharing for globalization within team for zero dependency.

● Delivered Custom client reports on ad-hoc basis.

Systems Analyst

Amdocs India Pvt Ltd
06.2016 - 08.2016

● Worked on job scheduling using cron-tab.

● Worked mainly on DML operations daily basis.

● Created Reporting tool using Oracle PL/SQL, Perl and Apex to automate the Client request.

● Modification and implementation of the databases using Oracle SQL, PL/SQL, UNIX/LINUX.

Trainee Software Engineer

Manthantec Consultancy Pvt Ltd
01.2015 - 03.2016

Smart Axiata

● Worked on data migration and data integrity check after migration.

● Tested all plans as per requirements in Product Design Documents by preparing fully planned test cases in complete details by covering all scenarios.

Celcom (Kenan Billing system)

● Worked on Account Move Tool (AMT) where data of a customer having a post-paid connection.

Built triggers which would prevent certain accounts getting scheduled for move which were basically not eligible for move

Education

Bachelor of Engineering - Information Technology

Shah And Anchor Kutchhi Engineering College
04.2001 - 07.2014

HSC - Science

Ramanand Arya D.A.V College
04.2001 - 06.2010

SSC -

Hindi Bal Vidya Mandir
04.2001 - 05.2008

Skills

SQL, PL/SQL, Python, React

Spark, Hadoop, Hive, Sqoop,

AWS S3, AWS Glue, EMR, EC2, RDS

Data Modeling, Databricks Datalake

Oracle, PostgreSql, MSSQL, Teradata, Cloudera, JavaScript

Data Warehousing, Performance Tuning, Big Data Processing

Certification

Microsoft Certified Data Analyst Associate.

Timeline

Senior Data Engineer

Unison Consulting (Client : Standard Chattered)
04.2024 - Current

Databricks Certified Data Engineer Associate

03-2023

Data Engineer

Unison Consulting, Singapore (Client: OCBC Singapore )
10.2022 - 03.2024

Data Engineer

Integrichain Pvt Ltd
11.2021 - 10.2022

Microsoft Certified Data Analyst Associate.

03-2021

Senior Software Engineer

Coforge Technologies (Client : Qatar Airways)
11.2018 - 10.2021

Software Engineer

Nextgen Clearing
11.2016 - 11.2018

Systems Analyst

Amdocs India Pvt Ltd
06.2016 - 08.2016

Trainee Software Engineer

Manthantec Consultancy Pvt Ltd
01.2015 - 03.2016

Bachelor of Engineering - Information Technology

Shah And Anchor Kutchhi Engineering College
04.2001 - 07.2014

HSC - Science

Ramanand Arya D.A.V College
04.2001 - 06.2010

SSC -

Hindi Bal Vidya Mandir
04.2001 - 05.2008
Susheelkumar PandeySenior Data Engineer