Data engineering professional with solid history of creating and managing efficient data systems. Known for delivering impactful solutions through collaboration and results-driven approach. Recognized for expertise in data warehousing and ETL processes, along with adaptability to evolving project needs.
Solid hands-on experience with Big Data Technologies like Apache Hadoop, Spark, Hive, Kafka, etc.
Solid experience in building batch processing, real-time and near real-time data ingestion pipelines with Apache Kafka, Apache Spark streaming using Python, Java, and Scala.
Proven experience in Design, Develop, Test, Build and deploy Big Data and machine learning solutions for handling Petabyte scale data using Python, Java, and Scala.
Good exposure to Azure cloud technologies like ADLS, Synapse Analytics, Azure Databricks, ADF, Azure functions, Keyvault, AzureML, logic apps, AKS, etc.
Hands-on experience with AWS tools like EC2, S3, SQS, SNS, Redshift, RDS, Lambda, Kinesis, etc.
Hands-on experience in writing HiveQL and SparkSQL queries for querying and analyzing large volumes of data.
Hands-on experience with NoSQL Databases like CosmosDB, MongoDB, and Cassandra.
Worked with various file formats like Flat files, images, GIS, Avro, ORC, Parquet, XML, JSON, etc.
Solid hands-on experience with containerization and orchestration using Docker and Kubernetes on various cloud platforms and on-premise.
Results-oriented achiever with proven ability to exceed targets and drive success in fast-paced environments. Combines strategic thinking with hands-on experience to deliver impactful solutions and enhance organizational performance.
Overview
12
12
years of professional experience
Work History
Senior Data Engineer
MODEC Offshore productions systems
02.2022 - Current
Collaborated with cross-functional teams to define requirements and develop end-to-end solutions for complex data engineering projects.
Reduced latency in real-time analytics applications by optimizing query performance through indexing strategies and proper database design principles.
Developed scalable infrastructure capable of handling vast amounts of structured and unstructured data, improving overall system performance.
Established strong working relationships with stakeholders across multiple departments, facilitating clear communication channels regarding project requirements and progress updates.
Delivered high-quality code reviews for colleagues'' contributions following established coding standards and best practices, maintaining consistency throughout projects.
Mentored junior engineers on best practices in data engineering, fostering a culture of continuous learning and improvement within the team.
Identified areas for improvement within existing ETL frameworks to maximize efficiency without sacrificing accuracy or integrity of results produced during transformations applied on ingested datasets.
Achieved cost-effective cloud-based storage solutions for handling vast amounts of data, resulting in reduced operational expenses and increased scalability.
Reengineered existing ETL workflows to improve performance by identifying bottlenecks and optimizing code accordingly.
Collaborated on ETL (Extract, Transform, Load) tasks, maintaining data integrity and verifying pipeline stability.
Developed Data ingestion Framework using Synapse, Azure Data factory, Azure functions, Key vaults to read and ingest data from various sources
Design, Develop, test, build and deploy cutting edge Spark applications to handle data ingestion and data analysis
Responsible for creating, configuring, managing, and maintaining the Azure resources using Terraform and azure DevOps git
Single point of contact for end to end SDLC including provisioning resources, developing pipelines, granting data access to other teams
Responsible for protecting the data from outside access, securing the azure resources, managing network configurations, etc
Evaluated new technologies and industry trends to identify opportunities for enhancing current systems or developing innovative solutions.
Senior Data Engineer
Vulcan AI pvt ltd
02.2020 - 02.2022
Company Overview: Vulcan AI pvt ltd, Singapore
Involved in the Design, Coding and Testing for the AI ops as per the client requirement
Designing, implementing, Developing and orchestrating Machine Learning pipelines for various machine learning models using tensorflow, cuda as per the requirement
Data preparation, Pre-processing and Ingestion from various data sources like Satellite images, XML, GIS files
Responsible for integrating, building & testing of Data pipelines in Azure Data factory
Involved in CI/CD pipelines using Azure devops, Containerization and orchestration using Docker and AKS on Azure cloud
Setting up the different environments in Azure for dev, testing and production
Vulcan AI pvt ltd, Singapore
Senior Data Engineer
Credit Agricole CIB
09.2018 - 02.2020
Working in building the end-to-end data pipeline to ingest the data to the centralized data lake
Coordinated with the business team directly to gather the requirements and translating the functional requirements to the technical designs
Well versed in designing the data flow architecture based on the business team inputs
Extensively worked on writing Spark codes based on java to cleanse, filter, normalize, nest, persist the data files and referential files from various sources to the data lake
Hands on experience with the data orchestration by on boarding the data ingestion as a part of automation
Great exposure in building the Jenkins build pipelines and master deploy jobs deployment in various environments
Helped in data ingestion pipeline performance improvements
Senior analyst – Big Data
Ernst and Young (EY)
01.2017 - 05.2018
Collaborated with cross-functional teams to identify areas of improvement, leading to increased operational effectiveness.
Developed and maintained data warehouses and data marts to support business operations.
Implemented business intelligence solutions to increase operational efficiency.
Optimized data access and storage to improve performance of analytics systems.
Developed and implemented data governance policies and procedures.
Collaborated with stakeholders to identify business needs and data sources.
Big Data Java Developer
Sierra Support Centre pvt ltd
04.2016 - 12.2016
Ensured seamless integration of Java-based applications with databases, APIs, and other systems.
Developed user-friendly interfaces for web applications, improving user experience and satisfaction.
Proved successful working within tight deadlines and a fast-paced environment.
Senior Systems Engineer
Cognizant Technology Solutions
02.2014 - 04.2016
Collaborated with cross-functional teams to deliver innovative solutions for business needs.
Led successful system upgrades by creating detailed implementation plans and coordinating resources effectively.
Collaborated with clients to determine need and devise appropriate software and hardware solutions.
Enhanced security measures by designing and deploying effective firewall configurations.
Education
Bachelor of Engineering - Computer Science and Engineering
Sriram Engineering College
05-2012
HSC - Science And Mathematics
Thavasumuthu Nadar Hr.Sec School
05-2008
SSLC - Mathematics
Thavasumuthu Nadar Hr.Sec School
06-2006
Skills
Core skills: ETL development, Data pipeline design,Data modeling
Big data framework: Apache Spark
Programming: Python/Java
Cloud computing: Microsoft Azure/AWS
Big data processing
Performance tuning
SQL
Kafka
MySQL/Oracle/SQL Server
NoSQL databases
Terraform/YAML
Docker/Kubernetes
Git version control
Metadata management
Continuous integration/Continuous Delivery
Interpersonal communication/Analytical skills
Languages
English
Tamil
Timeline
Senior Data Engineer
MODEC Offshore productions systems
02.2022 - Current
Senior Data Engineer
Vulcan AI pvt ltd
02.2020 - 02.2022
Senior Data Engineer
Credit Agricole CIB
09.2018 - 02.2020
Senior analyst – Big Data
Ernst and Young (EY)
01.2017 - 05.2018
Big Data Java Developer
Sierra Support Centre pvt ltd
04.2016 - 12.2016
Senior Systems Engineer
Cognizant Technology Solutions
02.2014 - 04.2016
Bachelor of Engineering - Computer Science and Engineering
Talent Acquisition Specialist at MODEC Offshore Production Systems (Singapore) / Offshore Frontier SolutionsTalent Acquisition Specialist at MODEC Offshore Production Systems (Singapore) / Offshore Frontier Solutions
Lead Planner (CIC Planning - Commercial) at MODEC Offshore Production Systems (Singapore) Pte LtdLead Planner (CIC Planning - Commercial) at MODEC Offshore Production Systems (Singapore) Pte Ltd
International School Science Teacher (Primary) at Siam Singapore International SchoolInternational School Science Teacher (Primary) at Siam Singapore International School