VIVEK Y J
Summary
Self-motivated Data Engineer with 13 years of experience in data warehouse and Business Intelligence projects. Collaborates effectively with multi-functional roles to identify and leverage areas for improvement in data systems and insights.
Overview
13
13
years of professional experience
1
1
Certification
Work History
Senior Data Engineer
Homage
06.2023 - Current
- Collaborated with cross-functional teams to define requirements and develop end-to-end solutions for complex data engineering projects.
- Develop data pipelines by implementing advanced ETL processes and streamlining data flow. Extract data from multiple source systems (db,api,gsheets etc ) accordingly using AWS Glue/Python and Workato to load data into AWS Redshift
- Reduced operational costs by automating manual processes and improving overall data management efficiency using Python and Pyspark
- Ensured data quality through rigorous testing, validation, and monitoring of all data assets, minimizing inaccuracies and inconsistencies.
- Reengineered existing ETL workflows to improve performance by identifying bottlenecks and optimizing code accordingly.
- Develop simple dashboards using Tableau for data analysis and insights
Manager,Data Engineer
Lazada
07.2020 - 06.2023
- Gather requirements from Product designer /Business user and study the scope and impacts
- Develop both Real time and Batch pipelines accordingly using Blink and ODPS from multiple source systems
- Performance optimization to address high campaign data volume and reduce resource used effectively
- Support Campaign runs and ensure data issues are minimal with less impact
- Perform campaign data analysis by building FBI dashboards
- Walkthrough of data ingestion part for newly joined teammates (Data flow, RCA and Development part)
Senior Software Engineer
Client : UOB Bank | Employer : WerkMindz Pte Ltd
08.2019 - 07.2020
- Involved in understanding the requirements of the End users/Business Analysts and developed Strategies for ETL processes.
- Identify the placeholders in FSLDM to map the attributes for various subject areas like CUSTOMER, LOANS, FACILITY, RISK and Finance areas.
- Ingestion of data into Data Lake and Subsequent Layers (FRR)
- Created Hive queries and analyze data that helped Business users for spot emerging trends and by comparing fresh data with EDW reference tables.
Senior Software Engineer
Client : OCBC Bank | Employer : Apar Technologies
04.2018 - 07.2019
- Developed and maintained data warehouses and data marts to support business operations.
- Employed vast knowledge of relational database concepts and PL/SQL coding to produce highly efficient data systems.
- Performed systems and data analysis
- · Supporting month end BAU runs and ensure the required reports are submitted to the users on time without any issues.
Senior Associate
Client : Apple | Employer : Cognizant
10.2013 - 04.2018
- Worked as an ETL Lead and involved in ETL design and development activities.
- Data Profiling, Feasibility and Technical Analysis of the Business Requirements.
- Developing stored procedures according to the design and performing data analysis and unit testing after each load.
- Enhanced data quality and accuracy through meticulous testing and validation procedures.
- Developed and maintained documentation for data warehouse architecture, processes, and user guides.
- Analyzed large amounts of data to identify trends and find patterns, signals and hidden stories within data.
- Trained junior developers on best practices in data warehousing techniques to facilitate team growth and knowledge sharing.
Senior Software Engineer
Client : OCBC Bank | Employer:HCL Technologies Ltd
09.2010 - 10.2013
- Extract Data from different source systems using Datastage.
- Create/Modify transformation views to build Datamarts as per business requirements in Teradata.
- Creation of interfaces, packages and scenarios in ODI.
- Create, schedule and run groups (jobs) in Control – M in Development and SIT environment.
- Perform Unit testing and debugging issues in SIT and UAT.
- Providing Production support and resolve issues within given SLA time.
Education
Bachelor of Engineering - Information Science
Dayananda Sagar College of Engineering.
Bengaluru - India06.2010
Skills
- AWS,ALI Cloud
- Bigdata and Hadoop
- SQL , Python and Pyspark
- Realtime : Blink/Flink , Kafka
- Database : Redshift,Snowflake,MongoDb,Hbase,Hive,Teradata
- ETL : Datastage and Informatica
- Visualization : Tableau and FBI
- Airflow
- ControlM
- Workato
- Vertica
- Unix
- Datawarehouse and Business Intelligence
Certification
- AWS Certified Solutions Architect – Associate
- IBM DB2
- Workato Automation Pro
Timeline
Senior Data Engineer
Homage
06.2023 - Current
Manager,Data Engineer
Lazada
07.2020 - 06.2023
Senior Software Engineer
Client : UOB Bank | Employer : WerkMindz Pte Ltd
08.2019 - 07.2020
Senior Software Engineer
Client : OCBC Bank | Employer : Apar Technologies
04.2018 - 07.2019
Senior Associate
Client : Apple | Employer : Cognizant
10.2013 - 04.2018
Senior Software Engineer
Client : OCBC Bank | Employer:HCL Technologies Ltd
09.2010 - 10.2013
Bachelor of Engineering - Information Science
Dayananda Sagar College of Engineering.
Similar Profiles
Joshua VargasJoshua Vargas
Freelance Care Professional at HomageFreelance Care Professional at Homage
Presha MaaranPresha Maaran
Operations Assistant at HomageOperations Assistant at Homage
Jun Hao KhooJun Hao Khoo
Therapy Assistant at HomageTherapy Assistant at Homage