Lead Data Engineer (BigData Platform Architect - AWS EMR, Databricks, AWS Athena , GCP Dataproc)
Are you a talented and experienced Data Engineer with a passion for building and maintaining cutting-edge Big Data platforms? Do you thrive in fast-paced, dynamic environments and have a deep understanding of AWS EMR, Databricks, AWS Athena, and GCP Dataproc? If so, we have an exciting opportunity for you to join our team at Visa as a Lead Data Engineer.In this role, you will have the opportunity to work with the latest technologies and design, develop, and implement scalable and efficient data solutions for our global business. As the Lead Data Engineer, you will play a crucial role in shaping and leading our Big Data platform architecture, working closely with cross-functional teams to ensure the success of our data initiatives.We are looking for a highly skilled and motivated individual with strong technical expertise and a proven track record of delivering complex data projects. If you have a passion for data, love working with cutting-edge technologies, and thrive in a collaborative and innovative environment, we want to hear from you! Join us at Visa and be a part of shaping the future of data-driven decision making.
- Design, develop, and maintain cutting-edge Big Data platforms using AWS EMR, Databricks, AWS Athena, and GCP Dataproc.
- Lead and manage the development of scalable and efficient data solutions for global business needs.
- Collaborate with cross-functional teams to understand data requirements and ensure successful implementation of data initiatives.
- Shape and lead the Big Data platform architecture, making strategic decisions to drive the future of data-driven decision making at Visa.
- Stay up-to-date with the latest technologies and trends in the data engineering field and bring innovative ideas to drive continuous improvement in data solutions.
- Ensure high quality, performance, and scalability of data solutions by conducting thorough testing and optimization.
- Mentor and guide other team members to improve their technical skills and deliver high-quality data solutions.
- Collaborate with data scientists, analysts, and other stakeholders to understand their needs and provide efficient data solutions to support their work.
- Identify and troubleshoot data issues, providing timely resolution and preventing future occurrences.
- Ensure compliance with data security and privacy regulations in all data solutions.
- Develop and maintain documentation for data solutions, including data models, architecture diagrams, and technical specifications.
- Participate in code reviews and provide constructive feedback to ensure high-quality coding standards are met.
- Monitor and analyze data pipelines to identify and resolve performance bottlenecks.
- Communicate effectively with team members, stakeholders, and management to provide updates on project progress and any potential issues.
- Continuously evaluate and improve data engineering processes to increase efficiency and productivity.
Extensive Experience With Aws Emr And Gcp Dataproc: The Ideal Candidate For This Role Should Have A Deep Understanding Of Amazon Web Services (Aws) Emr And Google Cloud Platform (Gcp) Dataproc. They Should Be Able To Design, Implement, And Manage Complex Big Data Solutions On These Platforms.
Strong Background In Big Data Architecture And Technologies: A Lead Data Engineer At Visa Should Have A Strong Background In Big Data Architecture And Technologies, Including Hadoop, Spark, And Other Related Tools. They Should Also Have Experience In Data Warehousing, Data Lakes, And Real-Time Data Processing.
Advanced Knowledge Of Databricks And Aws Athena: The Lead Data Engineer Should Have Advanced Knowledge Of Databricks And Aws Athena, As These Are The Primary Tools Used At Visa For Data Analytics And Processing. They Should Be Able To Design And Implement Efficient Data Pipelines Using These Tools.
Leadership And Project Management Skills: As A Lead Data Engineer, The Candidate Will Be Responsible For Leading A Team Of Data Engineers And Architects To Deliver High-Quality Solutions. They Should Have Strong Leadership Skills And Experience In Project Management To Ensure Timely And Successful Project Delivery.
Strong Communication And Collaboration Skills: The Candidate Should Have Excellent Communication Skills To Effectively Communicate With Stakeholders And Team Members. They Should Also Have The Ability To Collaborate With Cross-Functional Teams And Work In A Fast-Paced, Agile Environment.
Data Modeling
ETL
Data Management
Data warehousing
Databricks
Data governance
AWS Athena
AWS EMR
Bigdata Platform Architecture
Gcp Dataproc
Communication
Leadership
Time management
creativity
flexibility
Teamwork
collaboration
Adaptability
Problem-Solving
Empathy
According to JobzMall, the average salary range for a Lead Data Engineer (BigData Platform Architect - AWS EMR, Databricks, AWS Athena , GCP Dataproc) in Bengaluru, Karnataka, India is between ₹2,000,000 - ₹3,000,000 per year. This equates to approximately $28,000 - $42,000 USD per year. However, salaries can vary significantly depending on the individual's experience, skills, and the specific company they work for.
Apply with Video Cover Letter Add a warm greeting to your application and stand out!
Visa Inc. is an American multinational financial services corporation headquartered in Foster City, California, United States. It facilitates electronic funds transfers throughout the world, most commonly through Visa-branded credit cards, debit cards and prepaid cards.
Get interviewed today!
JobzMall is the world‘ s largest video talent marketplace.It‘s ultrafast, fun, and human.
Get Started