Lead Big Data Engineer
We are expanding our stellar engineering team to lead our customers through their most critical digital transformation journey, including cloud strategy, migrating from legacy systems, and application modernization through its products and services.
We are looking for someone who has experience working with cloud technologies.
- Build and optimize ‘big data’ pipelines, architectures and data sets.
- Build processes supporting data transformation, data structures, data ingestion, metadata, dependency and workload management.
- Lead business, systems & technical analysis and process design with high touch business and business change contact as required across back, middle and front office functions.
- Lead multiple end to end use case execution by being the SPOC for the technical team and ensuring they align with the overall Enterprise design and architecture
- Mentor team members both onshore and offshore to ensure timely and high quality deliverable
- Work closely with Solution Architect/ Architect to drive solutions architecture, systems and interface design, data analysis, scenario and use case analysis)
- At least 8 years of experience in IT and 5 years using Big Data on AWS.
- Hands-on experience with using Cloud Platform provided Big Data technologies and equally non-big data technologies e.g. EC2, EMR, RedShift, S3 in Amazon Web Services.
- Hands-on experience in setting up Cloud platforms for client’s use-cases.
- Good and practical scripting knowledge in Java ( core java ) , J2EE technologies and Python
- Good knowledge in Big Data technologies – Hadoop, Spark, Hive.
- Experience using programming frameworks - Spark, PySpark, Scala.
- Experience or learning ability in NoSQL Technologies – MongoDB, DynamoDB.
- Good knowledge of Big Data, Relational Database, Data Architecture concepts.
- Strong analytical, problem-solving, data analysis and research skills.
- Good people management skills to manage, guide and mentor a technical team.
- Ability to work with various business teams to resolve technical challenges and understand requirements.
- Demonstrable ability to interact, collaborate, drive consensus and confidence among different (of our) groups – both on-shore and off-shore.
- Demonstrable ability to think outside of the box and not be dependent on readily available tools.
- Experience with Data Science tools & technologies on Cloud (e.g. AWS SageMaker).
- Data analytics product development experience on Cloud.
- Knowledge of Palantir Foundry.