Data Engineer Lead / Architect
Jacksonville, FL / Irving, TX (Hybrid – 3 Days a Week)
Contract Type: W2 / Self Corporation
Long Term
Qualifications:
- Over 10+ years of total IT experience 5+ years of experience with Hadoop (Cloudera)/big data technologies
- Advanced knowledge of the Hadoop ecosystem and Big Data technologies Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Hive, Pig, Impala, Spark, Kafka, Kudu, Solr)
- Experience on designing and developing Data Pipelines for Data Ingestion or Transformation using Scala or Python.
- Experience with Spark programming (PySpark )
- Expert level building pipelines using Apache Spark Familiarity with core provider services from AWS.
- Hands-on experience with Python/Pyspark and basic libraries for machine learning is required
- Exposure to containerization and related technologies (e.g. Docker, Kubernetes) a plus
- Exposure to aspects of DevOps (source control, continuous integration, deployments, etc.)
- Proficient in programming in Python with prior Apache Beam/Spark experience a plus.
- System level understanding - Data structures, algorithms, distributed storage & compute
- Can-do attitude on solving complex business problems, good interpersonal and teamwork skills
- Possess team management experience and have led a team of data engineers and analysts
- Experience with Snowflake is highly preferred.
Education:
Bachelor’s degree/University degree or equivalent
Warm Regards,
Aravindharaj.A
________________________________________________________
Associate Lead – US TAG
Newt Global LLC