Job Title: Specialist - Data Engineering
Work Location: Tampa, Florida- Onsite
Full Time / Permanent
Python with PySpark
Job Description:
Mandatory Certification
Requisite Industry certification needed for this Position Candidate should have a valid relevant External Certification
MANDATORY
Senior Python LEAD Data Engineer 7 8 yrs of total IT experience having
1 Core Skills Min 5 yrs of Strong Core Python experience covering Oops concepts Unittest lib and Panda lib
2 Secondary Skills He She should have min 1 yrs experience on PySpark lib and Hive SQL
3 Good to have basic knowledge of Shell script
SKILLS REQUIRED
8 years of extensive work experience and deep knowledge in Core Python programming along with experience in writing python testcases with UnitTest library
Should have good Team Leadership capabilities and experience
Good knowledge and work experience in Unix commands Shell scripts etc
Good to have at least 1 year exp on Pyspark Hadoop and Hive as well
Expertise in software engineering principles such as design patterns code design testing and documentation
Writing effective and scalable codes implementing robust and maintainable Python applications
Extensive experience in all the below concepts to build frameworks
Modules Classes Methods Iterators Inheritance Polymorphism Abstract classes Decorators Generators Metaclasses
Magic methods Datastructures collections itertools
NumPy Pandas SciPy
Creating and working with Virtual Environment and installing required packages using pip
Regular expressions
Anaconda Working with Jupyter Notebook on AEN
Working with REST API Exception handling Regular expressions
multithreading multiprocessing Asynchronous programming
Lambda functionality
Strong problem solving skills
A deep understanding and multi process architecture and the threading limitations of Python
Good in database systems such as SQL NoSQL and Dev Ops tools like Git Jenkins etc
Developing and maintaining data pipelines and ETL processes Debugging applications to ensure low latency and high availability
Ability to collaborate on projects and work independently and provide technical solutions
Excellent communication skills
Banking domain knowledge is an added advantage
