Job Title: Data Engineer Databricks
Job Location: Atlanta, GA/ Tampa, FL/ NJ
Key Responsibilities:
- Work closely with analytics architects and AI engineers to support exploratory and production application data preparation needs. Data customers will include human analysts, analytics engineers, AI and analytical models, dashboarding applications, and data stores.
- Set up pipelines to ingest, store, clean, merge, transform, and aggregate data from multiple external and internal data sources
- Set up both batch and streaming pipelines
- Data pipelines and transformations will be built in Azure Databricks
- Data may need to be ported to or from graph data stores such as Neo4j or TigerGraph
Required Knowledge:
- Databricks data engineering (Databricks Professional Data Engineer and/or Spark Developer certification preferred)
- Understanding of batch and streaming data processing for reporting and model inferencing
- Understanding of structured, unstructured, and semi-structured data processing
- Exposure to graph database technologies
- Understanding of financial services risk and security data domain
- Highly proficient Python and SQL skills
- PySpark and Spark SQL
Key Talents
- Adaptability: Data engineering requirements will change frequently as analytical needs and priorities flex. The data engineer who fills this role must be able to understand and quickly adapt to new requests
- Conscientiousness: it is critical the person who fills this role takes in upon themselves to ensure that the output of their pipelines are what is practically needed by the business users and analytics engineers downstream. This person must be able to interpret specification in the context of business needs and work to clarify specifications to ensure that they serve the practical business need.
Preferred Experience:
- 2+ years of Databricks data engineering experience (Databricks Professional Data Engineer and/or Spark Developer certification preferred)
- 7+ years of experience in data engineering pipelines for analytics—must include batch and streaming data
- 3+ years working in a financial services or security related data domain
Life at Capgemini
Capgemini supports all aspects of your well-being throughout the changing stages of your life and career. For eligible employees, we offer:
- Flexible work
- Healthcare including dental, vision, mental health, and well-being programs
- Financial well-being programs such as 401(k) and Employee Share Ownership Plan
- Paid time off and paid holidays
- Paid parental leave
- Family building benefits like adoption assistance, surrogacy, and cryopreservation
- Social well-being benefits like subsidized back-up child/elder care and tutoring
- Mentoring, coaching and learning programs
- Employee Resource Groups
- Disaster Relief
Disclaimer
- Capgemini is an Equal Opportunity Employer encouraging diversity in the workplace. All qualified applicants will receive consideration for employment without regard to race, national origin, gender identity/expression, age, religion, disability, sexual orientation, genetics, veteran status, marital status or any other characteristic protected by law.
- This is a general description of the Duties, Responsibilities and Qualifications required for this position. Physical, mental, sensory or environmental demands may be referenced in an attempt to communicate the manner in which this position traditionally is performed. Whenever necessary to provide individuals with disabilities an equal employment opportunity, Capgemini will consider reasonable accommodations that might involve varying job requirements and/or changing the way this job is performed, provided that such accommodations do not pose an undue hardship.
- Capgemini is committed to providing reasonable accommodations during our recruitment process. If you need assistance or accommodation, please reach out to your recruiting contact.
- Click the following link for more information on your rights as an Applicant http://www.capgemini.com/resources/equal-employment-opportunity-is-the-law
