The Lead Databricks Administrator will provide problem resolution for referred incidents. The role will also provide administration and management of our internally developed Databricks platform. The role is typically 20% project work and 80% incident management. The role is focused around support of our Databricks platform and associated tools. The team is responsible for the Databricks platform, the enablement of new tools to ensure data governance, adherence to Security standards and the ability to scale our solutions.
RESPONSIBILITIES:
· Investigates and analyzes new technical possibilities, tools, and techniques that reduce complexity, create a more efficient and productive delivery process or create better technical solutions that increase business value.
· Responsible for meeting Service Level Agreement (SLA) targets, and collaboratively ensure team targets are met.
· Ensure all changes to Production systems are planned and approved in accordance with the Change Management process.
· Responsible for 2-3 technologies including complete ownership of all aspects of the technology (maintain, upgrade, train, inspire, and mentor).
· Work with internal customers on resolving Databricks issues. Ability to diagnose our standard deployments and investigate issues to determine root cause.
QUALIFICATIONS, SKILLS, AND EXPERIENCE:
· 3 or more years’ experience providing 2nd and/or 3rd line support in a fast-paced environment.
· Ability to solve problems and work to tight deadlines
· Ability to communicate with core infrastructure teams and non-technical business units
· Strong interpersonal and communication skills.
· Welcomes and embraces change.
· Ability to work in a highly collaborative environment.
· 2+ years of experience with Databricks and integration with other Azure technologies.
· 6+ months of experience with Unity Catalog and use of other Databricks tools such as Databricks Workflows, Delta Sharing, Serverless and SQL Warehouses.
· 2+ years of experience with coding in Python and Spark with performance tuning background
· 1+ years of solid experience with development of Terraform as IaaC
· 1+ years of experience in Terraform Cloud, Azure Functions, Github, CI/CD deployments preferred.
· Experience with cloud cost monitoring and tagging resources in a defined hierarchy
· Experience in troubleshooting data pipelines in Spark
· Experience in Azure Security and Databricks architecture is preferred.
