Principal, Infrastructure (network, storage, database) Architecture - Kafka Platform
This is an infrastructure position focused on redesigning and developing Kafka platforms, Flink
They need to be strong in Kafka, Flink, and have knowledge of Kubernetes, containerization and devops
SALARY: $200k - $210K - $215K AND 30% BONUS
LOCATION: CHICAGO, IL
3 DAYS ONSITE AND 2 DAYS REMOTE, HYBRID
You will drive the architectural vision of core real time data streaming thru the infrastructure platform. Kafka, flink and k8 hybrid onsite cloud aws or azure on premises. Networking infrastructures as a code, platform service, storage, database, network, security, IAM, microservices. 10 years java applications knowledge kafka flink kubernetes
- Collaborate with cross-functional teams to design, create and review software application architectures specifically tailored for streaming use cases.
- Ensure fault tolerance, scalability, and low-latency processing in streaming applications.
- Collaborate with DevOps teams to define deployment strategies and manage scalability.
- Drive optimization of streaming application performance by fine-tuning configurations, monitoring resource utilization, and identifying bottlenecks.
- Drive Implementation of best practices for efficient data serialization, compression, and network communication.
- Create and maintain architecture documentation, including system diagrams, data flow, and component interactions.
- Maintain vendor relationships and participate in escalation sessions and postmortems
- Evaluate and recommend tools and frameworks that enhance the performance and reliability of our streaming systems.
- Stay informed about industry trends related to Kafka, Flink, and Kubernetes.
Qualifications:
- Ability to execute spikes and provide code samples demonstrating best practices when developing solutions on Kafka and Flink.
- Experience with DevOps tools, e.g. Terraform, Ansible, Jenkins, Kubernetes, Helm and CI/CD pipeline etc.
Technical Skills:
- Expert level knowledge of Kafka
- Expert level knowledge of Flink
- In depth knowledge of on-premises networking as well as the hybrid connectivity to AWS and/or Azure
- Knowledge of Infrastructure as a Service (IaaS), Platform as a Service (PaaS), compute, storage, database, network, content distribution, security/IAM, microservices, management, and serverless services
- Knowledge of Infrastructure as Code (IaC) such as Terraform, CloudFormation, or Azure Resource Manager
- Knowledge of containerization technologies like Docker and orchestration tools like Kubernetes
- 10+ years of experience architecting of mission critical Cloud and On-Prem real-time data streaming and event-driven architectures
- 10+ years of experience with Java
- 5+ years of specific Kafka and Flink experience