Software Engineer, Data Infrastructure
Overview: We are a technology company focused on enhancing the partnership between humans and computers to achieve groundbreaking advancements. Our approach spans from user experience to model optimization, aiming to maximize user value per computational effort.
About the Company: We are a small, driven team committed to making significant strides in real-world AI applications. Supported by leading venture capital firms and major tech companies, we are financially stable and positioned for long-term success. Our team is diverse and multidisciplinary, dedicated to solving complex AI challenges.
About the Role: As a Data Infrastructure Engineer, you will play a crucial role in designing, implementing, and optimizing scalable systems to handle vast amounts of data for AI training. This role demands close collaboration with our data research and data crawling teams to ensure the infrastructure is both reliable and efficient.
What We Can Offer You:
- Competitive salary and comprehensive benefits package
- Opportunities for professional growth and development
- Collaborative and innovative work environment
- Relocation assistance for new employees
- Access to cutting-edge technology and resources
- Fully onsite work environment in San Francisco
Key Responsibilities:
- Develop and maintain petabyte-scale data processing systems for AI training datasets
- Manage workloads across large computing clusters
- Architect and sustain distributed computing environments
- Implement new data preparation methods in collaboration with the data research team
- Quickly troubleshoot and resolve infrastructure-related issues
Qualifications:
- Over 6 years of experience in data-intensive applications and software development
- Proficient in Kubernetes and containerization
- Experience with cloud services such as AWS, GCP, etc.
- Strong programming skills in languages like Go, Rust, or C++
- Background in building and maintaining infrastructure for ML model training data processing
Relevant Keywords: Data Infrastructure, Scalable Systems, AI, Kubernetes, Containerization, Cloud Services, Data Orchestration, Software Development, Go, Rust, C++, Machine Learning
