Work & Co, part of Accenture Song, is a global design and technology company with offices in the United States, Europe, and Latin America. We're known for uniting the industry’s best hands-on strategy, product and technical talent to generate tangible impact at scale. Client partners include IKEA, Apple, PGA TOUR, Gatorade, Google, Mercedes, Givenchy, the MTA and more. As Fast Company noted, our engineering and design teams are routinely entrusted with creating digital products for companies which rarely approach outside firms. Forrester Research calls our approach “a model to follow.”
Creating digital products is a team game. Our methodology emphasizes intense collaboration, early prototyping and testing, and so many iterations you’ll lose count. Design Directors work closely with Product Managers, Strategists, and Developers to create digital products and services that reshape industries while mentoring others in the Work & Co Design team. Our Partners are hands on and you’ll work directly with them — no middle management here.
What You Will Do
Partner with a supporting data scientist, product manager, and larger product team on a new-to-market generative AI product to define the Machine Learning evaluation & data science strategy informing content strategy decisions and technical optimizations.
Responsibilities
- Evaluate the performance of AI models that leverage Large Language Models (LLMs) in various applications.
- Build, improve, and validate evaluation models while ensuring they meet business requirements.
- Conduct thorough analyses to identify strengths and weaknesses in model performance.
- Develop and implement metrics to measure the accuracy, efficiency, and overall effectiveness of LLM-based AI models.
- Manage a small team of ML engineers and Data Scientists.
- Collaborate closely with the Product team to develop and execute a roadmap for evaluating the output of various AI models.
- Required daily participation in client-facing standup and workshops
- Required daily participation in client-facing discussions on Teams
- Required weekly participation in client-facing demos
- Required availability from 9:00am EST-6:00pm EST via normal communication channels (Slack, Zoom, Teams, E-mail, etc.)
Required Qualifications:
- Bachelor’s degree or equivalent in statistics, applied mathematics or equivalent practical experience
- Experience with Python
- Experience evaluating LLM results using different methodologies
- Knowledge of NLP techniques
- Domain knowledge in ML models evaluation and data curation techniques.
- Extensive experience in designing and implementing ML benchmarking, extending previous methods.
- Knowledge of machine learning libraries such as TensorFlow, PyTorch is essential.
- Familiarity with performance evaluation metrics and techniques for LLMs.
Preferred Qualifications:
- Prior experience working on a product involving generative AI that is live in market.
At Work & Co, we believe diversity is essential to creating the best digital products. We're proud to be an equal opportunity employer committed to diversity and inclusion. We do not discriminate on the basis of race, gender, ethnicity, citizenship, national origin of employees, religion, sexual orientation, age, marital status, disability, veteran status or any other legally protected status.
The base pay for the AI Research Scientist / Engineer (Consultant) role is between $125 and $135 per hour. The actual hourly rate is dependent upon many factors, such as: experience, knowledge, skills, and market demands. The base pay range is subject to change and may be modified in the future. Along with competitive pay, full-time employees are eligible for full benefits.
