Member of Technical Staff (MLE, Pre-Training Data)

• As a Machine Learning Engineer specializing in pretraining data, you will play a pivotal role in developing the data pipeline that underpins Cohere’s advanced language models
• Your responsibilities will encompass the end-to-end management of training data, including ingestion, cleaning, filtering, and optimization, as well as data modeling to ensure datasets are structured and formatted for optimal model performance
• You will work with diverse data sources—such as web data, code data, multilingual corpora, and synthetic data—to ensure their quality, diversity, and reliability
• In this role, you will design and implement scalable, robust pipelines for data processing, conduct data ablations to evaluate quality, and experiment with data mixtures to enhance model performance
• By combining research and engineering, you will bridge the gap between raw data and cutting-edge AI models, directly contributing to improvements in critical training metrics like throughput and accelerator utilization
• Design and build scalable data pipelines to ingest, clean, filter, and optimize diverse datasets, including web data, code data, multilingual corpora, and synthetic data
• Conduct data ablations to assess data quality and experiment with data mixtures to enhance model performance
• Develop robust data modeling techniques to ensure datasets are structured and formatted for optimal training efficiency
• Research and implement innovative data curation methods, leveraging Cohere’s infrastructure to drive advancements in natural language processing
• Collaborate with cross-functional teams, including researchers and engineers, to ensure data pipelines meet the demands of cutting-edge language models

Benefits

• Six weeks’ paid vacation
• Equity / stock options
• RRSP, 401(k), and Pension Scheme contributions
• Coverage for 100% of your insurance premiums across health, dental, vision, and travel
• Additional coverage for accessing mental health providers/services
• Six months of fully paid parental leave, including adoption and surrogacy
• Financial support for egg freezing and IVF in Canada and the UK
• A monthly fitness and wellness allowance
• Globally dispersed company that supports a remote work culture
• A $2,000 annual education benefit for professional development
• A weekly stipend for meals when working remotely and catered lunch when working from one of our global offices
• A monthly arts and culture allowance
• A monthly quality time allowance

• If you are passionate about transforming data into the foundation of AI systems, this role offers a unique opportunity to make a meaningful impact
• Experience working with large-scale datasets, including web data, code data, and multilingual corpora
• Strong software engineering skills, with proficiency in Python and experience building data pipelines
• Knowledge of data quality assessment techniques and experimentation with data mixtures
• A passion for bridging research and engineering to solve complex data-related challenges in AI model training
• Familiarity with data processing frameworks such as Apache Spark, Apache Beam, Pandas, or similar tools
• Bonus: paper at top-tier venues (such as NeurIPS, ICML, ICLR, AIStats, MLSys, JMLR, AAAI, Nature, COLING, ACL, EMNLP)