Senior Member of Technical Staff (Web Data)

• As a Senior Member of Technical Staff specializing in web data for pre-training, you will play a pivotal role in developing the large scale web data pipeline that underpins Cohere’s advanced language models
• In this role, you will work extensively with large-scale web corpora, transforming raw, noisy internet data into high-quality training data for pretraining
• You will own key components of the data pipeline, including extraction, parsing, deduplication, and filtering
• You will also analyze the composition and quality of web data, study its impact on downstream model performance, and collaborate closely with the broader data and evaluation teams to iterate on the training corpus
• Your work will be essential to Cohere’s mission of delivering efficient and reliable language understanding and generation capabilities, driving innovation in natural language processing
• Maintain large-scale pipelines for processing web corpora
• Work on filtering and quality-scoring systems to identify high-value web documents
• Analyze web data composition across domains, languages and time periods
• Develop and maintain highly-performant deduplication pipelines
• Collaborate with cross-functional teams, including researchers and engineers, to ensure data pipelines meet the demands of cutting-edge language models

Benefits

• Six weeks’ paid vacation
• Equity / stock options
• RRSP, 401(k), and Pension Scheme contributions
• Coverage for 100% of your insurance premiums across health, dental, vision, and travel
• Additional coverage for accessing mental health providers/services
• Six months of fully paid parental leave, including adoption and surrogacy
• Financial support for egg freezing and IVF in Canada and the UK
• A monthly fitness and wellness allowance
• Globally dispersed company that supports a remote work culture
• A $2,000 annual education benefit for professional development
• A weekly stipend for meals when working remotely and catered lunch when working from one of our global offices
• A monthly arts and culture allowance
• A monthly quality time allowance

• Experience working with large-scale web datasets
• A passion for bridging research and engineering to solve complex data-related challenges in AI model training
• Strong software engineering skills, with proficiency in Python and experience building data pipelines
• Knowledge of data quality assessment techniques and experimentation with data mixtures
• Familiarity with data processing frameworks such as Apache Spark, Apache Beam, Pandas, or similar tools
• If you are passionate about transforming data into the foundation of AI systems, this role offers a unique opportunity to make a meaningful impact
• If some of the above doesn’t line up perfectly with your experience, we still encourage you to apply!
• Paper at top-tier venues (such as NeurIPS, ICML, ICLR, AIStats, MLSys, JMLR, AAAI, Nature, COLING, ACL, EMNLP)