#7: Behavioral Testing with RecList for Recommenders with Jacopo Tagliabue

Episode number seven of Recsperts deals with behavioral testing for recommender systems. I talk to Jacopo Tagliabue, who is the founder of tooso and now director of artificial intelligence at Coveo. He made many contributions to various conferences like SIGIR, WWW, or RecSys. One of them is RecList, which provides behavioral, black-box testing for recommender systems.

In episode number seven, we meet Jacopo Tagliabue and discuss behavioral testing for recommender systems and experiences from ecommerce. Before Jacopo became the director of artificial intelligence at Coveo, he had founded tooso, which was later acquired by Coveo. Jacopo holds a PhD in cognitive intelligence and made many contributions to conferences like SIGIR, WWW, or RecSys. In addition, he serves as adjunct professor at NYU.

In this episode we introduce behavioral testing for recommender systems and the corresponding framework RecList that was created by Jacopo and his co-authors. Behavioral testing goes beyond pure retrieval accuracy metrics and tries to uncover unintended behavior of recommender models. RecList is an adaption of CheckList that applies behavioral testing to NLP and which was proposed by Microsoft some time ago. RecList comes with an open-source framework with ready set datasets for different recommender use-cases like similar, sequence-based and complementary item recommendations. Furthermore, it offers some sample tests to make it easier for newcomers to get started with behavioral testing. We also briefly touch on the upcoming CIKM data challenge that is going to focus on the evaluation of recommender systems.

In the end of this episode Jacopo also shares his insights from years of building and using diverse ML Ops tools and talk about what he refers to as the "post-modern stack".

Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.

Links from the Episode:

Papers:

General Links: