TechAid 2022 presents TechTalks: Benchmarks & The Evaluation of Research

How do we evaluate our research? The choice of how we evaluate the novelty and effectiveness of our research can have a great impact on whether and where the work is accepted. Not surprisingly, there are numerous cases where research has been published that used flawed methodology to evaluate the work. Nathan Sturtevant, Michael Bowling, and Richard Sutton cover the basics of where benchmarks come from and how they are selected, before moving towards a deeper debate about the right (and wrong) ways for us to evaluate our research and the broader purposes and goals of research in itself.