Lorenzo Cavallaro 'Transcending Transcend - Malware Classification in the Presence of Concept Drift'

No day goes by without reading machine learning success stories across different application domains. Systems security is no exception, where ML's tantalizing results leave one to wonder whether there are any unsolved problems left. However, machine learning has no clairvoyant abilities and once the magic wears off, we're left in uncharted territory. For instance, machine learning for malware classification shows encouraging results, but real deployments suffer from performance degradation as malware authors adapt their techniques to evade detection. This phenomenon, largely known as concept drift, occurs as new malware examples or, rather, our understanding of their representation, evolve and become less and less like the original training examples. A promising method to cope with this phenomenon is to equip classifiers with a rejection option in which examples that are likely to be misclassified are instead quarantined until they can be expertly analyzed.

In this talk, Prof. Lorenzo Cavallaro provides a smooth introduction to the field of malware classification leading up to the upcoming IEEE S&P 2022 paper, which proposes TRANSCENDENT. TRANSCENDENT is a rejection framework built around conformal prediction theory that fuels its statistical engine, which equips classifiers with the ability to discern whether examples should be rejected because they will likely be misclassified. Through several case studies, you can see how TRANSCENDENT outperforms state-of-the-art approaches while generalizing across various malware domains and classifiers. These insights support both old and new empirical findings, towards a sound and practical classification with rejection solution. TRANSCENDENT was released as open source, to aid the adoption of rejection strategies by the security community.