Architectures Beyond CNNs and Visual Scaling Laws (Neil Houlsby) | Tutorial (1/3)

ECCV CVinW Workshop Invited Talk: Architectures Beyond CNNs and Visual Scaling Laws (Neil Houlsby) | Google Brain Zurich Team Tutorial (1/3)

Abstract: I will present some of our work that has explored the capabilities of non-convolutional architectures for Computer Vision, such as Transformers, Mixers, and Mixture-of-expert based models. These architectures often demonstrate favourable properties in the context of transfer learning from a large source dataset to a small target datasets. In this context, I will discuss our exploration into these models' scaling laws, improved scaling law estimators, and the apparent saturation of larger vision models.