A new paper from researchers at Google DeepMind proposes Soft Mixture-of-Experts (Soft MoE), a novel sparse transformer architecture for image classification.