Vision Transformer Keras Code Examples

This video walks through the Keras Code Example implementation of Vision Transformers I see this as a huge opportunity for graduate students and researchers because this architecture has a serious room for improvement. I predict that Attention will outperform CNN models like ResNets, EfficientNets, etc. it will just take the discovery of complimentary priors, e. g. custom data augmentations or pretraining tasks. I hope you find this video useful, please check out the rest of the Keras Code Examples playlist Content Links: Keras Code Exampes Vision Transformers: Google AI Blog Visualization: Formal Paper describing this model: TensorFlow Addons: TensorFlow Addons AdamW: Chapters 0:00 Welcome to the Keras Code