We’ll be discussing how attention mechanisms (https://arxiv.org/abs/1706.03762) are used in transformer networks to significantly improve their performance.