Fourier Transform Replaces Transformer Self-Attention Layer
Machine Heart reports Machine Heart Editorial Team The research team from Google indicates that replacing the transformer self-attention layer with Fourier Transform can achieve 92% accuracy on the GLUE benchmark, with training times 7 times faster on GPU and 2 times faster on TPU. Since its introduction in 2017, the Transformer architecture has dominated the … Read more