Tapering Away Normalization in Transformers

Gated removal of normalization in Transformers for stable training and faster inference.

References

2026

  1. Gated Removal of Normalization in Transformers Enables Stable Training and Efficient Inference
    Andrei Kanavalau, Carmen Amo Alonso, and Sanjay Lall
    2026
    HTML