We present the Curse of Depth, a phenomenon in Large Language Models (LLMs) where deeper layers contribute less effectively to training due to the widespread use of Pre-Layer Normalization (Pre-LN).
Some results have been hidden because they may be inaccessible to you
Show inaccessible results