A place where I share my thoughts, findings and learnings

HOME
CATEGORIES
TAGS
ARCHIVES
ABOUT

Home Categories Architectures

Category

Architectures 1

Transformer showdown MHA vs MLA vs nGPT vs Differential Transformer Jan 22, 2025

Recently Updated

Transformer showdown MHA vs MLA vs nGPT vs Differential Transformer
Rethink LoRA initialisations for faster convergence

Trending Tags

activations Differential Transformer Fine tuning GQA kv cache LLM LoRA memory MHA MLA

© 2025 Datta Nimmaturi. Some rights reserved.

Using the Chirpy theme for Jekyll.

Trending Tags

activations Differential Transformer Fine tuning GQA kv cache LLM LoRA memory MHA MLA