A place where I share my thoughts, findings and learnings

HOME
CATEGORIES
TAGS
ARCHIVES
ABOUT

Home Tags trainig

Tag

trainig 1

Transformer showdown MHA vs MLA vs nGPT vs Differential Transformer Jan 22, 2025

Recently Updated

Understanding multi GPU Parallelism paradigms
Attention and Transformer Imagined
Rethink LoRA initialisations for faster convergence
Transformer showdown MHA vs MLA vs nGPT vs Differential Transformer

Trending Tags

Attention FFNN Transformer activations Data Parallelism Differential Transformer Fine tuning GPU GQA kv cache

© 2025 Datta Nimmaturi. Some rights reserved.

Using the Chirpy theme for Jekyll.

Trending Tags

Attention FFNN Transformer activations Data Parallelism Differential Transformer Fine tuning GPU GQA kv cache