A place where I share my thoughts, findings and learnings

HOME
CATEGORIES
TAGS
ARCHIVES
ABOUT

Home Categories Architectures

Category

Architectures 1

Transformer showdown MHA vs MLA vs nGPT vs Differential Transformer Jan 22, 2025

Recently Updated

The MathemaTricks behind FlashAttention
Exploring the Mixture of Experts
The lore behind LoRA
Rethink LoRA initialisations for faster convergence
Understanding multi GPU Parallelism paradigms

Trending Tags

Transformer Math Attention FFNN Finetuning GPU LoRA Training activations Data Parallelism

© 2026 Datta Nimmaturi. Some rights reserved.

Using the Chirpy theme for Jekyll.

Trending Tags

Transformer Math Attention FFNN Finetuning GPU LoRA Training activations Data Parallelism