A place where I share my thoughts, findings and learnings

HOME
CATEGORIES
TAGS
ARCHIVES
ABOUT

Home Tags Mixture of Experts

Tag

Mixture of Experts 1

Exploring the Mixture of Experts Feb 24, 2026

Recently Updated

The MathemaTricks behind FlashAttention
Exploring the Mixture of Experts
The lore behind LoRA
Rethink LoRA initialisations for faster convergence
Understanding multi GPU Parallelism paradigms

Trending Tags

Transformer Math Attention FFNN Finetuning GPU LoRA Training activations Data Parallelism

© 2026 Datta Nimmaturi. Some rights reserved.

Using the Chirpy theme for Jekyll.

Trending Tags

Transformer Math Attention FFNN Finetuning GPU LoRA Training activations Data Parallelism