A place where I share my thoughts, findings and learnings

HOME
CATEGORIES
TAGS
ARCHIVES
ABOUT

Home Tags Kernels

Tag

Kernels 1

The MathemaTricks behind FlashAttention Apr 12, 2026

Recently Updated

The MathemaTricks behind FlashAttention
Exploring the Mixture of Experts
The lore behind LoRA
Rethink LoRA initialisations for faster convergence
Understanding multi GPU Parallelism paradigms

Trending Tags

Transformer Math Attention FFNN Finetuning GPU LoRA Training activations Data Parallelism

© 2026 Datta Nimmaturi. Some rights reserved.

Using the Chirpy theme for Jekyll.

Trending Tags

Transformer Math Attention FFNN Finetuning GPU LoRA Training activations Data Parallelism