A place where I share my thoughts, findings and learnings

HOME
CATEGORIES
TAGS
ARCHIVES
ABOUT

Home Tags LLM

Tag

LLM 1

Rethink LoRA initialisations for faster convergence Jun 7, 2024

Recently Updated

Understanding multi GPU Parallelism paradigms
Attention and Transformer Imagined
Rethink LoRA initialisations for faster convergence
Transformer showdown MHA vs MLA vs nGPT vs Differential Transformer

Trending Tags

Attention FFNN Transformer activations Data Parallelism Differential Transformer Fine tuning GPU GQA kv cache

© 2025 Datta Nimmaturi. Some rights reserved.

Using the Chirpy theme for Jekyll.

Trending Tags

Attention FFNN Transformer activations Data Parallelism Differential Transformer Fine tuning GPU GQA kv cache