A place where I share my thoughts, findings and learnings

HOME
CATEGORIES
TAGS
ARCHIVES
ABOUT

Home Categories Fine tuning

Category

Fine tuning 1

Rethink LoRA initialisations for faster convergence Jun 7, 2024

Recently Updated

Transformer showdown MHA vs MLA vs nGPT vs Differential Transformer
Rethink LoRA initialisations for faster convergence

Trending Tags

activations Differential Transformer Fine tuning GQA kv cache LLM LoRA memory MHA MLA

© 2025 Datta Nimmaturi. Some rights reserved.

Using the Chirpy theme for Jekyll.

Trending Tags

activations Differential Transformer Fine tuning GQA kv cache LLM LoRA memory MHA MLA