SCAO — Optimizer
I built a 2nd-order optimizer for LLMs.
What is SCAO — Optimizer?
54% faster LLM training. SCAO is a sparse, second-order PyTorch optimizer designed as a high-throughput, drop-in replacement for AdamW. - whispering3/scao
Screenshots
?
No comments yet. Be the first!
Real conversations about SCAO — Optimizer on X
Post on X