Nonlinear Advantage
Recently, with the democratization of big open source fundation models such as Meta’s llama 2 or stable diffusion, many people gained interest in model pruni...
This is a blog about my research ideas. I try to keep things simple and visual.
Recent Posts
ChatGPT Claims It Can Do Zero-Shot Image Classification (I Called Its Bluff)
OpenAI’s latest large language model (LLM) ChatGPT is infamous for it’s zero-shot capacities: it can effortlessly complete tasks that it hasn’t explicitly be...
How ResNets fix the Loss Surface
In the last post of this series, we came to understand that harmonic distortions created by repeatedly applying ReLU layers are responsible for the rough los...
Why the Loss Surface of Deep Networks is broken
Studying loss surfaces is just a visual way of studying the optimization process of a neural network. My journey through the world of loss surfaces began whe...