TinyGPT: 50M Parameter LLM trained on TinyStories

[rasbt]

Just came a cross a nice, 50M small LLM by a reader (@NotShrirang) that works surprisingly well: https://github.com/NotShrirang/tinygpt

There's also a nice Streamlit app accompanying it to try it out: https://tinygpt.streamlit.app/

[NotShrirang]

Hey @rasbt, longtime fan here. I've been reading "Ahead of AI" magazine and following your YouTube for a while now. I’ve been working on implementing and pretraining LLMs from scratch since last year (for educational purposes of course), and your content has been a huge influence. Recently, I stumbled upon this "LLM From Scratch" repo and thought it’d be a great opportunity to try out some of the techniques I had learned from it. It’s been an amazing journey since! Thanks for all the inspiration, you truly are a visionary! 💯