LA-LLAMA3 is a minimalist implementation of the LLAMA3 algorithm, primarily using linear algebra operations, and its main program is confined to just 100 lines of code. This project serves as a tool to provide a clear understanding of the fundamental workings of the LLAMA3(and generally, Transformer) architecture.
And it actually works.
For a version with KV Cache, check out the kv_cache branch.