Llama3-Remake.py

🚀 Llama3 remake for educational purposes to help understand and experiment with the Llama3 model as described in the Llama3 release.

Structure

Behaviour

How to Get Started

1. Clone the Repository

   git clone https://github.com/llama3-py/llama3-remake.py
   cd llama3-remake.py
   

2. Install Dependencies Make sure you have Python installed, then run:

   pip install -r requirements.txt
   

Learning Resources and References

• Original Llama3 Release
• Llama3 Model Cards & Prompt formats
• Original Llama2 PAPER
• Rotary Position Embedding
• Root Mean Square Layer Normalization
• Attention Is All You Need
• PyTorch Official Documentation

Citation

If you use llama3-remake in your research, please cite the following work:

@misc{llama3-remake-2024,
  author = {Aakash Apoorv},
  title = {llama3-remake.py},
  year = {2024},
  howpublished = {\url{https://github.com/llama3-py/llama3-remake.py}},
}