Understanding Transformers Through Llama Model Architecture

Llama Nuts and Bolts is an open-source project on GitHub that rewrites the inference process of the Llama 3.1 8B-Instruct model (80 billion parameters) from scratch using the Go language. The author is Adil Alper DALKIRAN from Turkey.

If you are interested in how LLMs (Large Language Models) and Transformers work and have a basic understanding of the related concepts but still want to dive deeper, then this project is perfect for you!

The biggest features of this project are:

• • Developed from scratch using Go language, without relying on any machine learning libraries or mathematical computation libraries, stepping out of the comfort zone of the Python ecosystem.
• • Equipped with a complete flowchart of large model inference, providing insights into the details of how large models operate.
• • Comprehensive documentation and code explanations that allow you to experience the fundamentals of machine learning, Transformers models, attention mechanisms, RoPE (Rotary Position Embedding), and the underlying mathematical principles.

Here is the code and documentation link for the Llama Nuts and Bolts project:

• • https://github.com/adalkiran/llama-nuts-and-bolts
• • https://adalkiran.github.io/llama-nuts-and-bolts/

It should be noted that this project is developed solely for educational purposes and has not been tested in production or commercial environments. Its goal is to create an experimental project that can perform inference on the Llama 3.1 8B-Instruct model without relying on the Python ecosystem at all.

This project uses the Go language and does not utilize any existing machine learning or mathematical computation libraries, developing a console application from scratch that generates text output using pre-trained Llama 3.1 8B-Instruct model parameters.

The development process of this project allowed the author to delve deep into the internal structure of Transformers models and discover previously unrecognized details, including theoretical knowledge already understood by the author and new content that needed to be relearned, gaining new insights from it.

The first version of Llama Nuts and Bolts was released on March 12, 2024, compatible with the Llama 2 model, while its latest version supports the Llama 3.1 8B-Instruct model.

Without further ado, let’s take a look at the diagram.

Characteristics of Llama Transformers Architecture

Compared to the classic Transformers architecture, the Llama Transformers architecture has several notable features:

• • Decoder-Only Architecture: The Llama pure text model has only a decoder, without an encoder. This means it focuses solely on generating sequences based on the input context without the need for an encoder, relying primarily on the self-attention mechanism to capture dependencies in the input sequence. Llama is a decoder-only model, which means it focuses solely on generating sequences based on the input context. This contrasts with encoder-decoder models like BERT or T5, which utilize an encoder to understand the input and a decoder to generate output.
• • Self-Attention Mechanism: The Llama pure text model does not include cross-attention layers. The Llama self-attention layer is used within the decoder to process the input sequence, while cross-attention layers are more common in encoder-decoder models, where the encoder processes one input (e.g., source language) and the decoder generates output based on that processed information. Llama uses self-attention to capture dependencies in the input text without the need for cross-attention layers. This allows it to generate coherent and contextually relevant text.