← notes

AI from first principles

"Attention is all you need" paper

source

Basically, where it all started from.

Previous state of the art models, relied on recurrent neural networks, which had sequential processing due to the handling of internal hidden states, and while mechanisms to improved it exists (like factorization or conditional computation), the fundamental sequential constraint still exists. The paper proposes a better mechanism for "attention" with the "transformer" model, which allows for more parallelizable work.

3blue1brown Neural networks playlist

source

Transformers, the tech behind LLMs

source

The goal of a transformer is predicting the next word in a text. This prediction comes in the shape of a probabilistic distribution.

The first step in the model is to break up the input into chunks called tokens which can be words or character combinations. Each token is associated with a vector that somehow encode the meaning of this tokens.

The tokens go through an attention step, which allows each vector "talk" to each other, passing data back and forth to update their values. Basically gives context to each token. They then go through a "Multilayer perceptron" or "Feed-forward layer" in parallel. This process repeats, until the prediction is baked into the last vector.

The model needs a predefined vocabulary, and the first matrix in the model (Embedding matrix) has a single column for each word. This encodes the "meaning" of each token, and has seemingly random values at first, but is updated ("learned") based on data.

We build the input of the model by "plucking" the vector of each token into the attention step, which then modify each vector based on the distance to each other vector (giving context). The network can only process a fixed number of vectors at a time, known as the context size (GPT3 was 2048, GPT5 is 400k, MiniMax 2.5 is 205k)

Let's build GPT: from scratch, in code, spelled out.

source

recommends his "makemore" series as preamble

Tokenization

He starts getting his vocabulary size by getting a set of all characters found in the first 1000 characters of the shakespear file. The next step is to tokenize this vocabulary, which means convert the raw text to some sequence of integers. In the example, this integer is just the position of the character in the vocabulary.

I thought each token was a vector, but in here they are plain scalars

Some production used tokenizers examples are:

After getting the encoding, we tokenize the entire dataset, turning it into big array of integers. He splits this encoded array getting a training dataset and a validation data set. (90%/10% of the dataset) The training is done by "batches of chunks" as doing it with the entire dataset would be computationally prohibitive. Each chunk has a "mirror" one, which has its contents as the chunk shifted by one in the dataset

The training chunk is fed into the transformer and it aims to achieve the results of the validation chunk.

Training (@ 22:11)

BigramLanguageModel? He refers to the makemore playlist again

He somehow got a prediction from the torch.nn.Embedding(vocab_size, vocab_size) and afterwards calculated the loss function of the results (the quality of the prediciton) with cross_entropy of the validation chunk againts the prediciton

is pytorch the defacto library for all of this? -> Yes, PyTorch is the standard for deep learning research.

Ah! Ok, so the first torch.nn.Embedding is just a random matrix of weights. Its literally the untrained embedding matrix, no training has occured.

The library seems to do everything, the loss object has a backward function which seems to do the backpropagation(?) and then its just a matter of looping indefinitely, until the loss function reaches an optimal(?) (Guessing before watching the rest) At this point there's no "attention"

He goes over some matrix multiplication trick to optimize the computation of the average of the context. Basically:

[1   , 0   , 0   ]   [2]   [2] 
[0.5 , 0.5 , 0   ] x [4] = [3] 
[0.33, 0.33, 0.33]   [6]   [4]

With the last vector being the rolling average of the vector we are multiplying against

"The crux of the video" (@ 1:02:15)

He goes over the construction of the "attention", where the matrix multiplication gets new values where the weights get modified, not for the average but for their value in context.

RLHF & PPO 10,000ft overview

source

Step performed with a pretrained model where we finetune the results. There's supervided fine tuning and Instruction fine tuning.

General references:

Conclusion

I now have a better understanding of AI, still not motivated to build with it or dive deeper into the subject, but I can see its place as a new tool rather than blind hate.

It's token prediction based on available context and the relationships of the tokens within it.

The underlying math is more interesting to me, as it might be transferable to other things in game dev (physics, AI behaviors, procedural generation)