Self-attention is a way of reading.

Before attention, there are tokens.

Before a Transformer can decide what matters, it needs a way to represent what it has been given. A sentence is split into tokens: words, word fragments, punctuation, or other pieces of a sequence. Those tokens are mapped to numbers and then to vectors. The model does not receive a sentence as a human reader does; it receives a structured field of representations.

That first translation is easy to overlook, but it shapes everything that follows. A token is not the meaning of a word by itself. It is a point the network can compare with other points, and tokenization determines which pieces can meet inside the later attention calculation.

A machine that reads in parallel.

The Transformer architecture in the paper has two broad jobs. The encoder analyzes the input and builds an abstract representation of it. The decoder uses that representation to produce an output, one step at a time. Between the two, the model has a way to connect pieces of the input with the parts of the output they help explain.

The important change from older recurrent approaches is parallelism. The relationships inside a sequence can be calculated together, which lets GPUs and TPUs do the kind of simultaneous work they are built for. The output of a language model may still arrive word by word, but the context used to choose each word can be prepared in parallel.

Attention is context.

The paper's example is deliberately ordinary: “Die Bank liegt am Fluss.” The word Bank could describe a place to sit or a financial institution. The surrounding words do the disambiguating. Self-attention gives the model a way to ask which other tokens help decide what Bank means here.

For every token, the mechanism creates three learned projections: Query, Key, and Value. The Query represents what the current token is looking for. The Key represents what another token can offer as a match. The Value carries the information that will be mixed into the current representation if the match matters.

The formal operation compares Queries and Keys, turns the scores into weights, and applies those weights to the Values. In its familiar shorthand, scaled dot-product attention can be written as softmax(QKᵀ / √dₖ) · V. The equation is compact; the design idea is more intuitive: context is built by deciding which relationships deserve more weight.

Many heads, one sequence.

One attention calculation can notice one kind of relationship. Multi-Head Attention runs several of them in parallel, allowing different heads to specialize in different patterns. One may be more sensitive to grammatical structure, another to a semantic connection, while all of them still work with relationships between tokens rather than isolated words.

Those projections are not hand-written rules. During training, the model adjusts the matrices that create its Queries, Keys, and Values by measuring the error between a prediction and the expected result. Fine-tuning then adapts a model that already knows broad patterns to a narrower domain such as medicine, mathematics, or finance.

Pocket AI keeps the model close.

The paper becomes more concrete when it leaves the architecture diagram and asks where a model should run. Pocket AI is presented as a local AI app: a small language model runs on the user's own iPhone or iPad instead of sending every prompt to a remote server.

The app's model choices include Llama 3.2 in 1B and 3B sizes, as well as a 1.5B DeepSeek-R1-Distill-Qwen model. Quantization makes that range more practical by reducing the precision and footprint of the model. It is not magic compression; it is a trade between capability, memory, speed, and the hardware available in a pocket.

That choice changes the product. A local model can reduce the need to send personal prompts elsewhere, but the device now carries the cost of inference. Privacy, latency, battery, memory, and model quality become part of the interface rather than invisible infrastructure behind it.

A small performance study.

On an iPhone 13 Pro, the paper reports three different generation rates: 11, 22, and 42 tokens per second for the three small models. The order matters less than the shape of the result. A consumer device can run several useful model sizes locally, but each step up changes the balance between speed, memory, and capability.

The same experiment reports up to 1.5 GB of memory during text generation. These are measurements from one local study, not universal promises about every phone or every prompt. Their value is that they make the tradeoff visible: the model's architecture eventually becomes a physical experience of waiting, heat, and available space.

Reported throughput

11 / 22 / 42 tokens/s

Llama 3.2 1B / 3B / DeepSeek-R1-Distill-Qwen 1.5B

Generation memory

Up to 1.5 GB

Reported during local text generation on an iPhone 13 Pro

Reasoning is a loop.

The paper's final movement looks beyond the classic next-token model toward Reasoning Models. It describes systems such as DeepSeek R1 and OpenAI o1/o3 as models trained to work through problems in more deliberate steps. The supplied architecture diagram gives that idea a visible shape: a query becomes a plan, the plan is checked, steps are executed, findings are gathered, and the result is validated.

A loop like this is useful as a systems diagram, not as proof of human-like thought. It shows where planning and validation could live in a product. It also suggests why smaller models can be surprisingly capable on narrow tasks: the quality of the process and the fit of the training data matter alongside the raw parameter count.

The limit is not only compute.

Self-attention brings context, not truth. A language model can complete a pattern with a sentence that sounds confident and is still factually wrong. Running the model locally may change who handles a prompt, but it does not make the model accurate by default.

The paper also points toward the cost of longer context. More tokens create more possible relationships, and classic attention pays for those relationships with additional computation. Sparse Attention is one direction for reducing that cost. Longer context is therefore not a free upgrade; it is another architectural decision.

Privacy, transparency, accuracy, and resource use belong in the same conversation. The most useful local model is not simply the largest one that fits. It is the one whose behavior, limits, and costs a person can understand well enough to choose deliberately.

A way of reading, not a way of knowing.

A Transformer does not need to understand a sentence as a person does in order to be useful. It needs a powerful way to represent relationships, compare context, and turn a sequence of weighted signals into a next step. Self-attention is that way of reading: selective, parallel, and astonishingly scalable.

Pocket AI makes the idea tangible. The architecture does not stay in a diagram or a paper; it becomes a tradeoff on a device someone carries. That is the useful question to carry forward: not only what a model can generate, but what kind of relationship with the model a product makes possible.