Attention Is All You Need — Interactive Transformer Demo

ATTENTION IS ALL YOU NEED

Interactive Transformer Architecture Visualization

📄 Read Original Paper (Vaswani et al., 2017)

This is a working implementation of the Transformer neural network architecture from the groundbreaking paper "Attention Is All You Need" (Vaswani et al., NeurIPS 2017). Watch as the model learns sequence-to-sequence mappings using multi-head self-attention, positional embeddings, and feed-forward layers — no recurrence needed!
Architecture: Embedding Layer → Multi-Head Self-Attention (4 heads) → Feed-Forward Network → Softmax Output

TRANSFORMER COMPONENTS

🔹 Embeddings

Converts tokens into dense 16D vectors

🔹 Multi-Head Attention

4 attention heads compute Q·K^T to learn which tokens relate to each other

🔹 Attention Weights

Softmax over scaled scores creates probability distribution

🔹 Feed-Forward

2-layer network (ReLU activation) transforms attention output

🔹 Backpropagation

Cross-entropy loss guides gradient descent to update all weights

LEGEND:

⚪ White = Current token (focus)
🟡 Yellow = Target (training)
🟢 Green = Generated (prediction)
🔵 Cyan = Vocabulary token