Interactive RoPE Visualization

See how Rotary Position Embeddings encode token positions through rotation.

Controls

Max Token Position

Set the slider's maximum sequence length.

Token Position (0)

Slide to see how the rotation changes with position.

Hidden Dimensions (16)

The total dimensionality of the token embedding.

Theta Base

The base value used to calculate frequencies.

Key Observation

Notice how the first few dimensions on the left (high frequency) spin rapidly with each step in position. In contrast, the dimensions on the right (low frequency) barely move.

This allows the model to learn both short-range and long-range relationships simultaneously.