How Modern LLMs Evolved Beyond the Original Transformer Design
Discover how the Transformer architecture evolved from the 2017 translation model into today's decoder-only LLMs, optimizing memory, hardware efficiency, and context handling.

If you have ever tried to understand a modern large language model, you have likely encountered the familiar two-stack diagram from the 2017 paper Attention Is All You Need. That original architecture was built for translation, featuring an encoder on the left to read a source sentence, a decoder on the right to write the output, and cross-attention connections linking them together. Today, however, popular language models look quite different.
When tasks shifted from translating pairs of texts to generative pretraining—where a model reads a prefix and predicts whatever comes next—the separate text encoder on the left was removed. Causal attention stacks can process input prefixes directly without needing a separate source sentence to encode. Meanwhile, multimodal systems may still use specific encoders for images, and translation models can still use them, but text-only language models streamlined their structure to fit their new jobs.
Managing Memory and Hardware Pressures
As chatbots and text generators learned to handle longer conversations and larger documents, system memory became a major bottleneck. Attention mechanisms rely on keys and values to read information from previous tokens, which are kept in a KV cache during generation to avoid recomputing them. As sequences grow longer, that cache demands substantial memory.
To curb these costs, architectures adopted grouped-query attention, or GQA. By having several query heads share the same key and value head, models drastically reduce the cache payload without altering the basic attention calculation. Other designs like DeepSeek-V3 employ multi-head latent attention to store compressed representations of keys and values, tackling the same long-history memory pressure from a different angle.
Hardware execution constraints also drove structural changes. Standard attention implementations can waste valuable time writing and reading large intermediate score matrices in GPU memory. FlashAttention tackles this by processing data in tiles and tracking normalization statistics on the fly, avoiding the need to materialize the full attention matrix in high-bandwidth memory.
What it means for developers
For software engineers building applications on top of modern AI, understanding these underlying architectural shifts explains why certain models handle long contexts differently or incur varying serving costs. Knowing how components like rotary positional embeddings, SwiGLU feed-forward networks, RMSNorm, and mixture-of-experts routing function helps developers interpret model cards and choose the right tool for their specific performance and token budget requirements. For those looking to experiment, developers can try top AI models cheaply through one API at https://apixoai.online.
Ultimately, while the foundational concepts introduced in 2017 remain vital, today's LLMs have been heavily re-engineered for generative tasks, hardware efficiency, and scalable context handling.
Source: Why Modern LLMs Don’t Look Like the Original Transformer — Towards AI. Written by the Apixo team from that report.
One key for Claude, GPT, GLM, DeepSeek and more. Pay per token with crypto.
Get your API key

