dev-newsTransformers logohuggingface.co/docs/transformers

Latest stable Transformers: 5.20.2, released (release notes). 118 stable releases since 22 Jan 2024.

Transformers v5

minors since v5.0
20
within 7 days
6
new model entries
97
  1. 5.0major releaseno new models; API refactor, weight loading and tokenizers
  2. 5.110 d4EXAONE-MoEPP-DocLayoutV3Youtu-LLMGlmOcr
  3. 5.211 d4VoxtralRealtimeGLM-5Qwen3.5, Qwen3.5 MoeVibeVoice Acoustic Tokenizer
  4. 5.316 d9EuroBERTVibeVoice ASRTimesFM2.5PP-DocLayoutV2OlmoHybridModernVBertColModernVBertHiggs Audio V2Higgs Audio V2 Tokenizer
  5. 5.423 d13VidEoMTUVDocJina Embeddings v3Mistral4PI0SLANeXtPP-OCRv5_mobile_recPP-OCRv5_server_recPP-OCRv5_mobile_detPPLCNetPPLCNetV3PP-OCRv5_server_detCHMv2
  6. 5.56 d3Gemma4NomicBERTMusicFlamingo
  7. 5.620 d4OpenAI Privacy FilterQianfanOCRSAM3-LiteTextSLANet
  8. 5.76 d2LagunaDEIMv2
  9. 5.87 d6DeepSeek-V4Gemma 4 AssistantGraniteSpeechPlusGranite4VisionEXAONE-4.5PP-FormulaNet
  10. 5.915 d3Cohere2MoeParakeet TDTHRM-Text
  11. 5.10.114 d4Gemma4 unified + MTPSapiens2DeepSeek-OCR-2Mellum
  12. 5.117 d2DiffusionGemmaDeepSeek-V3.2
  13. 5.122 d3MiniMax-M3-VLPP-OCRv6Parakeet-RNNT
  14. 5.1321 d9KimiK 2.5, 2.6, and 2.7MiMo-V2-FlashNemotron 3.5 ASRNemotronAsrStreamingQwen3 ASRZAYAVideoPrismRADIOMiniCPM3
  15. 5.1412 d3InklingTIPSv2TIPSv2 DPT
  16. 5.1526 d4Meta Muse GlimmerGraniteMoeSWA & GraniteSWAA.X-K1 & A.X-K2Cosmos3 Edge
  17. 5.1616 d5Qwen4-ExpGraniteSpeech5Step3p7CohereCompassESMC and ESMFold2
  18. 5.1714 d7HYV4VibeVoiceNeoMMEFun-ASR-NanoKimiLinearCanaryNeuCodec
  19. 5.1821 d4Nemotron 3 DiarizationNemotronH OmniHyperCLOVAX Vision V2GTE
  20. 5.196 d1EmbeddingGemma2
  21. 5.209 d7Muse Spark OpenLing 3.0Kolibri1MiniCPM-V 4.7UnlimitedOcrUltraBERTMOSS-Transcribe-Diarize
Every minor release since v5.0.0 on 26 January 2026, from the GitHub releases of huggingface/transformers. Chips are the entries under “New Model additions” in each release's notes; some entries cover a model family. There was no 5.10.0 tag, so 5.10.1 stands in for it.

Transformers v5.0.0 came out on 26 January 2026, the first major release in five years, with a promise to ship a minor release every week instead of every five weeks so that new models land sooner. Since then 20 minors have shipped in 37 weeks, up to 5.20.0 on 15 October, and their release notes list 97 new model entries between them, from DeepSeek-V4 and Gemma 4 to OCR, speech and diarization models.

Only six of those 20 arrived within seven days of the one before. The longest gap was 26 days between 5.14 and 5.15 over July and August, then 23 days before 5.4 in March and 21 days before both 5.13 and 5.18. 5.20.0 opens its breaking changes with a warning that inference with expert parallelism has speed regressions, removes deprecated cache methods and raises the minimum kernels version to 0.17.2. Two patch releases followed within four days.

As of 24 October 2026. Transformers 5.20.0 release notes.

Latest releases

  1. 5.20.2Fixes input casting in generate that broke multimodal models.
  2. 5.20.1Patch fixes a day after 5.20.0.
  3. 5.20.0Seven new models; deprecated cache methods removed; kernels 0.17.2 or newer.
  4. 5.19.0Adds EmbeddingGemma2.
  5. 5.18.0Adds Nemotron 3 Diarization, NemotronH Omni, HyperCLOVAX Vision V2 and GTE.
  6. 5.17.0Adds HYV4, KimiLinear, Canary and four more.

Upgrading from Transformers v4 to v5

v5.0.0 (26 January 2026) is PyTorch only: the TensorFlow and Jax code is gone, so pin transformers<5 if you still need them. Most v4 code that loads a model with an Auto class and calls generate or Trainer keeps working; what breaks is a list of removed arguments, classes and pipelines.

  1. Rename use_auth_token to token everywhere, and drop safe_serialization=False: v5 always saves safetensors. The default shard size is now 50GB instead of 5GB.
  2. Replace load_in_4bit=True / load_in_8bit=True in from_pretrained with a quantization_config (see the next section). AutoModelWithLMHead is gone: use AutoModelForCausalLM, AutoModelForMaskedLM or AutoModelForSeq2SeqLM; AutoModelForVision2Seq becomes AutoModelForImageTextToText.
  3. Pass processing_class= instead of tokenizer= to Trainer, and move generation settings off the model config onto model.generation_config.
  4. The summarization, translation, question-answering, image-to-text and visual-question-answering pipelines were removed. Use text-generation with a chat model, or image-text-to-text with a vision-language model, and put the task in the prompt:
    from transformers import pipeline

summarizer = pipeline("text-generation", model="Qwen/Qwen3-4B-Instruct-2507")
msgs = [{"role": "user", "content": "Summarize the following text:\n\n..."}]
print(summarizer(msgs)[0]["generated_text"][-1]["content"])

torchscript and torch.fx support were dropped in favour of torch.compile/dynamo and torch.export, and head masking and head pruning were removed. get_text_features and the other get_*_features helpers now return an output object; take .pooler_output for the old tensor.

v5 migration guide v5.0.0 release notes

Loading a model quantized to 4 or 8 bits

Quantization stores weights at lower precision so a model fits in less memory. In v5 you ask for it with a config object passed to from_pretrained; for on-the-fly quantization without a calibration step, bitsandbytes is the usual choice, at 4 or 8 bits.

fp3232 GBbf1616 GB8-bit8 GB4-bit4 GB
Weights alone for an 8-billion-parameter model: parameters × bits ÷ 8. Activations, the KV cache and quantization constants come on top.
  1. Load a 4-bit model with bitsandbytes, computing in bfloat16:
    import torch
from transformers import AutoModelForCausalLM, BitsAndBytesConfig

q = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16)
model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.2-3B", device_map="auto", quantization_config=q
)
  2. For fine-tuning with QLoRA, use bnb_4bit_quant_type="nf4" and bnb_4bit_use_double_quant=True, and train LoRA adapters with PEFT on top of the frozen 4-bit weights.
  3. Models already quantized with GPTQ, AWQ or compressed-tensors carry their own config in the checkpoint, so plain from_pretrained loads them; installing the matching backend library is enough.

The quantization overview lists each method's bit widths, hardware support and whether it supports PEFT fine-tuning, torch.compile and saving with save_pretrained, and links a picker that filters them by model and hardware.

Quantization overview bitsandbytes guide

Choosing an attention backend

Every attention backend computes the same result; they differ in speed and memory. Pick one with attn_implementation when loading, or switch later with set_attn_implementation. "sdpa" is PyTorch's built-in scaled dot product attention; "flash_attention_2" and "flash_attention_3" need a supported GPU.

  1. Choose the backend at load time, or swap it without reloading:
    from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.2-1B", attn_implementation="flash_attention_2"
)
model.set_attn_implementation("sdpa")
  2. To skip building the flash-attn package, load a compiled kernel from the Hub with the Kernels library, for example attn_implementation="kernels-community/flash-attn2". Asking for FlashAttention by name also falls back to the Hub kernel.
  3. Multimodal models take a dict per backbone, for example {"vision_config": "sdpa", "text_config": "flash_attention_2"}. Some vision encoders run better in fp32, and FlashAttention doesn't support fp32.

Paged versions of each backend, such as "paged|flash_attention_2" and "paged|sdpa", are also registered.

Attention backends

Questions people search for

What is the latest version of Transformers?

Transformers 5.20.2, released 19 October 2026 (release notes). Upgrade with pip install -U transformers.

Does Transformers v5 still support TensorFlow?

No. v5 removed the TensorFlow and Jax code and supports PyTorch only. Stay on 4.x (pip install "transformers<5") if you need TensorFlow or Jax models.

Why does load_in_4bit no longer work in from_pretrained?

v5 removed the load_in_4bit and load_in_8bit arguments. Pass quantization_config=BitsAndBytesConfig(load_in_4bit=True) instead.

How often does Transformers release?

v5 promised a minor release every week. In practice 20 minors shipped in the 37 weeks after 5.0.0, and six of them came within seven days of the one before; the chart at the top of this page shows every gap.