Transformers Regular Expression Denial of Service (ReDoS) vulnerability
Moderate severity
GitHub Reviewed
Published
Apr 29, 2025
to the GitHub Advisory Database
•
Updated Apr 29, 2025
Description
Published by the National Vulnerability Database
Apr 29, 2025
Published to the GitHub Advisory Database
Apr 29, 2025
Reviewed
Apr 29, 2025
Last updated
Apr 29, 2025
A Regular Expression Denial of Service (ReDoS) vulnerability was identified in the huggingface/transformers library, specifically in the file
tokenization_gpt_neox_japanese.py
of the GPT-NeoX-Japanese model. The vulnerability occurs in the SubWordJapaneseTokenizer class, where regular expressions process specially crafted inputs. The issue stems from a regex exhibiting exponential complexity under certain conditions, leading to excessive backtracking. This can result in high CPU usage and potential application downtime, effectively creating a Denial of Service (DoS) scenario. The affected version is v4.48.1 (latest).References