Ecosyste.ms: Advisories
An open API service providing security vulnerability metadata for many open source software ecosystems.
Security Advisories: GSA_kwCzR0hTQS03ajdtLXY3bTMtanFtN84AA5Vh
Scrapy decompression bomb vulnerability
Impact
Scrapy limits allowed response sizes by default through the DOWNLOAD_MAXSIZE and DOWNLOAD_WARNSIZE settings.
However, those limits were only being enforced during the download of the raw, usually-compressed response bodies, and not during decompression, making Scrapy vulnerable to decompression bombs.
A malicious website being scraped could send a small response that, on decompression, could exhaust the memory available to the Scrapy process, potentially affecting any other process sharing that memory, and affecting disk usage in case of uncompressed response caching.
Patches
Upgrade to Scrapy 2.11.1.
If you are using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.11.1 is not an option, you may upgrade to Scrapy 1.8.4 instead.
Workarounds
There is no easy workaround.
Disabling HTTP decompression altogether is impractical, as HTTP compression is a rather common practice.
However, it is technically possible to manually backport the 2.11.1 or 1.8.4 fix, replacing the corresponding components of an unpatched version of Scrapy with patched versions copied into your own code.
Acknowledgements
This security issue was reported by @dmandefy through huntr.com.
Permalink: https://github.com/advisories/GHSA-7j7m-v7m3-jqm7JSON: https://advisories.ecosyste.ms/api/v1/advisories/GSA_kwCzR0hTQS03ajdtLXY3bTMtanFtN84AA5Vh
Source: GitHub Advisory Database
Origin: Unspecified
Severity: High
Classification: General
Published: 3 months ago
Updated: 16 days ago
CVSS Score: 7.5
CVSS vector: CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H
Identifiers: GHSA-7j7m-v7m3-jqm7, CVE-2024-3572
References:
- https://github.com/scrapy/scrapy/security/advisories/GHSA-7j7m-v7m3-jqm7
- https://github.com/scrapy/scrapy/commit/71b8741e3607cfda2833c7624d4ada87071aa8e5
- https://github.com/scrapy/scrapy/commit/809bfac4890f75fc73607318a04d2ccba71b3d9f
- https://docs.scrapy.org/en/latest/news.html#scrapy-2-11-1-2024-02-14
- https://nvd.nist.gov/vuln/detail/CVE-2024-3572
- https://huntr.com/bounties/c4a0fac9-0c5a-4718-9ee4-2d06d58adabb
- https://github.com/advisories/GHSA-7j7m-v7m3-jqm7
Blast Radius: 25.8
Affected Packages
pypi:scrapy
Dependent packages: 111Dependent repositories: 2,753
Downloads: 1,374,241 last month
Affected Version Ranges: < 1.8.4, >= 2.0.0, < 2.11.1
Fixed in: 1.8.4, 2.11.1
All affected versions: 0.14.1, 0.14.2, 0.14.3, 0.14.4, 0.16.0, 0.16.1, 0.16.2, 0.16.3, 0.16.4, 0.16.5, 0.18.0, 0.18.1, 0.18.2, 0.18.3, 0.18.4, 0.20.0, 0.20.1, 0.20.2, 0.22.0, 0.22.1, 0.22.2, 0.24.0, 0.24.1, 0.24.2, 0.24.3, 0.24.4, 0.24.5, 0.24.6, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.0.6, 1.0.7, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.4.0, 1.5.0, 1.5.1, 1.5.2, 1.6.0, 1.7.0, 1.7.1, 1.7.2, 1.7.3, 1.7.4, 1.8.0, 1.8.1, 1.8.2, 1.8.3, 2.0.0, 2.0.1, 2.1.0, 2.2.0, 2.2.1, 2.3.0, 2.4.0, 2.4.1, 2.5.0, 2.5.1, 2.6.0, 2.6.1, 2.6.2, 2.6.3, 2.7.0, 2.7.1, 2.8.0, 2.9.0, 2.10.0, 2.10.1, 2.11.0
All unaffected versions: 1.8.4, 2.11.1