Skip to content

Asset module binary files are stored in webpack cache #19359

@Knagis

Description

@Knagis

Bug report

Currently asset modules are stored in webpack filesystem cache, seemingly multiple times per asset, increasing the cache size significantly.

What is the current behavior?

Asset modules are cached in cache packs, seems both by the asset parser and then when dumping the compilation generated assets. In our large production build, webpack cache takes ~17GB.

We added the following patch in IdleFileCachePlugin.js and the cache size dropped to 1.8GB and cached build times improved.

diff --git a/lib/cache/IdleFileCachePlugin.js b/lib/cache/IdleFileCachePlugin.js
index 3ac59121bafafbdafae3a7dbe0db04242fb41352..fa67d0c0d64935dd4d53e650945a083521d4d748 100644
--- a/lib/cache/IdleFileCachePlugin.js
+++ b/lib/cache/IdleFileCachePlugin.js
@@ -57,6 +57,23 @@ class IdleFileCachePlugin {
 		compiler.cache.hooks.store.tap(
 			{ name: "IdleFileCachePlugin", stage: Cache.STAGE_DISK },
 			(identifier, etag, data) => {
+				if (identifier.includes("|asset/")) {
+					// avoid storing big binary blobs in cache
+					// we also skip anything directly related to them (like generated code) since that is invalidated every time because the file itself isn't in cache. 
+					return;
+				}
+				if (identifier.startsWith("Compilation/assets|")) {
+					// this skips storing generated complete assets in cache, since the asset resources are not cached, these files get invalidated anyway
+					return;
+				}

If the current behavior is a bug, please provide the steps to reproduce.

Enable filesystem cache, configure asset/resource modules.

What is the expected behavior?

Asset module binary data is not stored in cache. Perhaps add a size limit to allow caching (just an example) 2KB assets, but nothing bigger. Reading the asset from disk directly would be as fast as reading it from the cache.

Other relevant information:
webpack version: 5.97.1
Node.js version: 23.8.0
Operating System: Win11
Additional tools:

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions