[Detail Bug] Webpack assets can hang or stay broken when caching is enabled during compilation

[detail-app]

Summary

• Context: WebpackLoader in webpack_loader/loaders.py is responsible for loading asset metadata from a stats file and providing it to the bundle rendering logic.
• Bug: When the CACHE configuration option is enabled, the loader incorrectly caches the "compile" status of the bundle, causing it to enter an infinite loop in development or become permanently stuck in an error state in production.
• Actual vs. expected: In DEBUG mode, if the loader encounters a "compile" status while CACHE is enabled, it will poll the cache indefinitely instead of re-reading the stats file. In production, once the "compile" status is cached, the loader will persistently raise a WebpackLoaderBadStatsError even after compilation is complete. It should not cache the metadata if the status is "compile", or it should bypass the cache when polling.
• Impact: This bug can cause application processes to hang indefinitely in development or become permanently non-functional in production until they are restarted, specifically if they happen to load the stats file during an active Webpack compilation.

Code with bug

    def get_assets(self):
        if self.config["CACHE"]:
            if self.name not in self._assets:
                self._assets[self.name] = self.load_assets() # <-- BUG 🔴 [Caches "compile" status indefinitely]
            return self._assets[self.name]
        return self.load_assets()

...

    def get_bundle(self, bundle_name):
        assets = self.get_assets()

        # poll when debugging and block request until bundle is compiled
        # or the build times out
        if settings.DEBUG:
            timeout = self.config["TIMEOUT"] or 0
            timed_out = False
            start = time.time()
            while assets["status"] == "compile" and not timed_out:
                time.sleep(self.config["POLL_INTERVAL"])
                if timeout and (time.time() - timeout > start):
                    timed_out = True
                if not timeout:
                    warn(
                        message=_LOADER_POSSIBLE_LIMBO, category=RuntimeWarning
                    )
                assets = self.get_assets() # <-- BUG 🔴 [Always returns the same cached "compile" dict if CACHE is True]

Evidence

I have confirmed this bug by creating a reproduction test case that simulates a "compile" status in the stats file.

1. When DEBUG is True and CACHE is True, the get_bundle method enters a while loop. Because get_assets() returns the cached asset data (which has status: "compile"), the loop condition remains true until the timeout is reached (if configured) or indefinitely.
2. Even after the stats file on disk is updated to status: "done", get_assets() continues to return the cached compile status, making the loader unable to recover.
3. In a production-like environment (DEBUG: False), the first request that hits during compilation will cache the "compile" status. All subsequent requests will fail with WebpackLoaderBadStatsError because they use the cached data and never re-read the finished build stats.

The reproduction script tests/app/tests/test_bug_repro.py demonstrated this behavior:

    def test_cache_compile_status_hangs(self):
        with self.settings(DEBUG=True):
            config = settings.WEBPACK_LOADER["DEFAULT"].copy()
            config["CACHE"] = True
            config["TIMEOUT"] = 0.2
            ...
            # 1. Set status to 'compile'
            with open(stats_file, "w") as f:
                json.dump({"status": "compile"}, f)
            ...
            # 2. Try to get bundle. It times out after 0.2s because it's stuck in the loop.
            with self.assertRaises(WebpackLoaderTimeoutError):
                loader.get_bundle("main")
            ...
            # 3. Update file to 'done'
            with open(stats_file, "w") as f:
                json.dump({"status": "done", ...}, f)

            # 4. It STILL times out because it's using cached 'compile' status.
            with self.assertRaises(WebpackLoaderTimeoutError):
                loader.get_bundle("main")

Why has this bug gone undetected?

This bug is primarily triggered when CACHE is set to True while DEBUG is also True, or when a race condition occurs in production where the very first request to a worker process happens during an active build. The default configuration for CACHE is not settings.DEBUG, which means that in most development environments, caching is disabled, and the polling mechanism works correctly. In production, builds are usually completed before the application server starts, so the loader never encounters the "compile" status during its initial load.

Recommended fix

The get_assets method should not cache the assets if the status is "compile".

    def get_assets(self):
        if self.config["CACHE"]:
            if self.name not in self._assets:
                assets = self.load_assets()
                if assets.get("status") != "compile":
                    self._assets[self.name] = assets # <-- FIX 🟢 [Only cache if not compiling]
                return assets
            return self._assets[self.name]
        return self.load_assets()

Related bugs

• FakeWebpackLoader.get_assets returns an empty dictionary {}, which causes get_asset_by_source_filename to raise a KeyError: 'assets' because it attempts to access self.get_assets()["assets"].
• WebpackLoader.get_bundle also raises KeyError if a chunk listed in the "chunks" section of the stats file is missing from the "assets" section, instead of raising the intended WebpackBundleLookupError, because it uses direct key access assets["assets"][chunk] instead of .get().

History

This bug was introduced in commit 009d285 (@owais, 2016-02-21, PR #48). This commit introduced a class-level cache for the webpack stats file but failed to ensure that temporary states like "compiling" were not cached, leading to persistent errors in production and potential hangs in development.