Skip to content

Conversation

mrspeedyy
Copy link

@mrspeedyy mrspeedyy commented Jun 19, 2025

Cache warming takes too long (hours/days) on lower spec hw or non-ssd storage with large database. Changing startup sequence to optimize availability, gunicorn starts immediately, does not need to wait until the cache is warmed, and can serve (slower, but better than nothing) immediately after database is up.

On low spec hw, especially slow storage, database warming can take up hours before web api is available. This change tradeoffs initial performance for immediate availability.
@philipkozeny
Copy link
Collaborator

Looks good from my side, thanks for the PR.

Quick question for @mtmail before the approval:
What are your thoughts on triggering the warm command after the container is fully healthy and serving requests?

@mtmail
Copy link
Contributor

mtmail commented Jun 23, 2025

Goal of warming is to put database table and indices into RAM. The operating system system does some caching automatically, there's no direct method to force the operating system to cache it (*). Warming is helpful with slow discs and if you have a lot of RAM.
Question again is if the docker setup should optimize for users importing one region on average hardware, or those importing the whole planet on high performance hardware. And even the high performance setups might not need warming if they have enough servers online.
I'd make it optional (a environment setting) and do the warming before serving user queries.

*) Ages ago there was a kernel method on Debian/Ubuntu to load a path into memory. I can't find notes and it'd be probably too specific to one operating system. It was a hack.

@philipkozeny
Copy link
Collaborator

see #626

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants