bug: add FileBasedLock (NFS proof) #104
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Linear Issue
IDSSE-1164
Changes
utils.FileBasedLock
Example:
Explanation
These changes will make a blank
foo.<ext>.lock
file on the filesystem before reading from/writing to afoo.<ext>
file.max_age
seconds, we assume the thread that locked it is dead somehow, and we take over the file lock anyway.Doesn't every OS have locks built into their file I/O?
Yes, but IDSSe can't always rely on that. Our services can be running on different Kubernetes nodes, or different containers in that node, or different Python/Gunicorn processes, and the filesystem they're reading/writing files from could be local FS, or a Network File Share (NFS). This makes it hard to know for sure that no one else is tinkering with your file when your thread goes to do file I/O.