[CORE] Implement custom buffer options for File::open() #1301
Conversation
xezon
added
Enhancement
| else | ||
| { | ||
| const Int bufferMode = text | ||
| ? _IOLBF // Uses line buffering |
There was a problem hiding this comment.
We may only want line buffering on text && write ?
There was a problem hiding this comment.
Seems sensible to me, since line buffering works by buffering up till the relevant newline or return character is seen.
So you don't want that occurring outside of text mode with binary data.
There was a problem hiding this comment.
After some thinking I concluded that it is better separated from TEXT and BINARY, so I added new modes LINEBUF and FULLBUF. This way the caller has full control over the behavior.
|
I am wondering if we should even bother with the raw reading versions and that much of this could be simplified by just using the standard buffered I/O functions. |
|
What do you mean by raw reading versions? |
I think he means dump the posix io api stuff. |
|
Ah ok. Yes we can likely remove in the future. We can keep it for a moment just in case we have some strange file io behaviour discrepancy and want to test before/after. |
xezon
force-pushed
the
xezon/enable-buffered-io-2
branch
from
179975b to
e74ef90
Compare
|
Rebased on Main branch. |
| { | ||
| //USE_PERF_TIMER(RAMFile) | ||
| File *file = TheFileSystem->openFile( filename, access ); | ||
|
|
||
| bufferSize = 0; // RAM File needs no file buffer because it is read in one go. |
There was a problem hiding this comment.
I would say that it should use the default buffer size as the initial file access will make multiple calls to read individual bytes of data instead of a whole block of 512 bytes at a time from disc etc.
This will be a performance degredation after the previous changes to enable buffering of reads. And RAM files are used frequently within the game, if not they are the most commonly used file type when loading data.
There was a problem hiding this comment.
Can you show where these reads are that you refer to?
There was a problem hiding this comment.
Just bellow in this same function, when you first use open from the file system, you can either be given a ram file or a local file depending on if the file you want is from local storage or an archive.
The files contents are then read and copied to the calling ram file.
There was a problem hiding this comment.
oh wait a sec, it was some github funnyness and i was seeing part of the open( File *file) function bellow as though it was part of this function.
But since the file system can call and load from a local file or archive, that file will be the one affected by the buffersize call rather than the ram file itself.
xezon
force-pushed
the
xezon/enable-buffered-io-2
branch
from
8b5d716 to
6fddb34
Compare
This change is a follow up for #1299 and implements a new argument for File::open() and friends to specify a default or custom buffer size. Depending on the TEXT and BINARY flags, a buffer mode is picked. By default the default system buffer size is used (BUFSIZ aka 512 on Windows). Buffer size can be 0 for no buffer. RAMFile::open() always uses no buffer.
Functionality wise nothing really changes.