When using chunks callback, complete callback parameters are undefined #964
Description
Scenario
I'm attempting to use the chunk callback to obtain status updates on the parse operation for large files (similar to #385). I noticed that the step callback is called for each line which makes it inefficient for calculating progress, so the chunk callback seems to be the best fit for this scenario.
Problem
When calling parse with the chunk callback defined, both the results and the file parameter passed to the complete callback are undefined. The results and the file parameter passed to the chunk callback are defined as expected.
When calling parse without a chunk callback defined, both the results and the file parameter passed to the complete callback are defined as expected.
Expected Behavior
Whether or not the chunk callback is defined, the complete callback should fire exactly once with the result set.
Relevant Versions:
- "@types/papaparse": "^5.3.5",
- "papaparse": "^5.3.2",
- "typescript": "~4.6.4",
- "@angular/common": "~13.3.0",
This seems to behave identically on the following browsers:
- Firefox 107.0.1
- Edge 108.0.1462.54
This occurs both when importing the raw PapaParse module via import { parse } from "node_modules/papaparse/papaparse.js" as well as when importing the typed version via import { parse } from "papaparse"
Possibly related issues:
- chunkSize do not have any effect, defaults to around 65KB #616
- step + big local file = "Cannot read property 'meta' of undefined" #92
- Firefox stops after the first chunk #690
- chunk + worker does not work #955
- Suggestion for website FAQ: Working Code Example for Loading Bar using a Worker #385
- Progress bar #171
- chunkSize do not have any effect, defaults to around 65KB #616
Sample Code:
public ParseContactCSV(f: File, hasHeaderRow: boolean): Observable<ContactImportStatusUpdate> {
const statusUpdateDebounceMS = 250
let lastStatusTimestamp: number;
let lastPercent = 0;
return new Observable<ContactImportStatusUpdate>((o)=>{
o.next({
operation: "Parse Import",
statusText: "Beginning parse",
result: null
})
parse<File>(f, {
worker: true, // The chunk/complete callback bug happens whether or not we're using a worker
header: hasHeaderRow,
chunkSize: f.size/100, // The chunk/complete callback bug happens whether or not chunk size is defined.
chunk(results, parser) {
const percentDone = Math.ceil((results.meta.cursor / f.size)*100)
// limit our status emits to one-percent steps and to emit no more frequently than every statusUpdateDebounceMS
if (percentDone > lastPercent && (Date.now() - lastStatusTimestamp) < statusUpdateDebounceMS) {
o.next({
operation: "Parse Import",
statusText: "Parsed " + percentDone.toLocaleString(undefined,{maximumFractionDigits:2}) +"%",
result: null
})
lastPercent = percentDone
lastStatusTimestamp = Date.now();
}
},
complete(results, file) {
o.next({
operation: "Parse Import",
statusText: "Complete; parsed " + results.data.length + " contacts",
result: results
})
},
})
});
}