-
Notifications
You must be signed in to change notification settings - Fork 6
Open
Description
Trying intake-elasticsearch on ES7. Apparently ES now reports ['hits']['total'] back as a dict instead of a value. We would, or:
- need to test the version of ES (not the library, but the server)
- just try/catch it
The second approach leads to changing the following in _run_query:
try:
# elasticsearch >= version 7
scroll_size = s['hits']['total']['value']
except KeyError:
# elasticsearch <= version 6
scroll_size = s['hits']['total']
Okay for a PR like so, or would you prefer not to support ES7 just yet? Or want a different approach?
ps: unsure what to do when s['hits']['total']['relation'] != 'eq'
meaning that the total hits is not exact. We can force exact results by setting "track_total_hits": true
, see https://www.elastic.co/guide/en/elasticsearch/reference/7.4/search-request-body.html#request-body-search-track-total-hits for more info. (This can be passed via the catalog qargs probably)
Metadata
Metadata
Assignees
Labels
No labels