Can't get data cvs.

[sergii-rolskii]

Hi,
If I do:
curl -o /tmp/data.txt - XGET 'http://localhost:9200/my-index/_search' -d '{"size":0,"query":{"bool":{"must":[{"query_string":{"analyze_wildcard":true,"query":"*"}},{"range":{"@timestamp":{"gte":1497538800000,"lte":1500213705347,"format":"epoch_millis"}}}],"must_not":[]}},"_source":{"excludes":[]},"aggs":{"2":{"date_histogram":{"field":"@timestamp","interval":"1h","time_zone":"Europe","min_doc_count":1}}}}'

I get /tmp/data.txt (data has json format) but if I do:

curl -o /tmp/data.csv -XGET "http://localhost:9200/my-index/_data?format=csv&source={"size":0,"query":{"bool":{"must":[{"query_string":{"analyze_wildcard":true,"query":"*"}},{"range":{"@timestamp":{"gte":1497538800000,"lte":1500213705347,"format":"epoch_millis"}}}],"must_not":[]}},"_source":{"excludes":[]},"aggs":{"2":{"date_histogram":{"field":"@timestamp","interval":"1h","time_zone":"Europe","min_doc_count":1}}}}"

I have error:
curl: (3) [globbing] nested braces not supported at pos 67

Where am I wrong? This query successfully working on kibana(dev-tools): GET my-index/_search {query...}

[marevol]

" is wrong.

[sergii-rolskii]

@marevol what do you mean?

btw, if I do:
curl -o /tmp/data.csv -XGET "http://localhost:9200/my-index/_data?format=csv&source=%7B%22query%22%3A%7B%22match_all%22%3A%7B%7D%7D%7D"

I don't see any error, then open file data.csv and see:

{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"No endpoint or operation is available at [_data]"}],"type":"illegal_argument_exception","reason":"No endpoint or operation is available at [_data]"},"status":400}

elasticsearch version: 5.2.2
dataformat version 5.3.0

[marevol]

curl: (3) [globbing] nested braces not supported at pos 67

It's a general problem on shell, not elasticsearch and plugin problem.
The usage of " is wrong.

No endpoint or operation is available at [_data]

It seems that a plugin installation is failed.

elasticsearch version: 5.2.2
dataformat version 5.3.0

Use dataformat 5.2.x.

[sergii-rolskii]

@marevol
thank you for your answer, I installed version 5.2.2 and now it works. Maybe you know how correctly to decode the query?
Because if I decode my query, I don't see any data in file data.csv this file empty.

[sergii-rolskii]

OK, I think dataformat don't understand aggregations. I decode my query:

{"size":0,"query":{"bool":{"must":[{"query_string":{"analyze_wildcard":true,"query":"*"}},{"range":{"@timestamp":{"gte":1497538800000,"lte":1500213705347,"format":"epoch_millis"}}}],"must_not":[]}},"_source":{"excludes":[]},"aggs":{"2":{"date_histogram":{"field":"@timestamp","interval":"1h","time_zone":"Japan","min_doc_count":1}}}}

to

%7B%22size%22%3A0%2C%22query%22%3A%7B%22bool%22%3A%7B%22must%22%3A%5B%7B%22query_string%22%3A%7B%22analyze_wildcard%22%3Atrue%2C%22query%22%3A%22*%22%7D%7D%2C%7B%22range%22%3A%7B%22%40timestamp%22%3A%7B%22gte%22%3A1497538800000%2C%22lte%22%3A1500213705347%2C%22format%22%3A%22epoch_millis%22%7D%7D%7D%5D%2C%22must_not%22%3A%5B%5D%7D%7D%2C%22_source%22%3A%7B%22excludes%22%3A%5B%5D%7D%2C%22aggs%22%3A%7B%222%22%3A%7B%22date_histogram%22%3A%7B%22field%22%3A%22%40timestamp%22%2C%22interval%22%3A%221h%22%2C%22time_zone%22%3A%22Europe%22%2C%22min_doc_count%22%3A1%7D%7D%7D%7D

and try get csv:

curl -o /tmp/data.csv -XGET "http://localhost:9200/my-index/_data?format=csv&source=%7B%22size%2.....my_decode_query......7D%7D%7D%7D"

So, data.csv empty... my query returns json like format:

{"took":5,"timed_out":false,"_shards":{"total":18,"successful":18,"failed":0},"hits":{"total":40166,"max_score":0.0,"hits":[]},"aggregations":{"3":{"doc_count_error_upper_bound":27,"sum_other_doc_count":39532,"buckets":[{"key":"5540C447F2020.AC10E74.3009D347","doc_count":30},{"key":"5540C466558C8.AC12341.140D236E","doc_count":29},{"key":"55407D5EA1B78.A80E1C8.9207E6F3","doc_count":28},{"key":"5540C0A842F28.AC110D1.D40758B9","doc_count":28},{"key":"5540861E2C3A0.A802B44.3A08BADC","doc_count":27},{"key":"5540A7916FC88.A8052D6.8708E054","doc_count":27},{"key":"553FDCB75FD60.AC11CE4.33054683","doc_count":26},{"key":"553FDCE1D47D0.A802609.E5000DA7","doc_count":26},{"key":"554009A4F6D18.AC12B66.FD026BEA","doc_count":26},{"key":"5540D143142D0.A8033C6.AB026987","doc_count":26},{"key":"5540D170926A8.AC15726.8B04487E","doc_count":26},{"key":"5540D18EB3CB8.AC10223.33076FED","doc_count":26},{"key":"55402E5A77CE0.A0D8705.3703FE37","doc_count":25},{"key":"554085FC415D8.A802F8B.18063EDD","doc_count":25},{"key":"553FEDE25C0C0.AC14C61.4506D2C1","doc_count":24},{"key":"553FEE037C070.AC13047.D4026621","doc_count":24},{"key":"55402E03A3888.AC1358F.C20C2F6F","doc_count":24},{"key":"5540C0C865C10.AC12686.280614F6","doc_count":24},{"key":"5540D11FCABA8.AC11147.2E073ADA","doc_count":23},{"key":"55407D30D9808.A8077CC.E155B","doc_count":22}]}}}

dataformat don't understand it.

[k-graefenhain]

If I try to send the query I get the following exception:
[2017-10-11T10:43:50,952][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [] fatal error in thread [elasticsearch[UmQ8KQQ][search][T#7]], exiting java.lang.IncompatibleClassChangeError: Found class org.elasticsearch.search.SearchHits, but interface was expected at org.codelibs.elasticsearch.df.content.csv.CsvContent$OnLoadListener.onResponse(CsvContent.java:136) ~[?:?] at org.codelibs.elasticsearch.df.content.csv.CsvContent.write(CsvContent.java:102) ~[?:?] ...

I use ElasticSearch 5.5.1 and the plugin for 5.3.0

[marevol]

To use elasticsearch 5.5.x, you need to install plugin 5.5.x.