Allow to get the data from a function instead of neo4j cypher

[thebestnom]

Closes #188 and #185
Also closes #35 I think

[dolleyj]

Does the recent activity on this indicate this will be a new feature in an upcoming release, or is this ticket consolidation?

Neovis is a candidate of mine, but it's a non-starter if neo4j credentials are stored directly in the javascript. I'd prefer flask to serve the graph data to neovis.

[thebestnom]

I want to work on that, but I don't have a lot of free time and a library we dependent 9n (vis-data) have a broken types on newer typescript that I need to fix first

[dolleyj]

@thebestnom , thanks. I appreciate the quick response. Cheers

[thebestnom]

@dolleyj hey, just published 2.1.0-alpha2 with the PR code if you want to test, API not final but it's good enough for now

just add to the config dataFunction with a iterator function that yields Records and it should work

[nielsjansendk]

@thebestnom I would like to test this, but I am not super strong in javascript. As I understand it, instead of passing

const config = {
      neo4j: {
                serverUrl: "neo4j://locahost:7687",
                serverUser: "neo4j",
                serverPassword: "password",
      },
     }
     

 You would instead pass

     const config = {
                dataFunction: myDataFunction(),
     }

And the datafunction should basically do the same as this:

function myDataFunction() {
let recordCount = 0;
			const _query = query || this.#query;
			const session = this.#driver.session(this.#database ? { database: this.#database } : undefined);
			const dataBuildPromises: Promise<unknown>[] = [];
			session.run(_query, parameters)
				.subscribe({
					onNext: (record) => {
						recordCount++;
						dataBuildPromises.push(this.#createSingleRecord(record));
					},
					onCompleted: async () => {
						await Promise.all(dataBuildPromises);
						await session.close();

						this.#completeRun();
						this.#events.generateEvent(NeoVisEvents.CompletionEvent, { recordCount });
					},
					onError: (error) => {
						this.#consoleLog(error, 'error');
						this.#events.generateEvent(NeoVisEvents.ErrorEvent, { error });
					}					
				} as Neo4jCore.ResultObserver);
				}

But replacing all references to the driver with my own instance of neo4j-driver. Is that the gist of it? The dataFunction has run the query and return the result of the run. Or do I just return the record from the dataFunction? There seems to be some sort of housekeeping with the return result of the datafunction, where createSingleRecord is called on the result. So what is the data function expected to provide? Is it possible that you could add an example with a dataFunction that uses neo4j-driver to execute some cypher on the server and return the result. Even a simple example would help a lot.

I am also not sure how the query gets passed to the datafunction. If my configuration has an initialCypher definition, shouldn't I be able to pass that to my datafunction?

[thebestnom]

Close, but first, don't accidentally call in the config, so

const config = {
                dataFunction: myDataFunction //make sure without brackets
     }

And it need to return Iterable or async iterable, that means Array, or generator with yield or async generator with yield
So somthing like

import * as neo4j from "neo4j-driver-core"; // not sure if from "neo4j-driver" or from core but somthing like that
async function *myDataFunction() {
    const data = await (await fetch(url)).json(); //you're server should return original neo4j record 
   // I think that for now youll have to convert it back to record class so somthing like
   for (const re of data) {
       yield neo4j.Record(re));
   }
   // Or somthing like that, that's a test api but it should work
}

[nielsjansendk]

@thebestnom I am going to give it a shot.

[thebestnom]

Make sure to see what I edited in the function 😅

[nielsjansendk]

@thebestnom I think maybe there is a problem. It is trying to connect to localhost:7687. However, in my dataFunction, I have specifically instructed it to connect to: 'http://localhost:7474/db/neo4j/tx/commit'; I think maybe it tries to connect to localhost:7687 because I have removed the neo4j part of my config, and then it falls back to the default. But if there is a dataFunction it really shouldn't attempt to talk to directly to neo4j from neovis, right? If that part of the config is missing, it should look for a datafunction and if that is not there it should throw an error of some sort.It is quite possible I didn't program the datafunction properly, but I still think t should never attempt to connect directly to neo4j if it has not been configured to do so. That can lead to confusing errors.

[thebestnom]

Yeah, guessed somthing was not right by not testing that in #init but it shouldn't make the render to fail, it should just log railing to connect but continuing aniway, no?