Query Vector Indexes

[greengori11a]

Feature description

neomodel should support querying vector indexes using the neo4j procedures db.index.vector.queryNodes and db.index.vector.queryRelationships. #815 added support for creating vector indexes but querying does not appear to be currently supported.

How this feature can improve the project?

This would improve support for semantic ML/AI workflows in neo4j and neomodel.

[greengori11a]

I am happy to have a stab at this, any suggestions on implementation?

[aanastasiou]

@greengori11a Just to clarify, you still have the option to send the query using cypher_query.

There are some considerations about its use: The most straightforward example is queries that have both a vector and property part. In general, you would aim to first do the property based filtering of the nodes followed by the vector "filter". In other words, first query nodes by the desired property values and then order the result of that query by its similarity to a given vector. But that doesn't mean that the reverse might not be desireable in some cases.

In terms of adding something like this to neomodel:

Neomodel provides a composable interface to writing CYPHER queries. This basically means that when you write something like SomeEntity.nodes.filter(some_attribute__gt=10, neomodel builds up a cypher query sends it to the database (with cypher_query by the way) and then handles the data transformation for the returned results. The starting point for that is .nodes and the "composition" is possible via NodeSet. For more details please see match and 'match_q' (but notice that match has both sync and async versions).

You could look at adding a vector_filter to NodeSet for example that takes the same parameters (index name, vector, k-nearest points) and YIELDs at least the node (possibly the score as well but that would have to be explicitly mentioned in the documentation) to the next NodeSet.

But, I can't tell how general or composable something lke this would end up being right now. It would be useful to hear how you were going to go about it (?) That is, if you already have an idea of where / how to add the vector index part?

[greengori11a]

@aanastasiou thanks for getting back to me! Noted that neomodel does support vector index query with cypher_query.

This forum post from 2023 suggests that 'metadata filtering' (property based filtering of the nodes followed by the vector 'filtering') is not currently supported in neo4j. I believe that this is still the case as the official documentation still reflects this and all other documentation performs the filtering after the vector index query, or pairwise recalculates the cosine or euclidean distance between two vectors using vector.similarity.cosine or vector.similarity.euclidean on the filtered nodes - which does not scale well.

Obviously there is nothing that neomodel the project can do about this but, I think that it is probably a limitation to consider in the implementation of vector_filter.

With that in mind, I agree with your suggestion of SomeEntity.nodes being the right place to put vector_filter for nodes (I have no good ideas about where db.index.vector.queryRelationships should go). I think the function should go something like this: SomeEntity.nodes.vector_filter(indexname : str, vector : list(float), topk: int, score__gt: float) where score borrows the same Django filter format as in .filter() to allow a user to easily threshold the scores (if no score is given then we return topk regardless of score) and YIELD the node. As long as we order the nodes descending by their score we could probably get away without also YIELDing the scores?

In terms of composability, we could ensure that vector_filter could be used with the neomodel subquery allowing a user to 'post-query filter' their NodeSet.

What do you think?

[aanastasiou]

This forum post from 2023 suggests that 'metadata filtering' (property based filtering of the nodes followed by the vector 'filtering') is not currently supported in neo4j.

Yes...and no :) What I mean is that when you realise that you have a "mixed query" then you can still chain two types in a CYPHER query but you just have to do it under the stated constraints. So, that is one set of constraints, the constraints imposed by the underlying tech and then at the level of neomodel we have another set which is how are we going to represent this accurately in the API so that it accurately reflects what the dbms can do.

With that in mind, I agree with your suggestion of SomeEntity.nodes being the right place to put vector_filter for nodes (I have no good ideas about where db.index.vector.queryRelationships should go). I think the function should go something like this: SomeEntity.nodes.vector_filter(indexname : str, vector : list(float), topk: int, score__gt: float) where score borrows the same Django filter format as in .filter() to allow a user to easily threshold the scores (if no score is given then we return topk regardless of score) and YIELD the node. As long as we order the nodes descending by their score we could probably get away without also YIELDing the scores?

Given the above constraints, it might be better to make the vector search part of filter(). As in .filter(vector_name=, vector_ref=, vector_k=). In this way, we can control whether a vector of property filter has to be applied.

Regarding the propagation of score...From personal experience, I can say that it is something that sooner or later will definitely be required to have been propagated at some level of a given query.

In terms of composability, we could ensure that vector_filter could be used with the neomodel subquery allowing a user to 'post-query filter' their NodeSet.

Yes, that sounds sensible.

Sorry, this doesn't look very much like a definite plan but as you can see, this is not exactly the most straightforward thing to implement.

@mariusconjeaud might also have additional suggestions about this (?).

[mariusconjeaud]

Hello ! Slowish reaction from me, I couldn't monitor neomodel too much for a month since I left Neo4j. Now I hope to pay more attention again.

I tend to agree that the .filter() method could be specialized to support specific vector (and full-text) filtering. It's better to group everything into this method, because it can then optimize how it manages the various filters.

If you look here, you will see that @tonioo and I created objects you can pass to neomodel querying methods to provide context. This could be an idea for defining a VectorFilter() maybe ?

[greengori11a]

Ok, I will take a stab at placing the vector query into the .filter() method.

@mariusconjeaud please correct me if I am wrong but is your suggestion to have the api looking something like: SomeEntity.nodes.filter(VectorFilter(vector_indexname="foo", vector=[...], topk=x, score__gt= 0.6,)) similar to the Path filter here?

[mariusconjeaud]

Yes, that is my suggestion @greengori11a !

[greengori11a]

Hi @mariusconjeaud @aanastasiou,

I've put in the MR for this issue - it is not passing a code duplication quality gate, could you please have a look and tell me how I should rectify this?

Thanks