PgvectorDocumentStore fail to add documents

[phiweger]

I try to run the example in this tutorial:

https://docs.haystack.deepset.ai/docs/pgvectordocumentstore

But I cannot write to the document store:

KeyError                                  Traceback (most recent call last)
Cell In[4], line 1
----> 1 document_store.write_documents([
      2     Document(content="This is first", embedding=[0.1]*768),
      3     Document(content="This is second", embedding=[0.3]*768)
      4     ])

File ~/.../haystack_integrations/document_stores/pgvector/document_store.py:503, in PgvectorDocumentStore.write_documents(self, documents, policy)
    500 if policy == DuplicatePolicy.NONE:
    501     policy = DuplicatePolicy.FAIL
--> 503 db_documents = self._from_haystack_to_pg_documents(documents)
    505 sql_insert = SQL(INSERT_STATEMENT).format(
    506     schema_name=Identifier(self.schema_name), table_name=Identifier(self.table_name)
    507 )
    509 if policy == DuplicatePolicy.OVERWRITE:

File ~/.../haystack_integrations/document_stores/pgvector/document_store.py:559, in PgvectorDocumentStore._from_haystack_to_pg_documents(documents)
    556 db_document["blob_meta"] = Jsonb(blob.meta) if blob and blob.meta else None
    557 db_document["blob_mime_type"] = blob.mime_type if blob and blob.mime_type else None
--> 559 db_document["dataframe"] = Jsonb(db_document["dataframe"]) if db_document["dataframe"] else None
    560 db_document["meta"] = Jsonb(db_document["meta"])
    562 if "sparse_embedding" in db_document:

KeyError: 'dataframe'

haystack-ai==2.14.3
haystack-experimental==0.10.0
pgvector==0.4.1
pgvector-haystack==1.3.0

[phiweger]

Maybe related to this: #1371

@anakin87

[anakin87]

Hello, @phiweger...

The issue you mentioned is related, but if I install the latest version of pgvector-haystack (3.4.1), the example from docs works correctly.

You can update the integration with pip install -U pgvector-haystack.

Let me know if that works for you...