Returning irrelevant answers

[lipsa7]

The model is not giving me the answer I want.

What is HT:LT ratio of Andhra Pradesh East discom?
<Answer {'answer': '13%).', 'type': 'extractive', 'score': 0.9178509712219238, 'context': 'tically higher than the utility \nwith the next highest rate (MP East, at 13%). This \ncould be attributed to unbalanced load, lack of \na protection sys', 'offsets_in_document': [{'start': 235620, 'end': 235625}], 'offsets_in_context': [{'start': 73, 'end': 78}], 'document_ids': ['dc508056555e17df501653fd820cc485'], 'meta': {'name': 'SPI_Electrification_15.pdf'}}>

It's taking answer from the following paragraph which has got nothing to do with the question asked.
Text from pdf:
the surveyed utilities
reported an average DT failure rate of 11%, with
three utilities in Gujarat (Central, South, and
North) having low DT failure rates mainly due
to adoption of better maintenance practices.
MP West has the highest DT failure rate, which
at 38% is dramatically higher than the utility
with the next highest rate (MP East, at 13%). This
could be attributed to unbalanced load, lack of
a protection system, and poor workmanship in
the installed DTs.

In its other answers also, it's not giving me the relevant answer to my question.

[bogdankostic]

Hi @lipsa7! What model are you using and how does your Pipeline look like? It seems that the question you asked cannot be answered by the provided text snippet. Have you tried setting the parameter return_no_answer to True for FARMReader?

[bogdankostic]

Unfortunately, Haystack doesn't support answering question based on images and charts. FARMReader works only on text - that's the reason why it can't answer your question.

[lipsa7]

Hi @bogdankostic, I have a query please. I have table data in pdf, the tapas model should refer 'Sales figures by salesperson and year' when it answers from the table so as to give relevant answers. A table for example:
https://digitalaccessibility.uoregon.edu/sites/digitalaccessibility2.uoregon.edu/files/inline-images/Simple%20Table%20Starting.png. The table is in text format.
I have written the following code:
camelot.read_pdf(
"/content/DGVCL-1912-2020-True-up-FY-2019-20-Order-31.03.2021.pdf",
pages='all',
password=None,
flavor='lattice',
suppress_stdout=False,
layout_kwargs={},
**kwargs,
)

but it returns the error NameError: name 'kwargs' is not defined.
Can you please help?

Can you please help?

[bogdankostic]

It seems like your using camelot for that and not one of Haystack's nodes. I'm not very familiar with camelot, but I think you should remove **kwargs.

If you want to use a Haystack node for extracting tables from PDFs, you might want to have a look at our AzureConverter or ParseConverter.

[lipsa7]

This is how I wanted the output to be:
https://miro.medium.com/v2/resize:fit:796/1*ow-aF3lRRKi2XoTjsxj8NA.png
The heading of the table is included in the output.

Can I use haystack table qa model to differentiate among several tables like this, as each table may contain similar data but for a different year? I need model to answer as per the year then.

[bogdankostic]

The ParsrConverter and AzureConverter won't integrate the header into the table as in your attached picture, but will add the meta fields preceding_context and following_context that can be used for the retriever.