LLM language setting

[paultrebor]

Hello!

Thanks for making QualCoder! I've been exploring it across various projects over the past few months, including its AI feature.

The most recent project I work on with a group of students. We want to use QualCoder to analyze Chinese language official policy documents. Since we are a mixed language team, we're dealing with documents both in the original Chinese and in English translation. We're prompting the AI both in English and Chinese.

Against his background, it appears that the language setting for the AI has a rather limiting impact on the way that we can prompt and interact with the AI. If language is set to English, the AI will only return results from English translations. If language is set to Chinese, it will only return results from the Chinese-language part of documents. If I don't set a language, it either defaults to English or complains that no language setting is provided. There doesn't seem to be a reliable way to make the AI language agnostic, so to speak, that is, to have it engage with Chinese text when prompted in English and vice-versa.

There does not appear to be a reliable way to change this behavior through prompting. For example, when language is set to English and I tell it to also consider Chinese text in its search it just ignores that. When I tell it to provide Chinese excerpts with its results, what it actually does is to create its own Chinese translations from our English translations.

I'm attaching two screenshots as examples.

I guess my question is, is there a way to make the AI language-agnostic?

Again, thanks for your amazing work!
Robert

[kaixxx]

Hi Robert,

That's complicated, I'm not sure what I can do here.

The AI language setting in the QualCoder settings dialog will result in the following instruction send with every prompt: Always answer in the following language: "{ai_language}". It should only affect the output language. I have worked with English data and German output, which worked fine but sometimes also translated direct quotes in the text.

However, since your documents are a mix between English and Chinese, the situation is a little more complicated. Te easiest solution would probably be to work with documents in English and Chinese separately. As far as I understand, they have the same content, just translated?

Another option would be to give the AI more detailed instructions how to handle the mixed languages, as you have tried already. The best way to do this would be to edit the prompts that QualCoder uses for the analysis. You can do that in the built-in prompt library, more information here: https://github.com/ccbogel/QualCoder/wiki/6.2.-AI-Prompt-Library. It seems that you are using the "text analysis" feature, so these are the prompts you should look at. Feel free to ask if you have any questions.

[paultrebor]

Hi Kai! Thanks a lot for your quick reply!

Always answer in the following language: "{ai_language}"

This was super helpful. After seeing this I decided to simply write "English and Chinese" into the configuration field, and it now appears to do what it's supposed to (or maybe that's just a fluke).

Thanks a lot!

[kaixxx]

I decided to simply write "English and Chinese" into the configuration field

That's a creative solution, haven't thought about that. The result look impressive...