Multilingual Potion Models

[lhr0909]

Hi,

Thank you so much for your work on Model2Vec and Potion models. I have been having success using the Potion models on iOS using swift-embeddings library.

I would like to see a multilingual Potion model that is based on embedding models like BAAI/bge-m3. Do you think this is something that is planned by your team? I tried to distill the model into Model2Vec but unfortunately the performance is just not as good as the potion models because there is a level of fine-tuning added.

If I were to set one up myself, what dataset would you recommend to train on so the multilingual capabilities are preserved? Any advice would be helpful if this is currently not on your roadmap. Cheers!

[Pringled]

Hi @lhr0909,

That's great to hear! Multilingual Potion models is indeed something on our roadmap, though I can't give an estimate yet on when we will do this since it will require some experimentation first. My recommendation would be to use all the splits from https://huggingface.co/datasets/allenai/c4, we just used the English one for the Potion models. I will keep you in the loop here if we do have an estimate on when we will release this.

[Pringled]

Hi @lhr0909,

We have just released potion-multilingual-128M! We're still doing a few things (adding MMTEB results, running more benchmarks, etc), but the model is already open on HF Hub, so if you want, you can give it a try. Would be great to hear if it works well for your usecase!

[stephantul]

Closing this because we released the model, feel free to re-open