Converting into exe file through pyinstaller-> spacy cannot find factory for 'curated transformer'

[estherkim083]

import spacy
import spacy_curated_transformers
# import spacy_transformers
import curated_transformers
import spacy_alignments
import spacy_legacy
import spacy_loggers
import spacy_pkuseg
import os
nlp = spacy.load(os.getcwd()+'\\en_core_web_trf-3.7.3')

x= input()
doc= nlp(x)

result =[]
for sent in doc.sents:
    result.append(sent.text)

print(result)

I wanted to turn the above code into exe file.
However, [valueerror: [e002] can't find factory for 'curated transformer' for language english (en)] error occurs...
I used pyinstaller to convert it into exe file. In the pyinstaller, I included spacy, spacy_curated_transformers, curated_transformers into the hidden import.
I wonder how to make this executable file configure the curated transformer factory...
Please help me.

My Environment

• Operating System: Windows 11
• Python Version Used: 3.11.8
• spaCy Version Used: 3.7.4
• Environment Information:

[svlandeg]

Hi!

Sorry to hear you're running into issues with this.

So the way that the factories work, is that the curated_transformer factory should be automatically available in your environment as soon as curated_transformers is installed. I'm not clear about the details of how you've configured the pyinstaller, but it feels like it somehow didn't correctly install curated_transformers...

[estherkim083]

Could you test by making my code into exe file?

[estherkim083]

I want some reply please.

[JackBeauregard]

Exactly the same problem in my installation... Any Solution?

[estherkim083]

I installed curated_transformers correctly.
When I convert above code to exe file with auto-py-to-exe, the error cannot find factory for 'curated transformer' occurs.
@JackBeauregard Was your problem from making it to exe file (-pyinstaller)?

[JackBeauregard]

Yes, that's correct. The problem only exists with PyInstaller; otherwise, everything works fine. As soon as I compile an EXE, the error appears. I have already created a new virtual environment, but nothing has changed.

However, I am also working with an old version of spacy-curated-transformers (0.3.0) because, starting from version 2.0.0, I encounter incompatibilities with Thinc and spaCy.

[JackBeauregard]

Here is my solution:

Import the following modules at the beginning of your script:

import spacy_transformers
import curated_transformers
import spacy_curated_transformers
import spacy_alignments
import spacy_transformers.pipeline_component
import spacy_curated_transformers.pipeline
import spacy

Manually add the following directories in PyInstaller:

Since PyInstaller does not automatically include certain dependencies, you need to manually specify them. You can achieve this by modifying your PyInstaller build command to include the required folders.

# Path to site-packages in your virtual environment
base_dir = os.path.abspath(os.path.dirname(__file__))  # Adjust as needed
site_packages_dir = os.path.join(base_dir, ".venv", "Lib", "site-packages")  # Windows example

# Required directories
curated_transformers_dir = os.path.join(site_packages_dir, "curated_transformers")
curated_transformers_info_dir = os.path.join(site_packages_dir, "curated_transformers-0.1.1.dist-info")
spacy_curated_transformers_info_dir = os.path.join(site_packages_dir, "spacy_curated_transformers-0.3.0.dist-info")

# Adding data directories in PyInstaller command (Example)
pyinstaller_cmd = [
    "pyinstaller",
    "--onefile",
    "--add-data", f"{curated_transformers_dir};curated_transformers",
    "--add-data", f"{curated_transformers_info_dir};curated_transformers-0.1.1.dist-info",
    "--add-data", f"{spacy_curated_transformers_info_dir};spacy_curated_transformers-0.3.0.dist-info",
    "your_script.py"
]

# Execute PyInstaller command
os.system(" ".join(pyinstaller_cmd))

Explanation:

1. The necessary imports are included at the beginning of the script.
2. The script dynamically retrieves the site-packages directory from your virtual environment.
3. The required folders (curated_transformers, curated_transformers-0.1.1.dist-info, and spacy_curated_transformers-0.3.0.dist-info) are manually added to PyInstaller using --add-data.

This ensures that PyInstaller includes all necessary dependencies when building your standalone executable.

[estherkim083]

I tried your solution and it worked finally!! Thanks for your help.💛