[Bug][Plugins] I try to add a new plugin but it always show "plugin already loaded" during devlake startup

[andyli86]

Search before asking

• I had searched in the issues and found no similar issues.

What happened

I am writing a plugin for devlake to do some customized metrics calculation, I begin by writing a minimal plugin, I write PluginEntry in plugin.go and PluginMeta etc in impl/impl.go, and the plugin can success compile and generate .so file, after I make a image with official devlake and copy .so under /app/bin/plugins, devlake will try to load it during the startup, but it always show "plugin already loaded" error message like:

devlake-1  | panic: attached stack trace
devlake-1  |   -- stack trace:
devlake-1  |   | github.com/apache/incubator-devlake/core/runner.LoadGoPlugins
devlake-1  |   |        /app/core/runner/loader.go:88
devlake-1  |   | github.com/apache/incubator-devlake/core/runner.LoadPlugins
devlake-1  |   |        /app/core/runner/loader.go:38
devlake-1  |   | github.com/apache/incubator-devlake/server/services.Init
devlake-1  |   |        /app/server/services/init.go:123
devlake-1  |   | github.com/apache/incubator-devlake/server/api.Init
devlake-1  |   |        /app/server/api/api.go:58
devlake-1  |   | github.com/apache/incubator-devlake/server/api.CreateAndRunApiServer
devlake-1  |   |        /app/server/api/api.go:79
devlake-1  |   | main.main
devlake-1  |   |        /app/server/main.go:33
devlake-1  |   | runtime.main
devlake-1  |   |        /usr/local/go/src/runtime/proc.go:250
devlake-1  |   | runtime.goexit
devlake-1  |   |        /usr/local/go/src/runtime/asm_amd64.s:1598
devlake-1  | Wraps: (2) plugin.Open("bin/plugins/minimal_plugin/minimal_plugin"): plugin already loaded
devlake-1  | Wraps: (3) plugin.Open("bin/plugins/minimal_plugin/minimal_plugin"): plugin already loaded
devlake-1  | Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *errors.errorString

I check my container there is only one minimal_plugin.so in the container, but I don't know why the devlake plugin framework say it already loaded, from the github.com/apache/incubator-devlake/core/runner.LoadGoPlugins code it is pretty straight forward just look all the folder and find each .so file then load each one... so I really confused.

What do you expect to happen

the plugin can be loaded successfully

How to reproduce

Here is what I did: I put my plugin code in a separate repo, build it to a .so file, then use a Dockerfile to copy this .so file to /app/bin/plugins and build a new image, then use the default docker-compose-dev.yaml in devlake repo to launch this image

FROM golang:1.24 AS builder
COPY . /app
WORKDIR /app
RUN go mod tidy
RUN go build -buildmode=plugin -o /app/bin/plugins/minimal_plugin.so plugin.go

# Use the official DevLake image as base
FROM devlake.docker.scarf.sh/apache/devlake:latest AS devlake

# Switch to root to handle permissions
USER root

# Remove all existing plugins to ensure a clean slate
# This helps rule out conflicts with *any* built-in plugin
#RUN rm -rf /app/bin/plugins/*

# Ensure the directory exists and has correct permissions too (belt and suspenders)
RUN mkdir -p /app/bin/plugins/minimal_plugin && chmod -R 755 /app/bin/plugins/minimal_plugin

# Copy ONLY the minimal plugin
COPY --from=builder /app/bin/plugins/minimal_plugin.so /app/bin/plugins/minimal_plugin/minimal_plugin.so

# Switch back to devlake user
USER devlake

build this docker image as devlake-minimal-test and update the docker compose yml code like below:

......
 devlake:
    image: devlake-minimal-test:latest
    ports:
      - 8080:8080
    restart: always
    volumes:
      - devlake-log:/app/logs
    env_file:
      - ./.env
    environment:
      LOGGING_DIR: /app/logs
      TZ: UTC
    depends_on:
      - mysql
......

Anything else

No response

Version

v1.0.1

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[dosubot]

Hello @andyli86! I'm here to help you with bugs, answer your questions, and assist you in becoming a contributor. While you wait for a human maintainer, feel free to share more details or ask about anything specific!

The "plugin already loaded" error occurs because the LoadPlugins method in DevLake automatically loads all .so files in the specified plugin directory during startup. If your minimal_plugin.so is already present in this directory, it will be loaded automatically, and attempting to load it again manually will result in this error. To resolve this, ensure that your plugin is not being loaded twice by checking the contents of the plugin directory and the logic in the LoadPlugins method ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[andyli86]

I also tried latest v1.0.2-beta7 image still same issue.
I use the main branch devlake for docker-compose-dev.yml and start the devlake for my test

[andyli86]

Hi dosubot, I have checked there is only one .so file in the container, the directory is correct because it will load all the plugins under /app/bin/plugins/ from the log devlake try to load other plugins then it fails on mine plugin only.

[dosubot]

The "plugin already loaded" error might be related to the logic in the LoadGoPlugins function, which registers plugins by their names. If a plugin with the same name has already been registered, it could lead to this error ^[1].

Additionally, ensure that the PLUGIN_DIR environment variable is correctly set to the directory where your plugin is located. Also, check if the DEVLAKE_PLUGINS environment variable is set to load specific plugins, which might affect the loading process ^[2][3].

Since you mentioned using the main branch for docker-compose-dev.yml, ensure there are no configurations or conditions in that file affecting plugin loading. If the issue persists, consider checking the implementation of the RegisterPlugin function to ensure no duplicate registration logic is causing the error ^[4].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[andyli86]

the plugin name is unique, there would not any other so name register as minimal_plugin
also the PLUGIN_DIR is correct, it is default to /app/bin/plugins so all the plugins can be loaded, the fact is all other plugins in the same directly are loaded just my plugin is not
docker compose do not have condition or configuration change the plugin loading.