feat: Add Augmented Reality (AR) functionality.

[zhengyangliu]

I have fully implemented the AR function.Here is a demonstration video.

But there are a few regrets:

1. I really can't automate the process of generating the marker file .mind, because I can't find a way to get a PCB image without blank areas in pure Python.
2. I can't package the dependent ar scripts as local files, and they always report errors or work abnormally.

[qu1ck]

First of all, this is great work and I can see this being very useful for debugging and educational purposes.

However this functionality is not helpful with primary goal of ibom as being an assembly documentation/helper tool. In fact tracking will likely not work reliably as the look of the board changes with components being added during assembly.

Normally I would not object to useful functionality being added even if it is not directly helpful for assembly/documentation. But making this work is not straightforward and has serious usability concerns.

1. This breaks self contained nature of ibom by requiring internet access.
2. This breaks ease of use because instead of single html file that you can open anywhere you now need to host it on a server.
3. Process of generating mind files is tedious (but probably can be improved by generating screenshots from kicad-cli and possibly using some remote api endpoint to convert them)

So I don't think this should be in main ibom package and I will not merge this at this point.

A few pointers about how the issues can be fixed before I can revisit the decision to merge this upstream:

1. No internet should be required, figure out the issues with embedding ar libs. Also the additional javascript should be embedded into html only if the ar functionality is enabled.
2. Add a simple webserver into ibom python code that will host the file on localhost. User should be able to launch it if they choose to do so after the bom is generated and open it in browser from localhost protocol instead of file://. It should work as a detached process.
3. kicad-cli already has pcb render command, but some functionality may be missing like options to render components or not, projection selection, etc. Additional functions may need updates to kicad.
   Also look into possibly reimplementing the mind file generation logic in python.

[zhengyangliu]

Hi @qu1ck,

Thanks for your thoughtful feedback! I’ve actually been exploring those exact points:

1. Embedding AR scripts locally
   I agree it’s doable—it just requires effort. The current AR frontend library mandates iframe usage, which complicates bundling everything into a single HTML and introduces several tricky issues, but I’m confident it can be resolved.

2. Auto-launching a local server
   I considered adding that. The challenge is that once the user opens the HTML manually again, the AR won’t initialize properly. Plus, a background Python server might clash with any server the user starts later. I haven’t yet found a clean way to handle that.

3. Mind‑file generation using pcb render
   That command, which I only recently discovered, fits our needs perfectly—it should make automating mind‑file generation straightforward. Earlier AI suggestions led me to pcb export, and I ended up tangled in Python-based SVG handling, so discovering pcb render was a game-changer.

Regarding use cases, I agree it’s more valuable during debugging than step‑by‑step assembly—hence why I chose to generate a final 3D-rendered image for the AR system, so it can track the board as it would appear when fully assembled.

[zhengyangliu]

Do you have any fresh ideas for addressing point 2? Or maybe I could generate an auto-launch bash or somethings witch in the same path.

[qu1ck]

You could generate a shell script (bat for windows) that will launch the server alongside the html file, like you said. It's still not ideal because it's not really portable but only other option is to generate actual python bundle executable which is an overkill.

One more thing to note here, to be able to ship this plugin as a kicad addon all of it's python dependencies should be bundles with the code too. KiCad v9 new api supports pypi dependencies which it will automatically install in a dedicated virtual environment for each plugin but ibom is not yet ported to the new api.

About mind files, generating images is half of the issue, converting it to .mind files is also cumbersome. Have you looked into porting the node lib that does the conversion to python?

[zhengyangliu]

It seems that there is still no good way to start the web server.
For mind file generation, I am trying to automate the browser call directly to achieve this, rather than packaging the nodejs runtime.

[zhengyangliu]

Ok, I'll pay attention to the python dependency issue.

[zhengyangliu]

I have a new idea. Maybe we should modify the instructions for starting the server and teach users to manually add the file to the security exceptions so that they can use it directly through the file link.

[qu1ck]

Does chrome allow that? Can you share steps for adding security exceptions?