Compressed images

[go-sakayori]

Once again thank you for your great repository!
I have a question about compressed image visualization.
After the support of compressedVideo (introduced in #471), I found out compressed images with jpeg compressed bgr8 is not supported anymore.

Is this intended or a bug? If this is a bug, I'll create a bug report.
It worked before. (Check by checking out to 4ae691b)

Thanks for your support in advance!

[aneuwald-ctw]

@go-sakayori thanks for reaching us!

Yes we noticed that after the implementation of compressed video. Your messages have "jpg" as format? I guess jpg bgr8 should continue being usable. We notice that some MCAPs came with the "format" empty, and that should be defined. Can you confirm that?

[aneuwald-ctw]

So, actually it is part of the message.

Take your image topic /sensing/camera/camera1/image_rect_color/compressedand put this in a RawMessage panel.

You should see something like this:

If the format is not defined, you will definitely see an error:

Can you check if that is the case?

[go-sakayori]

@aneuwald-ctw Thank you, I got it now.
The format is "bgr8; jpeg compressed bgr8.
This is what it looks like:

[go-sakayori]

@aneuwald-ctw friendly ping.

[aneuwald-ctw]

Hey @go-sakayori, so you can give a look of the schema documentation: https://docs.foxglove.dev/docs/visualization/panels/image

If you are using CompressedImage you can use only "webpg", "jpeg", "jpg" , "png" or "avif". Probably before Lichtblick was not finding your format and falling back to"jpg". Please use only jpeg if it is compressed image or bgr8if you are using raw images.

Please give a feedback if it worked!

[go-sakayori]

@aneuwald-ctw Thank you for the reply. I'll try to convert the format and see what happens.
According to the the supported messages documentation, the ROS2 sensor_msgs/msg/CompressedImage is supported. Currently the format is likely to be specified in ORIG_PIXFMT; CODEC compressed. My question is would there be a future implementation to support this format type?

[aneuwald-ctw]

Yes, this is actually a problem while we didn't finish our documentation yet. So Lichtblick does not support ROS 2 yet, only ROS 1. If you check this link, this is the interface we expected from a ROS msg: https://docs.ros2.org/foxy/api/sensor_msgs/msg/CompressedImage.html

Are you using ROS2?

[go-sakayori]

@aneuwald-ctw Thanks for the information. Yes, I'm currently using ROS2 humble and collecting some driving data in ROS bag formats, and utilizing Lichtblick to construct dataset from the collected data.

[aneuwald-ctw]

Oh that's the problem. I will bring this to our team internally and put ROS2 in our internal roadmap, thanks for bringing that!

You can may create a user script / message converter to convert it, that way you don't need to change your source

[aneuwald-ctw]

Hello @go-sakayori ! how are you doing?

I was wrong about my conclusions, because Lichtblick does accept ros2 messages, but I would like to know how is your message.

Look that example, where I force a schema from ROS2 and do it accordingly. I put random values that's why the image is broken, but it works without the broken image. I used the "bgr8", showing that there is support to it.

But Lichtblick is waiting for these interface:

ROS2: https://docs.ros2.org/foxy/api/sensor_msgs/msg/CompressedImage.html

[go-sakayori]

@aneuwald-ctw Hi, thanks for reaching out, and sorry for my late reply.
So the camera message I'm using is rgb8; jpeg compressed bgr8 as shown in the image below.
I suppose this format is also supported in ROS2.

[aneuwald-ctw]

@go-sakayori can you try out with this branch: feature/accept-any-image-format

[go-sakayori]

@aneuwald-ctw
Sorry for my late reply. I was out of office for a while.
I've noticed that you have already merged it into the main branch! Thank you for your work.
I have confirmed that the image could be shown now!