[build] Update protobuf to 5.29.5

[DownerCase]

Description

Updates protobuf from 3.11.4 to 5.29.4.
Originally it would have been to 4.25.x but:
a. That major version just went out of maintenance support.
b. There is a regression in that version with msvc and /permissive- which impacts eCAL.
c. Jumping to 5.x didn't seem to need any additional work...
d. 5.x gives a year few months of maintenance support

• EDIT: 5.x is out of support after March 2026
• https://protobuf.dev/support/version-support/

Unfortunately, updating protobuf has some knock on effects.

1. Protobuf now depends on the abseil library collection.
2. So abseil needed to be added as a submodule, wired in, and gets vendored out into the install tree along with protobuf
3. Abseil had some target names that conflicted with those in eCAL so they had to be changed to avoid collisions.
4. Abseil has strict ABI rules and requires everything that uses it to be compiled with the same C++ version as it.
5. This means I had to add a mechanism to detect and enforce an appropriate C++ version for the project via CMAKE_CXX_STANDARD

• The majority of the time this will be set to C++17 due to the use of FTXUI or QT6
• Moving the project to strictly require C++17 is not something currently needed, but will be for protobuf 6.x

6. Some executables/plugins were requiring C++17 when not needed so they have been bumped back to C++14

• Without this they try and use protobuf with std::string_view APIs but as protobuf was compiled with C++14 they would be missing and cause link failures.

7. The update to protobuf removes the protobuf-protos target so it is removed from the innosetup script

• (The files it use to include are now part of protobuf-headers)

8. There was a typo in the C# CMake for determining the protobuf version I corrected.
9. To catch integration problems (like UPB needing to be disabled) I added a simple test to the Windows workflow

• Windows specifically as the others use system packages for protobuf.

See also:

• Protobuf version support: https://protobuf.dev/support/version-support/

WARNING I have not lent any attention to the Python or C# (beyond fixing an obvious bug) binding situation (but no CI job has started failing). If there's something that needs attention point me at it and I'll have a go 😄

Related issues

Related to #1187 (I'll let the team decide if this closes it or not)
Superceeds #1818
Plus a bunch of other open issues mainly relating to the Python situation.

[KerstinKeller]

It's a big one. What I find a bit strange - though nice: it's only build system changes. So apparently nothing of the API parts which we used have been changed. Which is great, and unexpected.

I am a bit confused though, especially regarding ftxui, since you downgraded the c++ standard there (mon_cli for example) - that it builds, because my understanding was that C++17 was a requirement there.
Also it was downgraded in some protobuf samples, I really though that there was a reason that I have upgraded it recently.

Also, Protobuf is no longer a dependency of eCAL core.
We use it for the apps, and then of course the service and pubsub protobuf extensions.
Having a change like this, I wonder if we should go into the direction that users using eCAL&Protobuf should build the protobuf extension themselves, because then they have full control of the protobuf version, their C++ standard, ... But I know that not all eCAL devs will share that point of view.

but I will take a proper look tomorrow.

[DownerCase]

It's a big one. What I find a bit strange - though nice: it's only build system changes. So apparently nothing of the API parts which we used have been changed. Which is great, and unexpected.

I am a bit confused though, especially regarding ftxui, since you downgraded the c++ standard there (mon_cli for example) - that it builds, because my understanding was that C++17 was a requirement there. Also it was downgraded in some protobuf samples, I really though that there was a reason that I have upgraded it recently.

Also, Protobuf is no longer a dependency of eCAL core. We use it for the apps, and then of course the service and pubsub protobuf extensions. Having a change like this, I wonder if we should go into the direction that users using eCAL&Protobuf should build the protobuf extension themselves, because then they have full control of the protobuf version, their C++ standard, ... But I know that not all eCAL devs will share that point of view.

but I will take a proper look tomorrow.

Thanks Kerstin.

For protobuf the major version changes on the C++ API are nominal and scheduled yearly*, so going from 3.x -> 5.x hasn't required eCAL to change anything. Going to 6.x will require changes because they move signatures from const std::string& to std::string_view (expect another patch set after this one addressing that).

Technically ecal_mon_tui uses FTXUI so it implicitly gains a C++17 requirement, I was just doing a grep for cxx_std_17 and making sure nothing was using both that and protobuf. Also, perftool could get away with using C++17 but turns out it doesn't need it 🤷
The protobuf samples absolutely do need to be bumped back.

I can't speak for anyone except me, but protobuf is pretty much the only serializer I have used with eCAL, I consider it a "core" feature.

Splitting off "ecal-protobuf" could work as its fairly simple, and wouldn't be too hard for people to integrate into their builds, one way or another.

That said if nothing is done, this change will, with great certainty, bite someone if the binaries continue to be supplied in the current methods. Personally I'm against including the SDK in the windows installer (just leave it as the app installer) and would direct people to Conan/vcpkg/others. I appreciate that's considerable extra effort, but am willing to help. On the Linux side Debian-based distros are stuck on the pre-abseil protobuf versions so not an imminent concern yet. I've taken up maintainership of eCAL for the AUR (where protobuf is updated) and am working through all the issues; but here there aren't exactly many users.

---

* Some extra reading on Protobuf versions if you care:

https://protobuf.dev/support/version-support/
https://protobuf.dev/support/migration/
https://protobuf.dev/support/migration/#cpp-22

[DownerCase]

Updated with latest master and added sbom.py, you'll want to double check that though.

[KerstinKeller]

We now do need C++17 for several eCAL internal components, but in a way that it will stay compatible with VS2017 compiler (v141).
That said, can't we force / inject the actual C++ standard through the CMake compiler flags injected from outside (CMakePresets, or workflow files)?
My understanding was that target_compile_features(${PROJECT_NAME} PRIVATE cxx_std_14) will set a minimum required standard, but we could always go higher?

[DownerCase]

My understanding was that target_compile_features(${PROJECT_NAME} PRIVATE cxx_std_14) will set a minimum required standard, but we could always go higher?

This is correct, it sets the minimum C++ version for that target.

That said, can't we force / inject the actual C++ standard through the CMake compiler flags injected from outside (CMakePresets, or workflow files)?

I appear to have accommodated that with this section in the root CMakeLists that only sets the C++ standard if eCAL is not a sub-project and the standard hasn't been externally set like how you suggest (by setting CMAKE_CXX_STANDARD):

if("${CMAKE_SOURCE_DIR}" STREQUAL "${eCAL_SOURCE_DIR}" AND NOT DEFINED CMAKE_CXX_STANDARD)
  # eCAL is being compiled as the root project, and no standard version is set
  set(CMAKE_CXX_STANDARD "${_ECAL_CXX_STD}")
  message(STATUS "eCAL is using C++ ${_ECAL_CXX_STD}")
else()
  # eCAL is part of a larger build or the standard was externally set
endif()

We now do need C++17 for several eCAL internal components, but in a way that it will stay compatible with VS2017 compiler (v141).

As mentioned in the OP, the key catch is that everything that uses protobuf headers needs to be using at least (but preferably exactly) the C++ version abseil/protobuf was built for, as the available interface differs depending on the C++ version.

For example, your C++17 component would see string_view protobuf interfaces whilst a C++14 one would see string based ones. You would then get linker errors depending on which version of the interface is actually compiled into the protobuf library itself.

The easiest solution is just to use the same version everywhere, but sounds like you want to keep a C++14 interface for now?

It's been a while since I looked at this, so will try and poke things a bit later this week to get a clearer explanation of the failure modes. The easiest way should be to disable the bit where CMAKE_CXX_STANDARD gets set.

[DownerCase]

Just did a quick test setting the CMAKE_CXX_STANDARD to C++14 and building (because my compilers default is >= C++17).
ecalsys_gui uses C++17 fails to link due to the string/string_view difference on ParseFromString

ecalsys_gui.cpp:(.text._ZNSt17_Function_handlerIFiRKN4eCAL25SServiceMethodInformationERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERS9_EZNS0_8protobuf14CServiceServerINS0_2pb3sys7ServiceEE15RegisterMethodsEvEUlS3_SB_SC_E_E9_M_invokeERKSt9_Any_dataS3_SB_SC_[_ZNSt17_Function_handlerIFiRKN4eCAL25SServiceMethodInformationERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERS9_EZNS0_8protobuf14CServiceServerINS0_2pb3sys7ServiceEE15RegisterMethodsEvEUlS3_SB_SC_E_E9_M_invokeERKSt9_Any_dataS3_SB_SC_]+0x153):
 undefined reference to `google::protobuf::MessageLite::ParseFromString(std::basic_string_view<char, std::char_traits<char> >)'

Just pushed a commit to see what CI thinks if you don't try and build with a consistent version.