[#709] Add initial set of end to end tests for CLI

[orecham]

Notes for Reviewer

Adds initial set of end-to-end tests for iceoryx2-cli.

Not all CLI functionality is covered, just the most straight-forward. Provides the foundation for adding more complex test cases in follow-ups.

Additionally:

1. A number of bugs that were discovered have been fixed:
   1. Print newline character after console output to prevent odd artifacts from being inserted by the terminal
   2. Print help for positional arguments
   3. Properly organize provided CLI options in iox2-tunnel
2. Some new features were added to facilitate testing and and some more QoL:
   1. Option to print summarized or detailed information of discovered services in iox2 service discovery
   2. Option to force overwrite in iox2 config generate
3. Each command was organized into their own modules for cleanliness
   1. Some commands are complex and utilize multiple helper functions
   2. Same convention applied to all commands (even the most simple ones) for consistency

Pre-Review Checklist for the PR Author

• Add sensible notes for the reviewer
• PR title is short, expressive and meaningful
• Consider switching the PR to a draft (Convert to draft)
  • as draft PR, the CI will be skipped for pushes
• Relevant issues are linked in the References section
• Every source code file has a copyright header with SPDX-License-Identifier: Apache-2.0 OR MIT
• Branch follows the naming format (iox2-123-introduce-posix-ipc-example)
• Commits messages are according to this guideline
  • Commit messages have the issue ID ([#123] Add posix ipc example)
  • Commit author matches Eclipse Contributor Agreement(and ECA is signed)
• Tests follow the best practice for testing
• Changelog updated in the unreleased section including API breaking changes
• Assign PR to reviewer
• All checks have passed (except task-list-completed)

Checklist for the PR Reviewer

• Commits are properly organized and messages are according to the guideline
• Unit tests have been written for new behavior
• Public API is documented
• PR title describes the changes

Post-review Checklist for the PR Author

• All open points are addressed and tracked via issues

References

Closes #709

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.52%. Comparing base (484f846) to head (68719b5).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1058      +/-   ##
==========================================
- Coverage   78.53%   78.52%   -0.02%     
==========================================
  Files         342      342              
  Lines       36845    36845              
  Branches      271      271              
==========================================
- Hits        28937    28932       -5     
- Misses       7331     7336       +5     
  Partials      577      577              

Flag	Coverage Δ
Rust	78.52% <ø> (-0.02%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 5 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[orecham]

Not sure if this is the best place to put this common script now that it is used outside of the examples directory ... accepting ideas.

[elBoberido]

We also did not like it when we created it but we had no better idea where to place it. Suggestions welcome.

[orecham]

@elfenpiff Thoughts on creating this top-level iceoryx2-testing directory for utility scripts like this? I was also thinking to move the types used in examples to here too so they can be re-used in tests (e.g. the ComplexData type, which I am currently duplicating in the iceoryx2-tunnel test.

iceoryx2-bb-testing did not seem like the right place for these to me. The nesting in iceoryx2-bb does not seem fitting.

[elBoberido]

@orecham in theory, this could as well go into internal/something. It is currently only meant to be used in the repo

[orecham]

@elBoberido I moved the script to internal/end-to-end-testing/.

[orecham]

@elBoberido FYI I would like to remove this from our structs because it causes the RON output to prepend #r before every field (since it is not valid Rust syntax):

        global: (
            r#root-path: "/tmp/iceoryx2/",
            prefix: "iox2_",
            service: (
                directory: "services",
                r#data-segment-suffix: ".data",
                r#static-config-storage-suffix: ".service",
                r#dynamic-config-storage-suffix: ".dynamic",
                r#creation-timeout: (
                    secs: 0,
                    nanos: 500000000,
                ),
                r#connection-suffix: ".connection",
                r#event-connection-suffix: ".event",
                r#blackboard-mgmt-suffix: ".blackboard_mgmt",
                r#blackboard-data-suffix: ".blackboard_data",
            ),
            node: (
                directory: "nodes",
                r#monitor-suffix: ".node_monitor",
                r#static-config-suffix: ".details",
                r#service-tag-suffix: ".service_tag",
                r#cleanup-dead-nodes-on-creation: true,
                r#cleanup-dead-nodes-on-destruction: true,
            ),
        )

[elBoberido]

I wouldn't do that. It would completely break every config setup out there.

[orecham]

@elBoberido Hmm, if we ever want to change it, now (before v1.0.0) would be the time. Otherwise we need to live with this not-so-nice output from the default formater, which is currently ron.

We could of course change the default format to something else... although ron seems the most appropriate, given we the core is Rust.

[elBoberido]

Isn't this just for testing? Shouldn't the default formatter be TOML, which is also used for the config files

[orecham]

@elBoberido Not everything output by the CLI is a config. The format can be selected by the user using the --format option - the motivation here is that users can choose what they prefer and if they are parsing the output programatically specify the output (e.g. if they decide they prefer to parse it as JSON). For instance they might want to parse the output of iox2 service subscribe from a script or other application.

The default format was chosen kind of arbitrarily, but the train of thought was to use ron since it is a Rust project and besides the r# it seemed to be the most human-readable (other output formats deal withValueEnums kind of strangely), but only Ron has this not-so-nice-to-read r# before strings.

Moving forward, we either:

1. Leave it as is and just live with the r# being prepended for ron
2. Change the default formatting to something else (although users will still be able to choose something else)
3. Remove the kebab-case modification to the output

[elBoberido]

Unfortunately, we did not version the config files. If this would be the case, we would be able to read the version and the print a warning to use iox2 config check --fix. We cold of course start with the versioning now, and if no version info is present, print the warning.

We will most probably also break the C++ API with the new Optional, so we might as well declare the v0.8.0 as broken records release 😬

@elfenpiff what is your opinion?

[orecham]

@elfenpiff Ping

[elfenpiff]

I side with @orecham on this one but I have no strong feelings. But this would have to be prepared a bit more in the release notes. We have to state search - and replace it with _

[orecham]

What about the idea of implementing a cli command to check and fix the config to make it easier for users to update their configs.

Something like iox2 config check and iox2 config fix.

It might take some time to actually get to this though. Not so sure it will be trivial to do it well.

[elBoberido]

Also no strong feeling about this. We will introduce breakages with the new C++ Optional so it might be a good idea to use the next release for other breaking changes as well.