Skip to content

Commit 8daf9cf

Browse files
ClemensLinnhoffClemensLinnhoff
authored
Separate sensor model behavior tests from integration tests (#29)
* Update workflows from template Signed-off-by: ClemensLinnhoff <Clemens.Linnhoff@partner.bmw.de> * Readd BSL-1.0 license Signed-off-by: ClemensLinnhoff <Clemens.Linnhoff@partner.bmw.de> * Add behavior test folder Signed-off-by: ClemensLinnhoff <Clemens.Linnhoff@partner.bmw.de> * Fix links in regression test Signed-off-by: ClemensLinnhoff <Clemens.Linnhoff@partner.bmw.de> * Check object in FOV Signed-off-by: ClemensLinnhoff <Clemens.Linnhoff@partner.bmw.de> * Specify model profile in object fov behavior test Signed-off-by: ClemensLinnhoff <Clemens.Linnhoff@partner.bmw.de> * Fix path in behavior test Signed-off-by: ClemensLinnhoff <Clemens.Linnhoff@partner.bmw.de> * Add trace file generation Signed-off-by: ClemensLinnhoff <Clemens.Linnhoff@partner.bmw.de> * Add license to trace file generator script Signed-off-by: ClemensLinnhoff <Clemens.Linnhoff@partner.bmw.de> * Fix if in checking for files Signed-off-by: ClemensLinnhoff <Clemens.Linnhoff@partner.bmw.de> * Try trace file naming convention Signed-off-by: ClemensLinnhoff <Clemens.Linnhoff@partner.bmw.de> * Add trace file generation to test 002 Signed-off-by: ClemensLinnhoff <Clemens.Linnhoff@partner.bmw.de> * Adjust analysis scripts Signed-off-by: ClemensLinnhoff <Clemens.Linnhoff@partner.bmw.de> * Add lidar occlusion test Signed-off-by: ClemensLinnhoff <Clemens.Linnhoff@partner.bmw.de> * Add radar behavior test Signed-off-by: ClemensLinnhoff <Clemens.Linnhoff@partner.bmw.de> * Minor fix in test readmes Signed-off-by: ClemensLinnhoff <Clemens.Linnhoff@partner.bmw.de> * Fix readme headlines Signed-off-by: ClemensLinnhoff <Clemens.Linnhoff@partner.bmw.de> --------- Signed-off-by: ClemensLinnhoff <Clemens.Linnhoff@partner.bmw.de>
1 parent 414ffeb commit 8daf9cf

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

49 files changed

+3587
-40
lines changed

‎.github/workflows/build.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,13 +10,13 @@ jobs:
1010

1111
steps:
1212
- name: Checkout Model
13-
uses: actions/checkout@v3
13+
uses: actions/checkout@v4
1414
with:
1515
submodules: true
1616

1717
- name: Cache Protobuf
1818
id: cache-protobuf
19-
uses: actions/cache@v3
19+
uses: actions/cache@v4
2020
with:
2121
path: protobuf-21.12
2222
key: ${{ runner.os }}-protobuf
@@ -55,7 +55,7 @@ jobs:
5555

5656
- name: Cache Model FMU
5757
id: cache-model-fmu
58-
uses: actions/cache@v3
58+
uses: actions/cache@v4
5959
with:
6060
path: /tmp/model_fmu
6161
key: ${{ runner.os }}-model-fmu-${{ github.sha }}

‎.github/workflows/build_esmini.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ jobs:
1111
steps:
1212
- name: Cache esmini FMU
1313
id: cache-esmini-fmu
14-
uses: actions/cache@v3
14+
uses: actions/cache@v4
1515
with:
1616
path: /tmp/esmini_fmu
1717
key: ${{ runner.os }}-esmini-fmu

‎.github/workflows/build_openmcx.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ jobs:
1111
steps:
1212
- name: Cache OpenMCx
1313
id: cache-openmcx
14-
uses: actions/cache@v3
14+
uses: actions/cache@v4
1515
with:
1616
path: openmcx
1717
key: ${{ runner.os }}-openmcx

‎.github/workflows/build_osi_field_checker.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ jobs:
1111
steps:
1212
- name: Cache Protobuf
1313
id: cache-protobuf
14-
uses: actions/cache@v3
14+
uses: actions/cache@v4
1515
with:
1616
path: protobuf-21.12
1717
key: ${{ runner.os }}-protobuf
@@ -31,7 +31,7 @@ jobs:
3131

3232
- name: Cache OSI Field Checker FMU
3333
id: cache-osi-field-checker
34-
uses: actions/cache@v3
34+
uses: actions/cache@v4
3535
with:
3636
path: /tmp/osi-field-checker
3737
key: ${{ runner.os }}-osi-field-checker

‎.github/workflows/build_tracefile_player.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ jobs:
1111
steps:
1212
- name: Cache Protobuf
1313
id: cache-protobuf
14-
uses: actions/cache@v3
14+
uses: actions/cache@v4
1515
with:
1616
path: protobuf-21.12
1717
key: ${{ runner.os }}-protobuf
@@ -31,7 +31,7 @@ jobs:
3131

3232
- name: Cache Tracefile Player FMU
3333
id: cache-tracefile-player-fmu
34-
uses: actions/cache@v3
34+
uses: actions/cache@v4
3535
with:
3636
path: /tmp/tracefile_player_fmu
3737
key: ${{ runner.os }}-tracefile-player-fmu

‎.github/workflows/build_tracefile_writer.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ jobs:
1111
steps:
1212
- name: Cache Protobuf
1313
id: cache-protobuf
14-
uses: actions/cache@v3
14+
uses: actions/cache@v4
1515
with:
1616
path: protobuf-21.12
1717
key: ${{ runner.os }}-protobuf
@@ -31,7 +31,7 @@ jobs:
3131

3232
- name: Cache Tracefile Writer FMU
3333
id: cache-tracefile-writer-fmu
34-
uses: actions/cache@v3
34+
uses: actions/cache@v4
3535
with:
3636
path: /tmp/tracefile_writer_fmu
3737
key: ${{ runner.os }}-tracefile-writer-fmu

‎.github/workflows/cl0.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ jobs:
99
runs-on: ubuntu-latest
1010
steps:
1111
- name: checkout
12-
uses: actions/checkout@v3
12+
uses: actions/checkout@v4
1313
- uses: enarx/spdx@master
1414
with:
1515
licenses: Apache-2.0 BSD-3-Clause BSD-2-Clause BSL-1.0 MIT MPL-2.0

‎.github/workflows/cl2.yml

Lines changed: 12 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ jobs:
2929
name: Generate Integration Test Paths
3030

3131
steps:
32-
- uses: actions/checkout@v3
32+
- uses: actions/checkout@v4
3333
- name: Generate matrix with all integration tests
3434
id: set-matrix
3535
working-directory: test/integration
@@ -52,55 +52,55 @@ jobs:
5252
run: echo ${{ github.workspace }}/${{ matrix.path }}
5353

5454
- name: Checkout Model
55-
uses: actions/checkout@v3
55+
uses: actions/checkout@v4
5656
with:
5757
submodules: true
5858

5959
- name: Cache Protobuf
6060
id: cache-protobuf
61-
uses: actions/cache@v3
61+
uses: actions/cache@v4
6262
with:
6363
path: protobuf-21.12
6464
key: ${{ runner.os }}-protobuf
6565

6666
- name: Cache Model FMU
6767
id: cache-model-fmu
68-
uses: actions/cache@v3
68+
uses: actions/cache@v4
6969
with:
7070
path: /tmp/model_fmu
7171
key: ${{ runner.os }}-model-fmu-${{ github.sha }}
7272

7373
- name: Cache Tracefile Player FMU
7474
id: cache-tracefile-player-fmu
75-
uses: actions/cache@v3
75+
uses: actions/cache@v4
7676
with:
7777
path: /tmp/tracefile_player_fmu
7878
key: ${{ runner.os }}-tracefile-player-fmu
7979

8080
- name: Cache Tracefile Writer FMU
8181
id: cache-tracefile-writer-fmu
82-
uses: actions/cache@v3
82+
uses: actions/cache@v4
8383
with:
8484
path: /tmp/tracefile_writer_fmu
8585
key: ${{ runner.os }}-tracefile-writer-fmu
8686

8787
- name: Cache esmini FMU
8888
id: cache-esmini-fmu
89-
uses: actions/cache@v3
89+
uses: actions/cache@v4
9090
with:
9191
path: /tmp/esmini_fmu
9292
key: ${{ runner.os }}-esmini-fmu
9393

9494
- name: Cache OSI Field Checker
9595
id: cache-osi-field-checker
96-
uses: actions/cache@v3
96+
uses: actions/cache@v4
9797
with:
9898
path: /tmp/osi-field-checker
9999
key: ${{ runner.os }}-osi-field-checker
100100

101101
- name: Cache OpenMCx
102102
id: cache-openmcx
103-
uses: actions/cache@v3
103+
uses: actions/cache@v4
104104
with:
105105
path: openmcx
106106
key: ${{ runner.os }}-openmcx
@@ -136,14 +136,13 @@ jobs:
136136
137137
- name: Archive Trace File
138138
if: steps.check-for-trace-files.outputs.found_trace == 1
139-
uses: actions/upload-artifact@v3
139+
uses: actions/upload-artifact@v4
140140
with:
141141
name: ${{ steps.check-for-trace-files.outputs.trace_file_name }}
142142
path: ./test/integration/${{ matrix.path }}/output/${{ steps.check-for-trace-files.outputs.trace_file_name }}
143143

144144
- name: Check For Python Scripts
145145
id: check-for-python-scripts
146-
if: steps.check-for-trace-files.outputs.found_trace == 1
147146
working-directory: ./test/integration/${{ matrix.path }}
148147
run: |
149148
( count=`ls -1 *.py 2>/dev/null | wc -l`
@@ -165,14 +164,14 @@ jobs:
165164
run: sudo make install && sudo ldconfig
166165

167166
- name: Install Analysis Dependencies
168-
if: steps.check-for-python-scripts.outputs.found_script == 1 && steps.check-for-trace-files.outputs.found_trace == 1
167+
if: steps.check-for-python-scripts.outputs.found_script == 1
169168
working-directory: ./lib/open-simulation-interface
170169
run: |
171170
python3 -m pip install protobuf==4.21.12
172171
python3 -m pip install .
173172
174173
- name: Run Trace File Analysis
175-
if: steps.check-for-python-scripts.outputs.found_script == 1 && steps.check-for-trace-files.outputs.found_trace == 1
174+
if: steps.check-for-python-scripts.outputs.found_script == 1
176175
working-directory: ./test/integration/${{ matrix.path }}
177176
run: python3 ${{ steps.check-for-python-scripts.outputs.python_file_name }} output/${{ steps.check-for-trace-files.outputs.trace_file_name }}
178177

‎.github/workflows/cl3.yml

Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
name: Credibility Assessment Level 3
2+
3+
on:
4+
workflow_call:
5+
6+
jobs:
7+
generate_behavior_test_paths:
8+
runs-on: ubuntu-latest
9+
name: Generate Behavior Test Paths
10+
11+
steps:
12+
- uses: actions/checkout@v4
13+
- name: Generate matrix with all behavior tests
14+
id: set-matrix
15+
working-directory: test/behavior
16+
run: |
17+
echo "matrix=$(ls -l | grep '^d' | awk -F ' ' '{print $9}' | grep -Po '\d{3}.*' | jq -R -s -c 'split("\n") | map(select(length > 0))')" >> $GITHUB_OUTPUT
18+
outputs:
19+
matrix: ${{ steps.set-matrix.outputs.matrix }}
20+
21+
run_behavior_test:
22+
needs: [generate_behavior_test_paths]
23+
name: Behavior Test
24+
runs-on: ubuntu-latest
25+
strategy:
26+
fail-fast: false
27+
matrix:
28+
path: ${{ fromJson(needs.generate_behavior_test_paths.outputs.matrix) }}
29+
30+
steps:
31+
- name: Print matrix path
32+
run: echo ${{ github.workspace }}/${{ matrix.path }}
33+
34+
- name: Checkout Model
35+
uses: actions/checkout@v4
36+
with:
37+
submodules: true
38+
39+
- name: Cache Protobuf
40+
id: cache-protobuf
41+
uses: actions/cache@v4
42+
with:
43+
path: protobuf-21.12
44+
key: ${{ runner.os }}-protobuf
45+
46+
- name: Cache Model FMU
47+
id: cache-model-fmu
48+
uses: actions/cache@v4
49+
with:
50+
path: /tmp/model_fmu
51+
key: ${{ runner.os }}-model-fmu-${{ github.sha }}
52+
53+
- name: Cache Tracefile Player FMU
54+
id: cache-tracefile-player-fmu
55+
uses: actions/cache@v4
56+
with:
57+
path: /tmp/tracefile_player_fmu
58+
key: ${{ runner.os }}-tracefile-player-fmu
59+
60+
- name: Cache Tracefile Writer FMU
61+
id: cache-tracefile-writer-fmu
62+
uses: actions/cache@v4
63+
with:
64+
path: /tmp/tracefile_writer_fmu
65+
key: ${{ runner.os }}-tracefile-writer-fmu
66+
67+
- name: Cache esmini FMU
68+
id: cache-esmini-fmu
69+
uses: actions/cache@v4
70+
with:
71+
path: /tmp/esmini_fmu
72+
key: ${{ runner.os }}-esmini-fmu
73+
74+
- name: Cache OSI Field Checker
75+
id: cache-osi-field-checker
76+
uses: actions/cache@v4
77+
with:
78+
path: /tmp/osi-field-checker
79+
key: ${{ runner.os }}-osi-field-checker
80+
81+
- name: Cache OpenMCx
82+
id: cache-openmcx
83+
uses: actions/cache@v4
84+
with:
85+
path: openmcx
86+
key: ${{ runner.os }}-openmcx
87+
88+
- name: Install dependencies
89+
run: sudo apt install -y libxml2-dev zlib1g-dev libzip-dev
90+
91+
- name: Install ProtoBuf
92+
working-directory: protobuf-21.12
93+
run: sudo make install && sudo ldconfig
94+
95+
- name: Check if analysis script exists
96+
id: check_analysis_script
97+
uses: andstor/file-existence-action@v3
98+
with:
99+
files: test/behavior/${{ matrix.path }}/analyze.py
100+
101+
- name: Check if trace file generator exists
102+
id: check_trace_generator
103+
uses: andstor/file-existence-action@v3
104+
with:
105+
files: test/behavior/${{ matrix.path }}/generate_test_trace.py
106+
107+
- name: Install OSI for python
108+
if: steps.check_analysis_script.outputs.files_exists == 'true' || steps.check_trace_generator.outputs.files_exists == 'true'
109+
working-directory: ./lib/open-simulation-interface
110+
run: |
111+
python3 -m pip install protobuf==4.21.12
112+
python3 -m pip install .
113+
114+
- name: Run test trace generation
115+
if: steps.check_trace_generator.outputs.files_exists == 'true'
116+
working-directory: ./test/behavior/${{ matrix.path }}
117+
run: python3 generate_test_trace.py
118+
119+
- name: Create Output Folder
120+
working-directory: ./test/behavior/${{ matrix.path }}
121+
run: mkdir output
122+
123+
- name: Run OpenMCx
124+
id: cosimulation
125+
run: ./openmcx/install/openmcx ./test/behavior/${{ matrix.path }}/SystemStructure.ssd
126+
127+
- name: Check For Trace Files
128+
id: check-for-trace-files
129+
working-directory: ./test/behavior/${{ matrix.path }}/output
130+
run: |
131+
( count=`ls -1 *.osi 2>/dev/null | wc -l`
132+
if [ $count == 0 ]; then
133+
echo No trace file found.
134+
echo "found_trace=0" >> $GITHUB_OUTPUT
135+
elif [ $count == 1 ]; then
136+
echo "found_trace=1" >> $GITHUB_OUTPUT
137+
filename=$(ls -1 *.osi)
138+
echo "trace_file_name=${filename}" >> $GITHUB_OUTPUT
139+
else
140+
echo More than one trace file.
141+
echo "found_trace=0" >> $GITHUB_OUTPUT
142+
fi
143+
)
144+
145+
- name: Archive Trace File
146+
if: steps.check-for-trace-files.outputs.found_trace == 1
147+
uses: actions/upload-artifact@v4
148+
with:
149+
name: ${{ steps.check-for-trace-files.outputs.trace_file_name }}
150+
path: ./test/behavior/${{ matrix.path }}/output/${{ steps.check-for-trace-files.outputs.trace_file_name }}
151+
152+
- name: Run Trace File Analysis
153+
if: steps.check_analysis_script.outputs.files_exists == 'true'
154+
working-directory: ./test/behavior/${{ matrix.path }}
155+
run: python3 analyze.py output/${{ steps.check-for-trace-files.outputs.trace_file_name }}
156+
157+
- name: Failed?
158+
if: steps.cosimulation.outputs.failed == 1
159+
run: exit 1

‎.github/workflows/cpp-linter.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,13 +9,13 @@ jobs:
99
runs-on: ubuntu-latest
1010
steps:
1111
- name: Checkout Model
12-
uses: actions/checkout@v3
12+
uses: actions/checkout@v4
1313
with:
1414
submodules: true
1515

1616
- name: Cache Protobuf
1717
id: cache-protobuf
18-
uses: actions/cache@v3
18+
uses: actions/cache@v4
1919
with:
2020
path: protobuf-21.12
2121
key: ${{ runner.os }}-protobuf

‎.github/workflows/credibility_assessment.yml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,3 +19,8 @@ jobs:
1919
name: CL 2
2020
needs: cl1
2121
uses: ./.github/workflows/cl2.yml
22+
23+
cl3:
24+
name: CL 3
25+
needs: cl2
26+
uses: ./.github/workflows/cl3.yml

0 commit comments

Comments
 (0)