Skip to content

Commit c70d35c

Browse files
ben-spillerben-spiller
authored
Merge pull request #14 from ApamaCommunity/gha
GitHub Actions instead of Travis
2 parents 043978b + fd49a99 commit c70d35c

File tree

5 files changed

+143
-108
lines changed

5 files changed

+143
-108
lines changed

‎.github/workflows/test.yml

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
name: Tests
2+
3+
on:
4+
push:
5+
branches: [ master ]
6+
7+
pull_request:
8+
branches: [ master ]
9+
10+
workflow_dispatch: # also allow manually invoking this
11+
12+
jobs:
13+
test:
14+
strategy:
15+
# Disable fail fast since it's useful to see test results for all platforms even if some failed
16+
fail-fast: false
17+
18+
matrix:
19+
include:
20+
- test-run-id: ubuntu-py3.9
21+
os: ubuntu-latest
22+
python-version: 3.9
23+
perf-test: true
24+
25+
- test-run-id: win-py3.6
26+
os: windows-latest
27+
python-version: 3.6
28+
perf-test: false
29+
30+
runs-on: ${{matrix.os}}
31+
32+
steps:
33+
# Install the desired version of Python and PySys
34+
- uses: actions/checkout@v2
35+
36+
- uses: actions/setup-python@v2
37+
with:
38+
python-version: ${{matrix.python-version}}
39+
40+
- name: Cache Python deps (get location)
41+
id: pip-cache
42+
run: |
43+
python -m pip install -U "pip>=20.1"
44+
echo "::set-output name=dir::$(pip cache dir)"
45+
- name: Cache Python deps
46+
uses: actions/cache@v2
47+
with:
48+
path: ${{ steps.pip-cache.outputs.dir }}
49+
key: ${{ matrix.test-run-id }}-pip
50+
51+
- name: Python dependencies - install
52+
id: deps
53+
shell: bash
54+
run: |
55+
pip install pysys==2.0 wheel
56+
57+
# Use older version of coverage as 5.0 requires an SQLite version that doesn't work on this macos image currently
58+
pip install coverage==4.5.4
59+
60+
- name: Test with PySys
61+
working-directory: tests/correctness
62+
shell: bash
63+
id: pysys
64+
run: |
65+
if [[ "${{ matrix.perf-test }}" = "true" ]]; then
66+
cd ..
67+
echo Running ALL tests from `pwd`
68+
fi
69+
70+
python -m pysys run --ci --outdir=${{matrix.test-run-id}}
71+
72+
- name: Upload Python code coverage
73+
uses: codecov/codecov-action@v1
74+
if: steps.pysys.outputs.artifact_PythonCoverageDir
75+
with:
76+
file: ${{ steps.pysys.outputs.artifact_PythonCoverageDir }}/coverage.xml
77+
78+
# If any tests fail, PySys will return an error code, so we need "if: always()" so the following steps will run
79+
80+
- name: Upload performance CSV artifacts
81+
uses: actions/upload-artifact@v2
82+
# Only do this if some performance results were generated
83+
if: always() && steps.pysys.outputs.artifact_CSVPerformanceReport
84+
85+
with:
86+
name: pysys_performance_${{matrix.test-run-id}}
87+
path: ${{ steps.pysys.outputs.artifact_CSVPerformanceReport }}
88+
89+
- name: Upload archive artifacts for any test failures
90+
uses: actions/upload-artifact@v2
91+
if: always() && steps.pysys.outputs.artifact_TestOutputArchiveDir
92+
93+
with:
94+
name: pysys_output_${{matrix.test-run-id}}
95+
path: ${{ steps.pysys.outputs.artifact_TestOutputArchiveDir }}

‎.travis.yml

Lines changed: 0 additions & 64 deletions
This file was deleted.

‎README.rst

Lines changed: 15 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
1-
.. image:: https://travis-ci.com/ApamaCommunity/apama-log-analyzer.svg?branch=master
2-
:target: https://travis-ci.com/ApamaCommunity/apama-log-analyzer
1+
.. image:: ../../workflows/Tests/badge.svg
2+
:target: ../../actions
33

44
.. image:: https://codecov.io/gh/ApamaCommunity/apama-log-analyzer/branch/master/graph/badge.svg
55
:target: https://codecov.io/gh/ApamaCommunity/apama-log-analyzer
@@ -34,7 +34,7 @@ Features:
3434

3535
This can be very useful for debugging slow receiver disconnections.
3636

37-
- Supported Apama releases: **Apama 4.3 through to latest** (10.5+). Also works with correlator logging from `Apama-ctrl`, downloaded from **Cumulocity**.
37+
- Supported Apama releases: **Apama 4.3 through to latest** (10.11+). Also works with correlator logging from `Apama-ctrl`, downloaded from **Cumulocity**.
3838

3939
- Licensed under the **Apache License 2.0**.
4040

@@ -195,26 +195,26 @@ Importing CSVs in a non-English locale (e.g. Germany)
195195
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
196196
Unfortunately the CSV file format (and Excel in particular) has fairly poor support for use in locales such as German that have different decimal, thousand and date formats to the US/UK format generated by this tool. It is therefore necessary to explicitly tell Excel how to interpret the numeric CSV columns. In Excel 365, the steps are:
197197

198-
#. Open Excel (it should be displaying an empty spreadsheet; don't open the CSV file yet).
199-
#. On the **Data** tab click **From Text/CSV** and select the CSV file to be imported.
200-
#. Ensure the **Delimiter** is set to **Comma**, then click **Edit**.
201-
#. On the **Home** tab of the Power Query Editor dialog, click **Use First Row as Headers**.
202-
#. Select all columns that contain numbers. To do this click the heading for ``epoch secs``, scroll right until you see ``# metadata:`` then hold down **SHIFT** and click the column before ``# metadata:``.
203-
#. (Optional: if you plan to use any values containing non-numeric data (e.g. slowest consumer or context name) then deselect those columns by holding down **CTRL** while clicking them; otherwise non-numeric values will show up as _Error_ or blank).
204-
#. Right-click the selected column headings, and choose **Change Type > Using Locale...**.
205-
#. Set the Data Type to **Decimal Number** and Locale to **English (Australia)** (or United States; any English locale should be fine), then click **OK**.
206-
#. On the **Home** tab click **Close & Load**.
198+
#. Open Excel (it should be displaying an empty spreadsheet; don't open the CSV file yet).
199+
#. On the **Data** tab click **From Text/CSV** and select the CSV file to be imported.
200+
#. Ensure the **Delimiter** is set to **Comma**, then click **Edit**.
201+
#. On the **Home** tab of the Power Query Editor dialog, click **Use First Row as Headers**.
202+
#. Select all columns that contain numbers. To do this click the heading for ``epoch secs``, scroll right until you see ``# metadata:`` then hold down **SHIFT** and click the column before ``# metadata:``.
203+
#. (Optional: if you plan to use any values containing non-numeric data (e.g. slowest consumer or context name) then deselect those columns by holding down **CTRL** while clicking them; otherwise non-numeric values will show up as _Error_ or blank).
204+
#. Right-click the selected column headings, and choose **Change Type > Using Locale...**.
205+
#. Set the Data Type to **Decimal Number** and Locale to **English (Australia)** (or United States; any English locale should be fine), then click **OK**.
206+
#. On the **Home** tab click **Close & Load**.
207207

208208
Resources
209209
=========
210210

211211
From the Apama documentation:
212212

213-
- `List of Correlator Status Statistics <http://www.apamacommunity.com/documents/10.5.0.2/apama_10.5.0.2_webhelp/apama-webhelp/index.html#page/apama-webhelp%2Fre-DepAndManApaApp_list_of_correlator_status_statistics.html>`_ - for understanding the meaning of the statistics available
213+
- `List of Correlator Status Statistics <https://www.apamacommunity.com/documents/10.11.1.1/apama_10.11.1.1_webhelp/apama-webhelp/#page/apama-onlinehelp%2Fre-DepAndManApaApp_list_of_correlator_status_statistics.html>`_ - for understanding the meaning of the statistics available
214214

215-
- `Inspecting correlator state <http://www.apamacommunity.com/documents/10.5.0.2/apama_10.5.0.2_webhelp/apama-webhelp/index.html#page/apama-webhelp%2Fre-DepAndManApaApp_inspecting_correlator_state.html%23>`_ - for using the engine_inspect tool to get detailed information on the number of monitor instances, listeners, etc, which can help to identify application memory leaks
215+
- `Inspecting correlator state <https://www.apamacommunity.com/documents/10.11.1.1/apama_10.11.1.1_webhelp/apama-webhelp/#page/apama-onlinehelp%2Fre-DepAndManApaApp_inspecting_correlator_state.html>`_ - for using the engine_inspect tool to get detailed information on the number of monitor instances, listeners, etc, which can help to identify application memory leaks
216216

217-
- `Shutting down and managing components <http://www.apamacommunity.com/documents/10.5.0.2/apama_10.5.0.2_webhelp/apama-webhelp/index.html#page/apama-webhelp/re-DepAndManApaApp_shutting_down_and_managing_components.html>`_ and its child topics - contain information on using `dorequest` to get detailed memory/CPU profiles, a string representation of the correlator queues, and various enhanced logging options
217+
- `Shutting down and managing components <https://www.apamacommunity.com/documents/10.11.1.1/apama_10.11.1.1_webhelp/apama-webhelp/#page/apama-onlinehelp%2Fre-DepAndManApaApp_shutting_down_and_managing_components.html>`_ and its child topics - contain information on using `dorequest` to get detailed memory/CPU profiles, a string representation of the correlator queues, and various enhanced logging options
218218

219219

220220

‎tests/correctness/Cor_008/run.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,3 +33,5 @@ def validate(self):
3333
line = 2
3434
self.assertEval("{value} == 0", value=s[line]['warns'], valueName=f'warns for {log}.log line {line}')
3535
self.assertEval("{value} == 1", value=s[line]['errors'], valueName=f'errors for {log}.log line {line}')
36+
37+
self.assertThatGrep('loganalyzer_output/logged_errors.txt', ' 3x (.*)', expected='from Tue 2019-07-30 18:04:32 to 18:04:32 (=0:00:00) in correlator-warns-and-errors')

‎tests/pysysproject.xml

Lines changed: 31 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -1,41 +1,44 @@
1-
<?xml version="1.0" standalone="yes"?>
1+
<?xml version="1.0" encoding="utf-8"?>
22
<pysysproject>
33
<requires-pysys>2.0</requires-pysys>
44
<requires-python>3.6</requires-python>
55

6-
<property root="testRootDir"/>
7-
<property environment="env"/>
8-
<property osfamily="osfamily"/>
9-
106
<!-- User-defined properties -->
117
<property name="verboseWaitForSignal" value="true"/>
12-
<property name="logAnalyzerScript" value="${testRootDir}/../apamax/log_analyzer.py"/>
13-
14-
<!-- Standard default settings. See sample project file and API docs for more details. -->
15-
<property name="defaultAbortOnError" value="true"/>
16-
<property name="defaultIgnoreExitStatus" value="false"/>
17-
<property name="defaultEnvironsTempDir" value="self.output"/>
18-
<property name="defaultAssertDiffStripWhitespace" value="false"/>
19-
<property name="redirectPrintToLogger" value="false"/>
20-
<property name="supportMultipleModesPerRun" value="true"/>
21-
22-
<property name="pythonCoverageDir" value="coverage-python-@OUTDIR@"/>
23-
<collect-test-output pattern=".coverage*" outputDir="${pythonCoverageDir}" outputPattern="@FILENAME@_@TESTID@_@UNIQUE@"/>
8+
9+
<property name="appHome" value="${env.PYSYS_APP_HOME}" default="${testRootDir}/.." pathMustExist="true"/>
10+
<property name="logAnalyzerScript" value="${appHome}/apamax/log_analyzer.py"/>
2411

2512
<!-- Custom test framework extensions, if needed -->
26-
<path value="framework" relative="true" />
13+
<pythonpath value="framework"/>
2714

2815
<performance-reporter classname="correlatorloganalyzer.customperfreporter.CustomPerfReporter"
2916
summaryfile="${testRootDir}/performance/performance_output/@OUTDIR@/perf_@HOSTNAME@_${startTime}.csv"
3017
/>
3118

3219
<writers>
33-
<writer classname="TravisCIWriter" module="pysys.writer.ci"></writer>
20+
<writer classname="pysys.writer.testoutput.TestOutputArchiveWriter">
21+
<property name="destDir" value="__pysys_output_archives/"/>
22+
<property name="maxTotalSizeMB" value="1024.0"/>
3423

35-
<writer classname="JUnitXMLResultsWriter" module="pysys.writer">
36-
<property name="outputDir" value="${testRootDir}/pysys-junit-report"/>
24+
<!-- Don't include any non-failure outcomes as REQUIRES INSPECTION generates a very large output which
25+
could use our GitHub Actions quota quickly -->
26+
<property name="includeNonFailureOutcomes" value=""/>
27+
</writer>
28+
29+
<writer classname="pysys.writer.outcomes.JUnitXMLResultsWriter">
30+
<property name="outputDir" value="__pysys_junit_xml"/>
3731
</writer>
3832

33+
<!-- Code coverage for .py files -->
34+
<writer classname="pysys.writer.coverage.PythonCoverageWriter">
35+
<property name="destDir" value="__coverage_python.${outDirName}"/>
36+
<property name="pythonCoverageArgs" value=""/>
37+
</writer>
38+
39+
<writer classname="pysys.writer.ci.GitHubActionsCIWriter"></writer>
40+
<writer classname="pysys.writer.ci.TravisCIWriter"></writer>
41+
3942
<writer classname="ConsoleSummaryResultsWriter" module="pysys.writer">
4043
<property name="showOutputDir" value="true"/>
4144
<property name="showOutcomeReason" value="true"/>
@@ -50,13 +53,12 @@
5053
<default-file-encoding pattern="*.yaml" encoding="utf-8"/>
5154
</default-file-encodings>
5255

53-
<formatters>
54-
<formatter classname="ColorLogFormatter" module="pysys.utils.logutils"
55-
name="stdout" messagefmt="%(asctime)s %(levelname)-5s %(message)s" datefmt="%H:%M:%S">
56-
</formatter>
57-
58-
<formatter classname="BaseLogFormatter" module="pysys.utils.logutils"
59-
name="runlog" messagefmt="%(asctime)s %(levelname)-5s %(message)s" datefmt=""/>
60-
</formatters>
56+
<pysysdirconfig>
57+
58+
</pysysdirconfig>
59+
60+
<!-- Add project-specific text to be appended to the "pysys run -h". -->
61+
<project-help>
62+
</project-help>
6163

6264
</pysysproject>

0 commit comments

Comments
 (0)