Skip to content

Commit 6cb5baa

Browse files
committed
updated README.md
1 parent 2568003 commit 6cb5baa

File tree

2 files changed

+60
-140
lines changed

2 files changed

+60
-140
lines changed

README.md

Lines changed: 58 additions & 138 deletions
Original file line numberDiff line numberDiff line change
@@ -5,81 +5,66 @@
55

66
# Simple Example for C/C++ Tests with Continuous Integration (CI)
77

8-
This repo covers setting up a basic testing suite with github badges for a C/C++ library. Its not meant to be deep tutorial on testing but just cover some basics of setting up unit tests, coverage tests, and continuous integration (in this case using either GitHub Actions or Travis-CI). The repo doesn't have a lot of code - there is a simple library which is tested for coverage and integration.
8+
This repository demonstrates setting up a basic testing suite with GitHub badges for a C/C++ library. It covers the basics of setting up unit tests, coverage tests, and continuous integration using GitHub Actions or Travis-CI. The repository contains a simple library tested for coverage and integration.
99

1010
## Motivation
1111

12-
I just wanted to make a small standalone test project to show continuous integration tools and workflow for C (or C++) language testing.
12+
This project provides a small standalone example to show continuous integration tools and workflows for C (or C++) language testing without the overhead of a much larger project.
1313

1414
## Features
1515

16-
The lib.h / lib.c files are broken out as examples of testing an embedded library. Most of the projects I work on are for embedded systems so I wanted a way to get a build badge for these embedded projects. Since many of those compilers and environments are not on Linux I wanted just a simple abstraction of how the CI process works without all the complexities of a "real" project.
16+
The lib.h and lib.c files are examples of testing an embedded library. Many of the projects I work on are for embedded systems, so I wanted a way to get a build badge for these projects. This example abstracts how the CI process works without the complexities of a "real" project.
1717

18-
## How it works
18+
## How It Works
1919

20-
In this demo project there is a C library (could also be C++ etc). The library code is just a few demo functions which are in the lib.h and lib.c files. They don't really do anything but allow for simple abstraction of what is necessary to build a larger project. This repo and project are meant to provide a quick overview of testing and how to get build badges working on Github.
20+
This demo project includes a C library with a few demo functions in lib.h and lib.c. The code doesn't do much but serves as a simple abstraction of what is necessary to build a larger project. This repository provides an overview of testing and how to get build badges working on GitHub.
2121

22-
### Quick Overview of Testing
22+
## Quick Overview of Testing
2323

24-
There are many different phases of testing. Here are a few areas but phrased as questions.
24+
Testing helps tell us if the software is working as intended. This means understanding what we want to test in to a few key areas.
2525

26-
Common Testing "Questions" about a project:
27-
* Does it run as intended? (is it functionally correct / does it do what its supposed to do?)
28-
* Does it have side effects when running? (are resources tied up such as ports blocked, thread contention? Are other programs or services affected unintentionally?)
29-
* Are all the possible permutations of execution tested? (code coverage, Is every piece of code - every if-then statement etc tested?)
30-
* How much memory or resources are used? (is memmory efficiently used? Is memory freed correctly after use? When the program is complete does it leave things intact?)
31-
* Does it exit gracefully? (are any resources requested released before the code exits?)
26+
### Common Testing Questions
3227

28+
* Does it run as intended?
29+
* Does it have side effects when running?
30+
** Are resources tied up such as ports blocked, thread contention?
31+
** Are other programs or services affected unintentionally?
32+
* Are all possible execution paths tested? (coverage)
33+
* How much memory or resources are used? Is memory freed correctly / are their leaks?
34+
* Does it exit gracefully?
3335

34-
### Unit Testing
36+
### Unit Testing
3537

36-
Unit Testing is a practice of writting small tests to see that piece of code, typically a full module or library, passes a set of tests to make sure it runs as intended. The simple unit tests are done after writing function. We then make a small program (the Unit test program) which calls our new function with as many different example parameters as we think are appropriate to make sure it works correctly. If the results returned match the expected results we can say the function passes the test. If the results for a given set of parameters don't agree we call an assertion (usually via a special ASSERT type macro) which records the failure and attempts to keep running then test in our test program. The goal is to be able to craft a set of these tests which test all the possible paths of execution in our code and passes all the test.
38+
Unit Testing involves writing small tests to ensure that a piece of code, typically a module or library, passes a set of tests to verify it runs as intended. The goal is to craft tests that cover all possible paths of execution in the code.
3739

38-
Note that its not the goal to create a test that passes every possible permutation of the input parameters - as this could be an impossibly large number or variations even for just a few parameters. This idea of testing all the possible paths of exeuction is called code coverage. Testing code coverage is done with tools which see if the test program has successfully "challenged" the target library code by examing whether each execution path (or line of code) has been run.
40+
Example function:
3941

40-
For example if there is a function like this:
41-
42-
```C
43-
int add5ifGreaterThan2 (int a) {
44-
int r;
45-
46-
if (a>2)
47-
r = a + 5;
48-
else
49-
r = a;
50-
51-
return r;
42+
```c
43+
int add5ifGreaterThan2(int a) {
44+
int r;
45+
if (a > 2)
46+
r = a + 5;
47+
else
48+
r = a;
49+
return r;
5250
}
5351
```
5452
55-
Our test program for add5ifGreaterThan2() needs to supply values of a that are both less and great than 2 so both paths of the if statement
56-
```C
57-
if (a<2)
58-
```
59-
60-
are tested.
61-
62-
We do this with test code such as this:
63-
64-
```C
65-
//code in test program ...
66-
ASSERT (add5ifGreaterThan2(1) == 1) // supplies value of 'a' that tests the if (a<2) case and tests for a result
67-
ASSERT (add5ifGreaterThan2(3) == 8) // supplies value of 'a' that tests the if (a>2) case and tests for a result
53+
Test code:
6854
55+
```c
56+
ASSERT(add5ifGreaterThan2(1) == 1) // tests if (a <= 2) case
57+
ASSERT(add5ifGreaterThan2(3) == 8) // tests if (a > 2) case
6958
```
7059

71-
Of course this example is very simple but it gives a general idea of how the parameters need to be chosen to make sure both sides of the if clause in the example are run by the test program. The ASSERT macro checks whether the result is logically true. If it is not then it triggers an error process. Depending on the testing framework used different types of logging and output can be examined if the statement fails.
72-
73-
74-
#### More info
60+
#### More Info
7561

7662
Here is a link to the wikipedia article for more depth on unit testing practice and history: [Unit_testing](https://en.wikipedia.org/wiki/Unit_testing).
7763

78-
### Frameworks
79-
To make Unit Testing easier to automate, unit testing frameworks have been written to help test results from function calls, gather statistics about passing/failing test cases, and
8064

81-
Unit testing frameworks are available in most languages and many have names like JUnit (for Java) or (PyUnit for Python etc). In C/C++ there are many excellent frameworks available many of which are open source. Here we'll be using the well regard googletest (an open source unit test framework). A few other well known C++ testing frameworks are shown in the list below. Many provide features well beyond unit testing and some work in special enviroments such as small microcontrollers or embedded systems.
65+
### Testing Frameworks
8266

67+
Unit testing frameworks make it easier to automate testing. Many languages have frameworks named like JUnit (for Java) or PyUnit (for Python). In C/C++, several frameworks are available, including:
8368

8469
* [CppUTest](http://cpputest.github.io/) - A unit test framework for C++
8570
* [Boost C++ Unit Test Framework](http://www.boost.org/doc/libs/1_35_0/libs/test/doc/components/utf/index.html) - This is the test framework used by the popular Boost library for C++
@@ -88,74 +73,44 @@ Unit testing frameworks are available in most languages and many have names like
8873
* [Google Test](https://github.com/google/googletest) - we'll be using this here. Google Test is a free opensource framework using special macros for asserting when code doesn't perform as expected.
8974
* [Catch](https://github.com/philsquared/Catch) - Catch is a test framework for C
9075

91-
92-
### Installing Google Test
93-
We'll be using Google Test (called gtest) here so first we need to install it.
76+
We'll be using Google Test (gtest) in this example.
9477

9578
Here is the link to the project source
9679
[Google Test](https://github.com/google/googletest)
9780

98-
Examples here are built using Ubuntu Linux, but should apply to most other operating systems.
99-
100-
On Ubuntu Linux you can install gtest using this command. If you are developing on another sytem refer to the documentation link for install procedures. Other than installing, all of the commands and test procedures we'll be using later will be the same (whether Windows / MacOS / POSIX / Linux).
81+
### Installing Google Test
10182

83+
On Ubuntu Linux, install gtest using the following commands:
10284

10385
```bash
10486
sudo apt-get install libgtest-dev
105-
106-
sudo apt-get install cmake # install cmake
107-
87+
sudo apt-get install cmake
10888
cd /usr/src/gtest
109-
11089
sudo cmake CMakeLists.txt
111-
11290
sudo make
113-
11491
sudo cp *.a /usr/lib
115-
11692
sudo mkdir /usr/local/lib/googletest
117-
11893
sudo ln -s /usr/lib/libgtest.a /usr/local/lib/gtest/libgtest.a
119-
12094
sudo ln -s /usr/lib/libgtest_main.a /usr/local/lib/gtest/libgtest_main.a
121-
12295
```
12396

12497
You can read more about the Google Test project here: [Testing Primer](https://github.com/google/googletest/blob/master/googletest/docs/Primer.md)
12598

99+
### Testing vs Continuous Integration
126100

127-
===========================
128-
129-
130-
## Features
131-
132-
The lib.h / lib.c files are broken out as examples of testing an embedded library. Most of the projects I work on are for embedded systems so I wanted a way to get a build badge for these embedded projects. Since many of those compilers and environments are not on Linux I wanted just a simple abstraction of how the Travis build or Github Actions works, but without all the complexities of a "real" project.
133-
134-
## Testing vs Continuous Integration
135-
136-
In this demo project there is a C library (could also be C++ etc). The library code is just a few demo functions which are in the lib.h and lib.c files. They don't really do anything but allow for simple abstraction of what is necessary to build a larger project.
137-
138-
Once you've made unit tests, and gotten your code to run using the local test suite the next step starts. How does an *outsider* know your code passes tests? This is where continuous integration (CI) starts. CI uses services (such as Travis-CI, Github Actions, Circle-CI, Jenkins and many others) to automatically run your test suites and then report the result. When a CI program runs your test suite it can be configured to accept or reject your code based on the tests passing. This in turn can be used to automatically deploy your code. This is called Continuous Deployment (CD) or pipelines. CD and pipelines are beyond the scope of this repo and example.
139-
140-
## Using Travis-CI as an example of build-badge and CI
141-
142-
### Travis-CI
143-
144-
Travis-CI looks in the .travis.yml (note that is dot travis dot yml) to see how to run the code. In this case it first calls make which compiles lib.c and example.c in to lib.o and example.o and then links them to produce the final executable called example.out.
101+
Once you've written unit tests and run them locally, CI services (such as Travis-CI, GitHub Actions, Circle-CI, Jenkins, and others) can automatically run your test suites and report the results **every time you check in**. CI can be configured to accept or reject your code based on the tests passing, and can even deploy your code automatically if it passes all the tests. This is called Continuous Deployment, or CD.
145102

146-
If you look inside the file example.c you will see there are few hand written test cases. They are not meant to be a complete example of how to write test cases just a simple view of how the tests will be run in general. The main() function calls local function run_tests() which in turn calls each individual test case. Rather than link in a larger test case environment such as cppUnit etc there is a trivial set of test functions, one for each function in the lib.c library. If run_tests() is able to run all the tests successfully it will return to main() with a value of S_OK else it will return some failure code. This value is then returned from the main() program in example.out on exit.
103+
### Using Travis-CI as a CI Provider
104+
Travis-CI
105+
Travis-CI looks in the .travis.yml file to see how to run the code. It compiles lib.c and example.c into lib.o and example.o, and then links them to produce the executable example.out. The test suite is run, and the exit code is used to determine if the build passes.
147106

148-
Travis-CI then runs the example.out and looks for the exit code from the main() function. Being a Posix style of system an exit code of zero from example.out is considered passing and hence Travis-ci will then declare the build passing. If a non zero value is returned travis will declare the build failing. So to sum up, the primary means for travis knowing whether the test suite passes is getting the proper exit code from the test suite executable which in our case here is running example.out.
107+
### Using Github Actions as CI Provider
149108

150-
Github Actions works in similar way. When the code is pushed to github (main branch in our case), github triggers and action to look at the file here:
109+
GitHub Actions work similarly. When code is pushed to the main branch, GitHub triggers an action to look at the ./github/workflows/ci.yml file, which specifies the environment, builds the code, runs the test script, and reports success or failure.
151110

152-
```bash
153-
./github/workflows/ci.yml
154-
```
155-
This very similar to the .travis.yml. It tells the github actions runner to load the correct environment, build the code, run the test script, and then report back success or failure. When github has completed running the ci.yml tasks it reports sucess or failure as shown in the badge.
111+
### Code Coverage
156112

157-
## Code Coverage
158-
Code coverage is achieved using gcov from the gcc test suite. The example.out test program is compiled with the flags -ftest-coverage -fprofile-arcs. To see the code coverage run gcov:
113+
Code coverage is achieved using gcov from the gcc test suite. To see the code coverage:
159114

160115
```bash
161116
make clean
@@ -164,59 +119,24 @@ make
164119
gcov lib.c
165120
```
166121

167-
which will generate the file
168-
169-
```bash
170-
lib.c.gcov
171-
```
172-
173-
This can be viewed in any text editor.
122+
This generates the file lib.c.gcov, which can be viewed in any text editor. Lines with #### have never been run.
174123

175-
Lines that appear with #### have never been run.
124+
## FAQ
176125

126+
Q: What's the point of this repo if it doesn't do anything?
127+
A: It's a simple example of using Travis-CI for testing purposes.
177128

129+
Q: How can I make it fail for testing purposes?
130+
A: Modify a line in main.c to make it fail, or use other methods to test failure cases.
178131

179-
### FAQ:
132+
Q: Why isn't there a proper unit test framework?
133+
A: This is a barebones test of integration and badge service.
180134

181-
Q: What's the point this repo doesn't do anything.
182-
A: Exactly! Its just a simple example of the travis service for test purposes. I was looking for a simple repo that just had this basic stuff but was having trouble finding one.
183-
184-
Q: I see the badge says passing can I make fail for simple purposes?
185-
A: Just clone it modify a commented line in the main.c to make it fail. Of course there are other ways to make it fail too but this is just for test purposes.
186-
187-
Q: Why isn't there a proper unit test framework?
188-
A: I just wanted a barebones test of the integration and badge service.
189-
190-
191-
## License
135+
## License
192136

193137
(OSI Approved BSD 2-clause)
194138

195-
All rights reserved.
196-
197-
Redistribution and use in source and binary forms, with or without
198-
modification, are permitted provided that the following conditions are met:
199-
200-
* Redistributions of source code must retain the above copyright notice, this
201-
list of conditions and the following disclaimer.
202-
203-
* Redistributions in binary form must reproduce the above copyright notice,
204-
this list of conditions and the following disclaimer in the documentation
205-
and/or other materials provided with the distribution.
206-
207-
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
208-
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
209-
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
210-
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
211-
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
212-
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
213-
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
214-
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
215-
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
216-
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
217-
218-
219-
220-
copyright (C) <2016 and onward> <M. A. Chatterjee> <deftio [at] deftio [dot] com>
221-
version 1.0.3 (added github actions support)
139+
version 1.0.3 (added GitHub Actions support)
222140
version 1.0.2 (updated for travis-ci.com transition) M. A. Chatterjee
141+
142+
(c) 2016 and later M. A. Chatterjee <deftio [at] deftio [dot] com>

test-library.c

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
11
/*
2-
@main.c - simple test file example file for travis-ci example library
2+
@main.c - simple test file example file for continuous integration example library
33
44
@copy Copyright (C) <2012> <M. A. Chatterjee>
55
@author M A Chatterjee <deftio [at] deftio [dot] com>
66
77
This file contains header defintions for travis-ci code testing example.
88
99
@license:
10-
Copyright (c) 2011-2016, M. A. Chatterjee <deftio at deftio dot com>
10+
Copyright (c) 2016 and later, M. A. Chatterjee <deftio at deftio dot com>
1111
All rights reserved.
1212
1313
Redistribution and use in source and binary forms, with or without

0 commit comments

Comments
 (0)