Overhaul test system

[charlesnicholson]

Rework the testing system to compile 100% of the tests against the 2^N flag matrix as well as C and C++ axes.

[charlesnicholson]

From #316 (comment)

===

I don't have an appetite for removing C++ from the tests entirely; a great deal of embedded systems code is written in C++ these days, and many users define NANOPRINTF_IMPLEMENTATION and include the header file in c++ source files. Nanoprintf must work in C++, and its tests must pass in C++! (I try to avoid C++ in embedded systems; it is the most complicated programming language ever designed by humans. Lots of folks like C++ firmware, and it provides conveniences over C in host-side code, so I support it..)

That said, you point out a very valid issue- nanoprintf has weak testing for pure C. I have had #132 open for quite as while, as you have noticed :)

I would happily accept a PR that hits the following notes:

Get rid of doctest entirely.
Replace / rewrite the tests in a pure C test driver that can be compiled as C++ as well.
Can do the (2^N) * 2 (one for C, one for C++) exhaustive test compilation and execution, so that every test can be run in both C and C++.
I would love to change the CMake setup such that instead of today's approach of creating 2^N binaries (and invoking the linker that many times), it stamps out (2^N) * 2 files that all get compiled and linked into the same executable, and all run in one invocation. Doctest gives you that automatically via some static registration magic; it "finds" all of the tests via what I'm assuming is the standard C++ "define a class at file scope whose constructors register the tests with a central runner" technique. Perhaps the equivalent in this "C and C++" world is to also simply generate a file that explicitly calls x_y_z_tests or something for every x, y, z etc in the 2^N configuration. Perhaps there are other ways as well, but the heart of the approach, I think, needs to be along the lines of "generate a file that contains the 2^N set of NANOPRINTF_ define values currently being iterated over, that also includes a common .inl file that defines all of the tests" for the axes of [(C x C++), (All 2^N flag combinations)]. The executable would be large, containing a LOT of static copies of nanoprintf compiled with the _STATIC flag enabled, but I don't think that's a big deal even if the executable hits a ridiculous 64MB; it's just host-side testing.

This is a medium-sized project but I think it's worth doing; it's got a ton of wins:

It gives users and implementors 100% confidence that new code works across all languages and flags
It speeds up the build - only linking once is much better than linking 2^N * 2 times
It speeds up the tests - running one optimized mega-test file will be faster than launching 2^N * 2 test processes, and leaves the door open for multithreading the tests if it ever matters.

[stefano-zanotti-88]

Our discussion in #312 (the strange numbers I get for the code size) got me thinking: should we add yet another dimension to the testing, by compiling as O3 as well as Os?
Certain bugs -- especially those related to the compiler exploiting UB in the code -- only surface at certain optimization levels.

[stefano-zanotti-88]

About the generation of the 2^N tests. I was thinking about generating 2^N files, each of which defines its own combination of NPF options, and includes nanoprintf.h.
It also defines all the test cases (let's say we start with the conformance tests, then we'll see how to handle mpaland etc), and the test cases call npf_vsnprintf directly.
The test cases are called inside a single test function, and the test function of all the 2^N files have a unique name, so that they can be called by a central main function, which would be the global test runner.

However, I also thought of two other things.
One, the tests need not be duplicated; each of the 2^N files also defines a uniquely-named wrapper for npf_vsnprintf, and the main function calls the test 2^N times, passing the appropriate npf_vsnprintf function as a function pointer. The test will be marginally slower since it must go through the wrapper, but this way we do not have 2^N copies of the test itself, which is bound to be the largest culprit for size growth, due to all the constant strings and arguments that make up the test cases. Such a test function should receive not just the function pointer, but also a struct indicating what it supports, so that it can skip the execution of non-applicable tests, or change the oracle string accordingly.

Secondly, compiling the 2^N files is slower than compiling all of them together in a "jumbo/unity build" (actually, the best thing would be to split the 2^N files into M translation units, with M equal to the number of threads used for the compilation). To do this, however, we would need to have multiple instances of NPF in the same translation unit, which means that:

1. all the macros must be cleaned up at the end of the file (or, we must disable warnings about redefined macros)
2. all npf functions, including the private ones, must be wrapped in a macro that allows each instance to have a different prefix (or other form of name mangling).
   See how it is done here:
   https://github.com/nothings/stb/blob/802cd454f25469d3123e678af41364153c132c2a/stb_sprintf.h#L208-L214
   In this way, all the instances can coexist in the same translation unit, and the main function can call each one of them by their unique name (which also removes one level of indirection introduced in the discussion above).