Baillehache Pascal's personal website

CUTest, easy unit tests in C

I regularly advocate for the importance of unit tests (and all tests in general). LibCapy of course has its own unit tests, but I'm currently not using a particular tool or framework for that. Consequently even if they deserve their purpose, they are dirty and need to be improved and cleaned up to bring LibCapy nearer to an hypothetical version 1.0. A first step was to create a tool (you weren't expecting I would not do it myself, right?) to simplify their implementation and execution, which I'm introducing and sharing in this article.

To improve over my current unit tests implementation, all I needed for now was a tool that would allow it to be shorten to the definition of unit tests function. The execution would be automated, as well as a nice and clean output. Here again I've used C macros to accomplish it in a super easy and straightforward way. It's done in less than a hundred lines and a short afternoon (incuding this article). It includes one unit test file and executes the functions defined in it. The only fancy trick consists of redefining macros so that the single definition of the user can be reused differently several time, and hide code behind some if(0) when need: once as it is to define the unit test functions, once to count the number unit test functions before executing them, and once to call them without their body interfering. The rest is just producing clean output for the user.

It's used as follow. Gather your unit tests into one or several files where each test function is defined with the macro CUTEST(testName, "Test description") { ... }. Ah, CUTest, that's the name of the tool, because "C Unit Test", and that's cute ! testName is the name of the unit test function, Test description is the descrition displayed the execution. The body of the function is the definition of the unit test. If you want, you can add other functions (for your fixtures typically), as long as you include-guard them. Only functions that are defined with CUTEST() will be considered as unit tests. You will almost certainly also need to include the module header of what you're testing. Inside the unit test body, the test condition is performed using another macro: CUTEST_ASSERT(cond, msg, msg_arg1, msg_arg2, ...). cond is the condition to be satisfied for the unit test to pass (you may have several one per test if you want). msg, msg_arg1, ... are the arguments passed to a 'printf' call in case the condition is not satisfy with the view to explain why it wasn't.

An example unit test file would look like this:

#ifndef FIXTURE
#define FIXTURE
void foo(int* a) {
  *a *= *a;
}
#endif

CUTEST(test001, "First unit test") {
  CUTEST_ASSERT(1 == 1, "ok");
}

CUTEST(test002, "Second unit test") {
  int a = 2;
  foo(&a);
  CUTEST_ASSERT(a == 4, "a = %d", a);
}

CUTEST(test003, "Third unit test") {
  CUTEST_ASSERT(1 == 0, "failing unit test");
}

That's all for the unit tests file creation. The execution is illustrated by the Makefile below, given that you've put all your unit test files in the UnitTests folder and they're named test_XXX.c. You'll probably need to modify the compilation command with compilation argument needed by your project.

test:
  for file in `ls UnitTests/test_*.c`; do
  gcc -o cutest /path/to/cutest.c -DCUTEST_FILE=$$file && cutest; \rm -f cutest; done

This will execute all the unit test functions in all the unit test files and print a summary looking as follow:

The code of CUTest is as follow (you can download the source by clicking here):

(Edited on 2022/10/11 to improve the output when there is several assert in the same test)

(Edited on 2022/11/22 to display the currently running test's label without waiting its completion)

(Edited on 2022/11/23 to correct bug when splitting long labels)

(Edited on 2023/12/10 to add the command line option for executing only one single test)

(Edited on 2023/12/12 to refactor 'unsigned int' into 'size_t')

(Edited on 2024/01/25 to correct a formatting bug when a label has a size such as it ends exactly at the end of the line)

/*
    CUTest - A unit test runner for C
    Copyright (C) 2022 Pascal Baillehache baillehache.pascal@gmail.com
    https://baillehachepascal.dev
    This program is free software: you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
    the Free Software Foundation, either version 3 of the License, or
    (at your option) any later version.
    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
    GNU General Public License for more details.
    You should have received a copy of the GNU General Public License
    along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <stdbool.h>
#include <stddef.h>
#include <time.h>

// Conversion from macro to string
#define CUTEST_TO_STR_(x) #x
#define CUTEST_TO_STR(x) CUTEST_TO_STR_(x)

// Escape sequence to colorize the output
#define CUTEST_START_RED "\033[38;2;255;0;0m"
#define CUTEST_END_RED "\033[39m"
#define CUTEST_START_GREEN "\033[38;2;0;255;0m"
#define CUTEST_END_GREEN "\033[39m"
#define CUTEST_START_BOLD "\033[1m"
#define CUTEST_END_BOLD "\033[0m"

// Print current date and time
#define CUTEST_TIME \
  { time_t now = time(NULL); printf("%.19s\n", asctime(localtime(&now))); }

// Variables to memorise the number of tests and failed test
size_t cutestIdxTest;
size_t cutestNbTest;
size_t cutestNbTestFailed;
bool cutestTestSuccess;

// Wrapper for the test condition in the unit tests. To be used as follow:
// CUTEST_ASSERT(cond, msg, msg_arg1, msg_arg2, ...)
// where 'cond' is the condition to be respected by the test, and 'msg', etc...
// are the arguments passed to a 'printf' call in case the condition
// is not satisfy with the view to explain why the condition wasn't satisfied.
#define CUTEST_ASSERT(cond, ...)                                      \
  if(!(cond)) {                                                       \
    if(cutestTestSuccess)                                             \
      printf(CUTEST_START_RED "NG" CUTEST_END_RED "\n");              \
    printf(                                                           \
      CUTEST_START_RED " -- Unsatisfied condition on line %u:\n%s\n", \
      __LINE__, CUTEST_TO_STR(cond));                                 \
    printf("While:\n");                                               \
    printf(__VA_ARGS__);                                              \
    printf(CUTEST_END_RED "\n");                                      \
    cutestTestSuccess = false;                                        \
  }

// Print the OK result for a successful test and increase the counter of
// failed test for an unsuccessful test
#define CUTEST_OK                                                \
  if(cutestTestSuccess) {                                        \
    printf(CUTEST_START_GREEN "OK" CUTEST_END_GREEN "\n");       \
  } else {                                                       \
    ++cutestNbTestFailed;                                        \
  }

// Unit tests declarations macro. To be used in the unit tests file as follow:
// CUTEST(test002, "Second unit test") {
//   int a = 2;
//   a *= a;
//   CUTEST_ASSERT(a == 4, "a equals to %d", a);
// }
#define CUTEST(name, lbl) void name(void)
#include CUTEST_TO_STR(CUTEST_FILE)

// Helper function to print and pad the label of the executed tests
void CutestPrintLbl(char const* const lbl) {
  size_t length = strlen(lbl);
  size_t lengthMax = 80;
  printf("%03lu/%03lu ", cutestIdxTest, cutestNbTest);
  size_t i = 0;
  for(; i < length; ++i) {
    if(i > 0 && (i % lengthMax) == 0) printf("\n        ");
    printf("%c", lbl[i]);
  }
  i %= lengthMax;
  if(i != 0) for(; i < lengthMax; ++i) printf(".");
  printf(" ");
  fflush(stdout);
}

// Main function
int main(int argc, char** argv) {

  // Variable to memorise the unit test to be executed
  // (if null, all are executed)
  char* nameTestToExecute = NULL;
  for(int iArg = 0; iArg < argc; ++iArg) {
    if(strcmp(argv[iArg], "-t") == 0 && (iArg + 1) < argc)
      nameTestToExecute = argv[iArg + 1];
  }

  // Print the unit test file name and start time
  printf(
    CUTEST_START_BOLD "\n  ==== %s ====" CUTEST_END_BOLD "\n",
    CUTEST_TO_STR(CUTEST_FILE));
  CUTEST_TIME

  // Init the counters
  cutestIdxTest = 0;
  cutestNbTest = 0;
  cutestNbTestFailed = 0;

  // Calculate the number of test
  #undef CUTEST
  #define CUTEST(name, lbl)                               \
    if(                                                   \
      nameTestToExecute == NULL ||                        \
      strcmp(nameTestToExecute, CUTEST_TO_STR(name)) == 0 \
    ) {                                                   \
      ++cutestNbTest;                                     \
    } if(0)
  #include CUTEST_TO_STR(CUTEST_FILE)

  // Unit tests call
  #undef CUTEST
  #define CUTEST(name, lbl)                               \
    if(                                                   \
      nameTestToExecute == NULL ||                        \
      strcmp(nameTestToExecute, CUTEST_TO_STR(name)) == 0 \
    ) {                                                   \
      ++cutestIdxTest; cutestTestSuccess = true;          \
      CutestPrintLbl(lbl); name(); CUTEST_OK              \
    }                                                     \
    if(0)
  #include CUTEST_TO_STR(CUTEST_FILE)

  // Variable to memorise the returned code
  int ret = 0;

  // Print a summary
  if(cutestNbTestFailed == 0) {
    printf(
      CUTEST_START_GREEN "All tests succeeded." CUTEST_END_GREEN "\n");
    ret = EXIT_SUCCESS;
  } else {
    printf(
      CUTEST_START_RED "%03lu/%03lu test(s) failed." CUTEST_END_RED "\n",
      cutestNbTestFailed, cutestNbTest);
    ret = EXIT_FAILURE;
  }

  // Print the end time and return the exit code
  CUTEST_TIME
  return ret;
}

// Example of Makefile:
// test:
//   for file in `ls UnitTests/test_*.c`; do
//   gcc -o cutest cutest.c -DCUTEST_FILE=$$file && cutest; \rm -f cutest; done
//
// To execute only the unit test 'test002', you can use:
// cutest -t test002

Now that I have the tool, I just have to refactor all the unit tests done so far ! (^_^;)

Edit on 2022/11/21: when using CUTest with LibCapy, the version below should be preferred as it properly handles uncaught exceptions (you can download the source by clicking here):

/*
    CUTest - A unit test runner for C
    Copyright (C) 2022 Pascal Baillehache baillehache.pascal@gmail.com
    https://baillehachepascal.dev
    This program is free software: you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
    the Free Software Foundation, either version 3 of the License, or
    (at your option) any later version.
    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
    GNU General Public License for more details.
    You should have received a copy of the GNU General Public License
    along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#include <capy.h>

// Conversion from macro to string
#define CUTEST_TO_STR_(x) #x
#define CUTEST_TO_STR(x) CUTEST_TO_STR_(x)

// Escape sequence to colorize the output
#define CUTEST_START_RED "\033[38;2;255;0;0m"
#define CUTEST_END_RED "\033[39m"
#define CUTEST_START_GREEN "\033[38;2;0;255;0m"
#define CUTEST_END_GREEN "\033[39m"
#define CUTEST_START_BOLD "\033[1m"
#define CUTEST_END_BOLD "\033[0m"

// Print current date and time
#define CUTEST_TIME \
  { time_t now = time(NULL); printf("%.19s\n", asctime(localtime(&now))); }

// Variables to memorise the number of tests and failed test
size_t cutestIdxTest;
size_t cutestNbTest;
size_t cutestNbTestFailed;
bool cutestTestSuccess;

// Wrapper for the test condition in the unit tests. To be used as follow:
// CUTEST_ASSERT(cond, msg, msg_arg1, msg_arg2, ...)
// where 'cond' is the condition to be respected by the test, and 'msg', etc...
// are the arguments passed to a 'printf' call in case the condition
// is not satisfy with the view to explain why the condition wasn't satisfied.
#define CUTEST_ASSERT(cond, ...)                                      \
  if(!(cond)) {                                                       \
    if(cutestTestSuccess)                                             \
      printf(CUTEST_START_RED "NG" CUTEST_END_RED "\n");              \
    printf(                                                           \
      CUTEST_START_RED " -- Unsatisfied condition on line %u:\n%s\n", \
      __LINE__, CUTEST_TO_STR(cond));                                 \
    printf("While:\n");                                               \
    printf(__VA_ARGS__);                                              \
    printf(CUTEST_END_RED "\n");                                      \
    cutestTestSuccess = false;                                        \
  }

// Print the OK result for a successful test and increase the counter of
// failed test for an unsuccessful test
#define CUTEST_OK                                                \
  if(cutestTestSuccess) {                                        \
    printf(CUTEST_START_GREEN "OK" CUTEST_END_GREEN "\n");       \
  } else {                                                       \
    ++cutestNbTestFailed;                                        \
  }

// Unit tests declarations macro. To be used in the unit tests file as follow:
// CUTEST(test002, "Second unit test") {
//   int a = 2;
//   a *= a;
//   CUTEST_ASSERT(a == 4, "a equals to %d", a);
// }
#define CUTEST(name, lbl) void name(void)
#include CUTEST_TO_STR(CUTEST_FILE)

// Helper function to print and pad the label of the executed tests
void CutestPrintLbl(char const* const lbl) {
  size_t length = strlen(lbl);
  size_t lengthMax = 80;
  printf("%03lu/%03lu ", cutestIdxTest, cutestNbTest);
  size_t i = 0;
  for(; i < length; ++i) {
    if(i > 0 && (i % lengthMax) == 0) printf("\n        ");
    printf("%c", lbl[i]);
  }
  i %= lengthMax;
  if(i != 0) for(; i < lengthMax; ++i) printf(".");
  printf(" ");
  fflush(stdout);
}

// Main function
int main(int argc, char** argv) {

  // Variable to memorise the unit test to be executed
  // (if null, all are executed)
  char* nameTestToExecute = NULL;
  for(int iArg = 0; iArg < argc; ++iArg) {
    if(strcmp(argv[iArg], "-t") == 0 && (iArg + 1) < argc)
      nameTestToExecute = argv[iArg + 1];
  }

  // Print the unit test file name and start time
  printf(
    CUTEST_START_BOLD "\n  ==== %s ====" CUTEST_END_BOLD "\n",
    CUTEST_TO_STR(CUTEST_FILE));
  CUTEST_TIME

  // Init the counters
  cutestIdxTest = 0;
  cutestNbTest = 0;
  cutestNbTestFailed = 0;

  // Calculate the number of test
  #undef CUTEST
  #define CUTEST(name, lbl)                               \
    if(                                                   \
      nameTestToExecute == NULL ||                        \
      strcmp(nameTestToExecute, CUTEST_TO_STR(name)) == 0 \
    ) {                                                   \
      ++cutestNbTest;                                     \
    } if(0)
  #include CUTEST_TO_STR(CUTEST_FILE)

  // Unit tests call
  #undef CUTEST
  #define CUTEST(name, lbl)                               \
    if(                                                   \
      nameTestToExecute == NULL ||                        \
      strcmp(nameTestToExecute, CUTEST_TO_STR(name)) == 0 \
    ) {                                                   \
      ++cutestIdxTest; cutestTestSuccess = true;          \
      CutestPrintLbl(lbl); try { name(); }                \
      catchDefault {CUTEST_ASSERT(false,                  \
        "Uncaught exception: %s",                         \
        CapyExcToStr(CapyGetLastExcId()));}               \
      endCatch; CUTEST_OK                                 \
    }                                                     \
    if(0)
  #include CUTEST_TO_STR(CUTEST_FILE)

  // Variable to memorise the returned code
  int ret = 0;

  // Print a summary
  if(cutestNbTestFailed == 0) {
    printf(
      CUTEST_START_GREEN "All tests succeeded." CUTEST_END_GREEN "\n");
    ret = EXIT_SUCCESS;
  } else {
    printf(
      CUTEST_START_RED "%03lu/%03lu test(s) failed." CUTEST_END_RED "\n",
      cutestNbTestFailed, cutestNbTest);
    ret = EXIT_FAILURE;
  }

  // Print the end time and return the exit code
  CUTEST_TIME
  return ret;
}

// Example of Makefile:
// test:
//   for file in `ls UnitTests/test_*.c`; do
//   gcc -o cutest cutest.c -DCUTEST_FILE=$$file -lcapy && cutest; \rm -f cutest; done
//
// To execute only the unit test 'test002', you can use:
// cutest -t test002

2022-10-10
in All, C programming,
230 views