Difference between pages "r7.1.1:Usage Guide (libtest)" and "r7.1.1:Usage Guide (test)"

From liblfds.org
(Difference between pages)
Jump to navigation Jump to search
m (1 revision imported)
 
m (1 revision imported)
 
Line 1: Line 1:
{{DISPLAYTITLE:Usage Guide (libtest)}}
{{DISPLAYTITLE:Usage Guide (test)}}
==Introduction==
==Introduction==
This page describes how to use the ''libtest'' library.
The ''test'' programme is a thin command line veneer to ''libtest''.  It runs a series of tests, first for the porting abstraction layer (and so can be used to validate any new ports) and then upon each data structure, and stops running on any test failure or error of any kind.


The library implements a great deal of functionality, almost all of which is used and only used by the ''libtest'' library itself.  From the point of view of an external caller to its API, there are only a few functions; a couple to init and run the entire test suite, a few more to handle the results from a test run, and one or two miscellanous functions.
Testing lock-free code is problematic since a large class of bugs are race conditions, which are impossible or nearly impossible to methodically reproduce.  As such testing consists in part of simple API tests to enure every function does what it is supposed to do at least when in the absence of race conditions, but in the main of best-effort attempts to provoke race conditions and detect them.


==Usage==
Make sure you run the release build - the debug build does so much extra work that it misses race conditions the release build will pick up on.
To use ''libtest'', include the header file ''libtest.h'' and link as normal to the library in your build.


==Dependencies==
==Running==
The ''libtest'' libraries depends on the ''libshared'' library and the ''liblfds711'' library.
The test programme is run from the command line.  Change directory into ''test_and_benchmark/test/bin/'' and type;


==Source Files==
  test -r
└── test_and_benchmark
    └── libtest
         ├── inc
        │   └── libtest
        │       ├── libtest_misc.h
         │       ├── libtest_results.h
        │       └── libtest_testsuite.h
        └── src
            ├── libtest_misc
            │   └── libtest_misc_determine_erg.c
            ├── libtest_results
             │   ├── libtest_results_cleanup.c
            │   ├── libtest_results_get_result.c
            │   ├── libtest_results_init.c
            │   └── libtest_results_internal.h
            └── libtest_testsuite
               ├── libtest_testsuite_cleanup.c
                 ├── libtest_testsuite_init.c
               ├── libtest_testsuite_internal.h
                 └── libtest_testsuite_run.c


This is a small subset of the full set of files, and shows only those files used by the publically exposed APIs.
The full command line (which is printed by ''-h'' or by running with no arguments) is as follows;


==Enums==
  test -e -h -i [n] -m [n] -r -v
  enum [[r7.1.1:enum lfds711_misc_validity|lfds711_misc_validity]]
  -e    : empirically determine Exclusive Reservation Granule
enum [[r7.1.1:enum libtest_misc_determine_erg_result|libtest_misc_determine_erg_result]]
            (currently supports only ARM32)
enum [[r7.1.1:enum libtest_test_id|libtest_test_id]]
  -h    : help
  -i [n] : number of iterations    (default : 1)
  -m [n] : memory for tests, in mb  (default : 512)
  -r    : run (causes test to run; present so no args gives help)
  -v    : version


==Opaque Structures==
===Arguments===
struct [[r7.1.1:struct libtest_results_state|libtest_results_state]]
'''-e''' : Attempts to determine the Exclusive Reservation Granule.  See [[r7.1.1:Usage Guide (liblfds)#Exclusive Reservation Granule (ARM, POWERPC)|Exclusive Reservation Granule (ARM, POWERPC)]] for details; this is only needed on ARM and POWERPC, and only currently supported for ARM32.
  struct [[r7.1.1:struct libtest_testsuite_state|libtest_testsuite_state]]
struct [[r7.1.1:struct libshared_memory_state|libshared_memory_state]]


==Prototypes==
'''-h''' : Show help.
void [[r7.1.1:function libtest_testsuite_init|libtest_testsuite_init]]( struct libtest_testsuite_state *ts,
                              struct libshared_memory_state *ms,
                              void (*callback_test_start)(char *test_name),
                              void (*callback_test_finish)(char *result) );
void [[r7.1.1:function libtest_testsuite_cleanup|libtest_testsuite_cleanup]]( struct libtest_testsuite_state *ts );
void [[r7.1.1:function libtest_testsuite_run|libtest_testsuite_run]]( struct libtest_testsuite_state *ts,
                            struct libtest_results_state *rs );
void [[r7.1.1:function libtest_results_init|libtest_results_init]]( struct libtest_results_state *rs );
void [[r7.1.1:function libtest_results_cleanup|libtest_results_cleanup]]( struct libtest_results_state *rs );
void [[r7.1.1:function libtest_results_get_result|libtest_results_get_result]]( struct libtest_results_state *rs,
                                  enum libtest_test_id test_id,
                                  enum lfds711_misc_validity *result );
void [[r7.1.1:function libtest_misc_determine_erg|libtest_misc_determine_erg]]( struct libshared_memory_state *ms,
                                  lfds711_pal_uint_t (*count_array)[10],
                                  enum libtest_misc_determine_erg_result *der,
                                  lfds711_pal_uint_t *erg_length_in_bytes );


==Overview==
'''-i [n]''' : The test programme will loop ''n'' times.  Any error (e.e. allocation failure) or test failure will cause the test programme to print and error and exit completely, rather than merely abort the current iteration.
To run the tests, take the following steps;


# Initialize a ''struct libshared_memory_state'', with enough store (at least 4mb).  The test code is not NUMA aware, so no special NUMA-aware allocations are required.
'''-m [n]''' : There are a large number of tests.  Mainly these tests run one thread per logical core, but there are a few API tests and a few single-threaded tests. Each test being run typically allocates a large number of data structure elements with which to perform the test. The value supplied to the ''-m'' argument is the amount of memory to be used for element allocation, in megabytes. The code does not properly or consistently check for memory exhaustion; a safe absolute minimum is 4mb, 64mb is a reasonable minimum for testing, and anything more than 1gb gives no advantage.
# Initialize a ''struct libtest_results_state''.
# Initialize a ''struct libtest_testsuite'', passing in the ''libshared_memory_state'' initialized earlier.
# Call ''libtest_testsuite_run'', passing in the ''struct libtest_testsuite_state'' and the ''struct libtest_results_state''.


When ''libtest_testsuite_run'' returns, the ''struct libtest_results_state'' will be populated, and it can be queried with ''libtest_results_get_result''.
'''-r''' : If run with no arguments, the help text is displayed.  As such, an argument is needed to inform the test programme that it is in fact to run tests, which is why the "-r" argument exists.


A given initialized ''struct libtest_testsuite'' can be passed any number of times to ''libtest_testsuite_run''.  The ''struct libtest_results'' however must be initialized, or a new struct provided, between calls, as this structure only stores the results of a single test run.
'''-v''' : Shows the build version, like so;
 
test 7.1.1 (release, user-mode)
libshared 7.1.1 (release, Linux, user-mode)
libtest 7.1.1 (release, Linux, user-mode)
  liblfds 7.1.1 (release, Linux, user-mode, x64, GCC >= 4.7.3)


==See Also==
==See Also==
* [[r7.1.1:Usage Guide (testing)|Usage Guide (testing)]]
* [[r7.1.1:Usage Guide (testing)|Usage Guide (testing)]]

Latest revision as of 20:16, 17 February 2017

Introduction

The test programme is a thin command line veneer to libtest. It runs a series of tests, first for the porting abstraction layer (and so can be used to validate any new ports) and then upon each data structure, and stops running on any test failure or error of any kind.

Testing lock-free code is problematic since a large class of bugs are race conditions, which are impossible or nearly impossible to methodically reproduce. As such testing consists in part of simple API tests to enure every function does what it is supposed to do at least when in the absence of race conditions, but in the main of best-effort attempts to provoke race conditions and detect them.

Make sure you run the release build - the debug build does so much extra work that it misses race conditions the release build will pick up on.

Running

The test programme is run from the command line. Change directory into test_and_benchmark/test/bin/ and type; 

 test -r

The full command line (which is printed by -h or by running with no arguments) is as follows; 

test -e -h -i [n] -m [n] -r -v
  -e     : empirically determine Exclusive Reservation Granule
           (currently supports only ARM32)
  -h     : help
  -i [n] : number of iterations     (default : 1)
  -m [n] : memory for tests, in mb  (default : 512)
  -r     : run (causes test to run; present so no args gives help)
  -v     : version

Arguments

-e : Attempts to determine the Exclusive Reservation Granule. See Exclusive Reservation Granule (ARM, POWERPC) for details; this is only needed on ARM and POWERPC, and only currently supported for ARM32.

-h : Show help.

-i [n] : The test programme will loop n times. Any error (e.e. allocation failure) or test failure will cause the test programme to print and error and exit completely, rather than merely abort the current iteration.

-m [n] : There are a large number of tests. Mainly these tests run one thread per logical core, but there are a few API tests and a few single-threaded tests. Each test being run typically allocates a large number of data structure elements with which to perform the test. The value supplied to the -m argument is the amount of memory to be used for element allocation, in megabytes. The code does not properly or consistently check for memory exhaustion; a safe absolute minimum is 4mb, 64mb is a reasonable minimum for testing, and anything more than 1gb gives no advantage.

-r : If run with no arguments, the help text is displayed. As such, an argument is needed to inform the test programme that it is in fact to run tests, which is why the "-r" argument exists.

-v : Shows the build version, like so; 

test 7.1.1 (release, user-mode)
libshared 7.1.1 (release, Linux, user-mode)
libtest 7.1.1 (release, Linux, user-mode)
liblfds 7.1.1 (release, Linux, user-mode, x64, GCC >= 4.7.3)

See Also

Retrieved from ‘https://www.liblfds.org/mediawiki/index.php?title=r7.1.1:Usage_Guide_(libtest)&oldid=1224’