Usage Guide (libbenchmark)
Introduction
This page describes how to use the libbenchmark library.
The library implements a great deal of functionality, almost all of which is used and only used by the libbenchmark library itself. From the point of view of an external caller to its API, there are only a few functions; a couple to init and run the entire benchmark suite, a few more to handle the results from a benchmark run and one or two miscellanous functions.
Usage
To use libbenchmark, include the header file libbenchmark.h and link as normal to the library in your build.
Dependencies
The libbenchmark libraries depends on the libshared library and the liblfds711 library.
Source Files
└── test_and_benchmark
└── libbenchmark
├── inc
│ └── libbenchmark
│ ├── libbenchmark_benchmarksuite.h
│ ├── libbenchmark_enums.h
│ ├── libbenchmark_gnuplot.h
│ ├── libbenchmark_results.h
│ ├── libbenchmark_topology.h
│ └── libbenchmark_topology_node.h
└── src
├── libbenchmark_benchmarksuite
│ ├── libbenchmark_benchmarksuite_cleanup.c
│ ├── libbenchmark_benchmarksuite_gnuplot.c
│ ├── libbenchmark_benchmarksuite_init.c
│ ├── libbenchmark_benchmarksuite_internal.h
│ └── libbenchmark_benchmarksuite_run.c
├── libbenchmark_results
│ ├── libbenchmark_results_cleanup.c
│ ├── libbenchmark_results_get_result.c
│ ├── libbenchmark_results_init.c
│ └── libbenchmark_results_internal.h
├── libbenchmark_topology
│ ├── libbenchmark_topology_cleanup.c
│ ├── libbenchmark_topology_init.c
│ ├── libbenchmark_topology_internal.h
│ ├── libbenchmark_topology_numa.c
│ └── libbenchmark_topology_lpsets.c
└── libbenchmark_topology_node
├── libbenchmark_topology_node_cleanup.c
└── libbenchmark_topology_node_init.c
This is a small subset of the full set of files, and shows only those files used by the publically exposed APIs.
Defines
#define LIBBENCHMARK_BENCHMARKSUITE_OPTION_DURATION
Enums
enum libbenchmark_benchmark_id; enum libbenchmark_datastructure_id; enum libbenchmark_lock_id; enum libbenchmark_topology_node_type; enum libbenchmark_topology_numa_mode;
Opaque Structures
struct lfds711_list_aso_element; struct lfds711_list_aso_state; struct lfds711_list_asu_state; struct libbenchmark_gnuplot_options; struct libbenchmark_results_state; struct libbenchmark_benchmarksuite_state; struct libbenchmark_topology_state; struct libbenchmark_topology_node_state; struct libshared_memory_state;
Macros
#define LIBBENCHMARK_TOPOLOGY_NODE_SET_TYPE( tns, node_type ) #define LIBBENCHMARK_TOPOLOGY_NODE_SET_LOGICAL_PROCESSOR_NUMBER( tns, processor_number ) #define LIBBENCHMARK_TOPOLOGY_NODE_SET_WINDOWS_GROUP_NUMBER( tns, win_group_number ) #define LIBBENCHMARK_TOPOLOGY_NODE_UNSET_WINDOWS_GROUP_NUMBER( tns ) #define LIBBENCHMARK_GNUPLOT_OPTIONS_INIT( gpo ) #define LIBBENCHMARK_GNUPLOT_OPTIONS_SET_Y_AXIS_SCALE_TYPE_LOGARITHMIC( gpo ) #define LIBBENCHMARK_GNUPLOT_OPTIONS_SET_WIDTH_IN_PIXELS( gpo, wip ) #define LIBBENCHMARK_GNUPLOT_OPTIONS_SET_HEIGHT_IN_PIXELS( gpo, wip )
Prototypes
int libbenchmark_topology_init( struct libbenchmark_topology_state *ts, struct libshared_memory_state *ms ); void libbenchmark_topology_cleanup( struct libbenchmark_topology_state *ts ); void libbenchmark_topology_generate_deduplicated_logical_processor_sets( struct libbenchmark_topology_state *ts, struct libshared_memory_state *ms, struct lfds711_list_asu_state *lp_sets ); void libbenchmark_topology_generate_numa_modes_list( struct libbenchmark_topology_state *ts, enum libbenchmark_topology_numa_mode numa_mode, struct libshared_memory_state *ms, struct lfds711_list_asu_state *numa_modes_list ); void libbenchmark_topology_node_init( struct libbenchmark_topology_node_state *tns ); void libbenchmark_topology_node_cleanup( struct libbenchmark_topology_node_state *tns, void (*element_cleanup_callback)(struct lfds711_list_aso_state *lasos, struct lfds711_list_aso_element *lasoe) ); void libbenchmark_results_init( struct libbenchmark_results_state *rs, struct libshared_memory_state *ms ); void libbenchmark_results_cleanup( struct libbenchmark_results_state *rs ); int libbenchmark_results_get_result( struct libbenchmark_results_state *rs, enum libbenchmark_datastructure_id datastructure_id, enum libbenchmark_benchmark_id benchmark_id, enum libbenchmark_lock_id lock_id, enum libbenchmark_topology_numa_mode numa_mode, struct lfds711_list_aso_state *lpset, struct libbenchmark_topology_node_state *tns, lfds711_pal_uint_t *result ); void libbenchmark_benchmarksuite_init( struct libbenchmark_benchmarksuite_state *bss, struct libbenchmark_topology_state *ts, struct libshared_memory_state *ms, enum libbenchmark_topology_numa_mode numa_mode, lfds711_pal_uint_t options_bitmask, lfds711_pal_uint_t benchmark_duration_in_seconds ); void libbenchmark_benchmarksuite_cleanup( struct libbenchmark_benchmarksuite_state *bss ); void libbenchmark_benchmarksuite_run( struct libbenchmark_benchmarksuite_state *bss, struct libbenchmark_results_state *rs ); void libbenchmark_benchmarksuite_get_list_of_gnuplot_strings( struct libbenchmark_benchmarksuite_state *bss, struct libbenchmark_results_state *rs, char *gnuplot_system_string, struct libbenchmark_gnuplot_options *gpo, struct lfds711_list_asu_state *list_of_gnuplot_strings );
Overview
It has become apparent from creating this page that the API interface to benchmarking is too complicated. The API to run the benchmarks and get hold of the results is actually simple enough - two init functions (one for the benchmark and one for the results) and then a function to run the benchmarks. The problem come when offering APIs to query the results.
There are many data structures, each of which can have many benchmarks, where each benchmark is run both for the liblfds lock-free version and then for the full range of system provided locking mechanisms, where each run of the benchmark for a given lock type runs over many combinations of logical cores, where for every combination a result is stored for each logical processor; and all of this on a NUMA system with more than one NUMA node is run twice, once NUMA aware and once not, to show how much difference this makes to performance.
As such, values for all of these parameters must be specified to the function which queries the result state - and this exposes a great deal of complexity and functionality, as can be seen above.
It will take a day to document this library, and the effort to do so is futile, because it will still be too complex to use. As such, the effort will go instead into simplifying the API. However, this should not delay the release of 7.1.1, so the functions in this library will for now go undocumented.