Usage Guide (libshared)
Introduction
This page describes how to use the libshared library.
The library implements a great deal of functionality, almost all of which is used and only used by other liblfds components. From the point of view of an external caller to its API, there is in fact only one API, which handles user-allocated memory.
Usage
To use libshared, include the header file libshared.h and link as normal to the library in your build.
Dependencies
The libtest libraries depends on the liblfds711 library.
Source Files
└── test_and_benchmark
└── libshared
├── inc
│ ├── libshared
│ │ └── libshared_memory.h
└── src
└── libshared_memory
├── libshared_memory_add.c
├── libshared_memory_cleanup.c
└── libshared_memory_init.c
This is a small subset of the full set of files, and shows only those files used by the publically exposed APIs.
Opaque Structures
struct libshared_memory_state;
Prototypes
void libshared_memory_init( struct libshared_memory_state *ms ); void libshared_memory_cleanup( struct libshared_memory_state *ms, void (*memory_cleanup_callback)(enum flag known_numa_node_flag, void *store, lfds711_pal_uint_t size) ); void libshared_memory_add_memory( struct libshared_memory_state *ms, void *memory, lfds711_pal_uint_t memory_size_in_bytes ); void libshared_memory_add_memory_from_numa_node( struct libshared_memory_state *ms, lfds711_pal_uint_t numa_node_id, void *memory, lfds711_pal_uint_t memory_size_in_bytes );
Overview
All liblfds libraries are written such that they perform no memory allocations. This is straightforward for liblfds, where the user passes in state structures and so on, but it is problematic for libtest and libbenchmark as the store they require varies depending on the number of logical cores in the system, where that number cannot be known in advance, and where the work being done is complex enough that it is impractical to require the user to pass in the required store to functions - rather, a generic method is needed, where the libraries can in effect perform dynamic memory allocation.
This is the purpose of the libshared_memory API. The caller of libtest or libbenchmark functionality initializes a struct libshared_memory_state, performs some memory allocation by whatever means are available and adds the pointer to that memory and its size in bytes to the memory state. This memory state is then passed into such function in libtest or libbenchmark which require it, which in turn draw upon the memory so provided for dynamic allocations.
The libtest library is not currently NUMA aware - it simply runs one thread per logical core and allocates everything from the allocation with the most free space at the time of the allocation request. The libbenchmark library is NUMA aware and on NUMA systems in fact requires an allocation from every NUMA node in the system.
On SMP systems, or on NUMA systems but where a non-NUMA aware allocator is used (e.g. malloc rather than say numa_alloc_onnode) memory is added by the libshared_memory_add_memory function. On NUMA systems, memory is added with the libshared_memory_add_memory_from_numa_node function. Any number of allocations from any number of nodes (or from the non-NUMA aware allocators) can be provided, although there's no obvious use case for this, since normal usage is to initialize and allocate per-NUMA node and then call a libtest or libenchmark function.
The libbenchmark_topology API offers an iterator API, which permits easy iteration over the NUMA nodes in a system, saving the caller the trouble of having to enumerate the processor/memory topology. Note that initializing a topology state requires an initialized and populated memory state; however, this state is not NUMA sensitive, and so it can be allocated using malloc and then, once obtained, a second memory state can be populated with per-NUMA node allocations.