Difference between pages "r7.1.1:Building Guide (libtest)" and "r7.1.1:Building Guide (test)"

From liblfds.org
(Difference between pages)
Jump to navigation Jump to search
m (1 revision imported)
 
m (1 revision imported)
 
Line 1: Line 1:
{{DISPLAYTITLE:Building Guide (libtest)}}
{{DISPLAYTITLE:Building Guide (test)}}
==Introducton==
==Introducton==
The ''libtest'' library is used by ''test'', and depends upon ''liblfds711'' and ''libshared''.  It is not used by, or needed to build, ''liblfds711''.
The ''test'' binary is a thin command line veneer which provides convenient access to the test functionality in ''libtest''.  The binary depends on ''liblfds711'', ''libtest'' and ''libshared''.


The ''libtest'' library uses a porting abstraction layer to mask platform differences.  Building ''libtest'' requires build files (makefile, etc) for your toolchain (GCC, gnumake, etc) and a port of the abstraction layer to your platform.
The ''test'' library uses a porting abstraction layer to mask platform differences.  Building ''test'' requires build files (makefile, etc) for your toolchain (GCC, gnumake, etc) and a port of the abstraction layer to your platform.


A small number of popular toolchains are supported out-of-the-box, along with porting abstraction layers which cover Windows (user-mode and kernel-mode) and Linux (user-mode).  A kernel-mode Linux port is not yet present as ''libtest'' uses the standard library function ''time'' to run tests and as such although the kbuild build files are present and correct, the library will fail to compile for that toolchain.  (Additionally, there is also not yet a Linux kernel port of ''libshared'', so even if ''libtest'' were ported, ''test'' still could not link).
A small number of popular toolchains are supported out-of-the-box, along with porting abstraction layers which cover Windows (user-mode) and Linux (user-mode).


==Supported Toolchains==
==Supported Toolchains==
Line 11: Line 11:


* GCC and gnumake
* GCC and gnumake
* GCC, kbuild and gnumake (Linux kernel toolchain)
* MSVC and gnumake (yes, gnumake, not nmake)
* MSVC and and gnumake (Windows user-mode toolchain - and, yup, gnumake, not nmake)
* WDK 7.1 (Windows XP/Vista/7 kernel toolchain)


Note that in 7.1.1, Visual Studio solution files are not provided.  This is because there are over ''liblfds'' and the ''benchmark'' and ''benchmark'' libraries and programmes (seven projects in all) in the end due to the multple build variants (debug, release, library, DLL, kernel library, kernel DLL, and repeated twice, once for VS2012 and once for VS2013) what comes to something like ten thousand settings, all of which have to be set using a mouse and a GUI, which is not only extraordinarily time-consuming and error-prone, but emotionally agonizing.
Note that in 7.1.1, Visual Studio solution files are not provided.  This is because there are over ''liblfds'' and the ''benchmark'' and ''benchmark'' libraries and programmes (seven projects in all) in the end due to the multple build variants (debug, release, library, DLL, kernel library, kernel DLL, and repeated twice, once for VS2012 and once for VS2013) what comes to something like ten thousand settings, all of which have to be set using a mouse and a GUI, which is not only extraordinarily time-consuming and error-prone, but emotionally agonizing.
Line 20: Line 18:


==Directory Structure==
==Directory Structure==
  └───test_and_benchmark
  └── test_and_benchmark
     └───libtest                                                      : libtest library
     └── test                  : test command line veneer
         ├───bin                                                      : output directory - the library file ends up here
         ├── bin              : output directory - the binary ends up here
         ├───build                                                    : build configuration directory - contains one directory per platform
         ├── build            : build configuration directory - contains one directory per platform
         │  ├───gcc_gnumake                                          : GCC and gnumake
         │  ├── gcc_gnumake   : GCC, gnumake, hosted implementation
        │   ├───gcc_gnumake_kbuild                                  : GCC, gnumake, and kbuild
         │  └── msvc_gnumake  : Microsoft Visual C (command line compiler) and gnumake
         │  ├───msvc_and_gnumake                                    : Microsoft Visual C (command line compiler) and gnumake
         ├── obj              : temporary directory for object files
         │  └───wdk_7.1                                              : WDK 7.1
         └── src              : the source files
        ├───inc                                                      : the public header files
        │  └───libtest
        ├───obj                                                      : temporary directory for object files
         └───src                                                      :
            ├───libtest_misc                                        : misc functions
            ├───libtest_porting_abstraction_layer                    : porting abstraction layer
            ├───libtest_results                                      : test results storage API
            ├───libtest_test                                        : API for running a single test
            ├───libtest_tests                                        : the tests themselves
            ├───libtest_testsuite                                    : convenience API to easily run all tests
            └───libtest_threadset                                    : convenience API for handling issuing threads


==Building==
==Building==
Line 45: Line 32:
This directory contains one directory per supported toolchain, where each such directory contains the files necessary to build for that toolchain.  Detailed descriptions of how to build for each toolchain are given below, with one heading per toolchain.
This directory contains one directory per supported toolchain, where each such directory contains the files necessary to build for that toolchain.  Detailed descriptions of how to build for each toolchain are given below, with one heading per toolchain.


The various other components (''test'', ''benchmark'', etc) in their build files know the location of the resultant library file, so it does not need to be installed.
The ''test'' binary depends on ''liblfds711'', ''libtest'' and ''libshared''.  The ''test'' build files are harcoded with relative paths to find the output from these libraries, so these libraries do not need to be installed.


On all platforms, you need to clean between changing build types (debug, release, static, dynamic, profiled, etc), as there is only one directory used to hold object files.
On all platforms, you need to clean between changing build types (debug, release, static, dynamic, profiled, etc), as there is only one directory used to hold object files.
Line 52: Line 39:


===GCC and gnumake===
===GCC and gnumake===
  └───test_and_benchmark
  └── test_and_benchmark
     └───libtest
     └── test
         └───build
         └── build
             └───gcc_gnumake
             └── gcc_gnumake
                     Makefile
                     Makefile


Line 64: Line 51:
The following targets are available;
The following targets are available;


  ar_cov      : archive (.a), coverage
  cov    : coverage
  ar_dbg      : archive (.a), debug
  dbg    : debug
  ar_prof      : archive (.a), profiling
  prof    : profiling
  ar_rel      : archive (.a), release
  rel    : release
  ar_tsan      : archive (.a), thread sanitizer
  tsan    : thread sanitizer
  ar_vanilla  : archive (.a), no specific-build arguments
  vanilla : no specific-build arguments
   
   
  so_cov      : shared (.so), coverage
  clean   : what you'd expect
so_dbg      : shared (.so), debug
so_prof      : shared (.so), profiling
so_rel      : shared (.so), release
so_tsan      : shared (.so), thread sanitizer
so_vanilla   : shared (.so), no specific-build arguments
clean        : what you'd expect


When switching from one target to another, ''clean'' must be made.
When switching from one target to another, ''clean'' must be made.
Line 84: Line 64:
If building ''*_ar_tsan'', ''libtsan'' must be installed.  This is not necessary if building ''*_so_tsan''.
If building ''*_ar_tsan'', ''libtsan'' must be installed.  This is not necessary if building ''*_so_tsan''.


===GCC, gnumake and kbuild===
If building for ARM32 with shared object versions of the ''liblfds'' libraries, linking will fail, with the error ''"/usr/bin/ld: ../../bin/test: hidden symbol `__aeabi_uidiv' in /usr/lib/gcc/arm-linux-gnueabihf/4.9/libgcc.a(_udivsi3.o) is referenced by DSO"''This means something has attempted to divide a 64 bit value by a 64 bit value, an operation which apparently is costly and usually unnecessary, a 32 bit divisor apparently is normally adequate, and as such is intentionally caught in a link-breaking way.
└───test_and_benchmark
    └───libtest
        └───build
            └───gcc_gnumake_kbuild
                    Kbuild
                    Makefile
 
Remember, although the build files are present, the porting abstraction layer is not fully implemented.  '''This platform will fail to build'''.
 
To build, install GCC, gnumake, kbuild (and the Linux kernel headers), enter the build directory and type;
 
make
 
The makefile is complimentory (mint-flavoured :-) and simply issues the necessary kbuild command.
 
There are three targets;
 
  clean
  help
  modules
 
Note that ''modules'' is the default build (i.e. with no arguments).
 
I do not properly understand kbuild. I've read the documentation; I find for example examples which have switches in which are simply not present in the documentation, and in general I don't see how libraries are built to fit in with larger projects. The most I can say about this build is that it compiles without warnings or errorsI am pretty sure it's not something which can be used as it is in a kernel build or driver, but anyone who knows enough to be developing such a thing should be able to take what is here (since it compiles against the kernel) and easily fit it into their work.


Any feedback on making this build proper is greatly appreciated.
Unfortunately, as far as can be determined, ''liblfds'' itself is not performing this operation - and as such, it is not obvious how to fix it.


===MSVC and gnumake===
===MSVC and gnumake===
  └───test_and_benchmark
  └── test_and_benchmark
     └───libtest
     └── test
         └───build
         └── build
             └───msvc_and_gnumake
             └── msvc_gnumake
                    libtest.def
                     makefile
                     makefile


Line 126: Line 81:
The following targets are available;
The following targets are available;


  libdbg : archive (.lib), debug
  dlldbg  : debug  (with ''liblfds'' libs built as DLLs)
  librel : archive (.lib), release
  dllrel  : release (with ''liblfds'' libs built as DLLs)
   
  libdbg : debug  (with ''liblfds'' libs built as libs)
  dlldbg : shared (.dll), debug
  librel  : release (with ''liblfds'' libs built as libs)
  dllrel : shared (.dll), release
   
   
  clean : what you'd expect
  clean   : what you'd expect


When switching from one target to another, clean must be made.
When switching from one target to another, clean must be made.
===WDK 7.1===
└───test_and_benchmark
    └───libtest
        └───build
            └───wdk_7.1
                    dirs
                    driver_entry_renamed_to_avoid_compiler_warning.c
                    libtest.def
                    readme_before_win_kernel_build.txt
                    runme_before_win_kernel_dynamic_lib_build.bat
                    runme_before_win_kernel_static_lib_build.bat
                    sources.dynamic
                    sources.static
All processor types are supported (x86, IA64, x64).
The WDK 7.1 kernel build environment is primitive and has a number of severe limitations; in particular, all source files must be in one directory and it is not possible to choose the output binary type (static or dynamic library) from the build command line; rather, a string has to be modified in a text file used by the build (!)
To deal with these limitations, it is necessary for a Windows kernel build to run a batch file prior to building.  There are two batch files, one for static library builds and the other for dynamic library builds.  They are idempotent; you can run them as often as you like and switch between them as often as you want.  It's all fine.
Both batch files copy all the sources file into a single directory, ''libtest/build/wdk7.1/single_dir_for_windows_kernel/''.
The static library batch file will then copy ''libtest/sources.static'' into ''libtest/build/wdk7.1/single_dir_for_windows_kernel/'', which will cause a static library to be built.
The dynamic library batch file will then copy ''libtest/sources.dynamic'' into ''libtest/build/wdk7.1/single_dir_for_windows_kernel/'', which will cause a dynamic library to be built.  It will also copy ''driver_entry_renamed_to_avoid_compiler_warning.c'' to ''libtest/build/wdk7.1/single_dir_for_windows_kernel/driver_entry.c'' (note the renaming), since the linker requires the ''DriverEntry'' function to exist, even though it's not used.
To build, start a build command line as usual, indicating through the command line you select whether you are making a debug or release build.  Then run the appropriate liblfds batch file to select a static or dynamic library.  Then, finally, run ''build.exe'' in the ''libtest/build/wdk_7.1/'' directory, with whatever arguments you prefer.
Note that due to limitations in the build environment, object files appear in a subdirectory in ''libtest/build/wdk7.1/single_dir_for_windows_kernel/'', rather than in the usual ''libtest/obj/''.  The final binary output however still appears almost as usual, in ''libtest/bin/[processor]/'', where processor might be ''i386''.  (The usual is just ''libtest/bin/'', with no subdirectories).
Note that the ''sources'' file used to compile the library asserts the define ''KERNEL_MODE''.  This appears to be necessary, due to the lack of a compiler-provided macro to differentiate in the code between a user-mode or kernel-mode build.


==See Also==
==See Also==
* [[r7.1.1:Building Guide (testing)|Building Guide (testing)]]
* [[r7.1.1:Building Guide (testing)|Building Guide (testing)]]

Latest revision as of 20:16, 17 February 2017

Introducton

The test binary is a thin command line veneer which provides convenient access to the test functionality in libtest. The binary depends on liblfds711, libtest and libshared.

The test library uses a porting abstraction layer to mask platform differences. Building test requires build files (makefile, etc) for your toolchain (GCC, gnumake, etc) and a port of the abstraction layer to your platform.

A small number of popular toolchains are supported out-of-the-box, along with porting abstraction layers which cover Windows (user-mode) and Linux (user-mode).

Supported Toolchains

The supported toolchains are;

  • GCC and gnumake
  • MSVC and gnumake (yes, gnumake, not nmake)

Note that in 7.1.1, Visual Studio solution files are not provided. This is because there are over liblfds and the benchmark and benchmark libraries and programmes (seven projects in all) in the end due to the multple build variants (debug, release, library, DLL, kernel library, kernel DLL, and repeated twice, once for VS2012 and once for VS2013) what comes to something like ten thousand settings, all of which have to be set using a mouse and a GUI, which is not only extraordinarily time-consuming and error-prone, but emotionally agonizing.

Mac support is not available due to lack of access to a Mac.

Directory Structure

└── test_and_benchmark
    └── test                  : test command line veneer
        ├── bin               : output directory - the binary ends up here
        ├── build             : build configuration directory - contains one directory per platform
        │   ├── gcc_gnumake   : GCC, gnumake, hosted implementation
        │   └── msvc_gnumake  : Microsoft Visual C (command line compiler) and gnumake
        ├── obj               : temporary directory for object files
        └── src               : the source files

Building

The library directory tree contains at its top level directory called build.

This directory contains one directory per supported toolchain, where each such directory contains the files necessary to build for that toolchain. Detailed descriptions of how to build for each toolchain are given below, with one heading per toolchain.

The test binary depends on liblfds711, libtest and libshared. The test build files are harcoded with relative paths to find the output from these libraries, so these libraries do not need to be installed.

On all platforms, you need to clean between changing build types (debug, release, static, dynamic, profiled, etc), as there is only one directory used to hold object files.

Per-Toolchain Build Instructions

GCC and gnumake

└── test_and_benchmark
    └── test
        └── build
            └── gcc_gnumake
                    Makefile

To build, install GCC and gnumake, enter the build directory and type;

make

The following targets are available;

cov     : coverage
dbg     : debug
prof    : profiling
rel     : release
tsan    : thread sanitizer
vanilla : no specific-build arguments

clean   : what you'd expect

When switching from one target to another, clean must be made.

If building *_ar_tsan, libtsan must be installed. This is not necessary if building *_so_tsan.

If building for ARM32 with shared object versions of the liblfds libraries, linking will fail, with the error "/usr/bin/ld: ../../bin/test: hidden symbol `__aeabi_uidiv' in /usr/lib/gcc/arm-linux-gnueabihf/4.9/libgcc.a(_udivsi3.o) is referenced by DSO". This means something has attempted to divide a 64 bit value by a 64 bit value, an operation which apparently is costly and usually unnecessary, a 32 bit divisor apparently is normally adequate, and as such is intentionally caught in a link-breaking way.

Unfortunately, as far as can be determined, liblfds itself is not performing this operation - and as such, it is not obvious how to fix it.

MSVC and gnumake

└── test_and_benchmark
    └── test
        └── build
            └── msvc_gnumake
                    makefile

To build, install an MSVC command line compiler, enter the build directory and type;

make

The following targets are available;

dlldbg  : debug   (with liblfds libs built as DLLs)
dllrel  : release (with liblfds libs built as DLLs)
libdbg  : debug   (with liblfds libs built as libs)
librel  : release (with liblfds libs built as libs)

clean   : what you'd expect

When switching from one target to another, clean must be made.

See Also