| [MU13] |
Hilbrich, Tobias and Protze, Joachim and Schulz, Martin and de Supinski, Bronis R. and Müller, Matthias S.
@@ -35,12 +46,14 @@ A brief summary is given in a subsequent section and more information can be fou
## Software dependencies
-TypeART requires [LLVM](https://llvm.org) version 10 and CMake version >= 3.12.
+TypeART requires [LLVM](https://llvm.org) version 10 and CMake version >= 3.14.
#### Building TypeART
TypeART uses CMake to build, cf. [GitHub CI build file](.github/workflows/basic-ci.yml) for a complete recipe to build.
-Example build recipe (debug build, installs to default prefix)
+Example build recipe (debug build, installs to default prefix
+`${typeart_SOURCE_DIR}/install/typeart`)
+
```{.sh}
$> git clone https://github.com/tudasc/TypeART
$> cd TypeART
@@ -50,35 +63,58 @@ $> cmake --build build --target install --parallel
#### CMake Configuration: Options for users
+##### Runtime
+
- `SOFTCOUNTERS` (default: **off**) : Enable runtime tracking of #tracked addrs. / #distinct checks / etc.
-- `USE_BTREE` (default: **on**) : Enable usage of btree-backed map instead of std::map for the runtime, typically resulting in higher performance.
+- `USE_ABSL` (default: **on**) : Enable usage of btree-backed map of the abseil project instead of std::map for the
+ runtime.
+- `USE_BTREE` (default: **off**) : Enable usage of a btree-backed map (alternative to abseil) instead of std::map for
+ the runtime.
+
+##### Logging and Passes
+
+- `SHOW_STATS` (default: **on**) : Passes show the statistics w.r.t. allocations etc.
+- `MPI_LOGGER` (default: **on**) : Enable better logging support in MPI execution context
+- `MPI_INTERCEPT_LIB` (default: **on**) : Library can be used by preloading to intercept MPI calls and check whether
+ TypeART tracks the buffer pointer
+- `LOG_LEVEL_` and `LOG_LEVEL_RT` (default **0**) : Granularity of logger. 3 ist most verbose, 0 is least.
+
+##### Testing
+
+- `TEST_CONFIG` (default: **off**) : Set (force) logging levels to appropriate levels for test runner to succeed
+- `ENABLE_CODE_COVERAGE` (default: **off**) : Enable code coverage statistics using LCOV 1.14 and genhtml (gcovr
+ optional)
+- `ENABLE_LLVM_CODE_COVERAGE` (default: **off**) : Enable llvm-cov code coverage statistics (llvm-cov and llvm-profdata
+ required)
## Using TypeART
-Making use of TypeART consists of two phases:
+Making use of TypeART consists of two phases:
- 1. Compile and instrument the target code with our LLVM passes, and,
- 2. execute the target program with a runtime library (based on the TypeART runtime) to accept the callbacks from the instrumented code and actually do some useful analysis.
+1. Compile and instrument the target code with our LLVM passes, and,
+2. execute the target program with a runtime library (based on the TypeART runtime) to accept the callbacks from the
+ instrumented code and actually do some useful analysis.
-To that end, the interface [RuntimeInterface.h](runtime/RuntimeInterface.h) can be used to query type information during the target code execution.
+To that end, the interface [RuntimeInterface.h](runtime/RuntimeInterface.h) can be used to query type information during
+the target code execution.
#### Example: MPI Demo
-The folder [demo](demo) contains an example of MPI related type errors that can be detected using TypeART.
-The code is compiled with our instrumentation, and executed by preloading the MPI related check library implemented in [tool.c](demo/tool.c), which is linked against the TypeART runtime and uses the aforementioned query interface.
-It overloads the required MPI calls and checks that the passed `void* buffer` is correct.
+The folder [demo](demo) contains an example of MPI related type errors that can be detected using TypeART. The code is
+compiled with our instrumentation, and executed by preloading the MPI related check library implemented
+in [tool.c](demo/tool.c), which is linked against the TypeART runtime and uses the aforementioned query interface. It
+overloads the required MPI calls and checks that the passed `void* buffer` is correct.
## LLVM pass
-The necessary allocation sites and type information are extracted in LLVM passes.
-TypeART analyzes:
+The necessary allocation sites and type information are extracted in LLVM passes. TypeART analyzes:
- Calls to ```malloc``` and ```free``` to keep track of active pointers referring to objects allocated on the heap,
-- relevant stack space allocations, i.e., allocations that cannot be proven to never lead to ```MPI``` functions,
+- relevant stack space allocations, i.e., allocations that cannot be proven to never lead to ```MPI``` functions,
- built-in as well as user-defined types to retrieve type size and the size of the allocation, e.g., for arrays.
-The type information is necessary to correlate the type of the buffer passed to an MPI call with the MPI datatype the user declared.
-In this prototype we restrict ourselves to:
+The type information is necessary to correlate the type of the buffer passed to an MPI call with the MPI datatype the
+user declared. In this prototype we restrict ourselves to:
+ primitive types (int, float, long, double, char, unsigned int, unsigned long)
+ arrays of primitive types
@@ -86,8 +122,9 @@ In this prototype we restrict ourselves to:
### Example of Instrumentation: Handling malloc
-To instrument relevant allocations and extract the necessary type information, the LLVM pass searches for specific patterns, e.g., how calls to ```malloc``` look like in LLVM IR.
-Calls to the ```malloc``` function are typically call instructions followed by a ```bitcast``` instruction to cast the returned pointer to the desired type.
+To instrument relevant allocations and extract the necessary type information, the LLVM pass searches for specific
+patterns, e.g., how calls to ```malloc``` look like in LLVM IR. Calls to the ```malloc``` function are typically call
+instructions followed by a ```bitcast``` instruction to cast the returned pointer to the desired type.
~~~{.ll}
; %0 == n * sizeof(float)
@@ -95,9 +132,9 @@ Calls to the ```malloc``` function are typically call instructions followed by a
%2 = bitcast i8* %1 to float *
~~~
-The patterns has all the information we require for our instrumentation.
-Our transformation first detects the type that the returned pointer is casted to, then it computes the extent of the allocation.
-The information is passed to our instrumentation function.
+The patterns has all the information we require for our instrumentation. Our transformation first detects the type that
+the returned pointer is casted to, then it computes the extent of the allocation. The information is passed to our
+instrumentation function.
~~~{.ll}
; %0 == n * sizeof(float)
diff --git a/cmake/ToolchainOptions.cmake b/cmake/ToolchainOptions.cmake
index c0652766..9a34592e 100644
--- a/cmake/ToolchainOptions.cmake
+++ b/cmake/ToolchainOptions.cmake
@@ -1,79 +1,82 @@
-find_package(LLVM REQUIRED CONFIG)
+find_package(LLVM 10 REQUIRED CONFIG)
message(STATUS "Found LLVM ${LLVM_PACKAGE_VERSION}")
list(APPEND CMAKE_MODULE_PATH "${LLVM_CMAKE_DIR}")
+set(LOG_LEVEL 0 CACHE STRING "Granularity of LLVM pass logger. 3 ist most verbose, 0 is least.")
+set(LOG_LEVEL_RT 0 CACHE STRING "Granularity of runtime logger. 3 ist most verbose, 0 is least.")
+option(SHOW_STATS "Passes show the statistics vars." ON)
+option(MPI_LOGGER "Whether the logger should use MPI." ON)
+option(MPI_INTERCEPT_LIB "Build MPI interceptor library for prototyping and testing." ON)
+option(SOFTCOUNTERS "Enable software tracking of #tracked addrs. / #distinct checks / etc." OFF)
+option(TEST_CONFIG "Set logging levels to appropriate levels for test runner to succeed" OFF)
+option(ENABLE_CODE_COVERAGE "Enable code coverage statistics" OFF)
+option(ENABLE_LLVM_CODE_COVERAGE "Enable llvm-cov code coverage statistics" OFF)
+
include(AddLLVM)
include(llvm-lit)
include(clang-tidy)
include(clang-format)
include(llvm-util)
include(log-util)
+include(coverage)
-set(LOG_LEVEL 0 CACHE STRING "Granularity of LLVM pass logger. 3 ist most verbose, 0 is least.")
-set(LOG_LEVEL_RT 0 CACHE STRING "Granularity of runtime logger. 3 ist most verbose, 0 is least.")
-option(SHOW_STATS "Passes show the statistics vars." OFF)
-option(MPI_LOGGER "Whether the logger should use MPI." OFF)
-option(MPI_INTERCEPT_LIB "Build MPI interceptor library, requires wrap.py generator file." OFF)
-option(SOFTCOUNTERS "Enable software tracking of #tracked addrs. / #distinct checks / etc." OFF)
-option(TEST_CONFIG "Set logging levels to appropriate levels for test runner to succeed" OFF)
-
-if(TEST_CONFIG)
+if (TEST_CONFIG)
set(LOG_LEVEL 2 CACHE STRING "" FORCE)
set(LOG_LEVEL_RT 3 CACHE STRING "" FORCE)
-endif()
+endif ()
-if(MPI_LOGGER)
+if (MPI_LOGGER)
find_package(MPI REQUIRED)
-endif()
+endif ()
-if(NOT CMAKE_BUILD_TYPE)
+if (NOT CMAKE_BUILD_TYPE)
# set default build type
set(CMAKE_BUILD_TYPE Debug CACHE STRING "" FORCE)
message(STATUS "Building as debug (default)")
-endif()
+endif ()
-if(CMAKE_INSTALL_PREFIX_INITIALIZED_TO_DEFAULT)
+if (CMAKE_INSTALL_PREFIX_INITIALIZED_TO_DEFAULT)
# set default install path
set(CMAKE_INSTALL_PREFIX "${typeart_SOURCE_DIR}/install/typeart" CACHE PATH "Default install path" FORCE)
message(STATUS "Installing to (default): ${CMAKE_INSTALL_PREFIX}")
-endif()
+endif ()
function(target_project_compile_options target)
cmake_parse_arguments(ARG "" "" "PRIVATE_FLAGS;PUBLIC_FLAGS" ${ARGN})
target_compile_options(${target} PRIVATE
- -Wall -Wextra -pedantic
- -Wunreachable-code -Wwrite-strings
- -Wpointer-arith -Wcast-align
- -Wcast-qual -Wno-unused-parameter
- )
+ -Wall -Wextra -pedantic
+ -Wunreachable-code -Wwrite-strings
+ -Wpointer-arith -Wcast-align
+ -Wcast-qual -Wno-unused-parameter
+ )
- if(ARG_PRIVATE_FLAGS)
+ if (ARG_PRIVATE_FLAGS)
target_compile_options(${target} PRIVATE
- "${ARG_PRIVATE_FLAGS}"
- )
- endif()
+ "${ARG_PRIVATE_FLAGS}"
+ )
+ endif ()
- if(ARG_PUBLIC_FLAGS)
+ if (ARG_PUBLIC_FLAGS)
target_compile_options(${target} PUBLIC
- "${ARG_PUBLIC_FLAGS}"
- )
- endif()
+ "${ARG_PUBLIC_FLAGS}"
+ )
+ endif ()
endfunction()
function(target_project_compile_definitions target)
cmake_parse_arguments(ARG "" "" "PRIVATE_DEFS;PUBLIC_DEFS" ${ARGN})
- if(ARG_PRIVATE_DEFS)
+ if (ARG_PRIVATE_DEFS)
target_compile_definitions(${target} PRIVATE
- "${ARG_PRIVATE_DEFS}"
- )
- endif()
+ "${ARG_PRIVATE_DEFS}"
+ )
+ endif ()
- if(ARG_PUBLIC_DEFS)
+ if (ARG_PUBLIC_DEFS)
target_compile_definitions(${target} PUBLIC
- "${ARG_PUBLIC_DEFS}"
- )
- endif()
+ "${ARG_PUBLIC_DEFS}"
+ )
+ endif ()
endfunction()
diff --git a/cmake/modules/coverage-gcovr.cmake b/cmake/modules/coverage-gcovr.cmake
new file mode 100644
index 00000000..566953bb
--- /dev/null
+++ b/cmake/modules/coverage-gcovr.cmake
@@ -0,0 +1,14 @@
+if (ENABLE_CODE_COVERAGE AND CMAKE_CXX_COMPILER_ID MATCHES "GNU")
+ find_program(GCOVR_COMMAND gcovr)
+ if(GCOVR_COMMAND)
+ add_custom_target(gcovr-report
+ COMMAND ${GCOVR_COMMAND} -r ${PROJECT_SOURCE_DIR} -e ${PROJECT_BINARY_DIR} -s -j 4
+ WORKING_DIRECTORY ${PROJECT_BINARY_DIR}
+ COMMENT "Make coverage report"
+ USES_TERMINAL
+ )
+ else()
+ add_custom_target(gcovr-report
+ COMMAND ${CMAKE_COMMAND} -E echo "gcovr-report does nothing, no gcovr executable found.")
+ endif()
+endif()
\ No newline at end of file
diff --git a/cmake/modules/coverage-lcov.cmake b/cmake/modules/coverage-lcov.cmake
new file mode 100644
index 00000000..c6e05294
--- /dev/null
+++ b/cmake/modules/coverage-lcov.cmake
@@ -0,0 +1,58 @@
+find_program(LCOV_COMMAND lcov)
+find_program(GENHTML_COMMAND genhtml)
+
+if(LCOV_COMMAND-NOTFOUND OR GENHTML_COMMAND-NOTFOUND)
+ message(WARNING "lcov and genhtml command needed for coverage.")
+endif()
+
+add_custom_target(
+ lcov-clean
+ COMMAND ${LCOV_COMMAND} -d ${CMAKE_BINARY_DIR} -z
+)
+
+if(CMAKE_CXX_COMPILER_ID MATCHES "Clang")
+ find_program(LLVMCOV_COMMAND REQUIRED
+ NAMES llvm-cov-10 llvm-cov
+ )
+ # workaround lcov and clang --coverage have a version mismatch
+ set(GCOV_TOOL --gcov-tool ${CMAKE_BINARY_DIR}/scripts/llvm-gcov.sh)
+ # this autogenerated file in build/* causes error when using llvm-cov gcov (should be fine with gcc)
+ set(GCOV_WORKAROUND --exclude *mpi_interceptor_rt.c)
+endif()
+
+add_custom_target(
+ lcov-make
+ COMMAND ${LCOV_COMMAND} ${GCOV_TOOL} ${GCOV_WORKAROUND} --no-external -c -d ${CMAKE_BINARY_DIR} -b ${CMAKE_SOURCE_DIR} -o typeart.coverage
+ COMMAND ${LCOV_COMMAND} --remove typeart.coverage '${CMAKE_BINARY_DIR}/*' -o typeart.coverage
+)
+
+add_custom_target(
+ lcov-html
+ COMMAND ${GENHTML_COMMAND} -o ${TYPEART_PROFILE_DIR} typeart.coverage
+ DEPENDS lcov-make
+)
+
+
+function(make_lcov_target target)
+# add_custom_target(
+# lcov-clean-${target}
+# COMMAND lcov -d ${CMAKE_BINARY_DIR} -z
+# WORKING_DIRECTORY ${PROJECT_BINARY_DIR}
+# )
+
+ get_target_property(LCOV_TARGET_SOURCE_DIR ${target} SOURCE_DIR)
+
+ add_custom_target(
+ lcov-make-${target}
+ COMMAND ${LCOV_COMMAND} ${GCOV_TOOL} ${GCOV_WORKAROUND} --no-external -c -d ${CMAKE_BINARY_DIR} -b ${LCOV_TARGET_SOURCE_DIR} -o counter-${target}.pro
+ COMMAND ${LCOV_COMMAND} --remove counter-${target}.pro '${CMAKE_BINARY_DIR}/*' -o counter-${target}.pro
+ WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
+ )
+
+ add_custom_target(
+ lcov-html-${target}
+ COMMAND ${GENHTML_COMMAND} -o ${TYPEART_PROFILE_DIR} counter-${target}.pro
+ WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
+ DEPENDS lcov-make-${target}
+ )
+endfunction()
diff --git a/cmake/modules/coverage-llvm-cov.cmake b/cmake/modules/coverage-llvm-cov.cmake
new file mode 100644
index 00000000..512059f3
--- /dev/null
+++ b/cmake/modules/coverage-llvm-cov.cmake
@@ -0,0 +1,65 @@
+find_program(LLVM_PROFDATA_COMMAND
+ NAMES llvm-profdata-10 llvm-profdata
+)
+find_program(LLVMCOV_COMMAND
+ NAMES llvm-cov-10 llvm-cov
+)
+
+if(LLVM_PROFDATA_COMMAND-NOTFOUND OR LLVMCOV_COMMAND-NOTFOUND)
+ message(WARNING "llvm-cov stack needed for coverage.")
+endif()
+
+
+add_custom_target(
+ cov-merge
+ COMMAND ${LLVM_PROFDATA_COMMAND} merge -sparse -o code.pro *.profraw
+ DEPENDS ${target}
+ WORKING_DIRECTORY ${TYPEART_PROFILE_DIR}
+ DEPENDS ${target}
+)
+
+add_custom_target(
+ cov-all-report
+ COMMAND ${LLVMCOV_COMMAND} report `cat -s ta-binaries.txt` --instr-profile=code.pro
+ WORKING_DIRECTORY ${TYPEART_PROFILE_DIR}
+ DEPENDS cov-merge
+)
+
+add_custom_target(
+ cov-clean
+ COMMAND rm ta-binaries.txt
+ COMMAND rm *.pro
+ WORKING_DIRECTORY ${TYPEART_PROFILE_DIR}
+)
+
+add_custom_target(
+ cov-all-clean
+ COMMAND rm *
+ WORKING_DIRECTORY ${TYPEART_PROFILE_DIR}
+)
+
+function(make_llvm_cov_target target)
+ add_custom_target(
+ cov-binary-list-${target}
+ COMMAND ${CMAKE_COMMAND} -E echo "-object $" >> ta-binaries.txt
+ WORKING_DIRECTORY ${TYPEART_PROFILE_DIR}
+ DEPENDS ${target}
+ )
+
+ add_custom_target(
+ cov-merge-${target}
+ COMMAND ${LLVM_PROFDATA_COMMAND} merge -sparse -o code-${target}.pro *.profraw
+ DEPENDS ${target}
+ WORKING_DIRECTORY ${TYPEART_PROFILE_DIR}
+ DEPENDS ${target}
+ )
+
+ add_custom_target(
+ cov-report-${target}
+ COMMAND ${LLVMCOV_COMMAND} report -object $ --instr-profile=code-${target}.pro
+ WORKING_DIRECTORY ${TYPEART_PROFILE_DIR}
+ DEPENDS ${target} cov-merge-${target}
+ )
+
+ add_dependencies(cov-all-report cov-binary-list-${target})
+endfunction()
\ No newline at end of file
diff --git a/cmake/modules/coverage.cmake b/cmake/modules/coverage.cmake
new file mode 100644
index 00000000..6ab7b8a6
--- /dev/null
+++ b/cmake/modules/coverage.cmake
@@ -0,0 +1,40 @@
+set(TYPEART_PROFILE_DIR ${CMAKE_BINARY_DIR}/profiles)
+file(MAKE_DIRECTORY ${TYPEART_PROFILE_DIR})
+
+if(NOT ENABLE_LLVM_CODE_COVERAGE AND ENABLE_CODE_COVERAGE)
+ include(coverage-gcovr)
+ include(coverage-lcov)
+endif()
+
+if(ENABLE_LLVM_CODE_COVERAGE)
+ include(coverage-llvm-cov)
+endif()
+
+function(target_project_coverage_options target)
+ if (NOT ENABLE_LLVM_CODE_COVERAGE AND ENABLE_CODE_COVERAGE AND CMAKE_CXX_COMPILER_ID MATCHES "GNU|Clang")
+ target_compile_options(${target} PUBLIC
+ -O0
+ -g
+ --coverage
+ )
+ target_link_options(${target} PUBLIC
+ --coverage
+ )
+ make_lcov_target(${target})
+ endif ()
+ if (ENABLE_LLVM_CODE_COVERAGE AND CMAKE_CXX_COMPILER_ID MATCHES "Clang")
+ target_compile_options(${target} PUBLIC
+ -O0
+ -g
+ -fprofile-instr-generate
+ -fcoverage-mapping
+ )
+ target_link_options(${target} PUBLIC
+ -fprofile-instr-generate
+ )
+ make_llvm_cov_target(${target})
+ endif ()
+endfunction()
+
+
+
diff --git a/cmake/modules/llvm-lit.cmake b/cmake/modules/llvm-lit.cmake
index 546233e0..33c4ce66 100644
--- a/cmake/modules/llvm-lit.cmake
+++ b/cmake/modules/llvm-lit.cmake
@@ -30,4 +30,6 @@ if(NOT LIT_COMMAND_I)
endif()
endif()
+mark_as_advanced(path_to_llvm_lit)
+
message(STATUS "llvm lit command is set to ${LIT_COMMAND_I}")
\ No newline at end of file
diff --git a/externals/CMakeLists.txt b/externals/CMakeLists.txt
index 40f733b6..b1496cfe 100644
--- a/externals/CMakeLists.txt
+++ b/externals/CMakeLists.txt
@@ -1,10 +1,16 @@
include(CMakeDependentOption)
-set(EXTERNAL_DIR ${CMAKE_CURRENT_BINARY_DIR})
-# TODO may remove later
set(FETCHCONTENT_UPDATES_DISCONNECTED ON CACHE STRING "" FORCE)
-option(USE_BTREE "Enable usage of btree-backed map instead of std::map for the runtime." ON)
-CMAKE_DEPENDENT_OPTION(USE_ABSL "Enable usage of abseil's btree-backed map instead of std::map for the runtime." ON "NOT USE_BTREE" OFF)
+include(FetchContent)
+
+mark_as_advanced(
+ FETCHCONTENT_BASE_DIR
+ FETCHCONTENT_QUIET
+ FETCHCONTENT_FULLY_DISCONNECTED
+)
+
+option(USE_ABSL "Enable usage of abseil's btree-backed map instead of std::map for the runtime." ON )
+CMAKE_DEPENDENT_OPTION(USE_BTREE "Enable usage of btree-backed map instead of std::map for the runtime." ON "NOT USE_ABSL" ON)
if(USE_BTREE)
add_subdirectory(btree)
diff --git a/externals/abseil/CMakeLists.txt b/externals/abseil/CMakeLists.txt
index 7000d141..0b0b2248 100644
--- a/externals/abseil/CMakeLists.txt
+++ b/externals/abseil/CMakeLists.txt
@@ -1,12 +1,20 @@
-include(FetchContent)
FetchContent_Declare(
cpp-abseil
GIT_REPOSITORY https://github.com/abseil/abseil-cpp.git
)
-FetchContent_GetProperties(cpp-abseil)
-if(NOT cpp-abseil_POPULATED)
- FetchContent_Populate(cpp-abseil)
- add_subdirectory(${cpp-abseil_SOURCE_DIR} ${cpp-abseil_BINARY_DIR})
-# set_target_properties(absl::btree PROPERTIES IMPORTED_GLOBAL TRUE)
-endif()
+FetchContent_MakeAvailable(cpp-abseil)
+
+mark_as_advanced(
+ ABSL_ENABLE_INSTALL
+ ABSL_GOOGLETEST_DOWNLOAD_URL
+ ABSL_LOCAL_GOOGLETEST_DIR
+ ABSL_RUN_TESTS
+ ABSL_USE_EXTERNAL_GOOGLETEST
+ ABSL_USE_GOOGLETEST_HEAD
+)
+
+mark_as_advanced(
+ FETCHCONTENT_SOURCE_DIR_CPP-ABSEIL
+ FETCHCONTENT_UPDATES_DISCONNECTED_CPP-ABSEIL
+)
diff --git a/externals/btree/CMakeLists.txt b/externals/btree/CMakeLists.txt
index 15b8dc7a..ec73ce12 100644
--- a/externals/btree/CMakeLists.txt
+++ b/externals/btree/CMakeLists.txt
@@ -1,15 +1,20 @@
-include(FetchContent)
FetchContent_Declare(
cpp-btree
GIT_REPOSITORY https://github.com/ahueck/cpp-btree.git
)
FetchContent_GetProperties(cpp-btree)
+
if(NOT cpp-btree_POPULATED)
FetchContent_Populate(cpp-btree)
-
+
add_library(cppbtree INTERFACE IMPORTED GLOBAL)
target_include_directories(cppbtree SYSTEM INTERFACE ${cpp-btree_SOURCE_DIR})
#target_compile_features(cppbtree INTERFACE cxx_std_11 )
add_library(google::btree ALIAS cppbtree)
endif()
+
+mark_as_advanced(
+ FETCHCONTENT_SOURCE_DIR_CPP-BTREE
+ FETCHCONTENT_UPDATES_DISCONNECTED_CPP-BTREE
+)
\ No newline at end of file
diff --git a/lib/CMakeLists.txt b/lib/CMakeLists.txt
index a7461810..568774f4 100644
--- a/lib/CMakeLists.txt
+++ b/lib/CMakeLists.txt
@@ -1,61 +1,4 @@
-set(PROJECT_NAME typeart_llvm)
-set(TARGETS_EXPORT_NAME ${PROJECT_NAME}-targets)
-
-add_subdirectory(analysis)
-
-set(PASS_SOURCES
- TypeARTPass.cpp
- support/TypeUtil.cpp
- support/InstrumentationHelper.cpp
- TypeManager.cpp
-)
-
-make_llvm_module(typeartpass
- "${PASS_SOURCES}"
- LINK_LIBS
- typelib
- DEPENDS
- meminstfinderpass
- INCLUDE_DIRS
- ${CMAKE_CURRENT_SOURCE_DIR}
- ${PROJECT_SOURCE_DIR}/typelib
- ${PROJECT_SOURCE_DIR}/runtime
-)
-
-target_project_compile_options(typeartpass)
-target_project_compile_definitions(typeartpass
- PRIVATE_DEFS
- LOG_LEVEL=${LOG_LEVEL}
-)
-
-if(SHOW_STATS)
- target_project_compile_definitions(typeartpass
- PRIVATE_DEFS
- LLVM_ENABLE_STATS
- )
-endif()
-
-install(
- TARGETS typeartpass
- EXPORT ${TARGETS_EXPORT_NAME}
- LIBRARY DESTINATION lib
- ARCHIVE DESTINATION lib
-)
-
-# also handles subdir meminstfinder
-install(
- EXPORT ${TARGETS_EXPORT_NAME}
- NAMESPACE typeart::
- DESTINATION lib/cmake
-)
-
-configure_package_config_file(
- ${PROJECT_SOURCE_DIR}/cmake/Config.cmake.in
- ${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}Config.cmake
- INSTALL_DESTINATION lib/cmake
-)
-
-install(FILES
- ${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}Config.cmake
- DESTINATION lib/cmake
-)
+add_subdirectory(passes)
+add_subdirectory(typelib)
+add_subdirectory(runtime)
+add_subdirectory(mpi_interceptor)
\ No newline at end of file
diff --git a/lib/TypeARTPass.cpp b/lib/TypeARTPass.cpp
deleted file mode 100644
index 3e8e37d3..00000000
--- a/lib/TypeARTPass.cpp
+++ /dev/null
@@ -1,403 +0,0 @@
-#include "TypeARTPass.h"
-
-#include "RuntimeInterface.h"
-#include "TypeIO.h"
-#include "TypeInterface.h"
-#include "analysis/MemInstFinderPass.h"
-#include "support/Logger.h"
-#include "support/TypeUtil.h"
-#include "support/Util.h"
-
-#include "llvm/ADT/Statistic.h"
-#include "llvm/IR/IRBuilder.h"
-#include "llvm/IR/Instructions.h"
-#include "llvm/IR/Module.h"
-#include "llvm/Support/CommandLine.h"
-#include "llvm/Support/Format.h"
-#include "llvm/Transforms/Utils/BasicBlockUtils.h"
-#include "llvm/Transforms/Utils/CtorUtils.h"
-#include "llvm/Transforms/Utils/EscapeEnumerator.h"
-#include "llvm/Transforms/Utils/ModuleUtils.h"
-
-#include
-#include
-#include
-
-using namespace llvm;
-
-#define DEBUG_TYPE "typeart"
-
-namespace {
-static llvm::RegisterPass msp("typeart", "TypeArt type information", false, false);
-} // namespace
-
-static cl::opt ClTypeArtStats("typeart-stats", cl::desc("Show statistics for TypeArt type pass."), cl::Hidden,
- cl::init(false));
-static cl::opt ClIgnoreHeap("typeart-no-heap", cl::desc("Ignore heap allocation/free instruction."), cl::Hidden,
- cl::init(false));
-static cl::opt ClTypeArtAlloca("typeart-alloca", cl::desc("Track alloca instructions."), cl::Hidden,
- cl::init(false));
-
-static cl::opt ClTypeFile("typeart-outfile", cl::desc("Location of the generated type file."), cl::Hidden,
- cl::init("types.yaml"));
-
-STATISTIC(NumInstrumentedMallocs, "Number of instrumented mallocs");
-STATISTIC(NumInstrumentedFrees, "Number of instrumented frees");
-STATISTIC(NumInstrumentedAlloca, "Number of instrumented (stack) allocas");
-STATISTIC(NumInstrumentedGlobal, "Number of instrumented globals");
-
-namespace tu = typeart::util::type;
-
-namespace typeart {
-namespace pass {
-
-// Used by LLVM pass manager to identify passes in memory
-char TypeArtPass::ID = 0;
-
-// std::unique_ptr TypeArtPass::typeMapping = std::make_unique();
-
-TypeArtPass::TypeArtPass() : llvm::ModulePass(ID), typeManager(ClTypeFile.getValue()) {
- assert(!ClTypeFile.empty() && "Default type file not set");
- EnableStatistics();
-}
-
-void TypeArtPass::getAnalysisUsage(llvm::AnalysisUsage& info) const {
- info.addRequired();
-}
-
-bool TypeArtPass::doInitialization(Module& m) {
- instr.setModule(m);
-
- LOG_DEBUG("Propagating type infos.");
- if (typeManager.load()) {
- LOG_DEBUG("Existing type configuration successfully loaded from " << ClTypeFile.getValue());
- } else {
- LOG_DEBUG("No valid existing type configuration found: " << ClTypeFile.getValue());
- }
-
- return true;
-}
-
-bool TypeArtPass::runOnModule(Module& m) {
- bool globalInstro{false};
- if (ClIgnoreHeap) {
- declareInstrumentationFunctions(m);
-
- DataLayout dl(&m);
-
- auto& c = m.getContext();
-
- const auto instrumentGlobal = [&](auto* global, auto& IRB) {
- auto type = global->getValueType();
-
- unsigned numElements = 1;
- if (type->isArrayTy()) {
- numElements = tu::getArrayLengthFlattened(type);
- type = tu::getArrayElementType(type);
- }
-
- int typeId = typeManager.getOrRegisterType(type, dl);
- auto* typeIdConst = instr.getConstantFor(IType::type_id, typeId);
- auto* numElementsConst = instr.getConstantFor(IType::extent, numElements);
- auto globalPtr = IRB.CreateBitOrPointerCast(global, instr.getTypeFor(IType::ptr));
-
- LOG_DEBUG("Instrumenting global variable: " << util::dump(*global));
-
- IRB.CreateCall(typeart_alloc_global.f, ArrayRef{globalPtr, typeIdConst, numElementsConst});
- return true;
- };
-
- const auto makeCtorFunc = [&]() -> IRBuilder<> {
- auto ctorFunctionName = "__typeart_init_module_" + m.getSourceFileName();
-
- FunctionType* ctorType = FunctionType::get(llvm::Type::getVoidTy(c), false);
- Function* ctorFunction = Function::Create(ctorType, Function::PrivateLinkage, ctorFunctionName, &m);
-
- BasicBlock* entry = BasicBlock::Create(c, "entry", ctorFunction);
-
- llvm::appendToGlobalCtors(m, ctorFunction, 0, nullptr);
-
- IRBuilder<> IRB(entry);
- return IRB;
- };
-
- const auto& globalsList = getAnalysis().getModuleGlobals();
- if (!globalsList.empty()) {
- auto IRB = makeCtorFunc();
- auto instrGlobalCount = llvm::count_if(globalsList, [&](auto g) { return instrumentGlobal(g, IRB); });
- NumInstrumentedGlobal += instrGlobalCount;
- globalInstro = instrGlobalCount > 0;
- IRB.CreateRetVoid();
- }
- }
- const auto instrumentedF = llvm::count_if(m.functions(), [&](auto& f) { return runOnFunc(f); }) > 0;
- return instrumentedF || globalInstro;
-}
-
-bool TypeArtPass::runOnFunc(Function& f) {
- using namespace typeart;
-
- if (f.isDeclaration() || f.getName().startswith("__typeart")) {
- return false;
- }
-
- if (!getAnalysis().hasFunctionData(&f)) {
- LOG_WARNING("No allocation data could be retrieved for function: " << f.getName());
- return false;
- }
-
- LOG_DEBUG("Running on function: " << f.getName())
-
- // FIXME this is required when "PassManagerBuilder::EP_OptimizerLast" is used as the function (constant) pointer are
- // nullpointer/invalidated
- declareInstrumentationFunctions(*f.getParent());
-
- bool mod{false};
- // auto& c = f.getContext();
- DataLayout dl(f.getParent());
-
- llvm::SmallDenseMap allocCounts;
-
- const auto& fData = getAnalysis().getFunctionData(&f);
- const auto& listMalloc = fData.listMalloc;
- const auto& listAlloca = fData.listAlloca;
- const auto& listFree = fData.listFree;
-
- const auto instrumentMalloc = [&](const auto& malloc) -> bool {
- const auto mallocInst = malloc.call;
- BitCastInst* primaryBitcast = malloc.primary;
-
- // Number of bytes allocated
- auto mallocArg = mallocInst->getOperand(0);
- int typeId = typeManager.getOrRegisterType(mallocInst->getType()->getPointerElementType(),
- dl); // retrieveTypeID(tu::getVoidType(c));
- if (typeId == TA_UNKNOWN_TYPE) {
- LOG_ERROR("Unknown allocated type. Not instrumenting. " << util::dump(*mallocInst));
- return false;
- }
-
- // Number of bytes per element, 1 for void*
- unsigned typeSize = tu::getTypeSizeInBytes(mallocInst->getType()->getPointerElementType(), dl);
- auto insertBefore = mallocInst->getNextNode();
-
- // Use the first cast as the determining type (if there is any)
- if (primaryBitcast) {
- auto* dstPtrType = primaryBitcast->getDestTy()->getPointerElementType();
-
- typeSize = tu::getTypeSizeInBytes(dstPtrType, dl);
-
- // Resolve arrays
- // TODO: Write tests for this case
- if (dstPtrType->isArrayTy()) {
- dstPtrType = tu::getArrayElementType(dstPtrType);
- }
-
- typeId = typeManager.getOrRegisterType(dstPtrType, dl);
- if (typeId == TA_UNKNOWN_TYPE) {
- LOG_ERROR("Target type of casted allocation is unknown. Not instrumenting. " << util::dump(*mallocInst));
- LOG_ERROR("Cast: " << util::dump(*primaryBitcast));
- LOG_ERROR("Target type: " << util::dump(*dstPtrType));
- return false;
- }
- }
-
- IRBuilder<> IRB(insertBefore);
- auto* typeIdConst = instr.getConstantFor(IType::type_id, typeId);
- auto* typeSizeConst = instr.getConstantFor(IType::extent, typeSize);
- // Compute element count: count = numBytes / typeSize
- Value* elementCount = nullptr;
- if (malloc.kind == MemOpKind::MALLOC) {
- elementCount = IRB.CreateUDiv(mallocArg, typeSizeConst);
- } else if (malloc.kind == MemOpKind::CALLOC) {
- elementCount = mallocInst->getOperand(0); // get the element count in calloc call
- } else if (malloc.kind == MemOpKind::REALLOC) {
- auto mArg = mallocInst->getOperand(1);
- elementCount = IRB.CreateUDiv(mArg, typeSizeConst);
-
- IRBuilder<> FreeB(mallocInst);
- auto addrOp = mallocInst->getOperand(0);
- FreeB.CreateCall(typeart_free.f, ArrayRef{addrOp});
-
- } else {
- LOG_ERROR("Unknown malloc kind. Not instrumenting. " << util::dump(*mallocInst));
- return false;
- }
-
- IRB.CreateCall(typeart_alloc.f, ArrayRef{mallocInst, typeIdConst, elementCount});
-
- return true;
- };
-
- const auto instrumentFree = [&](const auto& free_inst) -> bool {
- // Pointer address:
- auto freeArg = free_inst->getOperand(0);
- IRBuilder<> IRB(free_inst->getNextNode());
- IRB.CreateCall(typeart_free.f, ArrayRef{freeArg});
-
- return true;
- };
-
- const auto instrumentAlloca = [&](const auto& allocaData) -> bool {
- auto alloca = allocaData.alloca;
- Type* elementType = alloca->getAllocatedType();
- Value* numElementsVal = nullptr;
- // The length can be specified statically through the array type or as a separate argument.
- // Both cases are handled here.
- if (allocaData.is_vla) {
- numElementsVal = alloca->getArraySize();
- // This should not happen in generated IR code
- assert(!elementType->isArrayTy() && "VLAs of array types are currently not supported.");
- } else {
- size_t arraySize = allocaData.arraySize;
- if (elementType->isArrayTy()) {
- arraySize = arraySize * tu::getArrayLengthFlattened(elementType);
- elementType = tu::getArrayElementType(elementType);
- }
- numElementsVal = instr.getConstantFor(IType::extent, arraySize);
- }
-
- IRBuilder<> IRB(alloca->getNextNode());
-
- // unsigned typeSize = tu::getTypeSizeInBytes(elementType, dl);
- int typeId = typeManager.getOrRegisterType(elementType, dl);
-
- if (typeId == TA_UNKNOWN_TYPE) {
- LOG_ERROR("Type is not supported: " << util::dump(*elementType));
- }
-
- auto* typeIdConst = instr.getConstantFor(IType::type_id, typeId);
- auto arrayPtr = IRB.CreateBitOrPointerCast(alloca, instr.getTypeFor(IType::ptr));
-
- IRB.CreateCall(typeart_alloc_stack.f, ArrayRef{arrayPtr, typeIdConst, numElementsVal});
-
- allocCounts[alloca->getParent()]++;
-
- ++NumInstrumentedAlloca;
- return true;
- };
-
- if (!ClIgnoreHeap) {
- // instrument collected calls of bb:
- for (const auto& malloc : listMalloc) {
- ++NumInstrumentedMallocs;
- mod |= instrumentMalloc(malloc);
- }
-
- for (auto* free : listFree) {
- ++NumInstrumentedFrees;
- mod |= instrumentFree(free);
- }
- }
- if (ClTypeArtAlloca) {
- const bool instrumented_alloca = std::count_if(listAlloca.begin(), listAlloca.end(), instrumentAlloca) > 0;
- mod |= instrumented_alloca;
-
- if (instrumented_alloca) {
- // LOG_DEBUG("Add alloca counter")
- // counter = 0 at beginning of function
- IRBuilder<> CBuilder(f.getEntryBlock().getFirstNonPHI());
- auto* counter = CBuilder.CreateAlloca(instr.getTypeFor(IType::stack_count), nullptr, "__ta_alloca_counter");
- CBuilder.CreateStore(instr.getConstantFor(IType::stack_count), counter);
-
- // In each basic block: counter =+ num_alloca (in BB)
- for (auto data : allocCounts) {
- IRBuilder<> IRB(data.first->getTerminator());
- auto* load_counter = IRB.CreateLoad(counter);
- Value* increment_counter = IRB.CreateAdd(instr.getConstantFor(IType::stack_count, data.second), load_counter);
- IRB.CreateStore(increment_counter, counter);
- }
-
- // Find return instructions:
- // if(counter > 0) call runtime for stack cleanup
- EscapeEnumerator ee(f);
- while (IRBuilder<>* irb = ee.Next()) {
- auto* I = &(*irb->GetInsertPoint());
-
- auto* counter_load = irb->CreateLoad(counter, "__ta_counter_load");
- auto* cond = irb->CreateICmpNE(counter_load, instr.getConstantFor(IType::stack_count), "__ta_cond");
- auto* then_term = SplitBlockAndInsertIfThen(cond, I, false);
- irb->SetInsertPoint(then_term);
- irb->CreateCall(typeart_leave_scope.f, ArrayRef{counter_load});
- }
- }
- } else {
- NumInstrumentedAlloca += listAlloca.size();
- }
-
- return mod;
-} // namespace pass
-
-bool TypeArtPass::doFinalization(Module&) {
- /*
- * Persist the accumulated type definition information for this module.
- */
- LOG_DEBUG("Writing type file to " << ClTypeFile.getValue());
-
- if (typeManager.store()) {
- LOG_DEBUG("Success!");
- } else {
- LOG_FATAL("Failed writing type config to " << ClTypeFile.getValue());
- }
- if (ClTypeArtStats && AreStatisticsEnabled()) {
- auto& out = llvm::errs();
- printStats(out);
- }
- return false;
-}
-
-void TypeArtPass::declareInstrumentationFunctions(Module& m) {
- // Remove this return if problems come up during compilation
- if (typeart_alloc_global.f != nullptr && typeart_alloc_stack.f != nullptr && typeart_alloc.f != nullptr &&
- typeart_free.f != nullptr && typeart_leave_scope.f != nullptr) {
- return;
- }
-
- auto alloc_arg_types = instr.make_parameters(IType::ptr, IType::type_id, IType::extent);
- auto free_arg_types = instr.make_parameters(IType::ptr);
- auto leavescope_arg_types = instr.make_parameters(IType::stack_count);
-
- typeart_alloc.f = instr.make_function(typeart_alloc.name, alloc_arg_types);
- typeart_alloc_stack.f = instr.make_function(typeart_alloc_stack.name, alloc_arg_types);
- typeart_alloc_global.f = instr.make_function(typeart_alloc_global.name, alloc_arg_types);
- typeart_free.f = instr.make_function(typeart_free.name, free_arg_types);
- typeart_leave_scope.f = instr.make_function(typeart_leave_scope.name, leavescope_arg_types);
-}
-
-void TypeArtPass::printStats(llvm::raw_ostream& out) {
- const unsigned max_string{12u};
- const unsigned max_val{5u};
- std::string line(22, '-');
- line += "\n";
- const auto make_format = [&](const char* desc, const auto val) {
- return format("%-*s: %*u\n", max_string, desc, max_val, val);
- };
-
- out << line;
- out << " TypeArtPass\n";
- out << line;
- out << "Heap Memory\n";
- out << line;
- out << make_format("Malloc", NumInstrumentedMallocs.getValue());
- out << make_format("Free", NumInstrumentedFrees.getValue());
- out << line;
- out << "Stack Memory\n";
- out << line;
- out << make_format("Alloca", NumInstrumentedAlloca.getValue());
- out << line;
- out << "Global Memory\n";
- out << line;
- out << make_format("Global", NumInstrumentedGlobal.getValue());
- out << line;
- out.flush();
-}
-
-} // namespace pass
-} // namespace typeart
-
-#include "llvm/IR/LegacyPassManager.h"
-#include "llvm/Transforms/IPO/PassManagerBuilder.h"
-
-static void registerClangPass(const llvm::PassManagerBuilder&, llvm::legacy::PassManagerBase& PM) {
- PM.add(new typeart::pass::TypeArtPass());
-}
-static RegisterStandardPasses RegisterClangPass(PassManagerBuilder::EP_OptimizerLast, registerClangPass);
diff --git a/lib/analysis/MemInstFinderPass.cpp b/lib/analysis/MemInstFinderPass.cpp
deleted file mode 100644
index 386b9cb4..00000000
--- a/lib/analysis/MemInstFinderPass.cpp
+++ /dev/null
@@ -1,581 +0,0 @@
-/*
- * MemInstFinderPass.cpp
- *
- * Created on: Jun 3, 2018
- * Author: ahueck
- */
-
-#include "MemInstFinderPass.h"
-
-#include "MemOpVisitor.h"
-#include "support/Logger.h"
-#include "support/TypeUtil.h"
-#include "support/Util.h"
-
-#include "llvm/ADT/Statistic.h"
-#include "llvm/IR/Instructions.h"
-#include "llvm/Support/CommandLine.h"
-#include "llvm/Support/Format.h"
-
-using namespace llvm;
-
-#define DEBUG_TYPE "meminstanalysis"
-
-namespace {
-static RegisterPass X("mem-inst-finder", // pass option
- "Find heap and stack allocations in a program.", // pass description
- true, // does not modify the CFG
- true // and it's an analysis
-);
-} // namespace
-
-static cl::opt ClFilterNonArrayAlloca("alloca-array-only", cl::desc("Only use alloca instructions of arrays."),
- cl::Hidden, cl::init(true));
-
-static cl::opt ClFilterMallocAllocPair("malloc-store-filter",
- cl::desc("Filter allocs that get a store from a heap alloc."), cl::Hidden,
- cl::init(false));
-
-static cl::opt ClCallFilter("call-filter",
- cl::desc("Filter alloca instructions that are passed to specific calls."), cl::Hidden,
- cl::init(false));
-
-static cl::opt ClCallFilterDeep("call-filter-deep",
- cl::desc("If the CallFilter matches, we look if the value is passed as a void*."),
- cl::Hidden, cl::init(false));
-
-static cl::opt ClCallFilterGlob("call-filter-str", cl::desc("Filter alloca instructions based on string."),
- cl::Hidden, cl::init("MPI_*"));
-
-static cl::opt ClFilterGlobal("filter-globals", cl::desc("Filter globals of a module."), cl::Hidden,
- cl::init(true));
-
-STATISTIC(NumDetectedHeap, "Number of detected heap allocs");
-STATISTIC(NumFilteredDetectedHeap, "Number of filtered heap allocs");
-STATISTIC(NumDetectedAllocs, "Number of detected allocs");
-STATISTIC(NumCallFilteredAllocs, "Number of call filtered allocs");
-STATISTIC(NumFilteredMallocAllocs, "Number of filtered malloc-related allocs");
-STATISTIC(NumFilteredNonArrayAllocs, "Number of call filtered allocs");
-STATISTIC(NumDetectedGlobals, "Number of detected globals");
-STATISTIC(NumFilteredGlobals, "Number of filtered globals");
-STATISTIC(NumCallFilteredGlobals, "Number of filtered globals");
-
-namespace typeart {
-
-using namespace finder;
-
-namespace filter {
-
-class CallFilter::FilterImpl {
- const std::string call_regex;
- bool malloc_mode{false};
- llvm::Function* start_f{nullptr};
- int depth{0};
-
- public:
- explicit FilterImpl(const std::string& glob) : call_regex(util::glob2regex(glob)) {
- }
-
- void setMode(bool search_malloc) {
- malloc_mode = search_malloc;
- }
-
- void setStartingFunction(llvm::Function* start) {
- start_f = start;
- depth = 0;
- }
-
- bool filter(Value* in) {
- if (in == nullptr) {
- LOG_DEBUG("Called with nullptr");
- return false;
- }
-
- if (depth == 15) {
- return false;
- }
-
- const auto match = [&](auto callee) -> bool {
- const auto name = FilterImpl::getName(callee);
- return util::regex_matches(call_regex, name);
- };
-
- llvm::SmallPtrSet visited_set;
- llvm::SmallVector working_set;
- llvm::SmallVector working_set_calls;
-
- const auto addToWork = [&visited_set, &working_set](auto vals) {
- for (auto v : vals) {
- if (visited_set.find(v) == visited_set.end()) {
- working_set.push_back(v);
- visited_set.insert(v);
- }
- }
- };
-
- const auto peek = [&working_set]() -> Value* {
- auto user_iter = working_set.end() - 1;
- working_set.erase(user_iter);
- return *user_iter;
- };
-
- // Seed working set with users of value (e.g., our AllocaInst)
- addToWork(in->users());
-
- // Search through all users of users of .... (e.g., our AllocaInst)
- while (!working_set.empty()) {
- auto val = peek();
-
- // If we encounter a callsite, we want to analyze later, or quit in case we have a regex match
- CallSite c(val);
- if (c.isCall()) {
- const auto callee = c.getCalledFunction();
- const bool indirect_call = callee == nullptr;
-
- if (indirect_call) {
- LOG_DEBUG("Found an indirect call, not filtering alloca: " << util::dump(*val));
- return false; // Indirect calls might contain critical function calls.
- }
-
- const bool is_decl = callee->isDeclaration();
- // FIXME the MPI calls are all hitting this branch (obviously)
- if (is_decl) {
- LOG_DEBUG("Found call with declaration only. Call: " << util::dump(*c.getInstruction()));
- if (c.getIntrinsicID() == Intrinsic::not_intrinsic /*Intrinsic::ID::not_intrinsic*/) {
- if (ClCallFilterDeep && match(callee) && shouldContinue(c, in)) {
- continue;
- }
- return false;
- } else {
- LOG_DEBUG("Call is an intrinsic. Continue analyzing...")
- continue;
- }
- }
-
- if (match(callee)) {
- LOG_DEBUG("Found a call. Call: " << util::dump(*c.getInstruction()));
- if (ClCallFilterDeep && shouldContinue(c, in)) {
- continue;
- }
- return false;
- }
-
- working_set_calls.push_back(c);
- // Caveat: below at the end of the loop, we add users of the function call to the search even though it might be
- // a simple "sink" for the alloca we analyse
- } else if (auto store = llvm::dyn_cast(val)) {
- // If we encounter a store, we follow the store target pointer.
- // More inclusive than strictly necessary in some cases.
- LOG_DEBUG("Store found: " << util::dump(*store)
- << " Store target has users: " << util::dump(store->getPointerOperand()->users()));
- auto store_target = store->getPointerOperand();
- // FIXME here we check store operand, if target is another alloca, we already track that?:
- // Note: if we apply this to malloc filtering, this might become problematic?
- if (!malloc_mode && llvm::isa(store_target)) {
- LOG_DEBUG("Target is alloca, skipping!");
- } else {
- addToWork(store_target->users());
- }
- continue;
- }
- // cont. our search
- addToWork(val->users());
- }
- ++depth;
- return std::all_of(working_set_calls.begin(), working_set_calls.end(), [&](CallSite c) { return filter(c, in); });
- }
-
- private:
- bool filter(CallSite& csite, Value* in) {
- const auto analyse_arg = [&](auto& csite, auto argNum) -> bool {
- Argument& the_arg = *(csite.getCalledFunction()->arg_begin() + argNum);
- LOG_DEBUG("Calling filter with inst of argument: " << util::dump(the_arg));
- const bool filter_arg = filter(&the_arg);
- LOG_DEBUG("Should filter? : " << filter_arg);
- return filter_arg;
- };
-
- LOG_DEBUG("Analyzing function call " << csite.getCalledFunction()->getName());
-
- if (csite.getCalledFunction() == start_f) {
- return true;
- }
-
- // this only works if we can correlate alloca with argument:
- const auto pos = std::find_if(csite.arg_begin(), csite.arg_end(),
- [&in](const Use& arg_use) -> bool { return arg_use.get() == in; });
- // auto pos = csite.arg_end();
- if (pos != csite.arg_end()) {
- const auto argNum = std::distance(csite.arg_begin(), pos);
- LOG_DEBUG("Found exact position: " << argNum);
- return analyse_arg(csite, argNum);
- } else {
- LOG_DEBUG("Analyze all args, cannot correlate alloca with arg.");
- return std::all_of(csite.arg_begin(), csite.arg_end(), [&csite, &analyse_arg](const Use& arg_use) {
- auto argNum = csite.getArgumentNo(&arg_use);
- return analyse_arg(csite, argNum);
- });
- }
-
- return true;
- }
-
- bool filter(Argument* arg) {
- for (auto* user : arg->users()) {
- LOG_DEBUG("Looking at arg user " << util::dump(*user));
- // This code is for non mem2reg code (i.e., where the argument is stored to a local alloca):
- if (auto store = llvm::dyn_cast(user)) {
- // if (auto* alloca = llvm::dyn_cast(store->getPointerOperand())) {
- // LOG_DEBUG("Argument is a store inst and the operand is alloca");
- return filter(store->getPointerOperand());
- // }
- }
- }
- return filter(llvm::dyn_cast(arg));
- }
-
- bool shouldContinue(CallSite c, Value* in) const {
- LOG_DEBUG("Found a name match, analyzing closer...");
- const auto is_void_ptr = [](Type* type) {
- return type->isPointerTy() && type->getPointerElementType()->isIntegerTy(8);
- };
- const auto arg_pos = llvm::find_if(c.args(), [&in](const Use& arg_use) -> bool { return arg_use.get() == in; });
- if (arg_pos == c.arg_end()) {
- // we had no direct correlation for the arg position
- // Now checking if void* is passed, if not we can potentially filter!
- auto count_void_ptr = llvm::count_if(c.args(), [&is_void_ptr](const auto& arg) {
- const auto type = arg->getType();
- return is_void_ptr(type);
- });
- if (count_void_ptr > 0) {
- LOG_DEBUG("Call takes a void*, filtering.");
- return false;
- }
- LOG_DEBUG("Call has no void* argument");
- } else {
- // We have an arg_pos match
- const auto argNum = std::distance(c.arg_begin(), arg_pos);
- Argument& the_arg = *(c.getCalledFunction()->arg_begin() + argNum);
- auto type = the_arg.getType();
- if (is_void_ptr(type)) {
- LOG_DEBUG("Call arg is a void*, filtering.");
- return false;
- }
- LOG_DEBUG("Value* in is not passed as void ptr");
- }
- LOG_DEBUG("No filter necessary for this call, continue.");
- return true;
- }
-
- static inline std::string getName(const Function* f) {
- auto name = f->getName();
- // FIXME figure out if we need to demangle, i.e., source is .c or .cpp
- const auto f_name = util::demangle(name);
- if (f_name != "") {
- name = f_name;
- }
-
- return name;
- }
-};
-
-CallFilter::CallFilter(const std::string& glob) : fImpl{std::make_unique(glob)} {
-}
-
-bool CallFilter::operator()(AllocaInst* in) {
- LOG_DEBUG("Analyzing value: " << util::dump(*in));
- fImpl->setMode(/*search mallocs = */ false);
- fImpl->setStartingFunction(in->getParent()->getParent());
- const auto filter_ = fImpl->filter(in);
- if (filter_) {
- LOG_DEBUG("Filtering value: " << util::dump(*in) << "\n");
- } else {
- LOG_DEBUG("Keeping value: " << util::dump(*in) << "\n");
- }
- return filter_;
-}
-
-bool CallFilter::operator()(GlobalValue* g) {
- LOG_DEBUG("Analyzing value: " << util::dump(*g));
- fImpl->setMode(/*search mallocs = */ false);
- fImpl->setStartingFunction(nullptr);
- const auto filter_ = fImpl->filter(g);
- if (filter_) {
- LOG_DEBUG("Filtering value: " << util::dump(*g) << "\n");
- } else {
- LOG_DEBUG("Keeping value: " << util::dump(*g) << "\n");
- }
- return filter_;
-}
-
-CallFilter& CallFilter::operator=(CallFilter&&) noexcept = default;
-
-CallFilter::~CallFilter() = default;
-
-} // namespace filter
-
-char MemInstFinderPass::ID = 0;
-
-MemInstFinderPass::MemInstFinderPass() : llvm::ModulePass(ID), mOpsCollector(), filter(ClCallFilterGlob.getValue()) {
-}
-
-void MemInstFinderPass::getAnalysisUsage(llvm::AnalysisUsage& info) const {
- info.setPreservesAll();
-}
-
-bool MemInstFinderPass::runOnModule(Module& m) {
- mOpsCollector.visitModuleGlobals(m);
- auto& globals = mOpsCollector.listGlobals;
- NumDetectedGlobals += globals.size();
- if (ClFilterGlobal && !ClFilterNonArrayAlloca) {
- globals.erase(
- llvm::remove_if(
- globals,
- [&](const auto g) {
- const auto name = g->getName();
- if (name.startswith("llvm.") || name.startswith("__llvm_gcov") || name.startswith("__llvm_gcda")) {
- // 2nd and 3rd check: Check if the global is private gcov data.
- return true;
- }
-
- if (g->hasInitializer()) {
- auto* ini = g->getInitializer();
- StringRef ini_name = util::dump(*ini);
-
- if (ini_name.contains("std::ios_base::Init")) {
- return true;
- }
- }
- // if (!g->hasInitializer()) {
- // return true;
- // }
-
- if (g->hasSection()) {
- StringRef Section = g->getSection();
-
- // Globals from llvm.metadata aren't emitted, do not instrument them.
- if (Section == "llvm.metadata") {
- return true;
- }
- // Do not instrument globals from special LLVM sections.
- if (Section.find("__llvm") != StringRef::npos || Section.find("__LLVM") != StringRef::npos) {
- return true;
- }
- // Check if the global is in the PGO counters section.
- // auto OF = Triple(m.getTargetTriple()).getObjectFormat();
- // if (Section.endswith(getInstrProfSectionName(IPSK_cnts, OF,
- // /*AddSegmentInfo=*/false))) {
- // return true;
- // }
- }
-
- if (g->getLinkage() == GlobalValue::ExternalLinkage || g->getLinkage() == GlobalValue::PrivateLinkage) {
- return true;
- }
-
- Type* t = g->getValueType();
- if (!t->isSized()) {
- return true;
- }
-
- if (t->isArrayTy()) {
- t = t->getArrayElementType();
- }
- if (auto structType = dyn_cast(t)) {
- if (structType->isOpaque()) {
- LOG_DEBUG("Encountered opaque struct " << t->getStructName() << " - skipping...");
- return true;
- }
- }
- return false;
- }),
- globals.end());
-
- const auto beforeCallFilter = globals.size();
- NumFilteredGlobals = NumDetectedGlobals - beforeCallFilter;
-
- globals.erase(llvm::remove_if(globals, [&](const auto g) { return filter(g); }), globals.end());
-
- NumCallFilteredGlobals = beforeCallFilter - globals.size();
- NumFilteredGlobals += NumCallFilteredGlobals;
- }
-
- return llvm::count_if(m.functions(), [&](auto& f) { return runOnFunc(f); }) > 0;
-} // namespace typeart
-
-bool MemInstFinderPass::runOnFunc(llvm::Function& f) {
- if (f.isDeclaration() || f.getName().startswith("__typeart")) {
- return false;
- }
-
- mOpsCollector.visit(f);
-
- LOG_DEBUG("Running on function: " << f.getName())
-
- const auto checkAmbigiousMalloc = [](const MallocData& mallocData) {
- auto primaryBitcast = mallocData.primary;
- if (primaryBitcast) {
- const auto& bitcasts = mallocData.bitcasts;
- std::for_each(bitcasts.begin(), bitcasts.end(), [&](auto bitcastInst) {
- if (bitcastInst != primaryBitcast && (!typeart::util::type::isVoidPtr(bitcastInst->getDestTy()) &&
- primaryBitcast->getDestTy() != bitcastInst->getDestTy())) {
- // Second non-void* bitcast detected - semantics unclear
- LOG_WARNING("Encountered ambiguous pointer type in allocation: " << util::dump(*(mallocData.call)));
- LOG_WARNING(" Primary cast: " << util::dump(*primaryBitcast));
- LOG_WARNING(" Secondary cast: " << util::dump(*bitcastInst));
- }
- });
- }
- };
-
- NumDetectedAllocs += mOpsCollector.listAlloca.size();
-
- if (ClFilterNonArrayAlloca) {
- auto& allocs = mOpsCollector.listAlloca;
- allocs.erase(llvm::remove_if(allocs,
- [&](const auto& data) {
- if (!data.alloca->getAllocatedType()->isArrayTy() && data.arraySize == 1) {
- ++NumFilteredNonArrayAllocs;
- return true;
- }
- return false;
- }),
- allocs.end());
- }
-
- if (ClFilterMallocAllocPair) {
- auto& allocs = mOpsCollector.listAlloca;
- auto& mlist = mOpsCollector.listMalloc;
-
- const auto filterMallocAllocPairing = [&mlist](const auto alloc) {
- // Only look for the direct users of the alloc:
- // TODO is a deeper analysis required?
- for (auto inst : alloc->users()) {
- if (StoreInst* store = dyn_cast(inst)) {
- const auto source = store->getValueOperand();
- if (isa(source)) {
- for (auto& mdata : mlist) {
- // is it a bitcast we already collected? if yes, we can filter the alloc
- return std::any_of(mdata.bitcasts.begin(), mdata.bitcasts.end(),
- [&source](const auto bcast) { return bcast == source; });
- }
- } else if (isa(source)) {
- return std::any_of(mlist.begin(), mlist.end(),
- [&source](const auto& mdata) { return mdata.call == source; });
- }
- }
- }
- return false;
- };
-
- allocs.erase(llvm::remove_if(allocs,
- [&](const auto& data) {
- if (filterMallocAllocPairing(data.alloca)) {
- ++NumFilteredMallocAllocs;
- return true;
- }
- return false;
- }),
- allocs.end());
- }
-
- if (ClCallFilter) {
- auto& allocs = mOpsCollector.listAlloca;
- allocs.erase(llvm::remove_if(allocs,
- [&](const auto& data) {
- if (filter(data.alloca)) {
- ++NumCallFilteredAllocs;
- return true;
- }
- return false;
- }),
- allocs.end());
- // LOG_DEBUG(allocs.size() << " allocas to instrument : " << util::dump(allocs));
- }
-
- auto& mallocs = mOpsCollector.listMalloc;
- NumDetectedHeap += mallocs.size();
-
- for (const auto& mallocData : mallocs) {
- checkAmbigiousMalloc(mallocData);
- }
-
- FunctionData d{mOpsCollector.listMalloc, mOpsCollector.listFree, mOpsCollector.listAlloca};
- functionMap[&f] = d;
-
- mOpsCollector.clear();
-
- return false;
-} // namespace typeart
-
-bool MemInstFinderPass::doFinalization(llvm::Module&) {
- if (AreStatisticsEnabled()) {
- auto& out = llvm::errs();
- printStats(out);
- }
- return false;
-}
-
-void MemInstFinderPass::printStats(llvm::raw_ostream& out) {
- const unsigned max_string{28u};
- const unsigned max_val{5u};
- std::string line(42, '-');
- line += "\n";
- const auto make_format = [&](const char* desc, const auto val) {
- return format("%-*s: %*.1f\n", max_string, desc, max_val, val);
- };
-
- auto all_stack = double(NumDetectedAllocs.getValue());
- auto nonarray_stack = double(NumFilteredNonArrayAllocs.getValue());
- auto malloc_alloc_stack = double(NumFilteredMallocAllocs.getValue());
- auto call_filter_stack = double(NumCallFilteredAllocs.getValue());
-
- out << line;
- out << " MemInstFinderPass\n";
- out << line;
- out << "Heap Memory\n";
- out << line;
- out << make_format("Heap alloc", double(NumDetectedHeap.getValue()));
- out << make_format(
- "% call filtered",
- (double(NumFilteredDetectedHeap.getValue()) / std::max(1.0, double(NumDetectedHeap.getValue()))) * 100.0);
- out << line;
- out << "Stack Memory\n";
- out << line;
- out << make_format("Alloca", all_stack);
- out << make_format("% non array filtered", (nonarray_stack / std::max(1.0, all_stack)) * 100.0);
- out << make_format("% malloc-alloc filtered",
- (malloc_alloc_stack / std::max(1.0, all_stack - nonarray_stack)) * 100.0);
- out << make_format("% call filtered",
- (call_filter_stack / std::max(1.0, all_stack - nonarray_stack - malloc_alloc_stack)) * 100.0);
- out << line;
- out << "Global Memory\n";
- out << line;
- out << make_format("Global", double(NumDetectedGlobals.getValue()));
- out << make_format("Global total filtered", double(NumFilteredGlobals.getValue()));
- out << make_format("Global Call Filtered", double(NumCallFilteredGlobals.getValue()));
- out << make_format(
- "% global call filtered",
- (double(NumCallFilteredGlobals.getValue()) / std::max(1.0, double(NumDetectedGlobals.getValue()))) * 100.0);
- out << make_format(
- "% global filtered",
- (double(NumFilteredGlobals.getValue()) / std::max(1.0, double(NumDetectedGlobals.getValue()))) * 100.0);
- out << line;
- out.flush();
-}
-
-bool MemInstFinderPass::hasFunctionData(Function* f) const {
- auto iter = functionMap.find(f);
- return iter != functionMap.end();
-}
-
-const FunctionData& MemInstFinderPass::getFunctionData(Function* f) const {
- auto iter = functionMap.find(f);
- return iter->second;
-}
-
-const llvm::SmallVector& MemInstFinderPass::getModuleGlobals() const {
- return mOpsCollector.listGlobals;
-}
-
-} // namespace typeart
diff --git a/lib/analysis/MemOpVisitor.cpp b/lib/analysis/MemOpVisitor.cpp
deleted file mode 100644
index 89779e57..00000000
--- a/lib/analysis/MemOpVisitor.cpp
+++ /dev/null
@@ -1,122 +0,0 @@
-/*
- * MemOpVisitor.cpp
- *
- * Created on: Jan 3, 2018
- * Author: ahueck
- */
-
-#include "MemOpVisitor.h"
-
-#include "support/Logger.h"
-#include "support/TypeUtil.h"
-#include "support/Util.h"
-
-#include
-
-namespace typeart {
-namespace finder {
-using namespace llvm;
-
-MemOpVisitor::MemOpVisitor() = default;
-
-void MemOpVisitor::visitModuleGlobals(Module& m) {
- for (auto& g : m.globals()) {
- listGlobals.push_back(&g);
- }
-}
-
-void MemOpVisitor::visitCallInst(llvm::CallInst& ci) {
- const auto isInSet = [&](const auto& fMap) -> llvm::Optional {
- const auto* f = ci.getCalledFunction();
- if (!f) {
- // TODO handle calls through, e.g., function pointers? - seems infeasible
- LOG_INFO("Encountered indirect call, skipping.");
- return None;
- }
- const auto name = f->getName().str();
- const auto res = fMap.find(name);
- if (res != fMap.end()) {
- return {(*res).second};
- }
- return None;
- };
-
- if (auto val = isInSet(allocMap)) {
- visitMallocLike(ci, val.getValue());
- } else if (auto val = isInSet(deallocMap)) {
- visitFreeLike(ci, val.getValue());
- }
-}
-
-void MemOpVisitor::visitMallocLike(llvm::CallInst& ci, MemOpKind k) {
- // LOG_DEBUG("Found malloc-like: " << ci.getCalledFunction()->getName());
-
- SmallPtrSet bcasts;
-
- for (auto user : ci.users()) {
- // Simple case: Pointer is immediately casted
- if (auto inst = dyn_cast(user)) {
- bcasts.insert(inst);
- }
- // Pointer is first stored, then loaded and subsequently casted
- if (auto storeInst = dyn_cast(user)) {
- auto storeAddr = storeInst->getPointerOperand();
- for (auto storeUser : storeAddr->users()) { // TODO: Ensure that load occurs ofter store?
- if (auto loadInst = dyn_cast(storeUser)) {
- for (auto loadUser : loadInst->users()) {
- if (auto bcastInst = dyn_cast(loadUser)) {
- bcasts.insert(bcastInst);
- }
- }
- }
- }
- }
- }
-
- BitCastInst* primaryBitcast{nullptr};
- std::for_each(bcasts.begin(), bcasts.end(), [&](auto bcast) {
- if (!util::type::isVoidPtr(bcast->getDestTy())) {
- primaryBitcast = bcast;
- }
- });
-
- // LOG_DEBUG(" >> number of bitcasts found: " << bcasts.size());
-
- listMalloc.push_back(MallocData{&ci, primaryBitcast, bcasts, k});
-}
-
-void MemOpVisitor::visitFreeLike(llvm::CallInst& ci, MemOpKind) {
- // LOG_DEBUG(ci.getCalledFunction()->getName());
-
- listFree.insert(&ci);
-}
-
-// void MemOpVisitor::visitIntrinsicInst(llvm::IntrinsicInst& ii) {
-//
-//}
-
-void MemOpVisitor::visitAllocaInst(llvm::AllocaInst& ai) {
- // LOG_DEBUG("Found alloca " << ai);
- Value* arraySizeOperand = ai.getArraySize();
- size_t arraySize{0}; // FIXME avoid using int (
- bool is_vla{false};
- if (auto arraySizeConst = llvm::dyn_cast(arraySizeOperand)) {
- arraySize = arraySizeConst->getZExtValue();
- } else {
- is_vla = true;
- }
-
- listAlloca.push_back({&ai, arraySize, is_vla});
- // LOG_DEBUG("Alloca: " << util::dump(ai) << " -> lifetime marker: " << util::dump(lifetimes));
-} // namespace typeart
-
-void MemOpVisitor::clear() {
- listAlloca.clear();
- listMalloc.clear();
- listFree.clear();
-}
-
-MemOpVisitor::~MemOpVisitor() = default;
-
-} // namespace finder
-} // namespace typeart
diff --git a/lib/analysis/MemOpVisitor.h b/lib/analysis/MemOpVisitor.h
deleted file mode 100644
index 9801a46f..00000000
--- a/lib/analysis/MemOpVisitor.h
+++ /dev/null
@@ -1,67 +0,0 @@
-/*
- * MemOpVisitor.h
- *
- * Created on: Jan 3, 2018
- * Author: ahueck
- */
-
-#ifndef LIB_MEMOPVISITOR_H_
-#define LIB_MEMOPVISITOR_H_
-
-#include "llvm/IR/InstVisitor.h"
-
-#include
-
-namespace typeart {
-
-enum class MemOpKind { MALLOC, CALLOC, REALLOC, FREE };
-struct MallocData {
- llvm::CallInst* call{nullptr};
- llvm::BitCastInst* primary{nullptr}; // Non-null if non (void*) cast exists
- llvm::SmallPtrSet bitcasts;
- MemOpKind kind;
-};
-
-struct AllocaData {
- llvm::AllocaInst* alloca{nullptr};
- size_t arraySize;
- bool is_vla{false};
-};
-
-namespace finder {
-
-struct MemOpVisitor : public llvm::InstVisitor {
- MemOpVisitor();
- void clear();
- void visitModuleGlobals(llvm::Module& m);
- void visitCallInst(llvm::CallInst& ci);
- void visitMallocLike(llvm::CallInst& ci, MemOpKind k);
- void visitFreeLike(llvm::CallInst& ci, MemOpKind k);
- // void visitIntrinsicInst(llvm::IntrinsicInst& ii);
- void visitAllocaInst(llvm::AllocaInst& ai);
- virtual ~MemOpVisitor();
-
- llvm::SmallVector listGlobals;
- llvm::SmallVector listMalloc;
- llvm::SmallPtrSet listFree;
- llvm::SmallVector listAlloca;
-
- private:
- // clang-format off
- const std::map allocMap{{"malloc", MemOpKind::MALLOC},
- {"_Znwm", MemOpKind::MALLOC}, /*new*/
- {"_Znam", MemOpKind::MALLOC}, /*new[]*/
- {"calloc", MemOpKind::CALLOC},
- {"realloc", MemOpKind::REALLOC}
- };
- const std::map deallocMap{{"free", MemOpKind::FREE},
- {"_ZdlPv", MemOpKind::FREE}, /*delete*/
- {"_ZdaPv", MemOpKind::FREE} /*delete[]*/
- };
- // clang-format on
-};
-
-} // namespace finder
-} // namespace typeart
-
-#endif /* LIB_MEMOPVISITOR_H_ */
diff --git a/test/mpi_interceptor/CMakeLists.txt b/lib/mpi_interceptor/CMakeLists.txt
similarity index 96%
rename from test/mpi_interceptor/CMakeLists.txt
rename to lib/mpi_interceptor/CMakeLists.txt
index 145a8025..514cb887 100644
--- a/test/mpi_interceptor/CMakeLists.txt
+++ b/lib/mpi_interceptor/CMakeLists.txt
@@ -23,7 +23,7 @@ if(${MPI_C_FOUND} AND ${PYTHONINTERP_FOUND})
target_include_directories(interceptor-rt
PUBLIC
- ${PROJECT_SOURCE_DIR}/runtime
+ ${PROJECT_SOURCE_DIR}/lib
${CMAKE_CURRENT_SOURCE_DIR}
)
diff --git a/test/mpi_interceptor/InterceptorFunctions.h b/lib/mpi_interceptor/InterceptorFunctions.h
similarity index 91%
rename from test/mpi_interceptor/InterceptorFunctions.h
rename to lib/mpi_interceptor/InterceptorFunctions.h
index 1a5d8a1d..78a21b56 100644
--- a/test/mpi_interceptor/InterceptorFunctions.h
+++ b/lib/mpi_interceptor/InterceptorFunctions.h
@@ -8,7 +8,7 @@
#ifndef TEST_MPI_INTERCEPTOR_INTERCEPTORFUNCTIONS_H_
#define TEST_MPI_INTERCEPTOR_INTERCEPTORFUNCTIONS_H_
-#include "RuntimeInterface.h"
+#include "runtime/RuntimeInterface.h"
#include
#include
@@ -93,7 +93,7 @@ int ta_check_buffer(const char* mpi_name, const void* called_from, const void* b
return -1;
}
int typeId;
- size_t count = 0;
+ size_t count = 0;
typeart_status typeart_status_v = typeart_get_type(buf, &typeId, &count);
if (typeart_status_v != TA_OK) {
++mcounter.error;
@@ -137,9 +137,9 @@ void ta_exit() {
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
struct rusage end;
getrusage(RUSAGE_SELF, &end);
- printf("CCounter (%i) { Send: %i Recv: %i Send_Recv: %i Unsupported: %i MAX RSS[KBytes]: %ld }\n", rank, counter.send,
- counter.recv, counter.send_recv, counter.unsupported, end.ru_maxrss);
- printf("MCounter (%i) { Error: %i Null_Buf: %i Null_Count: %i }\n", rank, mcounter.error, mcounter.null_buff,
+ printf("CCounter (%i) { Send: %zu Recv: %zu Send_Recv: %zu Unsupported: %zu MAX RSS[KBytes]: %ld }\n", rank,
+ counter.send, counter.recv, counter.send_recv, counter.unsupported, end.ru_maxrss);
+ printf("MCounter (%i) { Error: %zu Null_Buf: %zu Null_Count: %zu }\n", rank, mcounter.error, mcounter.null_buff,
mcounter.null_count);
}
diff --git a/test/mpi_interceptor/mpi_interceptor_tmpl.impl b/lib/mpi_interceptor/mpi_interceptor_tmpl.impl
similarity index 100%
rename from test/mpi_interceptor/mpi_interceptor_tmpl.impl
rename to lib/mpi_interceptor/mpi_interceptor_tmpl.impl
diff --git a/test/mpi_interceptor/wrap.py b/lib/mpi_interceptor/wrap.py
similarity index 100%
rename from test/mpi_interceptor/wrap.py
rename to lib/mpi_interceptor/wrap.py
diff --git a/lib/passes/CMakeLists.txt b/lib/passes/CMakeLists.txt
new file mode 100644
index 00000000..6a33bb66
--- /dev/null
+++ b/lib/passes/CMakeLists.txt
@@ -0,0 +1,68 @@
+set(PROJECT_NAME typeart_llvm)
+set(TARGETS_EXPORT_NAME ${PROJECT_NAME}-targets)
+
+add_subdirectory(filter)
+add_subdirectory(analysis)
+
+set(PASS_SOURCES
+ TypeARTPass.cpp
+ support/TypeUtil.cpp
+ instrumentation/InstrumentationHelper.cpp
+ instrumentation/TypeARTFunctions.cpp
+ TypeManager.cpp
+ instrumentation/MemOpArgCollector.cpp
+ instrumentation/MemOpInstrumentation.cpp
+ instrumentation/Instrumentation.cpp
+)
+
+make_llvm_module(typeartpass
+ "${PASS_SOURCES}"
+ LINK_LIBS
+ typelib
+ DEPENDS
+ meminstfinderpass
+ INCLUDE_DIRS
+ ${CMAKE_CURRENT_SOURCE_DIR}
+ ${PROJECT_SOURCE_DIR}/lib/
+ ${PROJECT_SOURCE_DIR}/lib/passes/
+)
+
+mark_as_advanced(LLVM_TYPEARTPASS_LINK_INTO_TOOLS)
+
+target_project_compile_options(typeartpass)
+target_project_compile_definitions(typeartpass
+ PRIVATE_DEFS
+ LOG_LEVEL=${LOG_LEVEL}
+)
+
+if(SHOW_STATS)
+ target_project_compile_definitions(typeartpass
+ PRIVATE_DEFS
+ LLVM_ENABLE_STATS
+ )
+endif()
+
+install(
+ TARGETS typeartpass
+ EXPORT ${TARGETS_EXPORT_NAME}
+ LIBRARY DESTINATION lib
+ ARCHIVE DESTINATION lib
+)
+
+# also handles subdir meminstfinder
+install(
+ EXPORT ${TARGETS_EXPORT_NAME}
+ NAMESPACE typeart::
+ DESTINATION lib/cmake
+)
+
+configure_package_config_file(
+ ${PROJECT_SOURCE_DIR}/cmake/Config.cmake.in
+ ${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}Config.cmake
+ INSTALL_DESTINATION lib/cmake
+)
+
+install(FILES
+ ${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}Config.cmake
+ DESTINATION lib/cmake
+)
diff --git a/lib/passes/TypeARTPass.cpp b/lib/passes/TypeARTPass.cpp
new file mode 100644
index 00000000..842c3907
--- /dev/null
+++ b/lib/passes/TypeARTPass.cpp
@@ -0,0 +1,215 @@
+#include "TypeARTPass.h"
+
+//#include "RuntimeInterface.h"
+#include "analysis/MemInstFinderPass.h"
+#include "instrumentation/MemOpArgCollector.h"
+#include "instrumentation/MemOpInstrumentation.h"
+#include "instrumentation/TypeARTFunctions.h"
+#include "support/Logger.h"
+#include "support/Table.h"
+#include "typelib/TypeIO.h"
+
+#include "llvm/ADT/Statistic.h"
+#include "llvm/IR/Instructions.h"
+#include "llvm/IR/Module.h"
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/Format.h"
+#include "llvm/Transforms/Utils/CtorUtils.h"
+
+#include
+
+using namespace llvm;
+
+#define DEBUG_TYPE "typeart"
+
+namespace {
+static llvm::RegisterPass msp("typeart", "TypeArt type information", false, false);
+} // namespace
+
+static cl::opt ClTypeArtStats("typeart-stats", cl::desc("Show statistics for TypeArt type pass."), cl::Hidden,
+ cl::init(false));
+static cl::opt ClIgnoreHeap("typeart-no-heap", cl::desc("Ignore heap allocation/free instruction."), cl::Hidden,
+ cl::init(false));
+static cl::opt ClTypeArtAlloca("typeart-alloca", cl::desc("Track alloca instructions."), cl::Hidden,
+ cl::init(false));
+
+static cl::opt ClTypeFile("typeart-outfile", cl::desc("Location of the generated type file."), cl::Hidden,
+ cl::init("types.yaml"));
+
+STATISTIC(NumInstrumentedMallocs, "Number of instrumented mallocs");
+STATISTIC(NumInstrumentedFrees, "Number of instrumented frees");
+STATISTIC(NumInstrumentedAlloca, "Number of instrumented (stack) allocas");
+STATISTIC(NumInstrumentedGlobal, "Number of instrumented globals");
+
+namespace typeart::pass {
+
+// Used by LLVM pass manager to identify passes in memory
+char TypeArtPass::ID = 0;
+
+// std::unique_ptr TypeArtPass::typeMapping = std::make_unique();
+
+TypeArtPass::TypeArtPass() : llvm::ModulePass(ID), typeManager(ClTypeFile.getValue()) {
+ assert(!ClTypeFile.empty() && "Default type file not set");
+ EnableStatistics();
+}
+
+void TypeArtPass::getAnalysisUsage(llvm::AnalysisUsage& info) const {
+ info.addRequired();
+}
+
+bool TypeArtPass::doInitialization(Module& m) {
+ instrumentation_helper.setModule(m);
+
+ LOG_DEBUG("Propagating type infos.");
+ if (typeManager.load()) {
+ LOG_DEBUG("Existing type configuration successfully loaded from " << ClTypeFile.getValue());
+ } else {
+ LOG_DEBUG("No valid existing type configuration found: " << ClTypeFile.getValue());
+ }
+
+ auto arg_collector = std::make_unique(typeManager, instrumentation_helper);
+ auto mem_instrument = std::make_unique(functions, instrumentation_helper);
+ instrumentation_context =
+ std::make_unique(std::move(arg_collector), std::move(mem_instrument));
+
+ return true;
+}
+
+bool TypeArtPass::runOnModule(Module& m) {
+ bool instrumented_global{false};
+ if (ClTypeArtAlloca) {
+ declareInstrumentationFunctions(m);
+
+ const auto& globalsList = getAnalysis().getModuleGlobals();
+ if (!globalsList.empty()) {
+ const auto global_count = instrumentation_context->handleGlobal(globalsList);
+ NumInstrumentedGlobal += global_count;
+ instrumented_global = global_count > 0;
+ }
+ }
+
+ const auto instrumented_function = llvm::count_if(m.functions(), [&](auto& f) { return runOnFunc(f); }) > 0;
+ return instrumented_function || instrumented_global;
+}
+
+bool TypeArtPass::runOnFunc(Function& f) {
+ using namespace typeart;
+
+ if (f.isDeclaration() || f.getName().startswith("__typeart")) {
+ return false;
+ }
+
+ if (!getAnalysis().hasFunctionData(&f)) {
+ LOG_WARNING("No allocation data could be retrieved for function: " << f.getName());
+ return false;
+ }
+
+ LOG_DEBUG("Running on function: " << f.getName())
+
+ // FIXME this is required when "PassManagerBuilder::EP_OptimizerLast" is used as the function (constant) pointer are
+ // nullpointer/invalidated
+ declareInstrumentationFunctions(*f.getParent());
+
+ bool mod{false};
+ // auto& c = f.getContext();
+ DataLayout dl(f.getParent());
+
+ llvm::SmallDenseMap allocCounts;
+
+ const auto& fData = getAnalysis().getFunctionData(&f);
+ const auto& mallocs = fData.mallocs;
+ const auto& allocas = fData.allocas;
+ const auto& frees = fData.frees;
+
+ if (!ClIgnoreHeap) {
+ // instrument collected calls of bb:
+ const auto heap_count = instrumentation_context->handleHeap(mallocs);
+ const auto free_count = instrumentation_context->handleFree(frees);
+
+ NumInstrumentedMallocs += heap_count;
+ NumInstrumentedFrees += free_count;
+
+ mod |= heap_count > 0 || free_count > 0;
+ }
+ if (ClTypeArtAlloca) {
+ const auto stack_count = instrumentation_context->handleStack(allocas);
+ NumInstrumentedAlloca += stack_count;
+ mod |= stack_count > 0;
+ }
+
+ return mod;
+} // namespace pass
+
+bool TypeArtPass::doFinalization(Module&) {
+ /*
+ * Persist the accumulated type definition information for this module.
+ */
+ LOG_DEBUG("Writing type file to " << ClTypeFile.getValue());
+
+ if (typeManager.store()) {
+ LOG_DEBUG("Success!");
+ } else {
+ LOG_FATAL("Failed writing type config to " << ClTypeFile.getValue());
+ }
+ if (ClTypeArtStats && AreStatisticsEnabled()) {
+ auto& out = llvm::errs();
+ printStats(out);
+ }
+ return false;
+}
+
+void TypeArtPass::declareInstrumentationFunctions(Module& m) {
+ // Remove this return if problems come up during compilation
+ if (typeart_alloc_global.f != nullptr && typeart_alloc_stack.f != nullptr && typeart_alloc.f != nullptr &&
+ typeart_free.f != nullptr && typeart_leave_scope.f != nullptr) {
+ return;
+ }
+
+ TAFunctionDeclarator decl(m, instrumentation_helper, functions);
+
+ auto alloc_arg_types = instrumentation_helper.make_parameters(IType::ptr, IType::type_id, IType::extent);
+ auto free_arg_types = instrumentation_helper.make_parameters(IType::ptr);
+ auto leavescope_arg_types = instrumentation_helper.make_parameters(IType::stack_count);
+
+ typeart_alloc.f = decl.make_function(IFunc::heap, typeart_alloc.name, alloc_arg_types);
+ typeart_alloc_stack.f = decl.make_function(IFunc::stack, typeart_alloc_stack.name, alloc_arg_types);
+ typeart_alloc_global.f = decl.make_function(IFunc::global, typeart_alloc_global.name, alloc_arg_types);
+ typeart_free.f = decl.make_function(IFunc::free, typeart_free.name, free_arg_types);
+ typeart_leave_scope.f = decl.make_function(IFunc::scope, typeart_leave_scope.name, leavescope_arg_types);
+}
+
+void TypeArtPass::printStats(llvm::raw_ostream& out) {
+ const auto get_ta_mode = [&]() {
+ const bool heap = !ClIgnoreHeap.getValue();
+ const bool stack = ClTypeArtAlloca.getValue();
+ if (heap) {
+ if (stack) {
+ return " [Heap & Stack]";
+ }
+ return " [Heap]";
+ } else if (stack) {
+ return " [Stack]";
+ }
+ return " [Unknown]";
+ };
+
+ Table stats("TypeArtPass");
+ stats.wrap_header = true;
+ stats.title += get_ta_mode();
+ stats.put(Row::make("Malloc", NumInstrumentedMallocs.getValue()));
+ stats.put(Row::make("Free", NumInstrumentedFrees.getValue()));
+ stats.put(Row::make("Alloca", NumInstrumentedAlloca.getValue()));
+ stats.put(Row::make("Global", NumInstrumentedGlobal.getValue()));
+
+ stats.print(out);
+}
+
+} // namespace typeart::pass
+
+#include "llvm/IR/LegacyPassManager.h"
+#include "llvm/Transforms/IPO/PassManagerBuilder.h"
+
+static void registerClangPass(const llvm::PassManagerBuilder&, llvm::legacy::PassManagerBase& PM) {
+ PM.add(new typeart::pass::TypeArtPass());
+}
+static RegisterStandardPasses RegisterClangPass(PassManagerBuilder::EP_OptimizerLast, registerClangPass);
diff --git a/lib/TypeARTPass.h b/lib/passes/TypeARTPass.h
similarity index 73%
rename from lib/TypeARTPass.h
rename to lib/passes/TypeARTPass.h
index 47709724..9096bede 100644
--- a/lib/TypeARTPass.h
+++ b/lib/passes/TypeARTPass.h
@@ -1,9 +1,11 @@
#ifndef _LIB_MUSTSUPPORTPASS_H
#define _LIB_MUSTSUPPORTPASS_H
-#include "TypeDB.h"
#include "TypeManager.h"
-#include "support/InstrumentationHelper.h"
+#include "instrumentation/Instrumentation.h"
+#include "instrumentation/InstrumentationHelper.h"
+#include "instrumentation/TypeARTFunctions.h"
+#include "typelib/TypeDB.h"
#include "llvm/Pass.h"
@@ -16,13 +18,12 @@ class Function;
class AnalysisUsage;
} // namespace llvm
-namespace typeart {
-namespace pass {
+namespace typeart::pass {
class TypeArtPass : public llvm::ModulePass {
private:
struct TypeArtFunc {
- const std::string name{""};
+ const std::string name;
llvm::Value* f{nullptr};
};
@@ -33,7 +34,9 @@ class TypeArtPass : public llvm::ModulePass {
TypeArtFunc typeart_leave_scope{"__typeart_leave_scope"};
TypeManager typeManager;
- InstrumentationHelper instr;
+ InstrumentationHelper instrumentation_helper;
+ TAFunctions functions;
+ std::unique_ptr instrumentation_context;
public:
static char ID; // used to identify pass
@@ -50,7 +53,6 @@ class TypeArtPass : public llvm::ModulePass {
void printStats(llvm::raw_ostream&);
};
-} // namespace pass
-} // namespace typeart
+} // namespace typeart::pass
#endif
diff --git a/lib/TypeManager.cpp b/lib/passes/TypeManager.cpp
similarity index 94%
rename from lib/TypeManager.cpp
rename to lib/passes/TypeManager.cpp
index 3fcf5ac5..43dff118 100644
--- a/lib/TypeManager.cpp
+++ b/lib/passes/TypeManager.cpp
@@ -4,19 +4,20 @@
#include "TypeManager.h"
-#include "TypeIO.h"
-#include "TypeInterface.h"
#include "support/Logger.h"
#include "support/TypeUtil.h"
#include "support/Util.h"
+#include "typelib/TypeIO.h"
+#include "typelib/TypeInterface.h"
#include
+#include
namespace typeart {
using namespace llvm;
-TypeManager::TypeManager(std::string file) : file(file), structCount(0) {
+TypeManager::TypeManager(std::string file) : file(std::move(file)), structCount(0) {
}
bool TypeManager::load() {
@@ -99,8 +100,7 @@ int TypeManager::getOrRegisterVector(llvm::VectorType* type, const llvm::DataLay
// arrays. They are given special names and are marked with a TA_VEC flag to avoid confusion.
// Look up name
- auto it = structMap.find(name);
- if (it != structMap.end()) {
+ if (auto it = structMap.find(name); it != structMap.end()) {
if (!typeDB.isVectorType(it->second)) {
LOG_ERROR("Expected vector type: " << name);
return TA_UNKNOWN_TYPE;
@@ -142,9 +142,9 @@ int TypeManager::getOrRegisterStruct(llvm::StructType* type, const llvm::DataLay
return type->getStructName();
};
- auto name = getName(type);
- auto it = structMap.find(name);
- if (it != structMap.end()) {
+ const auto name = getName(type);
+
+ if (auto it = structMap.find(name); it != structMap.end()) {
if (!typeDB.isUserDefinedType(it->second)) {
LOG_ERROR("Expected user defined struct type: " << name);
return TA_UNKNOWN_TYPE;
diff --git a/lib/TypeManager.h b/lib/passes/TypeManager.h
similarity index 87%
rename from lib/TypeManager.h
rename to lib/passes/TypeManager.h
index b4a9cc58..cbd02d68 100644
--- a/lib/TypeManager.h
+++ b/lib/passes/TypeManager.h
@@ -1,36 +1,32 @@
#ifndef LLVM_MUST_SUPPORT_TYPEMANAGER_H
#define LLVM_MUST_SUPPORT_TYPEMANAGER_H
-#include "TypeDB.h"
+#include "typelib/TypeDB.h"
+#include
#include
#include
-#include |