Improving DynamicPoolList Performance with a Coalesce Heuristic

As needed, the DynamicPoolList memory pool (umpire::strategy::DynamicPoolList) will continue to allocate blocks to satisfy allocation requests that cannot be satisfied by blocks currently in the pool it is managing. Under certain application-specific memory allocation patterns, fragmentation within the blocks or allocations that are for sizes greater than the size of the largest available block can cause the pool to grow too large. For example, a problematic allocation pattern is when an application makes several allocations of incrementing size where each allocation is larger than the previous block size allocated.

The Umpire library provides Coalescing Heuristics to help manage the blocks of memory within a memory pool. The purpose of a coalescing heuristic is to ensure that the memory pool is properly maintained through the duration of an application so that it does not grow too large and prematurely run out of memory. There are two coalescing heuristics: Percent Releasable and Blocks Releasable. By default, a memory pool will use the Percent Releasable heuristic. Additionally, Umpire provides an optional tuning parameter. This tuning is called the HighWatermark tuning and it uses the pool’s HighWatermark value when coalescing the pool. By default, coalescing heuristics DO NOT use the HighWatermark tuning. Instead, they use the pool’s Actual Size value. Go to Umpire’s RZ Confluence page under “Design Documents” and refer to the “Umpire Pool Usage and Control” documentation for more information. You can also refer to Coalescing Pool Memory.

Note

To turn on the HighWatermark heuristic tuning, use the coalescing heuristic with the “_hwm” suffix (e.g. umpire::strategy::DynamicPoolList::blocks_releasable_hwm(num_blocks)()).

The umpire::strategy::DynamicPoolList::coalesce() method may be used to cause the umpire::strategy::DynamicPoolList to coalesce the releasable blocks into a single larger block. This is accomplished by: tallying the size of all blocks without allocations against them, releasing those blocks back to the memory resource, and creating a new block of the previously tallied size.

Note

If the HighWatermark heuristic tuning is used, then the new block will be allocated to the pool’s HighWatermark instead.

Applications may offer a heuristic function to the umpire::strategy::DynamicPoolList during instantiation that will return true whenever a pool reaches a specific threshold of releasable bytes (represented by completely free blocks) to the total size of the pool. The DynamicPoolList will call this heuristic function just before it returns from its umpire::strategy::DynamicPoolList::deallocate() method and when the function returns true, the DynamicPoolList will call the umpire::strategy::DynamicPoolList::coalesce() method.

The default heuristic of 100 will cause the DynamicPoolList to automatically coalesce when all of the bytes in the pool are releasable and there is more than one block in the pool.

A heuristic of 0 will cause the DynamicPoolList to never automatically coalesce.

Creation of the heuristic function is accomplished by:

  //
  // Create a heuristic function that will return true to the DynamicPoolList
  // object when the threshold of releasable size to total size is 75%.
  //
  auto heuristic_function = umpire::strategy::DynamicPoolList::percent_releasable(75);

In this example, we are using the Percent Releasable heuristic. The heuristic function is then provided as a parameter when the object is instantiated:

  //
  // Create a pool with an initial block size of 1 Kb and 1 Kb block size for
  // all subsequent allocations and with our previously created heuristic
  // function.
  //
  auto pooled_allocator = rm.makeAllocator<umpire::strategy::DynamicPoolList>("HOST_POOL", allocator, 1024ul, 1024ul,
                                                                              16, heuristic_function);

The complete example is included below:

//////////////////////////////////////////////////////////////////////////////
// Copyright (c) 2016-24, Lawrence Livermore National Security, LLC and Umpire
// project contributors. See the COPYRIGHT file for details.
//
// SPDX-License-Identifier: (MIT)
//////////////////////////////////////////////////////////////////////////////
#include "umpire/Allocator.hpp"
#include "umpire/ResourceManager.hpp"
#include "umpire/strategy/DynamicPoolList.hpp"
#include "umpire/util/Macros.hpp"
#include "umpire/util/wrap_allocator.hpp"

int main(int, char**)
{
  auto& rm = umpire::ResourceManager::getInstance();

  auto allocator = rm.getAllocator("HOST");

  // _sphinx_tag_tut_creat_heuristic_fun_start
  //
  // Create a heuristic function that will return true to the DynamicPoolList
  // object when the threshold of releasable size to total size is 75%.
  //
  auto heuristic_function = umpire::strategy::DynamicPoolList::percent_releasable(75);
  // _sphinx_tag_tut_creat_heuristic_fun_end

  // _sphinx_tag_tut_use_heuristic_fun_start
  //
  // Create a pool with an initial block size of 1 Kb and 1 Kb block size for
  // all subsequent allocations and with our previously created heuristic
  // function.
  //
  auto pooled_allocator = rm.makeAllocator<umpire::strategy::DynamicPoolList>("HOST_POOL", allocator, 1024ul, 1024ul,
                                                                              16, heuristic_function);
  // _sphinx_tag_tut_use_heuristic_fun_end

  //
  // Obtain a pointer to our specific DynamicPoolList instance in order to see the
  // DynamicPoolList-specific statistics
  //
  auto dynamic_pool = umpire::util::unwrap_allocator<umpire::strategy::DynamicPoolList>(pooled_allocator);

  void* a[4];
  for (int i = 0; i < 4; ++i)
    a[i] = pooled_allocator.allocate(1024);

  for (int i = 0; i < 4; ++i) {
    pooled_allocator.deallocate(a[i]);
    std::cout << "Pool has " << pooled_allocator.getActualSize() << " bytes of memory. "
              << pooled_allocator.getCurrentSize() << " bytes are used. " << dynamic_pool->getBlocksInPool()
              << " blocks are in the pool. " << dynamic_pool->getReleasableSize() << " bytes are releaseable. "
              << std::endl;
  }

  return 0;
}