Dynamic Pools¶
Frequently allocating and deallocating memory can be quite costly, especially when you are making large allocations or allocating on different memory resources. To mitigate this, Umpire provides allocation strategies that can be used to customize how data is obtained from the system.
In this example, we will look at the umpire::strategy::DynamicPoolList
strategy. This is a simple pooling algorithm that can fulfill requests for
allocations of any size. To create a new Allocator
using the
umpire::strategy::DynamicPoolList
strategy:
auto pooled_allocator = rm.makeAllocator<umpire::strategy::DynamicPoolList>(resource + "_pool", allocator);
We have to provide a new name for the Allocator, as well as the underlying Allocator we wish to use to grab memory.
Additionally, in the previous section on Allocators, we mentioned that you
could build a new allocator off of an existing one using the getAllocator
function. Here is another example of this, but using a strategy:
umpire::Allocator addon_allocator = rm.makeAllocator<umpire::strategy::SizeLimiter>(
resource + "_addon_pool", rm.getAllocator(pooled_allocator.getName()), 2098);
The purpose of this example is to show that the getAllocator
function
can be used more than just to get an initial allocator. The addon_allocator
will
be a dynamic pool allocator that is limited to 2098 bytes. Another good use case
for the getAllocator
function is grabbing each available allocator in a loop and
querying some property. (Note that addon_allocator
in the above example will be
created with the same memory resource as pooled_allocator
was.)
Once you have an Allocator
, you can allocate and deallocate memory as
before, without needing to worry about the underlying algorithm used for the
allocations:
double* data = static_cast<double*>(pooled_allocator.allocate(SIZE * sizeof(double)));
pooled_allocator.deallocate(data);
Don’t forget, these strategies can be created on top of any valid Allocator:
allocate_and_deallocate_pool("HOST");
#if defined(UMPIRE_ENABLE_DEVICE)
allocate_and_deallocate_pool("DEVICE");
#endif
#if defined(UMPIRE_ENABLE_UM)
allocate_and_deallocate_pool("UM");
#endif
#if defined(UMPIRE_ENABLE_PINNED)
allocate_and_deallocate_pool("PINNED");
#endif
Most Umpire users will make allocations that use the GPU via the
umpire::strategy::DynamicPoolList
, to help mitigate the cost of allocating
memory on these devices.
You can tune the way that umpire::strategy::DynamicPoolList
allocates
memory using two parameters: the initial size, and the minimum size. The
initial size controls how large the first underly allocation made will be,
regardless of the requested size. The minimum size controls the minimum size of
any future underlying allocations. These two parameters can be passed when
constructing a pool:
auto pooled_allocator = rm.makeAllocator<umpire::strategy::DynamicPoolList>(resource + "_pool", allocator,
initial_size, /* default = 512Mb*/
min_block_size /* default = 1Mb */);
Depending on where you are allocating data, you might want to use different sizes. It’s easy to construct multiple pools with different configurations:
allocate_and_deallocate_pool("HOST", 65536, 512);
#if defined(UMPIRE_ENABLE_DEVICE)
allocate_and_deallocate_pool("DEVICE", (1024 * 1024 * 1024), (1024 * 1024));
#endif
#if defined(UMPIRE_ENABLE_UM)
allocate_and_deallocate_pool("UM", (1024 * 64), 1024);
#endif
#if defined(UMPIRE_ENABLE_PINNED)
allocate_and_deallocate_pool("PINNED", (1024 * 16), 1024);
#endif
There are lots of different strategies that you can use, we will look at some of them in this tutorial. A complete list of strategies can be found here.
//////////////////////////////////////////////////////////////////////////////
// Copyright (c) 2016-23, Lawrence Livermore National Security, LLC and Umpire
// project contributors. See the COPYRIGHT file for details.
//
// SPDX-License-Identifier: (MIT)
//////////////////////////////////////////////////////////////////////////////
#include "umpire/Allocator.hpp"
#include "umpire/ResourceManager.hpp"
#include "umpire/strategy/DynamicPoolList.hpp"
void allocate_and_deallocate_pool(const std::string& resource)
{
auto& rm = umpire::ResourceManager::getInstance();
auto allocator = rm.getAllocator(resource);
// _sphinx_tag_tut_makepool_start
auto pooled_allocator = rm.makeAllocator<umpire::strategy::DynamicPoolList>(resource + "_pool", allocator);
// _sphinx_tag_tut_makepool_end
constexpr std::size_t SIZE = 1024;
// _sphinx_tag_tut_allocate_start
double* data = static_cast<double*>(pooled_allocator.allocate(SIZE * sizeof(double)));
// _sphinx_tag_tut_allocate_end
std::cout << "Allocated " << (SIZE * sizeof(double)) << " bytes using the " << pooled_allocator.getName()
<< " allocator...";
// _sphinx_tag_tut_deallocate_start
pooled_allocator.deallocate(data);
// _sphinx_tag_tut_deallocate_end
std::cout << " deallocated." << std::endl;
}
int main(int, char**)
{
// _sphinx_tag_tut_anyallocator_start
allocate_and_deallocate_pool("HOST");
#if defined(UMPIRE_ENABLE_DEVICE)
allocate_and_deallocate_pool("DEVICE");
#endif
#if defined(UMPIRE_ENABLE_UM)
allocate_and_deallocate_pool("UM");
#endif
#if defined(UMPIRE_ENABLE_PINNED)
allocate_and_deallocate_pool("PINNED");
#endif
// _sphinx_tag_tut_anyallocator_end
return 0;
}
//////////////////////////////////////////////////////////////////////////////
// Copyright (c) 2016-23, Lawrence Livermore National Security, LLC and Umpire
// project contributors. See the COPYRIGHT file for details.
//
// SPDX-License-Identifier: (MIT)
//////////////////////////////////////////////////////////////////////////////
#include "umpire/Allocator.hpp"
#include "umpire/ResourceManager.hpp"
#include "umpire/strategy/DynamicPoolList.hpp"
void allocate_and_deallocate_pool(const std::string& resource, std::size_t initial_size, std::size_t min_block_size)
{
constexpr std::size_t SIZE = 1024;
auto& rm = umpire::ResourceManager::getInstance();
auto allocator = rm.getAllocator(resource);
// _sphinx_tag_tut_allocator_tuning_start
auto pooled_allocator = rm.makeAllocator<umpire::strategy::DynamicPoolList>(resource + "_pool", allocator,
initial_size, /* default = 512Mb*/
min_block_size /* default = 1Mb */);
// _sphinx_tag_tut_allocator_tuning_end
double* data = static_cast<double*>(pooled_allocator.allocate(SIZE * sizeof(double)));
std::cout << "Allocated " << (SIZE * sizeof(double)) << " bytes using the " << pooled_allocator.getName()
<< " allocator...";
pooled_allocator.deallocate(data);
std::cout << " deallocated." << std::endl;
}
int main(int, char**)
{
// _sphinx_tag_tut_device_sized_pool_start
allocate_and_deallocate_pool("HOST", 65536, 512);
#if defined(UMPIRE_ENABLE_DEVICE)
allocate_and_deallocate_pool("DEVICE", (1024 * 1024 * 1024), (1024 * 1024));
#endif
#if defined(UMPIRE_ENABLE_UM)
allocate_and_deallocate_pool("UM", (1024 * 64), 1024);
#endif
#if defined(UMPIRE_ENABLE_PINNED)
allocate_and_deallocate_pool("PINNED", (1024 * 16), 1024);
#endif
// _sphinx_tag_tut_device_sized_pool_end
return 0;
}