Skip to content

Configuration ​

GTaP uses separate configuration structures for thread and block execution modes. The structures provide defaults for all runtime fields, while a few settings differ because the unit of execution is a CUDA thread or thread block.

Settings by execution mode ​

PurposeThread modeBlock mode
Grid sizegrid_size = 4096grid_size = 1024
CUDA block sizeblock_size = 32block_size = 256
Memory budgetmemory_budget_bytes = 0memory_budget_bytes = 0
Task capacitymax_tasks_per_warp = 0max_tasks_per_block = 0
Profile capacityprofile_capacity_per_warp = 15000profile_capacity_per_block = 15000
CUDA streamstream = nullptrstream = nullptr
DAQ queuesnum_queues = 1—
Dynamic shared memory per block—dynamic_shared_bytes = 0

The runtime memory budget defaults to 5 GiB. Set GTAP_MEMORY_BUDGET to change it, or set a nonzero memory_budget_bytes to override the environment variable.

sh
GTAP_MEMORY_BUDGET=5GiB ./app

GTAP_MEMORY_BUDGET accepts a decimal integer with an optional unit suffix. No suffix or B means bytes. K, M, G, T and their iB forms use powers of 1024; KB, MB, GB, TB use powers of 1000. An empty value is an error.

Task capacity limits the number of simultaneously live tasks assigned to each CUDA warp in thread mode or CUDA thread block in block mode, not the total number of tasks executed. Larger capacities consume more device memory. At 0, GTaP selects the largest valid capacity within the budget. A positive value specifies the capacity directly and overrides the budget. Profiling buffers are configured separately and excluded from the budget.

grid_size is the number of CUDA thread blocks launched by gtap_launch. nullptr selects the default CUDA stream. Profile capacity is used only when profiling is enabled.

Configuration structures ​

Thread mode ​

gtap_thread_config is defined by gtap_thread.cuh:

cpp
struct gtap_thread_config {
    int grid_size = 4096;
    int block_size = 32;
    size_t memory_budget_bytes = 0;
    int max_tasks_per_warp = 0;
    int num_queues = 1;
    int profile_capacity_per_warp = 15000;
    cudaStream_t stream = nullptr;
};

Task capacity is allocated to each CUDA warp and divided equally among the configured queues:

text
capacity per queue = max_tasks_per_warp / num_queues

Block mode ​

gtap_block_config is defined by gtap_block.cuh:

cpp
struct gtap_block_config {
    int grid_size = 1024;
    int block_size = 256;
    size_t memory_budget_bytes = 0;
    int max_tasks_per_block = 0;
    int profile_capacity_per_block = 15000;
    size_t dynamic_shared_bytes = 0;
    cudaStream_t stream = nullptr;
};

Task capacity is allocated to each CUDA thread block. dynamic_shared_bytes specifies the dynamic shared memory supplied to each block.

Accepted configurations ​

gtap_initialize rejects a configuration before it allocates runtime resources.

Both modes require:

  • grid_size > 0
  • a CUDA block size in (0, 1024]
  • block_size to be a multiple of 32
  • a non-negative task capacity (0 selects the memory budget)
  • when profiling is enabled, a profile capacity in (0, INT_MAX / 2]

Thread mode additionally requires:

  • num_queues > 0
  • max_tasks_per_warp % num_queues == 0

Invalid launch geometry returns cudaErrorInvalidConfiguration. An invalid task capacity, queue count, memory budget, or profiling capacity returns cudaErrorInvalidValue.

Compile-time settings ​

SettingKindPurpose
GTAP_ENABLE_PROFILINGPreprocessor macroEnables collection of profiling data
-fgtap-no-taskwaitGTaP Clang optionSelects the compact runtime for programs that do not use taskwait

For profiling, compile with -DGTAP_ENABLE_PROFILING; see the Profiling API Reference.

-fgtap-no-taskwait removes join-state support. Do not use it if the program contains #pragma gtap taskwait.

GTaP is a research prototype under active development.