Cupti: CUPTI Callback API

# CUPTI Callback API

## Data Structures
struct | CUpti_CallbackData
| Data passed into a runtime or driver API callback function. More...
struct | CUpti_GraphData
| CUDA graphs data passed into a resource callback function. More...
struct | CUpti_ModuleResourceData
| Module data passed into a resource callback function. More...
struct | CUpti_NvtxData
| Data passed into a NVTX callback function. More...
struct | CUpti_ResourceData
| Data passed into a resource callback function. More...
struct | CUpti_SynchronizeData
| Data passed into a synchronize callback function. More...
## Typedefs
typedef void(* | CUpti_CallbackFunc )(void *userdata, CUpti_CallbackDomain domain, CUpti_CallbackId cbid, const void *cbdata)
| Function type for a callback.
typedef uint32_t | CUpti_CallbackId
| An ID for a driver API, runtime API, resource or synchronization callback.
typedef CUpti_CallbackDomain * | CUpti_DomainTable
| Pointer to an array of callback domains.
typedef struct CUpti_Subscriber_st * | CUpti_SubscriberHandle
| A callback subscriber.
## Enumerations
enum | CUpti_ApiCallbackSite { CUPTI_API_ENTER = 0, CUPTI_API_EXIT = 1 }
| Specifies the point in an API call that a callback is issued. More...
enum | CUpti_CallbackDomain { CUPTI_CB_DOMAIN_INVALID = 0, CUPTI_CB_DOMAIN_DRIVER_API = 1, CUPTI_CB_DOMAIN_RUNTIME_API = 2, CUPTI_CB_DOMAIN_RESOURCE = 3, CUPTI_CB_DOMAIN_SYNCHRONIZE = 4, CUPTI_CB_DOMAIN_NVTX = 5 }
| Callback domains. More...
enum | CUpti_CallbackIdResource { CUPTI_CBID_RESOURCE_INVALID = 0, CUPTI_CBID_RESOURCE_CONTEXT_CREATED = 1, CUPTI_CBID_RESOURCE_CONTEXT_DESTROY_STARTING = 2, CUPTI_CBID_RESOURCE_STREAM_CREATED = 3, CUPTI_CBID_RESOURCE_STREAM_DESTROY_STARTING = 4, CUPTI_CBID_RESOURCE_CU_INIT_FINISHED = 5, CUPTI_CBID_RESOURCE_MODULE_LOADED = 6, CUPTI_CBID_RESOURCE_MODULE_UNLOAD_STARTING = 7, CUPTI_CBID_RESOURCE_MODULE_PROFILED = 8, CUPTI_CBID_RESOURCE_GRAPH_CREATED = 9, CUPTI_CBID_RESOURCE_GRAPH_DESTROY_STARTING = 10, CUPTI_CBID_RESOURCE_GRAPH_CLONED = 11, CUPTI_CBID_RESOURCE_GRAPHNODE_CREATE_STARTING = 12, CUPTI_CBID_RESOURCE_GRAPHNODE_CREATED = 13, CUPTI_CBID_RESOURCE_GRAPHNODE_DESTROY_STARTING = 14, CUPTI_CBID_RESOURCE_GRAPHNODE_DEPENDENCY_CREATED = 15, CUPTI_CBID_RESOURCE_GRAPHNODE_DEPENDENCY_DESTROY_STARTING = 16, CUPTI_CBID_RESOURCE_GRAPHEXEC_CREATE_STARTING = 17, CUPTI_CBID_RESOURCE_GRAPHEXEC_CREATED = 18, CUPTI_CBID_RESOURCE_GRAPHEXEC_DESTROY_STARTING = 19, CUPTI_CBID_RESOURCE_GRAPHNODE_CLONED = 20 }
| Callback IDs for resource domain. More...
enum | CUpti_CallbackIdSync { CUPTI_CBID_SYNCHRONIZE_INVALID = 0, CUPTI_CBID_SYNCHRONIZE_STREAM_SYNCHRONIZED = 1, CUPTI_CBID_SYNCHRONIZE_CONTEXT_SYNCHRONIZED = 2 }
| Callback IDs for synchronization domain. More...
## Functions
CUptiResult | cuptiEnableAllDomains (uint32_t enable, CUpti_SubscriberHandle subscriber)
| Enable or disable all callbacks in all domains.
CUptiResult | cuptiEnableCallback (uint32_t enable, CUpti_SubscriberHandle subscriber, CUpti_CallbackDomain domain, CUpti_CallbackId cbid)
| Enable or disabled callbacks for a specific domain and callback ID.
CUptiResult | cuptiEnableDomain (uint32_t enable, CUpti_SubscriberHandle subscriber, CUpti_CallbackDomain domain)
| Enable or disabled all callbacks for a specific domain.
CUptiResult | cuptiGetCallbackName (CUpti_CallbackDomain domain, uint32_t cbid, const char **name)
| Get the name of a callback for a specific domain and callback ID.
CUptiResult | cuptiGetCallbackState (uint32_t *enable, CUpti_SubscriberHandle subscriber, CUpti_CallbackDomain domain, CUpti_CallbackId cbid)
| Get the current enabled/disabled state of a callback for a specific domain and function ID.
CUptiResult | cuptiSubscribe (CUpti_SubscriberHandle *subscriber, CUpti_CallbackFunc callback, void *userdata)
| Initialize a callback subscriber with a callback function and user data.
CUptiResult | cuptiSupportedDomains (size_t *domainCount, CUpti_DomainTable *domainTable)
| Get the available callback domains.
CUptiResult | cuptiUnsubscribe (CUpti_SubscriberHandle subscriber)
| Unregister a callback subscriber.

## Detailed Description

Functions, types, and enums that implement the CUPTI Callback API.

## Typedef Documentation

typedef void( * CUpti_CallbackFunc)(void *userdata, CUpti_CallbackDomain domain, CUpti_CallbackId cbid, const void *cbdata)

Function type for a callback. The type of the data passed to the callback in `cbdata` depends on the `domain`. If `domain` is CUPTI_CB_DOMAIN_DRIVER_API or CUPTI_CB_DOMAIN_RUNTIME_API the type of `cbdata` will be CUpti_CallbackData. If `domain` is CUPTI_CB_DOMAIN_RESOURCE the type of `cbdata` will be CUpti_ResourceData. If `domain` is CUPTI_CB_DOMAIN_SYNCHRONIZE the type of `cbdata` will be CUpti_SynchronizeData. If `domain` is CUPTI_CB_DOMAIN_NVTX the type of `cbdata` will be CUpti_NvtxData.

Parameters:

  | userdata | User data supplied at subscription of the callback
  | domain | The domain of the callback
  | cbid | The ID of the callback
  | cbdata | Data passed to the callback.

typedef uint32_t CUpti_CallbackId

An ID for a driver API, runtime API, resource or synchronization callback. Within a driver API callback this should be interpreted as a CUpti_driver_api_trace_cbid value (these values are defined in cupti_driver_cbid.h). Within a runtime API callback this should be interpreted as a CUpti_runtime_api_trace_cbid value (these values are defined in cupti_runtime_cbid.h). Within a resource API callback this should be interpreted as a CUpti_CallbackIdResource value. Within a synchronize API callback this should be interpreted as a CUpti_CallbackIdSync value.

## Enumeration Type Documentation

enum CUpti_ApiCallbackSite

Specifies the point in an API call that a callback is issued. This value is communicated to the callback function via CUpti_CallbackData::callbackSite.

Enumerator:

  CUPTI_API_ENTER | The callback is at the entry of the API call.
  CUPTI_API_EXIT | The callback is at the exit of the API call.

enum CUpti_CallbackDomain

Callback domains. Each domain represents callback points for a group of related API functions or CUDA driver activity.

Enumerator:

  CUPTI_CB_DOMAIN_INVALID | Invalid domain.
  CUPTI_CB_DOMAIN_DRIVER_API | Domain containing callback points for all driver API functions.
  CUPTI_CB_DOMAIN_RUNTIME_API | Domain containing callback points for all runtime API functions.
  CUPTI_CB_DOMAIN_RESOURCE | Domain containing callback points for CUDA resource tracking.
  CUPTI_CB_DOMAIN_SYNCHRONIZE | Domain containing callback points for CUDA synchronization.
  CUPTI_CB_DOMAIN_NVTX | Domain containing callback points for NVTX API functions.

enum CUpti_CallbackIdResource

Callback IDs for resource domain, CUPTI_CB_DOMAIN_RESOURCE. This value is communicated to the callback function via the `cbid` parameter.

Enumerator:

  CUPTI_CBID_RESOURCE_INVALID | Invalid resource callback ID.
  CUPTI_CBID_RESOURCE_CONTEXT_CREATED | A new context has been created.
  CUPTI_CBID_RESOURCE_CONTEXT_DESTROY_STARTING | A context is about to be destroyed.
  CUPTI_CBID_RESOURCE_STREAM_CREATED | A new stream has been created.
  CUPTI_CBID_RESOURCE_STREAM_DESTROY_STARTING | A stream is about to be destroyed.
  CUPTI_CBID_RESOURCE_CU_INIT_FINISHED | The driver has finished initializing.
  CUPTI_CBID_RESOURCE_MODULE_LOADED | A module has been loaded.
  CUPTI_CBID_RESOURCE_MODULE_UNLOAD_STARTING | A module is about to be unloaded.
  CUPTI_CBID_RESOURCE_MODULE_PROFILED | The current module which is being profiled.
  CUPTI_CBID_RESOURCE_GRAPH_CREATED | CUDA graph has been created.
  CUPTI_CBID_RESOURCE_GRAPH_DESTROY_STARTING | CUDA graph is about to be destroyed.
  CUPTI_CBID_RESOURCE_GRAPH_CLONED | CUDA graph is cloned.
  CUPTI_CBID_RESOURCE_GRAPHNODE_CREATE_STARTING | CUDA graph node is about to be created
  CUPTI_CBID_RESOURCE_GRAPHNODE_CREATED | CUDA graph node is created.
  CUPTI_CBID_RESOURCE_GRAPHNODE_DESTROY_STARTING | CUDA graph node is about to be destroyed.
  CUPTI_CBID_RESOURCE_GRAPHNODE_DEPENDENCY_CREATED | Dependency on a CUDA graph node is created.
  CUPTI_CBID_RESOURCE_GRAPHNODE_DEPENDENCY_DESTROY_STARTING | Dependency on a CUDA graph node is destroyed.
  CUPTI_CBID_RESOURCE_GRAPHEXEC_CREATE_STARTING | An executable CUDA graph is about to be created.
  CUPTI_CBID_RESOURCE_GRAPHEXEC_CREATED | An executable CUDA graph is created.
  CUPTI_CBID_RESOURCE_GRAPHEXEC_DESTROY_STARTING | An executable CUDA graph is about to be destroyed.
  CUPTI_CBID_RESOURCE_GRAPHNODE_CLONED | CUDA graph node is cloned.

enum CUpti_CallbackIdSync

Callback IDs for synchronization domain, CUPTI_CB_DOMAIN_SYNCHRONIZE. This value is communicated to the callback function via the `cbid` parameter.

Enumerator:

  CUPTI_CBID_SYNCHRONIZE_INVALID | Invalid synchronize callback ID.
  CUPTI_CBID_SYNCHRONIZE_STREAM_SYNCHRONIZED | Stream synchronization has completed for the stream.
  CUPTI_CBID_SYNCHRONIZE_CONTEXT_SYNCHRONIZED | Context synchronization has completed for the context.

## Function Documentation

CUptiResult cuptiEnableAllDomains | ( | uint32_t | enable,
|  | CUpti_SubscriberHandle | subscriber |
| ) |  |  |

Enable or disable all callbacks in all domains.

Note:
  Thread-safety: a subscriber must serialize access to cuptiGetCallbackState, cuptiEnableCallback, cuptiEnableDomain, and cuptiEnableAllDomains. For example, if cuptiGetCallbackState(sub, d, *) and cuptiEnableAllDomains(sub) are called concurrently, the results are undefined.

Parameters:

  | enable | New enable state for all callbacks in all domain. Zero disables all callbacks, non-zero enables all callbacks.
  | subscriber | - Handle to callback subscription

Return values:

  | CUPTI_SUCCESS | on success
  | CUPTI_ERROR_NOT_INITIALIZED | if unable to initialized CUPTI
  | CUPTI_ERROR_INVALID_PARAMETER | if `subscriber` is invalid

CUptiResult cuptiEnableCallback | ( | uint32_t | enable,
|  | CUpti_SubscriberHandle | subscriber,
|  | CUpti_CallbackDomain | domain,
|  | CUpti_CallbackId | cbid |
| ) |  |  |

Enable or disabled callbacks for a subscriber for a specific domain and callback ID.

Note:
  Thread-safety: a subscriber must serialize access to cuptiGetCallbackState, cuptiEnableCallback, cuptiEnableDomain, and cuptiEnableAllDomains. For example, if cuptiGetCallbackState(sub, d, c) and cuptiEnableCallback(sub, d, c) are called concurrently, the results are undefined.

Parameters:

  | enable | New enable state for the callback. Zero disables the callback, non-zero enables the callback.
  | subscriber | - Handle to callback subscription
  | domain | The domain of the callback
  | cbid | The ID of the callback

Return values:

  | CUPTI_SUCCESS | on success
  | CUPTI_ERROR_NOT_INITIALIZED | if unable to initialized CUPTI
  | CUPTI_ERROR_INVALID_PARAMETER | if `subscriber`, `domain` or `cbid` is invalid.

CUptiResult cuptiEnableDomain | ( | uint32_t | enable,
|  | CUpti_SubscriberHandle | subscriber,
|  | CUpti_CallbackDomain | domain |
| ) |  |  |

Enable or disabled all callbacks for a specific domain.

Note:
  Thread-safety: a subscriber must serialize access to cuptiGetCallbackState, cuptiEnableCallback, cuptiEnableDomain, and cuptiEnableAllDomains. For example, if cuptiGetCallbackEnabled(sub, d, *) and cuptiEnableDomain(sub, d) are called concurrently, the results are undefined.

Parameters:

  | enable | New enable state for all callbacks in the domain. Zero disables all callbacks, non-zero enables all callbacks.
  | subscriber | - Handle to callback subscription
  | domain | The domain of the callback

Return values:

  | CUPTI_SUCCESS | on success
  | CUPTI_ERROR_NOT_INITIALIZED | if unable to initialized CUPTI
  | CUPTI_ERROR_INVALID_PARAMETER | if `subscriber` or `domain` is invalid

CUptiResult cuptiGetCallbackName | ( | CUpti_CallbackDomain | domain,
|  | uint32_t | cbid,
|  | const char ** | name |
| ) |  |  |

Returns a pointer to the name c_string in `**name`.

Note:
  Names are available only for the DRIVER and RUNTIME domains.

Parameters:

  | domain | The domain of the callback
  | cbid | The ID of the callback
  | name | Returns pointer to the name string on success, NULL otherwise

Return values:

  | CUPTI_SUCCESS | on success
  | CUPTI_ERROR_INVALID_PARAMETER | if `name` is NULL, or if `domain` or `cbid` is invalid.

CUptiResult cuptiGetCallbackState | ( | uint32_t * | enable,
|  | CUpti_SubscriberHandle | subscriber,
|  | CUpti_CallbackDomain | domain,
|  | CUpti_CallbackId | cbid |
| ) |  |  |

Returns non-zero in `*enable` if the callback for a domain and callback ID is enabled, and zero if not enabled.

Note:
  Thread-safety: a subscriber must serialize access to cuptiGetCallbackState, cuptiEnableCallback, cuptiEnableDomain, and cuptiEnableAllDomains. For example, if cuptiGetCallbackState(sub, d, c) and cuptiEnableCallback(sub, d, c) are called concurrently, the results are undefined.

Parameters:

  | enable | Returns non-zero if callback enabled, zero if not enabled
  | subscriber | Handle to the initialize subscriber
  | domain | The domain of the callback
  | cbid | The ID of the callback

Return values:

  | CUPTI_SUCCESS | on success
  | CUPTI_ERROR_NOT_INITIALIZED | if unable to initialized CUPTI
  | CUPTI_ERROR_INVALID_PARAMETER | if `enabled` is NULL, or if `subscriber`, `domain` or `cbid` is invalid.

CUptiResult cuptiSubscribe | ( | CUpti_SubscriberHandle * | subscriber,
|  | CUpti_CallbackFunc | callback,
|  | void * | userdata |
| ) |  |  |

Initializes a callback subscriber with a callback function and (optionally) a pointer to user data. The returned subscriber handle can be used to enable and disable the callback for specific domains and callback IDs.

Note:
  Only a single subscriber can be registered at a time. To ensure that no other CUPTI client interrupts the profiling session, it's the responsibility of all the CUPTI clients to call this function before starting the profling session. In case profiling session is already started by another CUPTI client, this function returns the error code CUPTI_ERROR_MULTIPLE_SUBSCRIBERS_NOT_SUPPORTED. Note that this function returns the same error when application is launched using NVIDIA tools like nvprof, Visual Profiler, Nsight Systems, Nsight Compute, cuda-gdb and cuda-memcheck.

  This function does not enable any callbacks.

  Thread-safety: this function is thread safe.

Parameters:

  | subscriber | Returns handle to initialize subscriber
  | callback | The callback function
  | userdata | A pointer to user data. This data will be passed to the callback function via the `userdata` paramater.

Return values:

  | CUPTI_SUCCESS | on success
  | CUPTI_ERROR_NOT_INITIALIZED | if unable to initialize CUPTI
  | CUPTI_ERROR_MULTIPLE_SUBSCRIBERS_NOT_SUPPORTED | if there is already a CUPTI subscriber
  | CUPTI_ERROR_INVALID_PARAMETER | if `subscriber` is NULL

CUptiResult cuptiSupportedDomains | ( | size_t * | domainCount,
|  | CUpti_DomainTable * | domainTable |
| ) |  |  |

Returns in `*domainTable` an array of size `*domainCount` of all the available callback domains.

Note:
  Thread-safety: this function is thread safe.

Parameters:

  | domainCount | Returns number of callback domains
  | domainTable | Returns pointer to array of available callback domains

Return values:

  | CUPTI_SUCCESS | on success
  | CUPTI_ERROR_NOT_INITIALIZED | if unable to initialize CUPTI
  | CUPTI_ERROR_INVALID_PARAMETER | if `domainCount` or `domainTable` are NULL

CUptiResult cuptiUnsubscribe | ( | CUpti_SubscriberHandle | subscriber | ) |

Removes a callback subscriber so that no future callbacks will be issued to that subscriber.

Note:
  Thread-safety: this function is thread safe.

Parameters:

  | subscriber | Handle to the initialize subscriber

Return values:

  | CUPTI_SUCCESS | on success
  | CUPTI_ERROR_NOT_INITIALIZED | if unable to initialized CUPTI
  | CUPTI_ERROR_INVALID_PARAMETER | if `subscriber` is NULL or not initialized