API Reference

Configuration

class tidl::Configuration

Specifies the configuration required for a network.

The Configuration object is used to specify various parameters required for network execution. Applications can directly initialize fields in an instance of Configuration or use the ReadFromFile method to read the configuration from a file.

Public Functions

tidl::ConfigurationConfiguration()

Default constructor.

bool tidl::ConfigurationValidate() const

Validate the fields in the configuration object.

void tidl::ConfigurationPrint(std::ostream &os = std::cout) const

Debug - Print the configuration.

bool tidl::ConfigurationReadFromFile(const std::string &file_name)

Read a configuration from the specified file and validate.

Public Members

int tidl::ConfigurationnumFrames

Number of frames of input data (can be 0 if data is not read from file)

int tidl::ConfigurationinHeight

Height of the input image. Used by the API, must be specified.

int tidl::ConfigurationinWidth

Width of the input image. Used by the API, must be specified.

int tidl::ConfigurationinNumChannels

Number of channels in the input frame (e.g. 3 for BGR) Used by the API, must be specified.

int tidl::ConfigurationpreProcType

Pre-processing type applied to the input frame Specific to each network, can take values from 0 to 4, default is 0 0 -> Caffe-Jacinto models 1 -> Caffe models (SqueezeNet) 2 -> TensorFlow (Inception, MobileNet) 3 -> CIFAR 10 4 -> JdetNet

bool tidl::ConfigurationrunFullNet

Force to run all layers, regardless of layersGroupId partitioning.

size_t tidl::ConfigurationNETWORK_HEAP_SIZE

Size of the device side network heap This heap is used for allocating memory required to run the network on the device. One per Execution Object.

size_t tidl::ConfigurationPARAM_HEAP_SIZE

Size of the device side heap used for parameter data. The size depends on the size of the parameter binary file. The constructor for Configuration sets PARAM_HEAP_SIZE to 9MB. There is one parameter heap for each instance of Executor .

std::string tidl::ConfigurationinData

Path to the input image file. This field is not used by the TIDL API itself. It can be used by applications to load an input image into a buffer. Can be empty if the application uses frameworks such as OpenCV to read images. Refer examples/test/main.cpp for usage.

std::string tidl::ConfigurationoutData

Path to the output image file. This field is not used by the TIDL API itself. It can be used by applications to specify a name for the output file. Can be empty if the application uses frameworks such as OpenCV to read images. Refer examples/test/main.cpp for usage.

std::string tidl::ConfigurationnetBinFile

Path to the TIDL network binary file. Used by the API, must be specified.

std::string tidl::ConfigurationparamsBinFile

Path to the TIDL parameter binary file Used by the API, must be specified.

std::map<int, int> tidl::ConfigurationlayerIndex2LayerGroupId

Map of layer index to layer group id. Used to override layer group assigment for layers. Any layer not specified in this map will retain its existing mapping.

bool tidl::ConfigurationenableOutputTrace

Enable tracing of output buffers associated with each layer.

bool tidl::ConfigurationenableApiTrace

Debug - Generates a trace of host and device function calls.

bool tidl::ConfigurationshowHeapStats

Debug - Shows total size of PARAM and NETWORK heaps. Also shows bytes available after all allocations. Can be used to adjust the heap size.

int tidl::ConfigurationquantHistoryParam1

Weight in percentage applied to previously processed input frame during application startup (first 10 frames of input).

TIDL maintains range statistics for previously processed frames. It quantizes the current inference activations using these range statistics from previous inputs (weighted average range). Therefore, the results observed when the input is processed on the device will not be identical to that observed during the import stage. Parameters to control quantization:

See

quantHistoryParam1, quantHistoryParam2, quantMargin

int tidl::ConfigurationquantHistoryParam2

Weight in percentage applied to previously processed input frames after the first 10 input frames.

int tidl::ConfigurationquantMargin

Margin added to the average in percentage.

std::vector<int> tidl::ConfigurationinConvType

subgraph data conversion type at subgraph inputs 0: float <-> Q, 1: float <-> float, 2: Q <-> Q

std::vector<int> tidl::ConfigurationinIsSigned

subgraph is signed data at subgraph inputs

std::vector<float> tidl::ConfigurationinScaleF2Q

subgraph scaleF2Q factor at subgraph inputs

std::vector<int> tidl::ConfigurationinIsNCHW

subgraph is external tensor NCHW layout at subgraph inputs

std::vector<int> tidl::ConfigurationoutConvType

subgraph data conversion type at subgraph outputs 0: float <-> Q, 1: float <-> float, 2: Q <-> Q

std::vector<int> tidl::ConfigurationoutIsSigned

subgraph is signed data at subgraph outputs

std::vector<float> tidl::ConfigurationoutScaleF2Q

subgraph scaleF2Q factor at subgraph outputs

std::vector<int> tidl::ConfigurationoutIsNCHW

subgraph is external tensor NCHW layout at subgraph outputs

Configuration file

TIDL API allows the user to create a Configuration object by reading from a file or by initializing it directly. Configuration settings supported by Configuration::ReadFromFile:

• numFrames
• inWidth
• inHeight
• inNumChannels
• preProcType
• layerIndex2LayerGroupId
• inData
• outData
• netBinFile
• paramsBinFile
• quantHistoryParam1
• quantHistoryParam2
• quantMargin
• enableTrace

An example configuration file:

numFrames     = 1
preProcType   = 0
inData        = ../test/testvecs/input/preproc_0_224x224.y
outData       = stats_tool_out.bin
netBinFile    = ../test/testvecs/config/tidl_models/tidl_net_imagenet_jacintonet11v2.bin
paramsBinFile = ../test/testvecs/config/tidl_models/tidl_param_imagenet_jacintonet11v2.bin
inWidth       = 224
inHeight      = 224
inNumChannels = 3

# Enable tracing of output buffers
enableTrace = true

Overriding layer group assignment

The TIDL device translation tool assigns layer group ids to layers during the translation process. TIDL API 1.1 and higher allows the user to override this assignment by specifying explicit mappings. There are two ways for the user to provide an updated mapping:

1. Specify a mapping in the configuration file to indicate that layers 12, 13 and 14 are assigned to layer group 2:

layerIndex2LayerGroupId = { {12, 2}, {13, 2}, {14, 2} }

2. User can also provide the layer index to group mapping in the code:

Configuration c;
c.ReadFromFile("test.cfg");
c.layerIndex2LayerGroupId = { {12, 2}, {13, 2}, {14, 2} };

Executor

class tidl::Executor

Manages the overall execution of a layersGroup in a network using the specified configuration and the set of devices available to the executor.

Public Functions

tidl::ExecutorExecutor(DeviceType device_type, const DeviceIds &ids, const Configuration &configuration, int layers_group_id = OCL_TIDL_DEFAULT_LAYERS_GROUP_ID)

Create an Executor object.

The Executor will create the required ExecutionObject’s and initialize them with the specified TI DL network. E.g.

Configuration configuration;
configuration.ReadFromFile("path to configuration file");
DeviceIds ids = {DeviceId::ID2, DeviceId::ID3};
Executor executor(DeviceType::EVE, ids, configuration);

Parameters

• device_type: DSP or EVE device
• ids: Set of devices uses by this instance of the Executor
• configuration: Configuration used to initialize the Executor
• layers_group_id: Layers group that this Executor should run

tidl::Executor~Executor()

Tear down an Executor and free resources used by the Executor object.

ExecutionObject *tidl::Executoroperator[](uint32_t index) const

Returns a single execution object at index.

uint32_t tidl::ExecutorGetNumExecutionObjects() const

Returns the number of ExecutionObjects associated with the Executor

Public Static Functions

static uint32_t tidl::ExecutorGetNumDevices(DeviceType device_type)

Returns the number of devices of the specified type available for TI DL.

Return

number of devices available

Parameters

• device_type: DSP or EVE/EVE device

static std::string tidl::ExecutorGetAPIVersion()

Returns a string corresponding to the API version.

Return

<major_ver>.<minor_ver>.<patch_ver>.<git_sha>

ExecutionObject

class tidl::ExecutionObject

Runs the TIDL network on an OpenCL device.

Inherits from ExecutionObjectInternalInterface

Public Functions

void tidl::ExecutionObjectSetInputOutputBuffer(const ArgInfo &in, const ArgInfo &out)

Specify the input and output buffers used by the EO

Parameters

• in: buffer used for input.
• out: buffer used for output.

char *tidl::ExecutionObjectGetInputBufferPtr() const

Returns a pointer to the input buffer set via SetInputOutputBuffer.

size_t tidl::ExecutionObjectGetInputBufferSizeInBytes() const

Returns size of the input buffer.

char *tidl::ExecutionObjectGetOutputBufferPtr() const

Returns a pointer to the output buffer.

size_t tidl::ExecutionObjectGetOutputBufferSizeInBytes() const

Returns size of the output buffer.

void tidl::ExecutionObjectSetFrameIndex(int idx)

Set the frame index of the frame currently processed by the ExecutionObject. Used for trace/debug messages.

Parameters

• idx: index of the frame

int tidl::ExecutionObjectGetFrameIndex() const

Returns the index of a frame being processed (set by SetFrameIndex)

bool tidl::ExecutionObjectProcessFrameStartAsync()

Start processing a frame. The call is asynchronous and returns immediately. Use ExecutionObject::ProcessFrameWait to wait.

bool tidl::ExecutionObjectProcessFrameWait()

Wait for the execution object to complete processing a frame

Return

false if ExecutionObject::ProcessFrameWait was called without a corresponding call to ExecutionObject::ProcessFrameStartAsync.

float tidl::ExecutionObjectGetProcessTimeInMilliSeconds() const

return the number of milliseconds taken on the device to execute the process call

Return

Number of milliseconds to process a frame on the device.

const std::string &tidl::ExecutionObjectGetDeviceName() const

Returns the device name that the ExecutionObject runs on.

void tidl::ExecutionObjectWriteLayerOutputsToFile(const std::string &filename_prefix = "trace_dump_") const

Write the output buffer for each layer to a file <filename_prefix>_<ID>_HxW.bin

const LayerOutput *tidl::ExecutionObjectGetOutputFromLayer(uint32_t layer_index, uint32_t output_index = 0) const

Returns a LayerOutput object corresponding to a layer. Caller is responsible for deleting the LayerOutput object.

See

LayerOutput

Parameters

• layer_index: The layer index of the layer
• output_index: The output index of the buffer for a given layer. Defaults to 0.

const LayerOutputs *tidl::ExecutionObjectGetOutputsFromAllLayers() const

Get output buffers from all layers.

int tidl::ExecutionObjectGetLayersGroupId() const

Returns the layersGrupId that the ExecutionObject is processing.

ExecutionObjectPipeline

class tidl::ExecutionObjectPipeline

Manages the pipelined execution using multiple ExecutionObjects. Each executor runs one layersGroup of the network. ExecutionObjects must run consecutive layersGroups to form a pipelined execution.

Inherits from ExecutionObjectInternalInterface

Public Functions

tidl::ExecutionObjectPipelineExecutionObjectPipeline(std::vector<ExecutionObject *> eos)

Create an ExecutionObjectPipeline object.

The ExecutionObjectPipeline will take the provided ExecutionObjects to create an execution pipeline. E.g.

Configuration config("path to configuration file");
DeviceIds ids = {DeviceId::ID0, DeviceId::ID1};
Executor exe_eve(DeviceType::EVE, ids, config, 1);
Executor exe_dsp(DeviceType::DSP, ids, config, 2);
ExecutionObjectPipeline ep0({exe_eve[0], exe_dsp[0]});
ExecutionObjectPipeline ep1({exe_eve[1], exe_dsp[1]});

Parameters

• eos: DSP or EVE ExecutionObjects forming a pipeline

tidl::ExecutionObjectPipeline~ExecutionObjectPipeline()

Tear down an ExecutionObjectPipeline and free used resources.

uint32_t tidl::ExecutionObjectPipelineGetNumExecutionObjects() const

Returns the number of ExecutionObjects associated with the ExecutionObjectPipeline

void tidl::ExecutionObjectPipelineSetInputOutputBuffer(const ArgInfo &in, const ArgInfo &out)

Specify the input and output buffers used by the EOP

Parameters

• in: buffer used for input.
• out: buffer used for output.

char *tidl::ExecutionObjectPipelineGetInputBufferPtr() const

Returns a pointer to the input buffer.

size_t tidl::ExecutionObjectPipelineGetInputBufferSizeInBytes() const

Returns size of the input buffer.

char *tidl::ExecutionObjectPipelineGetOutputBufferPtr() const

Returns a pointer to the output buffer.

size_t tidl::ExecutionObjectPipelineGetOutputBufferSizeInBytes() const

Returns the number of bytes written to the output buffer.

void tidl::ExecutionObjectPipelineSetFrameIndex(int idx)

Set the frame index of the frame currently processed by the ExecutionObjectPipeline. Used for trace/debug messages.

Parameters

• idx: index of the frame

int tidl::ExecutionObjectPipelineGetFrameIndex() const

Returns the index of a frame being processed (set by SetFrameIndex)

bool tidl::ExecutionObjectPipelineProcessFrameStartAsync()

Start processing a frame. The call is asynchronous and returns immediately. Use ProcessFrameWait() to wait.

bool tidl::ExecutionObjectPipelineProcessFrameWait()

Wait for the executor pipeline to complete processing a frame

Return

false if ProcessFrameWait() was called without a corresponding call to ExecutionObjectPipeline::ProcessFrameStartAsync().

const std::string &tidl::ExecutionObjectPipelineGetDeviceName() const

Return the combined device names that this pipeline runs on.

void tidl::ExecutionObjectPipelineWriteLayerOutputsToFile(const std::string &filename_prefix = "trace_dump_") const

Write the output buffer for each layer to a file <filename_prefix>_<ID>_HxW.bin

const LayerOutput *tidl::ExecutionObjectPipelineGetOutputFromLayer(uint32_t layer_index, uint32_t output_index = 0) const

Returns a LayerOutput object corresponding to a layer. Caller is responsible for deleting the LayerOutput object.

See

LayerOutput

Parameters

• layer_index: The layer index of the layer
• output_index: The output index of the buffer for a given layer. Defaults to 0.

const LayerOutputs *tidl::ExecutionObjectPipelineGetOutputsFromAllLayers() const

Get output buffers from all layers.