class cerebras.sdk.runtime.sdkruntimepybind.SdkRuntime(bindir: Union[pathlib.Path, str], **kwargs)
Bases: object
Manages the execution of SDK programs on the Cerebras Wafer Scale Engine
(WSE) or simfabric. The constructor analyzes the WSE ELFs in the bindir
and prepares the WSE or simfabric for a run.
Requires CM IP address and port for WSE runs.
Path to ELF files which is compiled by cslc. The runtime collects the I/O and fabric parameters automatically, including height, width, number of channels, width of buffers, etc.
If True, suppresses generation of simfab_traces when running. Default value is False, i.e., simfab_traces are produced.
Note that producing simfab_traces can greatly slow down the wall clock
time of a simulator run. If you are not using the SDK GUI with the output
of your run, consider setting this value to True.
Message logging output level. Available output levels are DEBUG, INFO, WARNING, and ERROR. Default value is WARNING.
Example:In the following example, an SdkRuntime runner object is instantiated. If args.cmaddr is non-empty, then the kernel code will run on the WSE pointed to by that address; otherwise, the kernel code will run on simfabric. The compiled kernel code in the directory args.name has exported symbols A and B pointing to arrays on the device. After loading the code and starting the run with load() and run(), data on the host stored in data is copied to A on the device, and then B on the device is copied back into data on the host.
Convert a logical coordinate to a physical coordinate. For a program with fabric offsets (offset_x, offset_y), and program rectangle coordinate (x, y), this function returns (offset_x + x, offset_y + y).
Dump the core of a simulator run, to be used for debugging with csdb. Note that the specified name of the corefile MUST be “corefile.cs1” to use with csdb, and this method can only be called after a blocking SdkRuntime API call, or after calling SdkRuntime.stop().
Retrieve the integer representation of an exported symbol which is exported in the kernel. Possible symbols include a data tensor or a host-callable function.
Receive a host tensor from the device. The data is received from the region of interest (ROI) which is a bounding box starting at coordinate (px, py) with width w and height h.
Number of elements per PE. The data type of an element is 16-bit and 32-bit only. If the tensor has k elements per PE, elem_per_pe is k even if the data type is 16-bit. If the data type is 16-bit, the user has to extend the tensor to a 32-bit one, with zero filled in the higher 16 bits.
32-bit if MemcpyDataType.MEMCPY_32BIT or 16-bit if MemcpyDataType.MEMCPY_16BIT. Note that this argument has no effect if streaming is True, and the user must handle the data appropriately in the receiving wavelet-triggered task. Additionally, the underlying type of the tensor dest must be 32-bit. The tensor must be extended to a 32-bit one with zero filled in the higher 16 bits.
Send a host tensor to the device. The data is distributed into the region of interest (ROI) which is a bounding box starting at coordinate (px, py) with width w and height h.
Number of elements per PE. The data type of an element is 16-bit and 32-bit only. If the tensor has k elements per PE, elem_per_pe is k even if the data type is 16-bit. If the data type is 16-bit, the user has to extend the tensor to a 32-bit one, with zero filled in the higher 16 bits.
Broadcast a row of host data down columns of PEs. The data is distributed across the first row in the region of interest (ROI), which is a bounding box starting at coordinate (px, py) with width w and height h, and then broadcast down each column of the ROI.
Number of elements per PE. The data type of an element is 16-bit and 32-bit only. If the tensor has k elements per PE, elem_per_pe is k even if the data type is 16-bit. If the data type is 16-bit, the user has to extend the tensor to a 32-bit one, with zero filled in the higher 16 bits.
Broadcast a column of host data across rows of PEs. The data is distributed across the first column in the region of interest (ROI), which is a bounding box starting at coordinate (px, py) with width w and height h, and then broadcast across each row of the ROI.
Number of elements per PE. The data type of an element is 16-bit and 32-bit only. If the tensor has k elements per PE, elem_per_pe is k even if the data type is 16-bit. If the data type is 16-bit, the user has to extend the tensor to a 32-bit one, with zero filled in the higher 16 bits.
Send a host tensor to the device with a stride pattern across receiving PEs. The data is distributed into the region of interest (ROI) which is a bounding box starting at coordinate (px, py) with width w and height h. Across a given row, row_stride determines the stride between receiving PEs within the ROI, and across a given column, col_stride determines the stride between receiving PEs.The first row and column to which data is sent is given by the PE (px, py) at the top-left of the ROI.We denote by xi and eta the number of columns and rows to which elements will be sent in the ROI, respectively. Since the ROI is w PEs wide and h PEs tall, xi and eta are given by xi = 1 + floor((w - 1) / row_stride) and eta = 1 + floor((h - 1) / col_stride).As an example, consider an ROI starting at (0, 0) with width 6 and height 8, and row and column strides 3 and 2, respectively. Then PEs with x coordinate 0 or 3 and y coordinate 0, 2, 4, 6 will receive data from the host. In this case, xi = 2 and eta = 4.
Number of elements per PE. The data type of an element is 16-bit and 32-bit only. If the tensor has k elements per PE, elem_per_pe is k even if the data type is 16-bit. If the data type is 16-bit, the user has to extend the tensor to a 32-bit one, with zero filled in the higher 16 bits.
Stride between PEs within a row in the ROI. Since the ROI is w PEs wide, the number of columns to which elements will be sent is xi = 1 + floor((w - 1) / row_stride).
Stride between PEs within a column in the ROI. Since the ROI is h PEs tall, the number of rows to which elements will be sent is eta = 1 + floor((h - 1) / col_stride).
Part of the SdkRuntime direct link API.Same as above when src.dtype is exactly np.int32, np.uint32 or np.float32. In that case, the runtime infers n_wavelets from len(src).
Wait for all pending commands (data transfers and kernel function calls) to complete and then stop simfabric or WSE. After this call is complete, no new commands will be accepted for this SdkRuntime object.SdkRuntime.stop() must be called to end a program. Otherwise, the runtime will emit an error.
Array returned from device containing elapsed timestamp data.
Returns: Elapsed cycle count.
Return type: numpy.int64
Example:Consider the following CSL snippet which records timestamps and produces a single array to copy back to the host, to generate an elapsed cycle count:
// import time module and create timestamp buffersconst timestamp = @import_module("<time>");var tsc_end_buf = @zeros([timestamp.tsc_size_words]u16);var tsc_start_buf = @zeros([timestamp.tsc_size_words]u16);// create elapsed timer buffer and advertise to hostvar timer_buf = @zeros([3]f32);var ptr_timer_buf: [*]f32 = &timer_buf;timestamp.enable_tsc();// record starting timestamptimestamp.get_timestamp(&tsc_start_buf);// perform some operation for which you want to calculate elapsed cycles// record ending timestamptimestamp.get_timestamp(&tsc_end_buf);timestamp.disable_tsc();var lo_: u16 = 0;var hi_: u16 = 0;var word: u32 = 0;lo_ = tsc_start_buf[0];hi_ = tsc_start_buf[1];timer_buf[0] = @bitcast(f32, (@as(u32,hi_) << @as(u16,16)) | @as(u32, lo_) );lo_ = tsc_start_buf[2];hi_ = tsc_end_buf[0];timer_buf[1] = @bitcast(f32, (@as(u32,hi_) << @as(u16,16)) | @as(u32, lo_) );lo_ = tsc_end_buf[1];hi_ = tsc_end_buf[2];timer_buf[2] = @bitcast(f32, (@as(u32,hi_) << @as(u16,16)) | @as(u32, lo_) );
Then the elapsed cycles can be calculated on the host with:
# Get symbol for timer_buf on devicesymbol_timer_buf = runner.get_id("timer_buf")# Copy back timer_buf from all width x height PEsdata = np.zeros((width*height*3, 1), dtype=np.uint32)runner.memcpy_d2h(data, symbol_timer_buf, 0, 0, width, height, 3, streaming=False, data_type=MemcpyDataType.MEMCPY_32BIT, order=MemcpyOrder.ROW_MAJOR, nonblock=False)elapsed_time_hwl = data.view(np.float32).reshape((height, width, 3))# Print elapsed cycles for each PEfor pe_x in range(width): for pe_y in range(height): cycle_cnt = sdk_utils.calculate_cycles(elapsed_time_hwl[pe_y, pe_x, :]) print("Elapsed cycles on PE ", pe_x, ", ", pe_y, ": ", cycle_cnt)
Converts a 16-bit tensor to a 32-bit tensor of type u32 for use with memcpy. The parameter sentinel distinguishes two different extensions of 16-bit data. If sentinel is None, zero-pad the upper 16 bits. If sentinel is not None, pack the index of the innermost dimension of the array into the upper 16-bits.
For 16-bit input data, if this parameter is not None, pack the index of the innermost dimension into the high bits of the 32-bit wavelet. If sentinel is None, then the high bits are zeros.
The numpy data type which should be used in the output view. The itemsize must be 1, 2, or 4 bytes.
Returns: Numpy view into arr with specified numpy data type.
Return type: numpy.ndarray.view
Example:memcpy_view() simplifies the use of various precision data types when copying between host and device. Consider the following Python host code which creates a float16 view into a numpy array. Note that this array must be 32-bit. The user can fill the array with float16 data, and copy it to an array on the device with CSL data type f16.
x_symbol = runner.get_symbol('x')# This container array must be 32-bitx_container = np.zeros(N, dtype=np.uint32)x = sdk_utils.memcpy_view(x_container, np.float16)x.fill(0.5)runner.memcpy_h2d(x_symbol, x_container, 0, 0, 1, 1, N, streaming=False, data_type=MemcpyDataType.MEMCPY_16BIT, order=MemcpyOrder.ROW_MAJOR, nonblock=False)
class cerebras.sdk.debug.debug_util.debug_util(bindir: Union[pathlib.Path, str])
Bases: object
Loads ELF files in bindir in order to dump symbols for debugging.The user does not need to export the symbols in the kernel. debug_util dumps the core and looks for the symbols in the ELFs. If the symbol at Px.y is not found in the corresponding ELF, debug_util emits an error.The most common errors are either: 1) a wrong coordinate passed in debug_util.get_symbol(), or 2) a correct coordinate, but the symbol has been removed due to compiler optimization. One can use readelf to check if the symbol exists or not. If not, the user can export the symbol in the kernel to keep the symbol in the ELF.The functionality of this class is only supported in the simulator.
from cerebras.sdk.debug.debug_util import debug_util# run the app# dirname is the path to ELFssimulator = SdkRuntime(dirname)simulator.load()simulator.run()...simulator.stop()# retrieve symbols after the rundebug_mod = debug_util(dirname)# assume the core rectangle starts at P4.1, the dimension is# width-by-height and we want to retrieve the symbol y for every PEcore_offset_x = 4core_offset_y = 1for py in range(height): for px in range(width): t = debug_mod.get_symbol(core_offset_x+px, core_offset_y+py, 'y', np.float32) print(f"At (py, px) = {py, px}, symbol y = {t}")
Read the value of symbol of given type at given PE coordinates. Note that each call to this function scans the whole fabric, so prefer debug_util.get_symbol_rect() over calling this in a loop.
Returns: Numpy array of output values read at symbol. The first two dimensions of the returned array are PE coordinates (column, row) relative to the rectangle.
