# SDK Appliance API Reference Source: https://sdk.cerebras.ai/api-docs/appliance-api Compile and run Cerebras SDK programs on a Wafer-Scale Cluster using the Appliance API — covers `SdkCompiler`, `SdkLauncher`, job submission, and appliance mode execution. See [Running SDK on a Wafer-Scale Cluster](/appliance-mode) for an introduction to appliance mode. ## SdkCompiler Python API for compiling SDK programs on a Cerebras Wafer-Scale Cluster. Manages the generation of compile artifacts on a Cerebras Wafer-Scale Cluster using the CSL compiler. [`SdkCompiler`](#sdkcompiler) must be used via a context manager. Appliance cluster namespace to which the job is submitted. Default is the default namespace. CPU cores on the WSC management node used by the compile job in units of 1/1000 CPU (default: 24000, or 24 cores) Memory in bytes requested from the management node for the compile job (default: `67 << 30`, or 67 GiB) If `True`, ignore version differences between appliance client and server. **Example**: In the following example, an [`SdkCompiler`](#sdkcompiler) object is instantiated via a context manager. The [`SdkCompiler.compile()`](#compile) function takes four arguments: * the directory containing the CSL code files, * the name of the top level CSL code file that contains the layout block, * the compiler arguments, * and the output directory or output file for the compile artifacts. ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} import json from cerebras.sdk.client import SdkCompiler # Instantiate compiler using a context manager with SdkCompiler(disable_version_check=True) as compiler: # Launch compile job artifact_path = compiler.compile( ".", "layout.csl", "--fabric-dims=8,3 --fabric-offsets=4,1 --memcpy --channels=1 -o out", "." ) # Write the artifact_path to a JSON file with open("artifact_path.json", "w", encoding="utf8") as f: json.dump({"artifact_path": artifact_path,}, f) ``` Generates compile artifacts using the CSL compiler. Directory containing CSL code files. Top-level CSL file containing the layout block. Arguments passed to the CSL compiler. Output directory or file for compile artifacts. > * **Returns**: String containing local path to compile artifacts. > * **Return type**: `str` ## SdkLauncher The SdkLauncher API can be used to upload artifacts, run custom commands in the appliance, and use custom scripts written as if the system was not in appliance mode and you were running directly from a worker node. You must use the `%CMADDR%` template string to pass the system address to a run script. Path to a compiled artifact which will be transferred and extracted in the appliance. If `True`, runs the program on the simulator using a worker node of the Wafer-Scale Cluster. Default value is `False`, i.e., allocates and runs on a WSE. Appliance cluster namespace to which the job is submitted. Default is the default namespace. CPU cores on the WSC management node used by the compile job in units of 1/1000 CPU (default: 24000, or 24 cores) Memory in bytes requested from the management node for the compile job (default: `67 << 30`, or 67 GiB) If `True`, ignore version differences between appliance client and server. **Example**: In the following example, an [`SdkLauncher`](#sdklauncher) object is instantiated via a context manager, with path to compile artifacts given by `artifact_path`. `launcher.stage` transfers an additional file `additional_artifact.txt` to the appliance. Next, `launcher.run` runs a command on the appliance worker node which writes the contents of that file to stdout. This example then uses `launcher.run` to run a host Python script `run.py`. Notice that it specifies the system's CM address passed to this script via the template string `%CMADDR`, which will be evaluated at runtime based on the system allocated to this job. It also uses `download_artifact` to transfer a file back from the appliance. ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} import json import os from cerebras.sdk.client import SdkLauncher # read the compile artifact_path from the json file with open("artifact_path.json", "r", encoding="utf8") as f: data = json.load(f) artifact_path = data["artifact_path"] # artifact_path contains the path to the compiled artifact. # It will be transferred and extracted in the appliance. # The extracted directory will be the working directory. # Set simulator=False if running on CS system within appliance. with SdkLauncher(artifact_path, simulator=False, disable_version_check=True) as launcher: # Transfer an additional file to the appliance, # then write contents to stdout on appliance launcher.stage("additional_artifact.txt") response = launcher.run( "echo \"ABOUT TO RUN IN THE APPLIANCE\"", "cat additional_artifact.txt", ) print("Test response: ", response) # Run the original host code as-is on the appliance, # using the same cmd as when using the Singularity container response = launcher.run("cs_python run.py --name out --cmaddr %CMADDR%") print("Host code execution response: ", response) # Fetch files from the appliance launcher.download_artifact("out.txt", "./output_dir/out.txt") ``` Downloads an artifact from the appliance. If the artifact is a directory, a tarball of that directory will be created and transferred. Name of the artifact to download. Local path where the artifact will be saved. > * **Returns**: The name of the file that has been written (can contain a `.tar.gz` > extension if the `artifact_name` was a directory.) > * **Return type**: `str` Stages additional artifacts in the remote working directory within the appliance. Local path to the artifact to be staged on the appliance. Run one or more shell commands on the appliance. One or more command strings. Use the special placeholder `%CMADDR%` wherever a CS system address should be substituted. **All** positional arguments must be strings. ## SdkRuntime The [`SdkRuntime`](#sdkruntime) appliance bindings are deprecated. Use [`SdkLauncher`](#sdklauncher) to wrap an SDK host Python script instead. Manages the execution of SDK programs on the Cerebras Wafer-Scale Cluster appliance. The constructor analyzes the WSE ELFs in the `bindir` and prepares the WSE or simfabric for a run. [`SdkRuntime`](#sdkruntime) must be used via a context manager. Path to ELF files compiled by [`SdkCompiler`](#sdkcompiler). The runtime collects the I/O and fabric parameters automatically, including height, width, number of channels, width of buffers, etc. If `True`, runs the program on simulator using a worker node of the Wafer-Scale Cluster. Default value is `False`, i.e., allocates and runs on a WSE. Appliance cluster namespace to which the job is submitted. Default is the default namespace. CPU cores on the WSC management node used by the compile job in units of 1/1000 CPU (default: 24000, or 24 cores) Memory in bytes requested from the management node for the compile job (default: `67 << 30`, or 67 GiB) If `True`, ignore version differences between appliance client and server. **Example**: In the following example, an [`SdkRuntime`](#sdkruntime) runner object is instantiated via a context manager, with path to compile artifacts given by `artifact_path`. The compiled kernel code has exported symbols `my_fn`, which names a function defined on all PEs in the program, and `A`, which points to an array on all PEs. The context manager loads and starts the program. Then, the function `my_fn` is launched. After this function is launched, `A` on the device is copied back into `data` on the host. ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} import json import os from cerebras.sdk.client import SdkRuntime # Read the artifact_path from the JSON file with open("artifact_path.json", "r", encoding="utf8") as f: data = json.load(f) artifact_path = data["artifact_path"] # Instantiate a runner object using a context manager. # Set simulator=False if running on CS system within appliance. with SdkRuntime(artifact_path, simulator=False, disable_version_check=True) as runner: # Launch my_fn on device runner.launch('my_fn', nonblock=False) # Copy A back from device symbol_A = runner.get_id("A") runner.memcpy_d2h(data, symbol_A, px, py, w, h, l, streaming=False, data_type=memcpy_dtype, order=memcpy_order, nonblock=False) ``` Like [`launch`](#launch), but without type checking on the arguments. The caller passes a list of integer-castable arguments which are packed as `uint32` before transfer. The exported name of the symbol corresponding to a host-callable function. List of arguments to pass to the function, cast to integers. Nonblocking if `True`, blocking otherwise. This kwarg is required. > * **Returns**: Handle to the task if `nonblock=True`, else `None`. > * **Return type**: `Optional[Task]` Downloads an artifact from the appliance worker node to the local host. If the artifact is a directory, a tarball of that directory will be created and transferred. Name of the artifact to download. Local path where the artifact will be saved. > * **Returns**: The name of the file that has been written (can contain a `.tar.gz` > extension if the `artifact_name` was a directory.) > * **Return type**: `str` See [SdkRuntime API Reference](/api-docs/sdkruntime-api#sdkruntime). Returns `True` if this runtime is running on the appliance simulator, `False` if running on a real CS system. See [SdkRuntime API Reference](/api-docs/sdkruntime-api#sdkruntime). See [SdkRuntime API Reference](/api-docs/sdkruntime-api#sdkruntime). See [SdkRuntime API Reference](/api-docs/sdkruntime-api#sdkruntime). See [SdkRuntime API Reference](/api-docs/sdkruntime-api#sdkruntime). Start program execution on the CS system or simulator. This step also encompasses loading the program. May be called at most once per [`SdkRuntime`](#sdkruntime) object. When using the context-manager form (`with SdkRuntime(...) as runner:`), `start()` is invoked implicitly on entry and explicit calls are unnecessary. Stop program execution. Must follow [`start()`](#start). Pending nonblocking operations are flushed before this call returns, so D2H destination buffers are guaranteed populated after `stop()` returns. When using the context-manager form, `stop()` is invoked implicitly on exit; calling it again afterwards is an error. See [SdkRuntime API Reference](/api-docs/sdkruntime-api#sdkruntime). ## Task Handle to a task launched by [`SdkRuntime`](#sdkruntime). ## MemcpyDataType Specifies the data size for transfers using [`SdkRuntime.memcpy_d2h()`](/api-docs/sdkruntime-api#memcpy_d2h) and [`SdkRuntime.memcpy_h2d()`](/api-docs/sdkruntime-api#memcpy_h2d) copy mode. > **Values**: > > * MEMCPY\_16BIT > * MEMCPY\_32BIT ## MemcpyOrder Specifies mapping of data for transfers using [`SdkRuntime.memcpy_d2h()`](/api-docs/sdkruntime-api#memcpy_d2h) and [`SdkRuntime.memcpy_h2d()`](/api-docs/sdkruntime-api#memcpy_h2d). > **Values**: > > * ROW\_MAJOR > * COL\_MAJOR ## sdk\_utils Utility functions for common operations with [`SdkRuntime`](#sdkruntime). Import from `cerebras.sdk.client.sdk_utils`. See [sdk\_utils module](/api-docs/sdkruntime-api#sdk_utils-module). ## debug\_util Utilities for parsing debug output and core files of a simulator run. Import from `cerebras.sdk.client.debug_util`. See [debug\_util module](/api-docs/sdkruntime-api#debug_util-module). # SdkLayout API Reference Source: https://sdk.cerebras.ai/api-docs/sdklayout-api Use `SdkLayout` to define code regions, configure colors, and specify program layout for Cerebras SDK kernels. This API is part of the `sdkruntimepybind` module documented in the [SdkRuntime API Reference](/api-docs/sdkruntime-api). ## Imports The classes and enums documented on this page live in `cerebras.sdk.runtime.sdkruntimepybind`. The canonical import block for an `SdkLayout`-based host program is: ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} from cerebras.sdk.runtime.sdkruntimepybind import ( SdkLayout, SdkTarget, Color, Route, Edge, RoutingPosition, ) ``` For host programs that mix `SdkLayout` with `SdkRuntime` (the typical case), import `SdkRuntime` and `SdkCompileArtifacts` from the same module: ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} from cerebras.sdk.runtime.sdkruntimepybind import ( SdkRuntime, SdkCompileArtifacts, ) ``` ## CodeRegion Specifies a code region. Create and return a new color that is scoped within a region. Name to be assigned to the color. > * **Returns**: A new color scoped within this region. > * **Return type**: [`Color`](#color) Create and return a new color (with a value) that is scoped within a region. Name to be assigned to the color. Value to be assigned to the color. > * **Returns**: A new color scoped within this region. > * **Return type**: [`Color`](#color) Given a color, an orientation (i.e., `edge`), a set of output routes, a size, and an optional prefix, create and return a new input communication port. The optional prefix can be used to create unique ports with the same color. Ports must be unique, otherwise an exception is thrown. Color of the input port. Edge on which to create the input port. List of output routes for the input port. The size of the port's data, used to verify port compatibility. Optional prefix to port's name, which allows creation of unique ports with the same color. > * **Returns**: Handle to the created input port. > * **Return type**: [`PortHandle`](#porthandle) Given a color, an orientation (i.e., `edge`), a set of input routes, a size, and an optional prefix, create and return a new output communication port. The optional prefix can be used to create unique ports with the same color. Ports must be unique, otherwise an exception is thrown. Color of the output port. Edge on which to create the output port. List of input routes for the output port. The size of the port's data, used to verify port compatibility. Optional prefix to port's name, which allows creation of unique ports with the same color. > * **Returns**: Handle to the created output port. > * **Return type**: [`PortHandle`](#porthandle) Set the routing for a given color on a single PE within this region. Coordinate on which color will be painted. Color to be painted on this coordinate. List of routing positions which will be applied to this color. Set the routing for a given color on all PEs of the region. Color to be painted on this region. List of routing positions which will be applied to this color. Set the routing for a given color on all PEs of the region with special routing on one or all region edges. Color to be painted on this region. List of routing positions which will be applied to this color. List of routing positions to be applied to this color on the region's edges. Set the routing for a given color on a contiguous rectangular subset of PEs within this region. Rectangular subset of PEs within this region on which color will be painted. Color to be painted on this rectangle. List of routing positions which will be applied to this color. Place code region at specific coordinates `(x, y)`. x-coordinate at which code region will be placed. y-coordinate at which code region will be placed. Set an unsigned integer parameter on a single PE within this region. Coordinate on which parameter will be set. Name of the parameter. Unsigned integer value of the parameter. Set a color parameter on a single PE within this region. Coordinate on which parameter will be set. Color value of the parameter. Set the value of parameter `name` on a single PE within this region with the value of color `value`. Coordinate on which parameter will be set. Name of the parameter. Color value of the parameter. Set an unsigned integer parameter on all PEs of the region. Name of the parameter. Unsigned integer value of the parameter. Set a color parameter on all PEs of the region. Color value of the parameter. Set the value of parameter `name` on all PEs of the region with the value of color `value`. Name of the parameter. Color value of the parameter. Set an unsigned integer parameter on a contiguous rectangular subset of PEs within this region. Rectangular subset of PEs within this region on which parameter will be set. Name of the parameter. Unsigned integer value of the parameter. Set a color parameter on a contiguous rectangular subset of PEs within this region. Rectangular subset of PEs within this region on which parameter will be set. Color value of the parameter. Set the value of parameter `name` on a contiguous rectangular subset of PEs within this region with the value of color `value`. Rectangular subset of PEs within this region on which parameter will be set. Name of the parameter. Color value of the parameter. Given a symbol name `symbol` and `data` with a given 2D shape specified by `width` and `height`, store `data` uniformly across the PEs of this code region. The 2D shape of `data` must be a multiple of the code region's dimensions or an error will be emitted. Name of the symbol. 2D data array to be applied to the symbol. Data must be of type `np.int32`, `np.uint32`, `np.float32`, `np.int16`, or `np.uint16`. Width of 2D data array. Height of 2D data array. ## Color Represents a color with an optional user-specified physical value. Objects of this class can be used for routing and can also be used to set microcode parameter values. If a physical value is not provided, then a physical value will be allocated automatically by the compiler. Name given to the color. Physical value given to the color. If not provided, then a value will be allocated automatically by the compiler. The maximum value is 23. Returns the global name of the color. If this color is attached to a region, then the name returned takes the form `region_name` + `_` + `name`. > * **Returns**: Global name of the color. > * **Return type**: `str` Returns the name of the color. > * **Returns**: Name of the color. > * **Return type**: `str` Returns the color's physical value if one has been assigned. Otherwise, returns `None`. > * **Returns**: Physical value of the color. > * **Return type**: `Optional[int]` ## Edge Represents edge positions along the boundary of a code region. > **Values**: > > * TOP > * BOTTOM > * LEFT > * RIGHT ## EdgeRouteInfo For a given code region, represents the routing positions in one of the region's four edges. ## FP16TYPE Specifies the 16-bit floating point format for compilation. > **Values**: > > * **F16**: IEEE 754 half-precision (`f16`) > * **BF16**: Brain floating point (`bf16`) > * **CB16**: Cerebras 16-bit floating point (`cb16`) ## SdkLayout Specifies a program layout. This API allows you to define rectangular code regions, define color routing and switching, automatically allocate colors, and automatically route between code regions. Execution platform specification. Message logging output level. Available output levels are `DEBUG`, `INFO`, `WARNING`, and `ERROR`. Default value is `WARNING`. Constructor variant that takes a path to a fabric JSON file which is used to define the compile target and execution platform. Takes same kwargs as above. Path to a fabric JSON file. Constructor variant that takes a target architecture. Takes same kwargs as above. Target architecture for compilation. Compile this layout and produce artifacts with a given path prefix. Path to which artifacts will be produced. List of additional library search paths for the compiler. Path prefix for the CSL compiler. If empty, the default compiler is used. If `True`, saves the port mapping to a file. Specifies the 16-bit floating point format for compilation. > * **Returns**: Compilation artifacts. > * **Return type**: [`SdkCompileArtifacts`](/api-docs/sdkruntime-api#sdkcompileartifacts) Automatically connect two ports. Transmitting output data port which will send data to `rx`. Receiving input data port which will receive data from `tx`. Create a code region. Path to code source file. Name of the created code region. Width in PEs of the created code region. Height in PEs of the created code region. > * **Returns**: The created code region object. > * **Return type**: [`CodeRegion`](#coderegion) Sets up an input stream from the host to a 1-PE region at `io_loc` and then to input port `port`. If `io_loc` is not provided, an available location will be automatically picked. `io_buffer_size` can be provided to specify the buffer size at `io_loc`. Returns the name of the stream's port which can be used by the [`SdkRuntime`](/api-docs/sdkruntime-api#sdkruntime) direct link API method [`SdkRuntime.send()`](/api-docs/sdkruntime-api#send). Handle to input port. PE location on wafer which receives input stream data. If not provided, a location is automatically chosen. Buffer size allocated at `io_loc`. Default is 1024. > * **Returns**: Name of created input stream's port. > * **Return type**: `str` Sets up an input stream from the host to `loc` on a given color `color` assuming that a code region is already defined at `loc` to consume the incoming data. An optional prefix can be provided to uniquely identify the stream in case of naming conflicts. Returns the name of the stream's port which can be used by the [`SdkRuntime`](/api-docs/sdkruntime-api#sdkruntime) direct link API method [`SdkRuntime.send()`](/api-docs/sdkruntime-api#send). PE location of existing input port into which data will be streamed. Color on which stream transmits data. Optional prefix prepended to created stream's name. > * **Returns**: Name of created input stream's port. > * **Return type**: `str` Sets up an output stream from output port `port` to a 1-PE region at `io_loc` and then to the host. If `io_loc` is not provided, an available location will be automatically picked. `io_buffer_size` can be provided to specify the buffer size at `io_loc`. Returns the name of the stream's port which can be used by the [`SdkRuntime`](/api-docs/sdkruntime-api#sdkruntime) direct link API methods [`SdkRuntime.receive()`](/api-docs/sdkruntime-api#receive) and [`SdkRuntime.receive_tofile()`](/api-docs/sdkruntime-api#receive_tofile). Handle to output port. PE location on wafer which sends out output stream data. If not provided, a location is automatically chosen. Buffer size allocated at `io_loc`. Default is 1024. > * **Returns**: Name of created output stream's port. > * **Return type**: `str` Sets up an output stream to the host from `loc` on a given color `color` assuming that a code region is already defined at `loc` to produce the outgoing data. An optional prefix can be provided to uniquely identify the stream in case of naming conflicts. Returns the name of the stream's port which can be used by the [`SdkRuntime`](/api-docs/sdkruntime-api#sdkruntime) direct link API methods [`SdkRuntime.receive()`](/api-docs/sdkruntime-api#receive) and [`SdkRuntime.receive_tofile()`](/api-docs/sdkruntime-api#receive_tofile). PE location of existing output port from which data will be streamed. Color on which stream transmits data. Optional prefix prepended to created stream's name. > * **Returns**: Name of created output stream's port. > * **Return type**: `str` Place child code regions horizontally and relative to the first child in the given list, and return the width of the resulting code region. Code regions to be placed. > * **Returns**: Width of resulting code region. > * **Return type**: `int` Place child code regions horizontally and relative to a specified origin, and return the width of the resulting code region. Code regions to be placed. PE coordinate which serves as origin for placed code regions. > * **Returns**: Width of resulting code region. > * **Return type**: `int` Place child code regions vertically and relative to the first child in the given list, and return the height of the resulting code region. Code regions to be placed. > * **Returns**: Height of resulting code region. > * **Return type**: `int` Place child code regions vertically and relative to a specified origin, and return the height of the resulting code region. Code regions to be placed. PE coordinate which serves as origin for placed code regions. > * **Returns**: Height of resulting code region. > * **Return type**: `int` ## PortHandle Handle to a program input or output data port. ## Route Represents route directions. > **Values**: > > * RAMP > * EAST > * WEST > * NORTH > * SOUTH ## RoutingPosition Represents a single routing position, which can consist of one or more route values for input and one or more route values for output. Set a list of routes as the input route position. List of routes to be set as the input route position. Set a list of routes as the output route position. List of routes to be set as the output route position. Add a route to the input route position. Route to be added to the input route position. Add a route to the output route position. Route to be added to the output route position. For this routing position object, return a list of all input routes in the input route position. > * **Returns**: List of all routes in the input route position. > * **Return type**: List\[[`Route`](#route)] For this routing position object, return a list of all output routes in the output route position. > * **Returns**: List of all routes in the output route position. > * **Return type**: List\[[`Route`](#route)] ## get\_edge\_routing Construct an edge routing info object from a given edge and routing positions. Edge of code region. List of routing positions to be applied to edge. > * **Returns**: Object containing edge routing info. > * **Return type**: [`EdgeRouteInfo`](#edgerouteinfo) ## Geometry ### IntRectangle Defines a rectangle of values. Origin of rectangle's northwest corner. Width and height of rectangle. ### IntVector Wraps a tuple of two integer values, often used to specify coordinates or offsets. x-coordinate y-coordinate # SdkRuntime API Reference Source: https://sdk.cerebras.ai/api-docs/sdkruntime-api Use `SdkRuntime` to load and run kernels, copy data between host and device, and manage execution on the Cerebras Wafer-Scale Engine. ## sdkruntimepybind Module Python API for [`SdkRuntime`](#sdkruntime) functions. ### MemcpyDataType Specifies the data size for transfers using [`SdkRuntime.memcpy_d2h()`](#memcpy_d2h) and [`SdkRuntime.memcpy_h2d()`](#memcpy_h2d) copy mode. > **Values**: > > * MEMCPY\_16BIT > * MEMCPY\_32BIT ### MemcpyOrder Specifies mapping of data for transfers using [`SdkRuntime.memcpy_d2h()`](#memcpy_d2h) and [`SdkRuntime.memcpy_h2d()`](#memcpy_h2d) copy mode. > **Values**: > > * ROW\_MAJOR > * COL\_MAJOR ### SdkCompileArtifacts Specifies compile artifacts for execution. Path to compile artifacts. ### SdkExecutionPlatform Specifies the simulator or system target and architecture for execution. Queries if the execution platform is a simulator. > * **Returns**: `True` if the execution platform is a simulator, `False` otherwise. > * **Return type**: `bool` Queries if the execution platform is a real system. > * **Returns**: `True` if the execution platform is a real system, `False` otherwise. > * **Return type**: `bool` ### SdkRuntime Manages the execution of SDK programs on the Cerebras Wafer-Scale Engine (WSE) or simfabric. The constructor analyzes the WSE ELFs in the `bindir` and prepares the WSE or simfabric for a run. Requires CM IP address and port for WSE runs. Path to ELF files which is compiled by `cslc`. The runtime collects the I/O and fabric parameters automatically, including height, width, number of channels, width of buffers, etc. `'IP_ADDRESS:PORT'` string of CM. Omit this `kwarg` to run on simfabric. If `True`, suppresses generation of `simfab_traces` when running. Default value is `False`, i.e., `simfab_traces` are produced. Note that producing `simfab_traces` can greatly slow down the wall clock time of a simulator run. If you are not using the SDK GUI with the output of your run, consider setting this value to `True`. Number of threads to use if running on simfabric. Maximum value is `64`. Default value is `5`, i.e., the simulator uses 5 threads. Message logging output level. Available output levels are `DEBUG`, `INFO`, `WARNING`, and `ERROR`. Default value is `WARNING`. Whether the program uses [`memcpy_h2d`](#memcpy_h2d) / [`memcpy_d2h`](#memcpy_d2h) for host-device data transfer. Default value is `True`. Set to `False` for programs that move data exclusively via [`SdkLayout`](/api-docs/sdklayout-api) streams. **Example**: In the following example, an [`SdkRuntime`](#sdkruntime) runner object is instantiated. If `args.cmaddr` is non-empty, then the kernel code will run on the WSE pointed to by that address; otherwise, the kernel code will run on simfabric. The compiled kernel code in the directory `args.name` has exported symbols `A` and `B` pointing to arrays on the device. After loading the code and starting the run with `load()` and `run()`, data on the host stored in `data` is copied to `A` on the device, and then `B` on the device is copied back into `data` on the host. ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} runner = SdkRuntime(args.name, cmaddr=args.cmaddr) symbol_A = runner.get_id("A") symbol_B = runner.get_id("B") runner.load() runner.run() runner.memcpy_h2d(symbol_A, data, px, py, w, h, l, streaming=False, data_type=memcpy_dtype, order=memcpy_order, nonblock=False) runner.memcpy_d2h(data, symbol_B, px, py, w, h, l, streaming=False, data_type=memcpy_dtype, order=memcpy_order, nonblock=False) ``` Constructor variant that takes a path to compiled ELF files and an execution platform specification. Takes same kwargs as above. Constructor variant that takes a compile artifacts specification and execution platform specification. Takes same kwargs as above. Convert a logical coordinate to a physical coordinate. For a program with fabric offsets (`offset_x`, `offset_y`), and program rectangle coordinate (`x`, `y`), this function returns (`offset_x + x`, `offset_y + y`). Two-element tuple `(x, y)` of logical coordinates. > * **Returns**: Two-element tuple `(physical_x, physical_y)`. > * **Return type**: `Tuple[int, int]` Dump the core of a simulator run, to be used for debugging with `csdb`. Note that the specified name of the corefile MUST be "corefile.cs1" to use with `csdb`, and this method can only be called after a blocking [`SdkRuntime`](#sdkruntime) API call, or after calling [`SdkRuntime.stop()`](#stop). Name of corefile. Must be "corefile.cs1" to use with `csdb`. Dump an ELF core of a simulator run, to be used for debugging. Name of ELF corefile. Retrieve the integer representation of an exported symbol which is exported in the kernel. Possible symbols include a data tensor or a host-callable function. The exported name of the symbol. > * **Returns**: Integer representation of exported symbol. > * **Return type**: `int` Part of the [`SdkRuntime`](#sdkruntime) direct link API. Retrieve the integer representation of a program port for streaming data via [`SdkRuntime.send()`](#send) or [`SdkRuntime.receive()`](#receive). The name of the port. > * **Returns**: Integer representation of program data port. > * **Return type**: `PortId` Query if task `task_handle` is complete. Handle to a task previously launched by [`SdkRuntime`](#sdkruntime). > * **Returns**: `True` if task is done, and `False` otherwise. > * **Return type**: `bool` Like [`launch`](#launch), but without type checking on the arguments. The caller is responsible for packing every argument into a single contiguous 1-D `numpy.ndarray` of `numpy.uint32`. Useful when the host already has arguments in a packed `u32` form, or to bypass the per-call type-checking overhead. The exported name of the symbol corresponding to a host-callable function. 1-D `uint32` array containing the packed arguments to pass to the function. Nonblocking if `True`, blocking otherwise. > * **Returns**: Handle to the task launched by [`SdkRuntime.call()`](#call). > * **Return type**: [`Task`](#task) Trigger a host-callable function defined in the kernel, with type checking for arguments. The exported name of the symbol corresponding to a host-callable function. > * **Positional Arguments**: Matches the arguments of the host-callable function. [`SdkRuntime.launch()`](#launch) will perform type checking on the arguments. Nonblocking if `True`, blocking otherwise. > * **Returns**: Handle to the task launched by [`SdkRuntime.launch()`](#launch). > * **Return type**: [`Task`](#task) **Example**: Consider a kernel which defines a host-callable function `fn_foo` by: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} comptime { @export_symbol(fn_foo); } ``` The host calls `fn_foo` by `runner.launch("fn_foo", nonblock=False)`. Load the binaries to simfabric or WSE. It may take 80+ seconds to load the binaries onto the WSE. Receive a host tensor from the device. The data is received from the region of interest (ROI) which is a bounding box starting at coordinate (`px`, `py`) with width `w` and height `h`. A 3-D host tensor `A[h][w][elem_per_pe]`, wrapped in a 1-D array according to keyword argument `order`. A user-defined color if keyword argument `streaming=True`, symbol of a device tensor otherwise. x-coordinate of start point of the ROI. y-coordinate of start point of the ROI. Width of the ROI. Height of the ROI. Number of elements per PE. The data type of an element is 16-bit and 32-bit only. If the tensor has `k` elements per PE, `elem_per_pe` is `k` even if the data type is 16-bit. If the data type is 16-bit, you have to extend the tensor to a 32-bit one, with zero filled in the higher 16 bits. Streaming mode if `True`, copy mode otherwise. In streaming mode, `src` is interpreted as a *color ID* and wavelets received on that color land in `dest` in arrival order. In copy mode, `src` is the symbol ID of a device tensor exported via `@export_symbol`. 32-bit if `MemcpyDataType.MEMCPY_32BIT` or 16-bit if `MemcpyDataType.MEMCPY_16BIT`. Has no effect when `streaming=True`; in streaming mode the host receives raw 32-bit wavelets and the caller is responsible for any reinterpretation. The underlying numpy dtype of `dest` must be 32-bit-wide (e.g. `int32`/`uint32`/`float32`); 16-bit values must be packed into the low 16 bits of a 32-bit container. Row-major if `MemcpyOrder.ROW_MAJOR` or column-major if `MemcpyOrder.COL_MAJOR`. Nonblocking if `True`, blocking otherwise. > * **Returns**: Handle to the task launched by [`SdkRuntime.memcpy_d2h()`](#memcpy_d2h). > * **Return type**: [`Task`](#task) Raised if any of the four mandatory kwargs (`streaming`, `data_type`, `order`, `nonblock`) is omitted, or if the underlying numpy dtype of `dest` is not 32-bit-wide. Send a host tensor to the device. The data is distributed into the region of interest (ROI) which is a bounding box starting at coordinate (`px`, `py`) with width `w` and height `h`. A user-defined color if keyword argument `streaming=True`, symbol of a device tensor otherwise. A 3-D host tensor `A[h][w][elem_per_pe]`, wrapped in a 1-D array according to parameter `order`. x-coordinate of start point of the ROI. y-coordinate of start point of the ROI. Width of the ROI. Height of the ROI. Number of elements per PE. The data type of an element is 16-bit and 32-bit only. If the tensor has `k` elements per PE, `elem_per_pe` is `k` even if the data type is 16-bit. If the data type is 16-bit, you have to extend the tensor to a 32-bit one, with zero filled in the higher 16 bits. > * **Keyword Arguments**: See [`SdkRuntime.memcpy_d2h()`](#memcpy_d2h) keyword arguments. > * **Returns**: Handle to the task launched by [`SdkRuntime.memcpy_h2d()`](#memcpy_h2d). > * **Return type**: [`Task`](#task) Broadcast a row of host data down columns of PEs. The data is distributed across the first row in the region of interest (ROI), which is a bounding box starting at coordinate (`px`, `py`) with width `w` and height `h`, and then broadcast down each column of the ROI. A user-defined color if keyword argument `streaming=True`, symbol of a device tensor otherwise. A 2-D host tensor `A[w][elem_per_pe]`, wrapped in a 1-D array according to parameter `order`. x-coordinate of start point of the ROI. y-coordinate of start point of the ROI. Width of the ROI. Height of the ROI. Number of elements per PE. The data type of an element is 16-bit and 32-bit only. If the tensor has `k` elements per PE, `elem_per_pe` is `k` even if the data type is 16-bit. If the data type is 16-bit, you have to extend the tensor to a 32-bit one, with zero filled in the higher 16 bits. > * **Keyword Arguments**: See [`SdkRuntime.memcpy_d2h()`](#memcpy_d2h) keyword arguments. > * **Returns**: Handle to the task launched by [`SdkRuntime.memcpy_h2d_colbcast()`](#memcpy_h2d_colbcast). > * **Return type**: [`Task`](#task) Broadcast a column of host data across rows of PEs. The data is distributed across the first column in the region of interest (ROI), which is a bounding box starting at coordinate (`px`, `py`) with width `w` and height `h`, and then broadcast across each row of the ROI. A user-defined color if keyword argument `streaming=True`, symbol of a device tensor otherwise. A 2-D host tensor `A[h][elem_per_pe]`, wrapped in a 1-D array according to parameter `order`. x-coordinate of start point of the ROI. y-coordinate of start point of the ROI. Width of the ROI. Height of the ROI. Number of elements per PE. The data type of an element is 16-bit and 32-bit only. If the tensor has `k` elements per PE, `elem_per_pe` is `k` even if the data type is 16-bit. If the data type is 16-bit, you have to extend the tensor to a 32-bit one, with zero filled in the higher 16 bits. > * **Keyword Arguments**: See [`SdkRuntime.memcpy_d2h()`](#memcpy_d2h) keyword arguments. > * **Returns**: Handle to the task launched by [`SdkRuntime.memcpy_h2d_rowbcast()`](#memcpy_h2d_rowbcast). > * **Return type**: [`Task`](#task) Send a host tensor to the device with a stride pattern across receiving PEs. The data is distributed into the region of interest (ROI) which is a bounding box starting at coordinate (`px`, `py`) with width `w` and height `h`. Across a given row, `row_stride` determines the stride between receiving PEs within the ROI, and across a given column, `col_stride` determines the stride between receiving PEs. The first row and column to which data is sent is given by the PE (`px`, `py`) at the top-left of the ROI. `xi` and `eta` denote the number of columns and rows to which elements will be sent in the ROI, respectively. Since the ROI is `w` PEs wide and `h` PEs tall, `xi` and `eta` are given by `xi = 1 + floor((w - 1) / row_stride)` and `eta = 1 + floor((h - 1) / col_stride)`. As an example, consider an ROI starting at (0, 0) with width 6 and height 8, and row and column strides 3 and 2, respectively. Then PEs with x coordinate 0 or 3 and y coordinate 0, 2, 4, 6 will receive data from the host. In this case, `xi = 2` and `eta = 4`. A user-defined color if keyword argument `streaming=True`, symbol of a device tensor otherwise. A 3-D host tensor `A[xi][eta][elem_per_pe]`, wrapped in a 1-D array according to parameter `order`. x-coordinate of start point of the ROI. y-coordinate of start point of the ROI. Width of the ROI. Height of the ROI. Number of elements per PE. The data type of an element is 16-bit and 32-bit only. If the tensor has `k` elements per PE, `elem_per_pe` is `k` even if the data type is 16-bit. If the data type is 16-bit, you have to extend the tensor to a 32-bit one, with zero filled in the higher 16 bits. Stride between PEs within a row in the ROI. Since the ROI is `w` PEs wide, the number of columns to which elements will be sent is `xi = 1 + floor((w - 1) / row_stride)`. Stride between PEs within a column in the ROI. Since the ROI is `h` PEs tall, the number of rows to which elements will be sent is `eta = 1 + floor((h - 1) / col_stride)`. > * **Keyword Arguments**: See [`SdkRuntime.memcpy_d2h()`](#memcpy_d2h) keyword arguments. > * **Returns**: Handle to the task launched by [`SdkRuntime.memcpy_h2d_stride()`](#memcpy_h2d_stride). > * **Return type**: [`Task`](#task) Read the value of a symbol on a specific PE. This method is only supported in the simulator, and requires that the [`SdkRuntime`](#sdkruntime) was configured with [`SimfabConfig(dump_core=True)`](#simfabconfig) and that [`stop()`](#stop) has already been called (the core dump is flushed at that point). x-coordinate of the PE. y-coordinate of the PE. Name of the symbol to read. Numpy dtype string for interpreting the returned data. Default is `"uint8"`. > * **Returns**: Numpy array containing the symbol's data, viewed as the specified dtype. > * **Return type**: `numpy.ndarray` Part of the [`SdkRuntime`](#sdkruntime) direct link API. Receive `n_wavelets` wavelets via the program port `port` into array `dest`. Program port from which data will be received. Can be specified by a numerical port ID or by name. Destination array into which the data will be received. Number of wavelets to receive. Nonblocking if `True`, blocking otherwise. > * **Returns**: Handle to the task launched by [`SdkRuntime.receive()`](#receive). > * **Return type**: [`Task`](#task) Part of the [`SdkRuntime`](#sdkruntime) direct link API. Receive data via the program port `port` and write to a file named `outfile`. Program port from which data will be received. Can be specified by a numerical port ID or by name. Name of file to which received output is written. Nonblocking if `True`, blocking otherwise. > * **Returns**: Handle to the task launched by [`SdkRuntime.receive_tofile()`](#receive_tofile). > * **Return type**: [`Task`](#task) Part of the [`SdkRuntime`](#sdkruntime) direct link API. Reports the port name, color and absolute coordinate of every program data port. Start the simfabric or WSE run and wait for commands from the host runtime. Part of the [`SdkRuntime`](#sdkruntime) direct link API. Stream `n_wavelets` wavelets from `src` to the device via the port `port`. Target program port in which to stream data. Can be specified by a numerical port ID or by name. Input source array whose contents will be streamed to the device. Number of wavelets to send. Nonblocking if `True`, blocking otherwise. > * **Returns**: Handle to the task launched by [`SdkRuntime.send()`](#send). > * **Return type**: [`Task`](#task) Part of the [`SdkRuntime`](#sdkruntime) direct link API. Same as above when `src.dtype` is exactly `np.int32`, `np.uint32` or `np.float32`. In that case, the runtime infers `n_wavelets` from `len(src)`. Wait for all pending commands (data transfers and kernel function calls) to complete and then stop simfabric or WSE. After this call is complete, no new commands will be accepted for this [`SdkRuntime`](#sdkruntime) object. Nonblocking D2H destination buffers are fully populated before `stop()` returns. If the simulator was constructed with `dump_core=True`, the core dump is flushed at this point; calls to [`read_symbol()`](#read_symbol) must therefore occur after `stop()` rather than before. [`SdkRuntime.stop()`](#stop) must be called to end a program. Otherwise, the runtime will emit an error. Wait for the task `task_handle` to complete. Handle to a task previously launched by [`SdkRuntime`](#sdkruntime). ### SdkTarget Specifies a target compilation architecture. > **Values**: > > * WSE2 > * WSE3 ### SimfabConfig Specifies simfab configuration for simulator runs. Number of CPU threads used by the simulator. Default is 16; maximum is 64. (Note: the legacy `SdkRuntime(bindir, simfab_numthreads=N)` constructor instead defaults to 5; `SimfabConfig` itself defaults to 16.) If `True`, suppresses generation of `simfab_traces` when running. If `True`, produces a coredump after execution ends. Name of produced coredump. `None` (default) is `out.core`. ### Task Handle to a task launched by [`SdkRuntime`](#sdkruntime). ### get\_platform Constructs an [`SdkExecutionPlatform`](#sdkexecutionplatform) object configured by simulator or system settings and target architecture. CM address in `"IP_ADDRESS:PORT"` format. `None` (default) or the empty string chooses the simulator. Simulator configuration object. Ignored when `cmaddr` is provided. Target architecture for the simulator or system. > * **Returns**: A configured execution platform object. > * **Return type**: [`SdkExecutionPlatform`](#sdkexecutionplatform) ### get\_simulator Constructs an [`SdkExecutionPlatform`](#sdkexecutionplatform) object for simulator. Simulator configuration object. Target architecture for the simulator. > * **Returns**: A configured execution platform object. > * **Return type**: [`SdkExecutionPlatform`](#sdkexecutionplatform) ### get\_system Constructs an [`SdkExecutionPlatform`](#sdkexecutionplatform) object for a real system. CM address in `"IP_ADDRESS:PORT"` format. > * **Returns**: A configured execution platform object. > * **Return type**: [`SdkExecutionPlatform`](#sdkexecutionplatform) ## sdk\_utils Module Utility functions for common operations with [`SdkRuntime`](#sdkruntime). Import from `cerebras.sdk.sdk_utils`. ### calculate\_cycles Converts values in `timestamp_buf` returned from device into a human-readable elapsed cycle count. Array returned from device containing elapsed timestamp data. > * **Returns**: Elapsed cycle count. > * **Return type**: `numpy.int64` **Example**: Consider the following CSL snippet which records timestamps and produces a single array to copy back to the host, to generate an elapsed cycle count: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // import time module and create timestamp buffers const timestamp = @import_module(" ### input\_array\_to\_u32 Converts a 16-bit tensor to a 32-bit tensor of type `u32` for use with `memcpy`. The parameter `sentinel` distinguishes two different extensions of 16-bit data. If `sentinel` is `None`, zero-pad the upper 16 bits. If `sentinel` is not `None`, pack the index of the innermost dimension of the array into the upper 16-bits. A flat (1-D) numpy array with 2 or 4 bytes per element. If your data is multi-dimensional, flatten it first (for example, with `arr.ravel()`). For 16-bit input data, if this parameter is not `None`, pack the index of the innermost dimension into the high bits of the 32-bit wavelet. If sentinel is `None`, then the high bits are zeros. If `sentinel` is not `None`, specifies size of fastest-changing dimension for generating the index. > * **Returns**: Numpy view into `arr` with specified numpy data type. > * **Return type**: `numpy.ndarray.view` ### memcpy\_view Returns a 32, 16 or 8 bit view of a 32 bit numpy array (only the lower 16 or 8 bits of each 32 bit word in the last two cases). A numpy array with 4 bytes per element on which the numpy view will be created. The numpy data type which should be used in the output view. The itemsize must be 1, 2, or 4 bytes. > * **Returns**: Numpy view into `arr` with specified numpy data type. > * **Return type**: `numpy.ndarray.view` **Example**: [`memcpy_view()`](#memcpy_view) simplifies the use of various precision data types when copying between host and device. Consider the following Python host code which creates a `float16` view into a numpy array. Note that this array *must* be 32-bit. You can fill the array with `float16` data, and copy it to an array on the device with CSL data type `f16`. ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} x_symbol = runner.get_symbol('x') # This container array must be 32-bit x_container = np.zeros(N, dtype=np.uint32) x = sdk_utils.memcpy_view(x_container, np.float16) x.fill(0.5) runner.memcpy_h2d(x_symbol, x_container, 0, 0, 1, 1, N, streaming=False, data_type=MemcpyDataType.MEMCPY_16BIT, order=MemcpyOrder.ROW_MAJOR, nonblock=False) ``` ## debug\_util Module Utilities for parsing debug output and core files of a simulator run. Import from `cerebras.sdk.debug.debug_util`. ### debug\_util Loads ELF files in `bindir` in order to dump symbols for debugging. You do not need to export the symbols in the kernel. [`debug_util`](#debug_util) dumps the core and looks for the symbols in the ELFs. If the symbol at `Px.y` is not found in the corresponding ELF, [`debug_util`](#debug_util) emits an error. The most common errors are either: 1) a wrong coordinate passed in [`debug_util.get_symbol()`](#get_symbol), or 2) a correct coordinate, but the symbol has been removed due to compiler optimization. One can use `readelf` to check if the symbol exists or not. If not, you can export the symbol in the kernel to keep the symbol in the ELF. The functionality of this class is only supported in the simulator. Path to ELF files. **Example**: ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} from cerebras.sdk.debug.debug_util import debug_util # run the app # dirname is the path to ELFs simulator = SdkRuntime(dirname) simulator.load() simulator.run() ... simulator.stop() # retrieve symbols after the run debug_mod = debug_util(dirname) # assume the core rectangle starts at P4.1, the dimension is # width-by-height, and the symbol y is retrieved for every PE core_offset_x = 4 core_offset_y = 1 for py in range(height): for px in range(width): t = debug_mod.get_symbol(core_offset_x+px, core_offset_y+py, 'y', np.float32) print(f"At (py, px) = {py, px}, symbol y = {t}") ``` Read the value of `symbol` of given type at given PE coordinates. Note that each call to this function scans the whole fabric, so prefer [`debug_util.get_symbol_rect()`](#get_symbol_rect) over calling this in a loop. x-coordinate of the PE, indexed from the northwest corner of the entire fabric (NOT the program rectangle). y-coordinate of the PE, indexed from the northwest corner of the entire fabric (NOT the program rectangle). Name of the symbol to be read. Numpy data type of values contained by symbol. > * **Returns**: Numpy array of output values read at symbol. > * **Return type**: `numpy.ndarray` Read the value of `symbol` of given type for a rectangle of PEs. Rectangle specified as `((col, row), (width, height))`, indexed from the northwest corner of the entire fabric (NOT the program rectangle). Name of the symbol to be read. Numpy data type of values contained by symbol. > * **Returns**: Numpy array of output values read at symbol. The first two dimensions of the returned array are PE coordinates `(column, row)` relative to the rectangle. > * **Return type**: `numpy.ndarray` Parse a CSL trace buffer with name `name` at the given PE coordinates. x-coordinate of the PE, indexed from the northwest corner of the entire fabric (NOT the program rectangle). y-coordinate of the PE, indexed from the northwest corner of the entire fabric (NOT the program rectangle). Name of the trace buffer to be read. > * **Returns**: Heterogeneous list of trace values. > * **Return type**: `list` **Example**: Consider a device kernel which initializes a trace buffer with the CSL `debug` library and uses it to record values: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const debug_mod = @import_module("", .{.key = "my_trace", .buffer_size = 100}); fn foo() void { debug_mod.trace_timestamp(); debug_mod.trace_string("Bar"); debug_mod.trace_i16(1); } ``` Then the trace can be read in the host code with: ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} trace_output = debug_mod.read_trace(4, 1, 'my_trace') print(trace_output) ``` If `foo` was executed only once, then `trace_output` will be a heterogeneous list containing a timestamp, the string "Bar", and the number 1. # Running SDK on a Wafer-Scale Cluster Source: https://sdk.cerebras.ai/appliance-mode Learn how to compile and run Cerebras SDK programs on a Wafer-Scale Cluster running in appliance mode. The current and future versions of the SDK match the ML software versioning scheme: * Cerebras ML Software 2.10 supports SDK 2.10, the current version of SDK software. * Cerebras ML Software 2.5 supports SDK 1.4. See the [SDK 1.4 documentation](https://cerebras-sdk-docs-140.netlify.app). * Cerebras ML Software 2.4 supports SDK 1.3. See the [SDK 1.3 documentation](https://cerebras-sdk-docs-130.netlify.app). In addition to the containerized Singularity build of the Cerebras SDK (see [Install the Cerebras SDK](/installation-guide)), Cerebras Wafer-Scale Clusters (WSC) running in appliance mode also support the SDK. This page documents some modifications needed to your code to run on a Wafer-Scale Cluster. For more information, see the [Wafer-Scale Cluster setup and installation documentation](https://training-docs.cerebras.ai/rel-2.5.0/getting-started/setup-and-installation). ## Wafer-Scale Cluster Overview The [Cerebras Wafer-Scale Cluster](https://cerebras.ai/product-cluster/) is our solution to training massive neural networks with near-linear scaling. The Wafer-Scale Cluster consists of one or more CS systems, together with special CPU nodes, memory servers, and interconnects, presented to the end user as a single system, or appliance. The appliance is responsible for job scheduling and allocation of the systems. There are two types of SDK jobs that can run on the appliance: compile jobs, which compile code on a worker node, and run jobs, which either run the compiled code on a worker node using the simulator, or run the code on a real CS system within the appliance. This guide walks through some changes necessary to compile and run your code on a Wafer-Scale Cluster. To request modified code examples for a Wafer-Scale Cluster, contact [developer@cerebras.net](mailto:developer%40cerebras.net). Unlike ML jobs, SDK jobs on a Wafer-Scale Cluster currently have a limitation: they can only use a single worker node and CS system. See [SDK Appliance API Reference](/api-docs/appliance-api) for the full API documentation. ## Set Up the Environment First, learn about [the components of the Cerebras Wafer-Scale Cluster](https://training-docs.cerebras.ai/rel-2.5.0/concepts/cerebras-wafer-scale-cluster#cerebras-wafer-scale-cluster). You interact with the Wafer-Scale Cluster via a user node. Start by setting up a Python virtual environment on the user node: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} $ python3.8 -m venv sdk_venv $ source sdk_venv/bin/activate ``` Next, install the `cerebras_appliance` and `cerebras_sdk` Python wheels in the virtual environment, specifying the proper Cerebras Software release: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} (sdk_venv) $ pip install --upgrade pip (sdk_venv) $ pip install cerebras_appliance==2.10.0 (sdk_venv) $ pip install cerebras_sdk==2.10.0 ``` ## Compile As an example, this guide walks through porting the [Complete Program](/csl/tutorials/gemv-01-complete-program) tutorial. In the containerized SDK setup, this code is compiled with the following command: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} cslc ./layout.csl --fabric-dims=8,3 --fabric-offsets=4,1 --memcpy --channels=1 -o out ``` To compile for the Wafer-Scale Cluster, use a Python script which launches a compile job: ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} import json from cerebras.sdk.client import SdkCompiler # Instantiate compiler using a context manager # Disable version check to ignore appliance client and server version differences. with SdkCompiler(disable_version_check=True) as compiler: # Launch compile job artifact_path = compiler.compile( ".", "layout.csl", "--fabric-dims=8,3 --fabric-offsets=4,1 --memcpy --channels=1 -o out", "." ) # Write the artifact_path to a JSON file with open("artifact_path.json", "w", encoding="utf8") as f: json.dump({"artifact_path": artifact_path,}, f) ``` The `SdkCompiler::compile` function takes four arguments: * the directory containing the CSL code files, * the name of the top level CSL code file that contains the layout block, * the compiler arguments, * and the output directory or output file for the compile artifacts. The last argument can either be a directory, specifying the location to which compile artifacts will be copied with default file name; or a file name, explicitly specifying the name and location for the compile artifacts. The function returns the compile artifact path. This artifact path is written to a JSON file, which is read by the runner object in the Python host code. Just as before, simply pass the full dimensions of the target system to the `--fabric-dims` argument to compile for a real hardware run. The `SdkCompiler()` constructor can take a few optional `kwargs`, including: * `resource_cpu`: number of CPU cores on the WSC's management node used by the compile job in units of 1/1000 CPU (default: 24000, or 24 cores) * `resource_mem`: number of bytes of memory requested from the management node for the compile job (default: `64 << 30`, or 64 GiB) * `disable_version_check`: specifies whether to ignore version differences between appliance client and server If SDK compilation jobs on the WSC are often waiting in the queue behind other jobs, such as ML execute or run jobs, this is typically because not enough resources are available on the management node. These `kwargs` can be used to request fewer resources from the management node and increase the number of simultaneously running jobs. ## Run with SdkLauncher In the containerized SDK setup, the Python host code for running is as follows: ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} import argparse import numpy as np from cerebras.sdk.runtime.sdkruntimepybind import SdkRuntime, MemcpyDataType, MemcpyOrder # Read arguments parser = argparse.ArgumentParser() parser.add_argument('--name', help="the test compile output dir") parser.add_argument('--cmaddr', help="IP:port for CS system") args = parser.parse_args() # Matrix dimensions M = 4 N = 6 # Construct A, x, b A = np.arange(M*N, dtype=np.float32).reshape(M, N) x = np.full(shape=N, fill_value=1.0, dtype=np.float32) b = np.full(shape=M, fill_value=2.0, dtype=np.float32) # Calculate expected y y_expected = A@x + b # Construct a runner using SdkRuntime runner = SdkRuntime(args.name, cmaddr=args.cmaddr) # Load and run the program runner.load() runner.run() # Launch the init_and_compute function on device runner.launch('init_and_compute', nonblock=False) # Copy y back from device y_symbol = runner.get_id('y') y_result = np.zeros([1*1*M], dtype=np.float32) runner.memcpy_d2h(y_result, y_symbol, 0, 0, 1, 1, M, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) # Stop the program runner.stop() # Ensure that the result matches expectations np.testing.assert_allclose(y_result, y_expected, atol=0.01, rtol=0) print("SUCCESS!") ``` If this file is named `run.py` and the compilation output is in the directory `out`, run it with the command: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} cs_python run.py --name out ``` To run on hardware, specify an IP address with the `--cmaddr` flag. The SDK provides an `SdkLauncher` class for running host code directly from a worker node within the appliance. This class allows files to be staged on the appliance before running the same host code that you would with the Singularity container. The following example demonstrates using `SdkLauncher` to run the host code for the example above, including a demonstration of `stage` for transferring a file to the appliance. To pass a system address to a run script when using `SdkLauncher`, you must use the `%CMADDR%` template string, as demonstrated below. ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} import json import os from cerebras.sdk.client import SdkLauncher # read the compile artifact_path from the json file with open("artifact_path.json", "r", encoding="utf8") as f: data = json.load(f) artifact_path = data["artifact_path"] # artifact_path contains the path to the compiled artifact. # It will be transferred and extracted in the appliance. # The extracted directory will be the working directory. # Set simulator=False if running on CS system within appliance. # Disable version check to ignore appliance client and server version differences. with SdkLauncher(artifact_path, simulator=True, disable_version_check=True) as launcher: # Transfer an additional file to the appliance, # then write contents to stdout on appliance launcher.stage("additional_artifact.txt") response = launcher.run( "echo \"ABOUT TO RUN IN THE APPLIANCE\"", "cat additional_artifact.txt", ) print("Test response: ", response) # Run the original host code as-is on the appliance, # using the same cmd as when using the Singularity container response = launcher.run("cs_python run.py --name out --cmaddr %CMADDR%") print("Host code execution response: ", response) # Fetch files from the appliance launcher.download_artifact("sim.log", "./output_dir/sim.log") ``` ## Run with SdkRuntime Bindings The `SdkRuntime` appliance bindings are deprecated. Use `SdkLauncher` to wrap an SDK host Python script instead. For appliance mode, some modifications can also be made to the original `run.py` script to run it on the appliance: ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} import json import os import numpy as np from cerebras.appliance.pb.sdk.sdk_common_pb2 import MemcpyDataType, MemcpyOrder from cerebras.sdk.client import SdkRuntime # Matrix dimensions M = 4 N = 6 # Construct A, x, b A = np.arange(M*N, dtype=np.float32).reshape(M, N) x = np.full(shape=N, fill_value=1.0, dtype=np.float32) b = np.full(shape=M, fill_value=2.0, dtype=np.float32) # Calculate expected y y_expected = A@x + b # Read the artifact_path from the JSON file with open("artifact_path.json", "r", encoding="utf8") as f: data = json.load(f) artifact_path = data["artifact_path"] # Instantiate a runner object using a context manager. # Set simulator=False if running on CS system within appliance. # Disable version check to ignore appliance client and server version differences. with SdkRuntime(artifact_path, simulator=True, disable_version_check=True) as runner: # Launch the init_and_compute function on device runner.launch('init_and_compute', nonblock=False) # Copy y back from device y_symbol = runner.get_id('y') y_result = np.zeros([1*1*M], dtype=np.float32) runner.memcpy_d2h(y_result, y_symbol, 0, 0, 1, 1, M, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) # Ensure that the result matches expectations np.testing.assert_allclose(y_result, y_expected, atol=0.01, rtol=0) print("SUCCESS!") ``` In particular: * The imports have changed to reflect appliance modules. * You read the path of the compile artifacts from the JSON file generated when compiling. * You no longer need to specify a CM address when running on real hardware. Instead, you simply pass a flag to the `SdkRuntime` constructor specifying whether to run in the simulator or on hardware. * `load()` and `run()` are replaced by `start()`. * You must use a [context manager](https://docs.python.org/3/reference/datamodel.html#context-managers) for the runner object. Doing so makes the `start()` and `stop()` functions implicit, so you don't need to explicitly call them. ## Control Appliance Logging When running with the appliance, you can control the level of appliance-related logging printed to the console. By default, the appliance logger uses the `WARNING` level, so only `WARNING` and higher level messages appear. You can set the level of the logger directly to enable other levels, such as `INFO` or `DEBUG`. For example: ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} import logging from cerebras.appliance import logger logging.basicConfig(level=logging.INFO) ``` ## Monitor and Manage Appliance Jobs Monitor jobs on the cluster with the `csctl` CLI tool. Find more information on cluster job monitoring and `csctl` in the [Cerebras training docs](https://training-docs.cerebras.ai/rel-2.5.0/cluster-monitoring/cerebras-job-scheduling-and-monitoring/cli-for-job-monitoring-csctl#job-monitoring-cli). Use `Ctrl-C` to cancel a running job. # Wafer-Scale Engine Architecture Source: https://sdk.cerebras.ai/computing-with-cerebras Learn how the Wafer-Scale Engine architecture works, how processing elements communicate, and how the host and device interact. Programming the Cerebras Wafer-Scale Engine (WSE) calls for a different mental model than programming a CPU or GPU. Instead of coordinating a handful of powerful cores, you're directing a vast mesh of small processors that compute and communicate independently. This page covers the hardware concepts you need before writing CSL: how the WSE is structured, how processing elements execute and talk to each other, and how the host and device interact. ## The WSE at a Glance The WSE is a wafer-parallel compute accelerator containing hundreds of thousands of independent processing elements (PEs), interconnected in a two-dimensional mesh on a single silicon wafer. Each PE has its own memory, program counter, and executable code, and communicates with its neighbors by sending and receiving 32-bit messages called wavelets in a single clock cycle. Each PE also has dataflow control: an instruction can terminate the currently running task, at which point the hardware selects the next runnable task — one that has been *activated* and *unblocked* (covered in [Programs and Tasks](#programs-and-tasks) below). Incoming wavelets travel along virtual channels called colors; congestion on one color doesn't block traffic on another. The Cerebras System (CS) is a self-contained rack-mounted system that houses a single WSE, along with its packaging, power, cooling, and I/O. It connects to a host CPU cluster over parallel 100 Gigabit ethernet. Throughout this documentation, the CS is called the "device," the host CPU cluster the "host," and the ethernet connection "host I/O." The SDK uses host I/O to move data between host and device and to launch functions on the device — the figure below shows the mesh of PEs and how they connect to the outside world. Diagram of the WSE's 2D mesh of processing elements and their connections to host I/O ## Processing Elements A PE contains three key elements: 1. **A processor**, also referred to as a compute engine (CE). 2. **A router**, directly connected via bidirectional links to its own CE and to the routers of the four nearest neighboring PEs in the mesh. The link to its own CE is called the RAMP, and the links to the four neighboring PEs are referred to by their cardinal directions. The router is the only communication device the PEs use to send and receive data. 3. **The local PE memory** where all of the PE’s data and code are stored. Neither the CE nor the local memory of a PE is directly accessible by other PEs. Diagram of a processing element showing its compute engine, router, and local memory ## The Programming Model To develop code for the WSE, you write device code in the Cerebras Software Language (CSL), and host code in Python. You then compile the device code, and run your program on either the **Cerebras fabric simulator**, or the actual network-attached device. The host code is responsible for copying data to and from the device, and launching discrete programs referred to as kernels. CSL gives you full control of the WSE. The following sections introduce the key concepts you'll need to structure device code in CSL. ### Programs and Tasks A CSL program consists of one or more subprograms. Some of these are callable functions, and some are tasks. A task is a procedure that cannot be called from other code. Rather, tasks are started by the PE hardware and run until they complete. At that point, the hardware chooses a new task to run. Tasks cannot be *called*, only *activated*. They cannot return values. For example, in this code snippet, `main_task` will set the value of the global variable `result` to `5.0` when it is activated and runs. In CSL this task is represented as: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var result: f32 = 0.0; task main_task() void { result = 5.0; } ``` Each task is bound to a **task ID**, which serves as a handle for identifying the task. ### Task IDs and Types The term “task identifier” or “task ID” is used to refer to a numerical value from 0 to 63 that can be associated with a task. Within this range there are two properties that further distinguish a task ID: *routable* and *activatable*. There are three types of tasks, each with an associated task ID handle type: * **Data tasks** are associated with a `data_task_id`, which on the WSE-2 architecture is created from a routable identifier associated with a color. On the WSE-3 architecture, a `data_task_id` is created from an *input queue*, which also must be associated with a color. An input queue is a hardware buffer where data is temporarily stored before entering the compute engine (CE) of a PE. * **Local tasks** are associated with a `local_task_id`, which is created from an activatable identifier. * **Control tasks** are associated with a `control_task_id`, which can be created from any identifier, including those that are neither routable nor activatable. Both data tasks and control tasks are wavelet-triggered tasks (WTT): their activation is triggered by the arrival of a wavelet. This introduction explores what it means for a task ID to be routable or activatable, and the usage of data tasks and local tasks. * On WSE-2, task IDs 0 to 23 are the routable task IDs, as data task IDs are created from one of the 24 routable colors (see below). * On WSE-3, task IDs 0 to 7 are the routable task IDs, as data task IDs are created from one of the 8 input queues. ### Communication The WSE provides efficient, fine-grained communication between PEs. A PE must be able to quickly, within a few cycles, respond to the arrival of a wavelet, update its internal state, and send out wavelets. The hardware uses 24 virtual communication channels, called *routable colors* or simply *colors*, to pass wavelets between PEs. Each color has an ID between 0 and 23. Each wavelet has a 5-bit tag that encodes its color. This color determines both the wavelet’s routing through the fabric and what task, if any, will consume the wavelet when received. (On WSE-3, the consuming task is determined by which input queue the color is bound to; see [`@initialize_queue`](/csl/language/builtins#@initialize_queue).) In the following code block, the task now takes an argument, named `wavelet_data`. This task is an example of a *data task*. The builtin CSL function `@bind_data_task` creates a binding between a task ID associated with the color of the incoming wavelet and the task `main_task`. On WSE-2, this task ID is the same as the color ID: it takes on a value between 0 and 23. On WSE-3, this task ID is instead the ID of an input queue which is bound to the color: it takes on a value between 0 and 7. When a red wavelet arrives, the task `main_task` is *activated*, which allows it to be selected by the task picker. The red wavelet sits in a buffer until the task picker selects the associated task `main_task`, at which point the wavelet is moved into a register so that the task can get instant access to that data. Syntactically, the wavelet’s data is an argument of the task. `main_task` is also called a *wavelet-triggered task*, since it is activated by the arrival of a wavelet. Note that the `@bind_data_task` operation occurs within a `comptime` block: everything in a `comptime` block is evaluated at compile-time. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // 7 is the ID of a color with some defined data routing const red: color = @get_color(7); // On WSE-3, 2 is the ID of an input queue which will // be bound to our data task. const iq: input_queue = @get_input_queue(2); // For WSE-2, the ID for this task is created from a color. // For WSE-3, the ID for this task is created from an input queue. const red_task_id: data_task_id = if (@is_arch("wse3")) @get_data_task_id(iq) else @get_data_task_id(red); var result: f32 = 0.0; task main_task(wavelet_data: f32) { result = wavelet_data; } comptime { @bind_data_task(main_task, red_task_id); // For WSE-3, input queue to which our data task is bound // must be bound to color red. if (@is_arch("wse3")) @initialize_queue(iq, .{ .color = red }); } ``` Colors can also be used in a manner such that wavelets arriving along the associated virtual communication channel do *not* activate a task, but are received by a construct known as a *fabric DSD*. The coming tutorials explore this usage. ### Task Activation and Control Flow As shown above, a task becomes available for selection by the task picker when its associated task ID is activated. The task in the above block was a data task bound to a `data_task_id`, associated with a color which defines a route taken by wavelets tagged with that color. You can also create tasks that do not take wavelets as arguments, and instead are explicitly activated by other tasks or functions. These are called local tasks, and the associated task ID type is `local_task_id`. On WSE-2, you can create a `local_task_id` from the range of task IDs 0 to 30. These IDs are the *activatable* IDs. On WSE-3, you can create a `local_task_id` from the range of task IDs 8 to 30. In the following example, `main_task` activates the task ID `foo_task_id`. This task ID is bound to `foo_task`, and so activating `foo_task_id` will cause `foo_task` to execute next. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const red: color = @get_color(7); const iq: input_queue = @get_input_queue(2); // Used only on WSE-3 const red_task_id: data_task_id = if (@is_arch("wse3")) @get_data_task_id(iq) else @get_data_task_id(red); const foo_task_id: local_task_id = @get_local_task_id(8); var result: f32 = 0.0; var sum: f32 = 0.0; task main_task(wavelet_data: f32) { result = wavelet_data; @activate(foo_task_id); } task foo_task() { sum += result; } comptime { @bind_data_task(main_task, red_task_id); @bind_local_task(foo_task, foo_task_id); if (@is_arch("wse3")) @initialize_queue(iq, .{ .color = red }); } ``` ### Block and Unblock Tasks You can also *block* a task ID to provide further control over task execution. A task must be unblocked and activated for it to be scheduled by the task picker. If a task is activated while blocked, it will not run until it has become unblocked. By default, all IDs are unblocked and inactive. The following example introduces one additional task, `bar_task`, and blocks its ID at compile time with `@block(bar_task_id)`. When `main_task` executes, it activates both `foo_task_id` and `bar_task_id`. However, because `bar_task_id` is blocked, `foo_task` always executes first. When `foo_task` executes, it unblocks `bar_task_id`, allowing `bar_task` to begin execution once `foo_task` finishes. If `bar_task_id` were not blocked at compile time, then the execution of `foo_task` and `bar_task` could occur in any order. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const red: color = @get_color(7); const iq: input_queue = @get_input_queue(2); // Used only on WSE-3 const red_task_id: data_task_id = if (@is_arch("wse3")) @get_data_task_id(iq) else @get_data_task_id(red); const foo_task_id: local_task_id = @get_local_task_id(8); const bar_task_id: local_task_id = @get_local_task_id(9); var result: f32 = 0.0; var sum: f32 = 0.0; task main_task(wavelet_data: f32) { result = wavelet_data; @activate(foo_task_id); @activate(bar_task_id); } task foo_task() { sum += result; @unblock(bar_task_id); } task bar_task() { sum *= 2.0; } comptime { @block(bar_task_id); @bind_data_task(main_task, red_task_id); @bind_local_task(foo_task, foo_task_id); @bind_local_task(bar_task, bar_task_id); if (@is_arch("wse3")) @initialize_queue(iq, .{ .color = red }); } ``` ### Layout Layout blocks are how you connect up the multiple PEs in your program rectangle in a way your computation requires. For example, see the following 2-PE rectangle: Diagram of a two-PE rectangle showing one possible way to interconnect the PEs The diagram shows only one of the many ways you can interconnect the two PEs. You connect a PE to another PE by specifying routes for colors. Using CSL you can define the specific colors and routes by which your rectangle is stitched up. This configuration of colors and routes forms an essential aspect of your computation, transforming the wavelets as they enter and pass through your rectangle. See the following example of a `layout` block showing a layout of two PEs in a single row: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // layout.csl // Top-level program source const main_color: color = @get_color(0); layout { @set_rectangle(2, 1); // A row containing two PEs. @set_tile_code(0, 0, "send.csl", .{ .send_color = main_color }); @set_tile_code(1, 0, "recv.csl", .{ .recv_color = main_color }); const send_route = .{ .rx = .{ RAMP }, .tx = .{ EAST } }; const recv_route = .{ .rx = .{ WEST }, .tx = .{ RAMP } }; @set_color_config(0, 0, main_color, send_route); @set_color_config(1, 0, main_color, recv_route); } ``` The `@set_tile_code()` builtin CSL function specifies the `.csl` file containing the program for the individual PE within the program rectangle denoted by the indices in the first two parameters of the function. For example, the program `send.csl` contains the task description that only the PE at the coordinate (0, 0) will perform. Each user program can define only one layout, specifying a rectangle of active PEs and a code file assigned to each PE. This layout must be defined at compile time, and a user program cannot define multiple layouts. Hence, zero or multiple `@set_rectangle()` calls are illegal. Additionally, the built-in `@set_tile_code()` must be called after `@set_rectangle()`. For example, if your rectangle contains five PEs, then you can configure each PE with a different program by having five `@set_tile_code()` calls after a single `@set_rectangle()`, with each `@set_tile_code()` function call specified with a separate `.csl` file. The `@set_color_config` calls assign the routing associated with `main_color` for each PE, from the perspective of the PE’s router. For instance, the router of PE (0, 0) will receive wavelets along color `main_color` from the `RAMP`, which connects the router to the CE. It will then transmit wavelets to the `EAST`, where it will be received by the router of the neighboring PE. ## Next Steps Now that we’ve introduced a high-level overview of the architecture and the programming model of CSL, continue on to [Tutorials](/csl/tutorials) for step-by-step walkthroughs on writing, compiling, and running complete programs with the Cerebras SDK. # Migrating from comptime_struct and @concat_structs Source: https://sdk.cerebras.ai/csl/comptime-struct-migration How to migrate existing CSL code from comptime_struct and @concat_structs to named struct types. The `comptime_struct` type and the `@concat_structs` builtin were removed in SDK 2.10.0. See [Version 2.10.0](/sdk-release-notes/sdk-rel-notes-cumulative#version-2100). This guide explains how to migrate existing code to use named `struct` types instead. For a full reference on struct types in CSL, see [Types](/csl/language/types). ## Replace `comptime_struct` with `struct` Because `comptime_struct` was removed, you can no longer use `@concat_structs`. **Before (removed):** ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // Error: `comptime_struct` is no longer supported in CSL const structA: comptime_struct; const structB: comptime_struct; const structC = @concat_structs(structA, structB); ``` **After:** CSL now requires `struct` types to be declared explicitly. These can be used at both compile time and runtime. The following code achieves the same result as the example above: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const structA = struct {...}; const structB = struct {...}; const structC = struct { a: structA, b: structB, }; ``` ## Partial Initialization Another key difference between `comptime_struct` and `struct` is that all fields must be declared and set. No partial initialization can be done, sometimes requiring the use of temporary placeholders. **Before (removed):** ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var mystruct: comptime_struct = .{ .foo = true }; mystruct = @concat_structs(mystruct, .{ .bar = 42 }); ``` **After:** ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const mystruct_t = struct { foo: bool, bar: u16, }; var mystruct: mystruct_t = .{ .foo = true, .bar = 0, // Temporary placeholder value }; mystruct.bar = 42; ``` ## Parameterized Structs Sometimes, the members of a given `struct` need to vary based on compile-time parameters or constants. This can be achieved by writing a comptime function that returns a `type`. The returned type is a `struct` whose fields depend on the arguments passed to the function. For example, a function that returns a point type whose coordinate type depends on the scale type requested: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // Functions returning a type must have comptime arguments only. fn point_type(comptime scalar_type: type) type { return struct { x: scalar_type, y: scalar_type, }; } fn make_point(x_val: anytype) point_type(@type_of(x_val)) { return .{ .x = x_val, .y = 0 }; } ``` See also the [parameterized struct types example](/csl/language/types#parameterized-struct-types-example) in the Language Guide for additional patterns. # CSL Compiler Source: https://sdk.cerebras.ai/csl/csl-compiler Use `cslc` to compile CSL programs with options for output, parameters, colors, target architecture, fabric dimensions, memcpy support, and more. The CSL compiler is invoked with the command `cslc` on your terminal. See [Compiling and Running Examples](/csl/working-with-code-samples) for usage examples. ## Synopsis ```text theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} cslc [-o OUTPUT_NAME] [--arch=ARCH] [--fabric-dims=W,H] [--fabric-offsets=X,Y] [--params NAME:VALUE[,...]] [--colors NAME:VALUE[,...]] [--memcpy] [--channels N] [--fp16-format=FORMAT] [--import-path DIR] [--width-west-buf N] [--width-east-buf N] [--output-json[=PATH]] [--out-routes] [--max-inlined-iterations N] [--comptime-func-depth-limit N] [--max-parallelism N] [-g0] [--warnings-as-errors] [--verbose] [-h] ``` `` is the top-level CSL source file (typically `layout.csl`). ## Output ### `-o OUTPUT_NAME` Output directory name. ELF files and other build artifacts are written under this directory. Default: `out`. ### `--output-json[=PATH]` Write a JSON file containing the resolved compilation parameters. If `PATH` is omitted, the file is named based on `-o` with a `.json` extension. This is the file host code typically reads to recover `--params` values at run time. ### `--out-routes` Print an ASCII table of per-PE color and routing information during compilation. Useful for debugging routes. ## Target Architecture and Fabric ### `--arch=ARCH` Select the target architecture. Valid values: `wse2`, `wse3`. Default: `wse2`. The architecture you compile for must match the system you intend to run on. ### `--fabric-dims=W,H` Width and height of the target fabric in PE units. For simulator runs, any bounding box large enough to contain your program rectangle suffices; for hardware, these must match the actual fabric dimensions of your CS system. ### `--fabric-offsets=X,Y` Offset of the upper-left corner of your program rectangle within the fabric. Together with `--fabric-dims`, this defines where the program lives on the fabric. ### `--fp16-format=FORMAT` Choose which 16-bit floating-point format may be used at runtime. Valid values: * `f16` — IEEE half precision (5-bit exponent, 10-bit mantissa). The default. * `cb16` — Cerebras float16 (6-bit exponent, 9-bit mantissa, customized bias). * `bf16` — Brain float16 (8-bit exponent, 7-bit mantissa). The other two FP16 types require comptime-known values. See [FP16 Types](/csl/language/types#fp16-types). ## Compile-Time Parameters and Colors ### `--params NAME:VALUE[,...]` Set the values of CSL source `param` declarations. The argument is a comma-separated list of `name:value` pairs where `name` matches a `param` in the source and `value` is an unsigned integer. May be specified multiple times; all values are concatenated. ### `--colors NAME:VALUE[,...]` Set the values of CSL source color declarations. Same format as `--params`. May be specified multiple times. ## Memcpy ### `--memcpy` Add memcpy infrastructure to the program, enabling host-device data transfer and host-side function launches via [`SdkRuntime`](/api-docs/sdkruntime-api). Almost all SDK programs use this. ### `--channels N` Number of memcpy I/O channels. Required when `--memcpy` is used; the value must be at least 1 and no larger than the height of the program rectangle (maximum 16). Higher values increase host-device throughput; performance improvements are typically minimal past 8. ## Module Imports ### `--import-path DIR` Add `DIR` to the list of directories searched for `<...>` paths in `@import_module` and `@set_tile_code` statements. May be specified multiple times; directories are searched in command-line order, before the compiler's built-in library directory. ## Fabric I/O Buffering ### `--width-west-buf N` Width of the west buffer. Default: `0` (no buffer). Increase to mitigate slow input from the west edge of the fabric. ### `--width-east-buf N` Width of the east buffer. Default: `0` (no buffer). Increase to mitigate slow output to the east edge of the fabric. ## Compile-Time Limits ### `--max-inlined-iterations N` Maximum number of iterations allowed when the compiler unrolls inline loops. If exceeded, compilation fails. Default: `0`, interpreted as no limit. Set this if a comptime loop is expanding more than you expect. ### `--comptime-func-depth-limit N` Maximum depth of the comptime function-call stack. If exceeded, compilation fails. Default: `0`, interpreted as no limit. Set this if a recursive comptime function is exploring an unbounded space. ### `--max-parallelism N` Limit the compiler's internal parallelism to `N` workers. Default: `0`, interpreted as no limit. Lower this on shared build machines. ## Debug and Diagnostics ### `-g0` Disable DWARF debug-information generation. Without this option, the compiler always emits debug info in ELF files. ### `--warnings-as-errors` Treat all compiler warnings as errors. ### `--verbose` Verbose output. Prints information about each compiler phase and additional diagnostics if linking fails. ### `-h`, `--help` Print usage information and exit. # Advanced Hardware Features Source: https://sdk.cerebras.ai/csl/language/advanced-features Learn how to use advanced WSE hardware features including color swapping and CE injection for fine-grained data routing control. ## Color Swapping The hardware supports color “swapping,” a feature allowing incoming data on one color to be sent out on another color. Two colors are swappable if they differ only in the low bit (i.e. `x ^ y == 1`). Color swapping is set per-color East-West (horizontal) or North-South (vertical). Consider a color pair consisting of colors 2 and 3. If East-West color swapping is enabled for color 3, but not color 2, then wavelets arriving from either the East or West on color 2 will be received on both color 2 and color 3. If both color 2 and color 3 have East-West color swapping enabled, then wavelets arriving from either the East or West on color 2 will be received on color 3, and wavelets arriving from either the East or West on color 3 will be received by color 2. Note that the transmission is still subject to which directions are enabled in the receive and transmit fields for the color. The behavior described above also applies to North-South color swapping. If *both* East-West color swap and North-South color swap are enabled for a given color, color swap is also enabled for wavelets arriving from the CE (onramp). ## Compute Element (CE) Injection This feature is only available on WSE-2, and is not supported on WSE-3. CE inject mode is a per-color setting that can be enabled at compile-time. A CE inject configured color is hard wired to the tile’s output queue corresponding to the color number divided by 4. E.g. color 4 in this mode is connected to output queue 1. This is an integer floor division, so e.g. if color 5 is in this mode it is also connected to output queue 1. Note that this means in practice only one of every four colors can be used in CE inject mode at once. Either low priority or high priority can be selected when configuring this mode. When either the fabric buffers are empty or the relevant output queue is empty, these modes behave identically. When both are non-empty, low priority mode will preference the fabric buffers, whereas high priority mode will preference the relevant output queue. To explain this more concretely, in low priority mode, the switch for this color flips to position 1 when the fabric buffers are empty and begins transmitting from the relevant output queue. It switches back to position 0 if the fabric buffers ever become non-empty. In high priority mode, the switch will flip to position 1 whenever the relevant output queue is non-empty and will only flip back to position 0 when the output queue becomes empty. In either mode, control wavelets on this color have no effect; the switch position is controlled entirely by buffer and queue occupancy. # Appendix Source: https://sdk.cerebras.ai/csl/language/appendix Reference SIMD mode behavior and performance characteristics for CSL builtin operations on WSE-2 and WSE-3 architectures. ## SIMD Mode Many of the [builtins for DSD operations](/csl/language/builtins#builtins-for-dsd-operations) have a SIMD (single instruction, multiple data) mode, in which multiple operations can be performed in a single cycle. Under appropriate conditions, these builtins will automatically execute in SIMD mode when operating on DSDs, if possible. In particular, builtins can only operate at their full SIMD width if no bank conflicts occur when fetching the operands from memory. The 48 KB of memory in a PE are laid out into 8 banks of 6 KB each. Each successive 16 bits are located in successive banks. In a single cycle, the PE can perform two 32-bit reads and one 32-bit write. However, the reads must occur from separate banks. More specifically, if the 8 banks are numbered 0 to 7, then the bank IDs `bank_1` and `bank_2` of the two reads must be such that `bank_1 % 4 != bank_2 % 4`. For best results in avoiding bank conflicts with SIMD operations, the operand addresses should be 32-bit aligned, and `(src0_addr % 8) == ((src1_addr + 4) % 8)`, where `src0_addr` and `src1_addr` are the addresses of the operands. Dumping the ELF file’s symbol table with the `--sym` option of `cs_readelf` provides addresses and banking information for all symbols in a compiled CSL program, and can be useful for determining if bank conflicts may occur. Additionally, if the DSD operands have non-contiguous strided accesses, the SIMD width may be limited: * Strides of 0 and 1 can operate at full SIMD width. * Strides such that `stride % 8` is 2, 3, 5, or 6 can operate at full SIMD width. * Unless stride is 1, strides such that `stride % 8` is 1 or 7 is limited to two operations per cycle, or a SIMD width of 2. * Unless stride is 0, strides such that `stride % 8` is 0 is limited to one operation per cycle, or a SIMD width of 1. * Strides such that `stride % 8` is 4 are limited to two operations per cycle, or a SIMD width of 4. The maximum width of builtins which can operate in SIMD mode are given in the table below. | Builtin | WSE-2 SIMD Width | WSE-3 SIMD Width | | ------------------------------------------- | ---------------- | ---------------- | | [@add16](/csl/language/builtins#@add16) | 4 | 8 | | [@addc16](/csl/language/builtins#@addc16) | 1 | 8 | | [@and16](/csl/language/builtins#@and16) | 4 | 8 | | [@fabsh](/csl/language/builtins#@fabsh) | 4 | 8 | | [@fabss](/csl/language/builtins#@fabss) | 2 | 4 | | [@faddh](/csl/language/builtins#@faddh) | 4 | 8 | | [@faddhs](/csl/language/builtins#@faddhs) | 2 | 4 | | [@fadds](/csl/language/builtins#@fadds) | 2 | 4 | | [@fnormh](/csl/language/builtins#@fnormh) | 4 | 8 | | [@fnorms](/csl/language/builtins#@fnorms) | 2 | 4 | | [@fh2s](/csl/language/builtins#@fh2s) | 1 | 4 | | [@fh2xp16](/csl/language/builtins#@fh2xp16) | 1 | 8 | | [@fmach](/csl/language/builtins#@fmach) | 4 | 8 | | [@fmachs](/csl/language/builtins#@fmachs) | 2 | 4 | | [@fmaxh](/csl/language/builtins#@fmaxh) | 1 | 8 | | [@fmaxs](/csl/language/builtins#@fmaxs) | 1 | 4 | | [@fmovh](/csl/language/builtins#@fmovh) | 4 | 8 | | [@fmovs](/csl/language/builtins#@fmovs) | 2 | 4 | | [@fmulh](/csl/language/builtins#@fmulh) | 4 | 8 | | [@fnegh](/csl/language/builtins#@fnegh) | 4 | 8 | | [@fnegs](/csl/language/builtins#@fnegs) | 2 | 4 | | [@fs2h](/csl/language/builtins#@fs2h) | 1 | 4 | | [@fs2xp16](/csl/language/builtins#@fs2xp16) | 1 | 4 | | [@fscaleh](/csl/language/builtins#@fscaleh) | 4 | 8 | | [@fscales](/csl/language/builtins#@fscales) | 2 | 4 | | [@fsubh](/csl/language/builtins#@fsubh) | 4 | 8 | | [@fsubs](/csl/language/builtins#@fsubs) | 2 | 4 | | [@mov16](/csl/language/builtins#@mov16) | 4 | 8 | | [@mov32](/csl/language/builtins#@mov32) | 2 | 4 | | [@or16](/csl/language/builtins#@or16) | 4 | 8 | | [@sar16](/csl/language/builtins#@sar16) | 1 | 4 | | [@sll16](/csl/language/builtins#@sll16) | 1 | 4 | | [@slr16](/csl/language/builtins#@slr16) | 1 | 4 | | [@sub16](/csl/language/builtins#@sub16) | 4 | 8 | | [@xor16](/csl/language/builtins#@xor16) | 4 | 8 | | [@xp162fh](/csl/language/builtins#@xp162fh) | 1 | 8 | | [@xp162fs](/csl/language/builtins#@xp162fs) | 1 | 4 | # Builtins Source: https://sdk.cerebras.ai/csl/language/builtins Reference the complete set of CSL builtin functions for task activation, DSD operations, type conversion, memory management, and hardware configuration. This section documents the builtins available in CSL. Builtins related to remote procedure calls (RPC) are documented in [Builtins for Supporting Remote Procedure Calls (RPC)](#builtins-for-supporting-remote-procedure-calls-rpc), and builtins for operation on DSDs are documented in [Builtins for DSD Operations](#builtins-for-dsd-operations). ## @activate Set the status of a local task to *Active*, allowing it to be picked by the task picker if it is also unblocked. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @activate(id); ``` Where: * `id` is an expression of type `local_task_id` that is bound to a local task. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const task_id: local_task_id = @get_local_task_id(10); comptime { @bind_local_task(my_task, task_id); } fn foo() void { // make my_task eligible to be picked by task picker @activate(task_id); } ``` ## @allocate\_fifo Create a FIFO DSD value. See [Data Structure Descriptors](/csl/language/dsds) for details. See [Data Structure Registers](/csl/language/dsrs) for details about use with DSRs. ## @as Coerce an input value from one numeric or boolean type to another. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @as(result_type, value); ``` Where: * `result_type` is a numeric (i.e. boolean, integer, or float) type. * `value` is numeric value. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // Convert the integer literal 10 into the 16-bit float value 10.0. @as(f16, 10); // Convert the float literal 10.2 into the 16-bit integer value 10. @as(i16, 10.2); ``` ### Semantics Float-to-integer type coercion rounds the value towards zero. For example: * `@as(i16, 11.2) == 11` * `@as(i16, 10.8) == 10` * `@as(i16, -10.8) == -10` Float-to-bool and integer-to-bool type coercions are equivalent to (unordered, not-equals) comparisons with zero. Thus: * `@as(bool, 0) == false` * `@as(bool, -0.0) == false` * `@as(bool, -5) == true` * `@as(bool, nan) == true` ## @assert Asserts that a condition is true. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @assert(cond); ``` Where: * `cond` is an expression of type `bool`. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} task t(wavelet : i16) void { @assert(wavelet > 10); } ``` ### Semantics Causes program execution to abort if the assert condition is false. Note: aborting will only happen in simulation, hardware executions of the program will ignore the assert. If the assert expression is encountered in a `comptime` context, the builtin is equivalent to `@comptime_assert`. ## @bitcast Reinterpret the raw bits of the input value as a value of another type. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @bitcast(result_type, value); ``` Where: * `result_type` is a pointer or numeric (i.e. boolean, integer, or float) type. * `value` is numeric value. Must be an integer if `result_type` is a pointer. * the bit width of `value` matches the bit width of values of type `result_type`. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // Convert the IEEE-754 binary16 value 1.0 into its hex representation // 0x3c00. const one: f16 = 1.0; @bitcast(u16, one); // Produce a 16-bit NaN value. const all_ones: u16 = 0xffff; @bitcast(f16, all_ones); ``` ## @bind\_control\_task Bind a task to a `control_task_id`, so that each time a control wavelet containing this ID in its payload is received, the task is activated and can be scheduled for activation. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @bind_control_task(this_control_task, this_control_task_id); ``` Where: * `this_control_task` is the name of a task. * `this_control_task_id` is an identifier of type `control_task_id`. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const my_task_id: control_task_id = @get_control_task_id(35); task my_task() void {} comptime { @bind_control_task(my_task, my_task_id); } ``` ### Semantics The `@bind_control_task` builtin must appear in a top-level `comptime` block. ## @bind\_data\_task Bind a task to a `data_task_id`, so that each time a wavelet is received along the routable color underlying `data_task_id`, the task is activated and can be scheduled for execution. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @bind_data_task(this_data_task, this_data_task_id); ``` Where: * `this_data_task` is the name of a task. * `this_data_task_id` is an identifier of type `data_task_id`. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // On WSE-2, data_task_ids are created from routable colors const my_task_id: data_task_id = @get_data_task_id(@get_color(0)); // A data task takes payload of wavelet as an argument task my_task(data: f32) void {} comptime { @bind_data_task(my_task, my_task_id); } ``` ### Semantics The `@bind_data_task` builtin must appear in a top-level `comptime` block. Tasks passed into this builtin must take at least one argument. ## @bind\_local\_task Bind a task to a `local_task_id`, so that each time that `local_task_id` is unblocked and activated, the task is activated and can be scheduled for execution. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @bind_local_task(this_local_task, this_local_task_id); ``` Where: * `this_local_task` is the name of a task. * `this_local_task_id` is an identifier of type `local_task_id`. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const my_task_id: local_task_id = @get_local_task_id(10); task my_task() void {} comptime { @bind_local_task(my_task, my_task_id); } ``` ### Semantics The `@bind_local_task` builtin must appear in a top-level `comptime` block. Tasks passed into this builtin cannot take any arguments. ## @bind\_rotating\_tasks Available on WSE-3 only. Bind a pair of tasks, a data task and a control task, and enable rotation between those tasks. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @bind_rotating_tasks(main, alt, task_id, config); ``` Where: * `main` is a data task handler. * `alt` is a control task handler. * `task_id` is a comptime-known expression of type `data_task_id`. * `config` is a comptime-known anonymous struct with the following fields: * `init` (optional) is a comptime-known non-negative integer that may not exceed `limit`. * `limit` (required) is a comptime-known non-negative integer. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const iq = @get_input_queue(0); const main_id = @get_data_task_id(iq); task main(data: u32) void {} task alt() void {} comptime { // Task 'alt' will start after '10' starts of task 'main'. // Task 'alt' is bound to control task id '0' (see 'semantics' for // more details). @bind_rotating_tasks(main, alt, main_id, .{.limit = 10}); // Input queue must be initialized to allow us to bind 'main' // to 'main_id'. @initialize_queue(iq, .{.color = c}); // Must be called to prevent 'main' and 'alt' from clashing. @set_control_task_table(); } const iq1 = @get_input_queue(0); const main_id1 = @get_data_task_id(iq); task main1(data: u32) void {} task alt1() void {} const iq2 = @get_input_queue(0); const main_id2 = @get_data_task_id(iq); task main2(data: u32) void {} task alt2() void {} comptime { // Task 'alt1' will start after '4' starts of task 'main'. // After that, 'alt1' will start after '10' starts of task 'main'. // Task 'alt' is bound to control task id '0' that is associated with // input queue 'iq1'. @bind_rotating_tasks(main1, alt1, main_id1, .{.init = 5, .limit = 10}); // Task 'alt' is bound to control task id '0' that is associated with // input queue 'iq2'. @bind_rotating_tasks(main2, alt2, main_id2, .{.limit = 10}); // 'iq1' and 'iq2' must have separate control task tables to allow the // two pairs of rotating tasks to function correctly. @initialize_queue(iq1, .{.color = c1, .ctrl_table_id = 0}); @initialize_queue(iq2, .{.color = c2, .ctrl_table_id = 1}); // This separates the control task table from the data task table // which will guarantee that alternate tasks will not overlap // with data tasks. @set_control_task_table(); } ``` ### Semantics The `@bind_rotating_tasks` builtin enables task rotation for a pair of tasks. Task rotation is a WSE-3 hardware feature that allows us to execute a different task every Nth data task start. Task `main` must be a data task, which means that it must have input parameters corresponding to input wavelets, while task `alt` must be a control task. WSE-3 can only support two concurrent pairs of rotating tasks. Therefore, the builtin can only be called at most twice during the evaluation of a top-level comptime block. When `@bind_rotating_tasks` is called, task `main` gets bound to `task_id`, which must be of type `data_task_id`, while task `alt` gets bound to control task id zero. In other words, a call to `@bind_rotating_tasks` is equivalent to `@bind_data_task(main, task_id)` and `@bind_control_task(alt, @get_control_task_id(0))`. Task `alt` is the same task that would be started if a control wavelet with control task id zero arrived from the input queue that is associated with `task_id`. This means that if `@bind_rotating_tasks` is called more than once, then the input queue associated with the `task_id` of each call must have its own dedicated control task table (see [`@set_control_task_table`](#@set_control_task_table)) in order to prevent multiple control tasks bound to control task id zero from clashing. Every time task `main` starts, a counter is compared against `limit`. If the counter equals `limit` then task `alt` starts instead of `main` and the counter resets to zero. The initial value of the counter can be optionally specified through the `init` field. If it is not specified then it defaults to zero. ## @block Block the task associated with the input `color`, `data_task_id`, or `local_task_id` so that the task is prevented from running when the task identifier is activated. For `color` and `data_task_id` inputs, `@block` prevents incoming wavelets on the associated color from activating tasks. This also applies to control wavelets carried by a color, preventing a control task bound to the ID in a control wavelet’s payload from activating. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @block(id); ``` Where: * `id` is an expression of type * WSE-2: `color`, `data_task_id`, or `local_task_id`. * WSE-3: `input_queue`, `data_task_id`, `local_task_id`, or `ut_id`. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // WSE-2 example const task_id: local_task_id = @get_local_task_id(10); comptime { @bind_local_task(my_task, task_id); } fn foo() void { // Prevent my_task from running whenever my_task is activated @block(task_id); } ``` ## @comptime\_assert Assert a compile-time condition to be true; abort compilation if otherwise. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @comptime_assert(cond); @comptime_assert(cond, message); ``` Where: * `cond` is a `comptime` expression of type `bool` and * `message` is an expression of type `comptime_string`. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} param size: u16; fn foo() void { @comptime_assert(size > 0 and size < 16); @comptime_assert(size > 0 and size < 16, "size should be between 0 and 16"); } ``` ## @comptime\_print Prints values at compile-time whenever the compiler evaluates this statement. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @comptime_print(val1, val2, ...); ``` Where: * all arguments are `comptime` expressions. This builtin is overloaded for an arbitrary number of arguments. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn foo() void { const my_struct = .{.x = 0, .dir = NORTH}; @comptime_print(my_struct); // prints the contents of `my_struct`. if (false) { @comptime_print("hello"); // not printed. } for ([2]i16 {1, 2}) |val| { @comptime_print("hello world"); // printed once. @comptime_print(val); // error: val is not comptime. } comptime { for ([2]i16 {1, 2}) |val| { @comptime_print(val); // all values are printed. } }; } ``` ### Semantics A `comptime_print` statement causes the compiler to print information for its arguments whenever the compiler evaluates such builtin. During `comptime` evaluation, the builtin is evaluated whenever control-flow reaches that line. Whenever the compiler is analysing reachable non-`comptime` code, the builtin is evaluated exactly once. For instance, a `@comptime_print` builtin inside a non-`comptime` loop causes the compiler to evaluate it exactly once. ## @constants Initialize a tensor with a value. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @constants(tensor_type, value); ``` Where: * `tensor_type` is a `comptime` tensor type. * The type of `value` is the same as the base type of `tensor_type`. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // Initialize a tensor of four rows and five columns with the same value 10. const matrix = @constants([4,5]i16, 10); ``` ## @dimensions Returns a 1D array in which the i’th element equals the size of the i’th dimension of the input array type. The length of the returned array equals the rank of the input array type. The type of each element in the returned array is `u32`. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @dimensions(array_type); ``` Where: * `array_type` is a `type` defining an array. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const array_type_3d = [3, 5, 7]f16; const dims = @dimensions(array_type_3d); // dims is a 1D array of length 3 with values [3, 5, 7] ``` ## @element\_count Returns the total number of elements in the input array type as an `u32`. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @element_count(array_type); ``` Where: * `array_type` is a `type` defining an array. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const array_type_3d = [3, 5, 7]f16; const num_elements = @element_count(array_type_3d); // num_elements == 105 ``` ## @element\_type Returns the element type of the input array type as a `type`. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @element_type(array_type); ``` Where: * `array_type` is a `type` defining an array. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const array_type_3d = [3, 5, 7]f16; const elem_type = @element_type(array_type_3d); // elem_type == f16 ``` ## @export Creates a symbol in the output object file that refers to a global function or variable. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @export(ptr, options); ``` Where: * `ptr` is a pointer to a global function or variable. * `options` is a struct literal containing the required field: * `.name: comptime_string` is the name of the object file symbol. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn my_func() void { ... } var my_var: i16 = 42; comptime { @export(&my_func, .{ .name = "exported_func" }); @export(&my_var, .{ .name = "exported_var" }); } ``` ### Semantics The `@export` builtin must be called within a `comptime` block. The first argument must be a direct pointer to a global function or variable. The second argument must be a struct literal containing the required `.name` field with a `comptime_string` value. Calling `@export` has the same effect as declaring a symbol with the `export` storage class. This builtin is useful when the exported name needs to be computed at compile-time or when symbols need to be conditionally exported based on compile-time parameters. When `@export` is called on a symbol that is already declared via `export`, the builtin's name takes precedence. In addition, a symbol may only be exported once via `@export`. Multiple `@export` calls on the same symbol result in an error. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} param debug_level: comptime_int = 0; fn debug_hook() void { ... } comptime { if (debug_level > 0) { const suffix = if (debug_level == 1) "basic" else "verbose"; const debug_hook_name = @strcat("debug_hook_", suffix); @export(&debug_hook, .{ .name = debug_hook_name }); } } ``` ## @field Access the value of a given struct field. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @field(some_struct, field_name); ``` where: * `some_struct` is a value of a struct type. * `field_name` is a string. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var my_struct = .{.a = 10}; // returns the value of field 'a' const a = @field(my_struct, "a"); // The 'a' field of 'my_struct' will be assigned the value '20' @field(my_struct, "a") = 20; ``` ### Semantics The builtin returns the value stored in the `field_name` field of `some_struct` if and only if such field exists. A call to `@field` can also be used as the left-hand side of an assignment as shown in the example. In this scenario, the underlying field of `some_struct` named `field_name` will be updated. ## @fp16 Returns the selected runtime FP16 format. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fp16(); ``` ### Semantics The builtin returns a `type` representing the runtime FP16 format specified by the `--fp16-format` command line option: `f16`, `cb16`, or `bf16`. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} if (@fp16() == cb16) { @comptime_print("compiling with --fp16-format=cb16"); } // A simple function defined in terms of the selected runtime FP16 format. // For example, if the code is compiled with --fp16-format=bf16, square will // have type 'fn(bf16) bf16' fn square(x: @fp16()) @fp16() { return x * x; } ``` ## @get\_array Convert a string to an array of bytes. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @get_array(string); ``` Where: * `string` is an expression of type `comptime_string`. ### Semantics Given a value `s` of type `comptime_string`, `@get_array` returns an array of type `[@strlen(s)]u8`. This array contains the bytes inside the string. Note that: * Strings in CSL are *not* null-terminated, so the length of the array returned by `@get_array(s)` is `@strlen(s)`, not `@strlen(s)+1`. If a null-terminated array is required, this can be constructed by concatenating the string `"\x00"` onto the end of the string before passing it to `@get_array`. * Strings in CSL are strings of *bytes*, not of characters. String literals are interpreted as UTF-8, so if a string contains non-ASCII Unicode characters, the length of the array returned by `@get_array` will not match the number of characters in the string. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const s = "abc"; // The type of 'arr' will be [3]u8. const arr = @get_array(s); // 'a', 'b', 'c' are 97, 98, 99 in UTF-8. @comptime_assert(arr[0] == 97); @comptime_assert(arr[1] == 98); @comptime_assert(arr[2] == 99); // The type of 'arr_with_terminator' will be [4]u8. const arr_with_terminator = @get_array(@strcat(s, "\x00")); @comptime_assert(arr[0] == 97); @comptime_assert(arr[1] == 98); @comptime_assert(arr[2] == 99); @comptime_assert(arr[3] == 0); // Although 'has_unicode' only has five characters, its UTF-8 encoding is 15 // bytes in length, because each Japanese hiragana character takes 3 bytes // in UTF-8 encoding. const has_unicode = "こんにちは"; // The type of 'unicode_arr' will be [15]u8. const unicode_arr = @get_array(has_unicode); // The first character in the string is "HIRAGANA LETTER KO", which happens // to be encoded in UTF-8 as the bytes E3 81 93. @comptime_assert(unicode_arr[0] == 0xe3); @comptime_assert(unicode_arr[1] == 0x81); @comptime_assert(unicode_arr[2] == 0x93); // The last character in the string is "HIRAGANA LETTER HA", which happens // to be encoded in UTF-8 as the bytes E3 81 AF. @comptime_assert(unicode_arr[12] == 0xe3); @comptime_assert(unicode_arr[13] == 0x81); @comptime_assert(unicode_arr[14] == 0xaf); ``` ## @get\_color Create a value of type `color` with the provided identifier. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @get_color(color_id); ``` Where: * `color_id` is an integer value ### Semantics If `color_id` is comptime-known then it must be within the range of valid routable colors as defined by the target architecture. If `color_id` is not comptime-known its type must be a 16-bit unsigned integer. No runtime checks are performed in this case to ensure that the color id is within the range of valid colors. ## @get\_config Read the value of a PE configuration register. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // The type of 'config' becomes a machine-word-sized unsigned // integer type. var config = @get_config(addr); @get_config(addr, accessed_range); @get_config(addr, accessed_ranges); ``` Where: * `addr` is a machine-word-sized unsigned integer expression that represents the word-address of the configuration register. * `access_range`, if specified, is a comptime-known 2-element tuple of integers specifying an *inclusive* range of addresses that `addr` falls within. * `accessed_ranges`, if specified, is a tuple of comptime-known 2-element tuples of integers that each specify an *inclusive* range of addresses that `addr` may fall within. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} comptime { // The value '42' will be stored in the configuration address '0x7e00' // before the program begins execution. @set_config(0x7e00, 42); // The previously-set value '42' will be retrieved from the configuration // address '0x7e00'. const old_value = @get_config(0x7e00); // This call will overwrite the previously set value in the configuration // address `0x7e00`. const new_value = old_value + 3; @set_config(0x7e00, new_value); } var config: [N]u16; const base_addr = 0x7e00; task foo(i: u16) void { // Read a configuration register at runtime. config[i] = @get_config(base_addr + i); // Read a configuration register that is known to occur within the range // [base_addr, base_addr + N]. config[i] = @get_config(base_addr + i, .{base_addr, base_addr + N}); // Read a configuration register that is known to occur within the ranges // [base_addr, base_addr + 4] or [base_addr + 8, base_addr + N]. config[i] = @get_config(base_addr + i, .{.{base_addr, base_addr + 4}, .{base_addr + 8, base_addr + N}}); } ``` ### Semantics The `@get_config` builtin can only be called at runtime and during the evaluation of a top-level `comptime` block. It cannot be evaluated at comptime unless it is during the evaluation of a top-level `comptime` block. If `@get_config` is encountered during the evaluation of a top-level `comptime` block then it will retrieve any configuration value that was previously stored at `addr`. If no user-defined value has previously been written to `addr`, and a default value exists for the register at `addr`, the default value will be returned. Otherwise, `@get_config` will raise an error at compile time. A call to `@get_config` at runtime will become a volatile runtime read operation (i.e., a read that should never be optimized by the compiler) that will return any configuration value stored to `addr`. In that scenario the `addr` expression does not have to be comptime-known. If `addr` is comptime-known then it must be a comptime-known integer value that falls within the valid configuration address range for the selected target architecture. In cases where `addr` may be runtime but is known to occur within a specific range or set of ranges, a second argument can be provided to communicate this assumption. A single, contiguous range may be specified as a tuple of integers, `.{access_start, access_end}`. In this case, it is required that `access_start <= addr <= access_end`. Multiple ranges may be specified as a nested tuple, `.{.{access_start_1, access_end_1}, ..., .{access_start_N, access_end_N}}`. In this case, each inner tuple `.{access_start_i, access_end_i}` must satisfy `access_start_i <= access_end_i`, and `addr` must fall within one of the specified ranges. An error is emitted if the compiler is able to detect a violation of the above requirements. If a violation occurs at runtime that the compiler cannot detect, behavior is undefined. In addition, if `@get_config` is called during the evaluation of a top-level `comptime` block then it is not allowed to specify an address that falls within a configuration range that is reserved by the compiler. These ranges correspond to the following configurations: * All DSRs * Filters * Basic routing * Switches * Input queues * Task table `addr` must be coercible to a machine-word-sized unsigned integer expression regardless of whether it’s comptime-known or not. ## @get\_config\_unchecked Read the value of a PE configuration register unsafely. `@get_config_unchecked` is, by design, dangerous to use. `@get_config` (see [@get\_config](#@get_config)) should generally be preferred. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // The type of 'config' becomes a machine-word-sized unsigned // integer type. var config = @get_config_unchecked(addr); ``` Where: * `addr` is a machine-word-sized unsigned integer expression that represents the word-address of the configuration register. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var config: [N]u16; const base_addr = 0x7e00; task foo(i: u16) void { config[i] = @get_config_unchecked(base_addr + i); } ``` ### Semantics `@get_config_unchecked` is identical to `@get_config` (see [@get\_config](#@get_config)) with two exceptions: * The compiler will not attempt to check if a reserved address is accessed by `@get_config_unchecked`. Accessing a reserved address may result in undefined behavior. * The code generated for `@get_config` may insert delays to guarantee that it observes effects of preceding writes to configuration space. In some cases, this may be overly conservative. `@get_config_unchecked` will not cause such delays to be inserted. It is the programmer’s responsibility to ensure that `@get_config_unchecked` does not observe indeterminate states of configuration space. ## @get\_control\_task\_id Create a value of type `control_task_id` with the provided identifier. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @get_control_task_id(id); ``` Where: * `id` is a comptime-known expression of any unsigned integer type, or a runtime expression of type `u16`. ### Semantics The builtin will only accept integers in the corresponding target architecture’s valid range for control task IDs. If `id` is comptime-known, the builtin will only accept integers in the corresponding target architecture’s valid range for control task IDs. If `id` is not comptime-known, its type must be `u16`. No runtime checks are performed in this case to ensure that `id` is within the range of valid control task IDs. ## @get\_data\_task\_id Create a value of type `data_task_id` with the provided identifier. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @get_data_task_id(id); ``` Where: * `id` is an expression of type * WSE-2: `color`. * WSE-3: `input_queue`. ### Semantics On WSE-2, if `id` is comptime-known, it must be within the range of valid routable colors as defined by the target architecture. If `id` is not comptime-known, no runtime checks are performed in this case to ensure that `id` is within the range of valid routable colors. ## @get\_dsd Create either a memory or fabric DSD value. See [Data Structure Descriptors](/csl/language/dsds) for details. ## @get\_dsr Create a unique DSR identifier value. This value will uniquely identify a physical DSR along with its DSR file. See [Data Structure Registers](/csl/language/dsrs) for details. ## @get\_filter\_id Get the integer identifier of the filter associated with a given color. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @get_filter_id(color_value); ``` Where: * `color_value` is a value of type `color` ### Semantics The input `color_value` must be comptime-known and the builtin is guaranteed to be evaluated at compile-time. It returns the filter’s identifier (if any) as an unsigned 16-bit integer value. If there is no filter set for `color_value`, an error is emitted. An error is also emitted if the compiler is unable to determine a unique filter identifier for all the PEs that share the same code and parameter values. ## @get\_input\_queue Create a value of type ‘input\_queue’ with the provided identifier. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @get_input_queue(queue_id); ``` Where: * `queue_id` is a comptime-known non-negative integer expression ### Semantics The provided comptime-known `queue_id` must be a non-negative comptime-known integer expression that is within the range of valid input queue ids as defined by the target architecture. ## @get\_int For types containing an underlying integer, return that integer value. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @get_int(value); ``` Where: * `value` is an expression with any of the following types: * `color` * `control_task_id` * `data_task_id` * `dsr_dest` * `dsr_fifo_dest` * `dsr_fifo_src1` * `dsr_src0` * `dsr_src1` * any `enum` type * `input_queue` * any integer type * `local_task_id` * `output_queue` * `sr` * `ut_id` * `xdsr` ### Semantics The `@get_int` builtin must have a single argument `value` having one of the types listed above. The underlying integer value of `value` is returned. `@get_int` can be evaluated at both comptime and runtime. * If `value` has `enum` type, a value of the enum’s underlying integer type is returned. * If `value` has integer type, it is returned unchanged. * A `u16` is returned if `value` has type `color`, `control_task_id`, `data_task_id`, `input_queue`, `local_task_id`, or `output_queue`. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const an_id = @get_local_task_id(29); const a_num : i8 = -10; const an_enum = enum(u32) { FOO = 1, BAR = 2, BAZ = 3 }; comptime { const an_id_int = @get_int(an_id); @comptime_assert(@type_of(an_id_int) == u16); @comptime_assert(an_int == 29); const another_num = @get_int(a_num); @comptime_assert(@type_of(another_num) == i8); @comptime_assert(another_num == -10); const foo = @get_int(an_enum.FOO); @comptime_assert(@type_of(foo) == u32); @comptime_assert(foo == 1); } ``` ## @get\_local\_task\_id Create a value of type `local_task_id` with the provided identifier. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @get_local_task_id(id); ``` Where: * `id` is a comptime-known expression of any unsigned integer type, or a runtime expression of type `u16`. ### Semantics If `id` is comptime-known, the builtin will only accept integers in the corresponding target architecture’s valid range for local task IDs. If `id` is not comptime-known, its type must be `u16`. No runtime checks are performed in this case to ensure that `id` is within the range of valid local task IDs. ## @get\_output\_queue Create a value of type ‘output\_queue’ with the provided identifier. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @get_output_queue(queue_id); ``` Where: * `queue_id` is a comptime-known non-negative integer expression ### Semantics The provided comptime-known `queue_id` must be a non-negative comptime-known integer expression that is within the range of valid output queue ids as defined by the target architecture. ## @get\_rectangle Access the size of the rectangular region that was given to `set_rectangle`, and other layout information. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @get_rectangle(); ``` Returns a struct with `u16` fields `width` and `height`, and additional information about the underlying fabric and offsets. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} comptime { const rectangle = @get_rectangle(); // rectangle = .{ // .width = , .height = , // .fabric = { // .width = , .height = , // }, // .offsets = { // .width = , .height = , // } // } } ``` ### Semantics `get_rectangle` returns the `width` and `height` provided to `set_rectangle` as a struct. This struct also contains `fabric`, a nested struct that contains the `width` and `height` of the underlying fabric, and `offsets`, a nested struct that contains the `width` and `height` of the offset of the rectangle. The `@get_rectangle` builtin can be used anywhere. In a `layout` block, `@get_rectangle` is only valid *after* the call to `@set_rectangle`. ## @get\_string\_from\_byte Given a comptime-known, non-negative integer small enough to fit in one byte, returns a one-byte `comptime_string` containing only that byte. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @get_string_from_byte(byte); ``` where: * `byte` is a non-negative integer that fits in one byte (i.e., is in the range \[0, 255]). ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const s = @get_string_from_byte(65); // s == "A" const t = @get_string_from_byte('A'); // t == "A" const u = @get_string_from_byte('\n'); // u == "\n" const v = @get_string_from_byte(0); // v == "\x00" ``` ## @get\_ut\_id Available on WSE-3 only. Create a value of type `ut_id` from the provided integer identifier. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @get_ut_id(id); ``` Where: * `id` is a comptime-known expression of any unsigned integer type, or a runtime expression of type `u16`. ### Semantics If `id` is comptime-known, the builtin will only accept integers in the target architecture’s valid range for microthread IDs. If `id` is not comptime-known, its type must be `u16`. In this case no runtime checks are performed to ensure that `id` is within the range of valid microthread IDs. ## @has\_field Checks whether a given struct value or struct type has a field with a given name. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @has_field(some_struct, field_name); ``` where: * `some_struct` is a value of a struct type, or a struct type. * `field_name` is a string. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @has_field(.{.blah = 10}, "blah"); // returns true (struct value). const MyStruct = struct { foo: u32, bar: bool }; @has_field(MyStruct, "foo"); // returns true (struct type). @has_field(MyStruct, "baz"); // returns false. ``` ### Semantics The builtin returns true if and only if the struct `some_struct` has a field called `field_name`. The builtin is guaranteed to be evaluated at compile-time. The input expressions are guaranteed to have no run-time effects. ## @import\_module Import a group of global symbols defined in a CSL file, while optionally initializing parameters in the imported file. See [Modules](/csl/language/modules) for details. ## @increment\_dsd\_offset Set the offset of a memory DSD value. See [Data Structure Descriptors](/csl/language/dsds) for details. ## @initialize\_queue Associates a routable color with a queue ID, and optionally sets the priority of the microthread associated with the queue. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @initialize_queue(queue); @initialize_queue(queue, config); ``` Where: * `queue` is a comptime-known expression of type `input_queue` or `output_queue`. * `config` is a comptime-known struct expression with the following fields: * `color` is a comptime-known expression of type `color`. Required for input queues on both architectures and for output queues on WSE-3; optional for output queues on WSE-2. * `priority` is an optional field that can be either `.{ .high = true }`, `.{ .medium = true }`, or `.{ .low = true }`. * `ctrl_table_id` is * WSE-2: not supported. * WSE-3: an optional field that must be a comptime-known integer expression. * `dense_mode` is * WSE-2: not supported. * WSE-3: an optional field that must be a comptime-known boolean expression. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const rx = @get_color(4); const tx = @get_color(5); comptime { // associates queue ID 3 with the color rx (4) @initialize_queue(@get_input_queue(3), .{ .color = rx }); // ensures that output queue 5 is properly initialized at startup if (@is_arch("wse2")) { @initialize_queue(@get_output_queue(5)); } else if (@is_arch("wse3")) { // Associates output queue ID 4 with the color tx (5) @initialize_queue(@get_output_queue(4), .{ .color = tx }); // Associates input queue ID 0 with color rx (4). In addition, // it assigns a control task table ID of '4' to this input queue. // Control wavelets arriving through this input queue will be // directed to a separate control task table which is identified // by the `.ctrl_table_id` field. @initialize_queue(@get_input_queue(0), .{ .color = rx, .ctrl_table_id = 4 }); // Enables input queue '1' in dense mode. // Similarly for output queues. @initialize_queue(@get_input_queue(1), .{ .color = rx, .dense_mode = true }); } ``` ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const rx = @get_color(4); const rxq = @get_input_queue(3); comptime { // associates the queue rxq (3) with the color rx (4) and high microthread // priority if (@is_arch("wse2")) @initialize_queue(rxq, .{ .color = rx, .priority = .{ .high = true } }); } ``` ### Semantics The `@initialize_queue` builtin will initialize the input or output queue configuration associated with the input or output queue ID `queue` respectively. The builtin can only be called at most once per queue during the evaluation of a top-level comptime block. On WSE-2: * If the argument `queue` is an expression of type `output_queue` then the builtin must have no more than a single argument (i.e., the `queue` argument). * If the argument `queue` is an expression of type `input_queue` then the `config` argument must be supplied, and must be a comptime-known struct with fields `color` (required) and `priority` (optional). * The `color` field is required and specifies the routable fabric color to which the input queue with ID `queue` will be bound. * The `priority` field is optional and can be used to specify the priority of the microthread that will be attached to the respective input queue with ID `queue`. See [Microthread Priority](/csl/language/dsds#microthread-priority) for more information on microthread priority. The default value is `.{ .high = true }`. On WSE-3: * Both input and output queues require both `queue` and `config` arguments. * The `config` comptime-known struct argument must have the `color` field but not the `priority` field. * The `color` field specifies the routable fabric color that the input or output queue with ID `queue` will be bound to. * The `ctrl_table_id` field is optional and allowed on input queues only. It can be used to specify an index identifier that represents a per-queue local control task table. What this means is that control wavelets arriving through the input queue with ID `queue` will be associated with a per-queue local control task table identified by the `ctrl_table_id` value. The default value is `0`. * Multiple input queues can have the same value for `ctr_table_id` which means that they will be sharing the same control task table. For example, if we never use the `ctrl_table_id` for any of our input queues then the default behavior is that they will all share the same control task table with `ctrl_table_id=0` which is the same behavior as on WSE-2. * When `dense_mode` is enabled on an output queue, 16-bit data are sent as half wavelets rather than full wavelets. A half wavelet is a special kind of wavelet that is processed more efficiently by the hardware by allowing queues (input and output) to operate on a finer granularity. By default, `dense_mode` is disabled, meaning that data is sent as full wavelets. * An input queue must have `dense_mode` enabled in order to process half wavelets. Otherwise, the behavior is undefined. Input queues have `dense_mode` disabled by default. ## @is\_arch Returns true if the current CSL program is being compiled for the given target architecture. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @is_arch(an_arch); ``` Where: * `an_arch` is a comptime-known string value that represents the architecture mnemonic. The available mnemonics are: * `"wse2"`: for the WSE-2 architecture * `"wse3"`: for the WSE-3 architecture ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // Enable logic that is only valid if we are compiling // for the WSE-2 architecture. if (@is_arch("wse2")) { ... WSE-2-specific logic ... } ``` ## @is\_comptime Returns `true` if this expression is being evaluated as a comptime expression, and `false` otherwise. (See [`is_constant_evaluated`](https://en.cppreference.com/w/cpp/types/is_constant_evaluated) for the details about the same function in C++). ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @is_comptime(); ``` ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn foo() i32 { if (@is_comptime()) { // This branch is always comptime. return 1; } else { // This branch is never comptime, // so can call an external library, // and use architecture primitives. return 2; } } var comptime_result = @as(i32, 0); var non_comptime_result = @as(i32, 0); var init_result = foo(); // returns 1 task mytask() void { comptime_result = comptime foo(); // returns 1 non_comptime_result = foo(); // returns 2 } comptime { const comptime_block_result = foo(); // return 1 ... } ``` ## @is\_same\_type Returns true if the two type arguments to this function are the same. `@is_same_type` is deprecated. Use the `==` or `!=` operators instead. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @is_same_type(this_type, another_type); ``` Where: * `this_type` and `another_type` are values of type `type`. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} param myType: type; // This function uses the appropriate DSD operation based on the `myType` // param. fn mov(dst: mem1d_dsd, src: mem1d_dsd) void { if (@is_same_type(myType, f16)) { @fmovh(dst, src); } else { @comptime_assert(@is_same_type(myType, f32)); @fmovs(dst, src); } } ``` ## @load\_to\_dsr Load a DSD value into a DSR. See [Data Structure Registers](/csl/language/dsrs) for details. ## @map Given a function, a list of input arguments and an optional output argument, perform a mapping of the input arguments to the output argument (if any) using the provided function. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @map(callback, Input...); @map(callback, Input..., Output); ``` Where: * `callback` is a function that accepts as many arguments as the number of `Input` arguments. It may optionally produce a value. * `Input` is a list of zero or more input arguments. * `Output` is an output argument. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const math_lib = @import_module(""); const memDSD = @get_dsd(mem1d_dsd, .{.tensor_access = |i|{size} -> A[i, i]); const faboutDSD = @get_dsd(fabout_dsd, .{.extent = size, .fabric_color = blue}); task foo() void { // Compute the square-root of each element of `memDSD` and // send it out to `faboutDSD`. @map(math_lib.sqrt_f16, memDSD, faboutDSD); } ``` ### Semantics The `@map` builtin requires at least one of its arguments to be a DSD or DSR (input or output). If `callback` returns a non-void value, the `Output` argument is mandatory and must be either a DSD, DSR of type `dsr_dest`, or a non-const pointer value whose base-type must match the return type of `callback`. A `fabin_dsd` value is not allowed as the `Output` argument. The `Input` arguments may include non-DSD/DSR values whose types must be compatible with the corresponding parameter types of `callback`. Values of type `fabout_dsd` are not allowed as `Input` arguments. If a DSR is used as an `Input` argument, it must be of type `dsr_src1`. For each DSD or DSR argument to `@map`, the corresponding parameter type or return type of `callback` must be an ABI-compatible numeric type. Currently, these types are: `i16`, `i32`, `u16`, `u32`, `@fp16()`, `f32`. Note that `@fp16()` gives the type of the selected runtime FP16 format (see [@fp16](#@fp16)). DSR arguments to `@map` are expected to be loaded with the `single_step` property (see [single\_step](/csl/language/dsrs#single_step)). ### Execution Semantics The `@map` builtin repeatedly calls the `callback` function for each element of the DSD/DSR argument(s). Before each call to `callback`, the next available value from each `Input` DSD/DSR is read and passed to `callback` while the non-DSD/DSR `Input` arguments are forwarded to `callback`. The value returned from the `callback` call - if any - is written back to the `Output` DSD/DSR or to the memory address that is specified by the `Output` non-const pointer. After reading or writing a DSD/DSR element value the *length* (or *extent* for fabric DSDs) of the respective DSD/DSR is decremented by one. If the length/extent is zero then the read/write operation fails and the implicit `@map` loop terminates. If DSD/DSR operands have different lengths/extents, it is possible for values to be read and discarded. Similarly, the computed value from `callback` may be discarded. ## @ptrcast Casts a value of pointer type to a different pointer type. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @ptrcast(destination_ptr_type, ptr); ``` Where: * `destination_ptr_type` is a pointer type. * `ptr` is a value of pointer type. ### Semantics The builtin returns a pointer with the same memory address as `ptr`, but whose type is `destination_ptr_type`. The `destination_ptr_type` must not be a pointer whose base type is only valid in `comptime` expressions. See [Comptime](/csl/language/comptime). This builtin is not valid in `comptime` expressions. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const x:u32 = 10; const new_ptr = @ptrcast(i16*, &x); ``` ## @queue\_flush Available on WSE-3 only. Activates the teardown task ID (see [teardown](/csl/language/libraries#teardown)) once the given queue becomes empty. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @queue_flush(queue_id); ``` Where: * `queue_id` is an expression of type `input_queue` or `output_queue`. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const out_q = @get_output_queue(2); const fabout = @get_dsd(fabout_dsd, .{..., .output_queue = out_q}); task foo() void { @mov16(fabout, ..., .{.async = true}); // Will activate the teardown task once 'out_q' is empty. // Note that 'out_q' may not be empty when the microthread // above is done. @queue_flush(out_q); } ``` ### Semantics The builtin can only be evaluated at runtime, and therefore it cannot appear during comptime evaluation or during the evaluation of a top-level comptime or layout block. The builtin’s return type is `void`. Calling the builtin will cause the teardown task to be activated once the input (or output) queue associated with `queue_id` becomes empty or is already empty. It is guaranteed that if the teardown task is activated because of a call to `@queue_flush` then the respective queue will be empty. From within the teardown task the `queue_flush` library (see [queue\_flush](/csl/language/libraries#queue_flush)) can be used to check whether the teardown task was activated due to a call to `queue_flush` for a given queue or not. ## @random16 Generates a 16-bit pseudo-random value. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @random16(); ``` ### Semantics The builtin returns a `u16` value drawn from the currently active PRNG (see [`@set_active_prng`](#@set_active_prng)). The default active PRNG generates through the LFSR algorithm with polynomial (x^ + x^ + 1); other PRNGs available on the target use different fixed polynomials. The LFSR state of the active PRNG is advanced 128 iterations after every use. The initial state of each PRNG, when the program starts, is set to `0xdeadbeef`. State is not shared between PEs, but it is shared between tasks. The builtin is not valid in `comptime` expressions. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const x:u16 = @random16(); const y:i16 = @as(i16, @random16()); ``` ## @range Generates a sequence of evenly spaced numbers. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @range(elem_type, start, stop, step) @range(elem_type, stop) ``` where: * `elem_type` is an integer type. * `start`, `stop` and `step` are numeric values. ### Examples ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @range(i16, 1, 5, 1) // generates the sequence 1, 2, 3, 4. Note that 5 // is not included @range(i32, 2, -3, 1) // generates 2, 1, 0, -1, -2 @range(u16, 2, 7, -1) // empty sequence @range(comptime_int, 4) // generates 0, 1, 2, 3 ``` ### Semantics The range of elements is defined as follows: * `start` defines the first element of the sequence. * `step` defines how to generate the next element of the sequence given the previous element: `next = previous + step`. * `stop` defines an upper bound on the sequence such that all elements in the sequence are strictly less than `stop`. `start`, `stop` and `step` are coerced to the type `elem_type`. If this it not possible, a compilation error is issued. `step != 0` is required. If `step > 0 and stop <= start` or `step < 0 and stop >= start`, then the resulting sequence is empty. The two-argument version of `@range` is equivalent to the common scenario where `start == 0` and `step == 1`. ## @range\_start, @range\_stop, @range\_step Returns the `start`, `stop` or `step` value of a given range. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var r = @range(elem_type, start, stop, step) var first = @range_start(r); var last = @range_stop(r); var inc = @range_step(r); ``` ### Examples ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const r = @range(i32, 3, 9, 2); const start = @range_start(r); // start == 3 of type i32 var stop_arg : u32 = 13; var r2 = @range(u32, stop_arg); var stop = @range_stop(r2); // stop == stop_arg ``` ## @rank Returns the rank (number of dimensions) of the input array type as a `u16`. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @rank(array_type); ``` Where: * `array_type` is a `type` defining an array. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const array_type_3d = [3, 5, 7]f16; const rank = @rank(array_type_3d); // rank == 3 ``` ## @set\_active\_prng Sets the active PRNG (Pseudo-Random Number Generator). ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @set_active_prng(prng_id); ``` Where: * `prng_id` is a 16-bit unsigned integer expression that specifies the PRNG ID to be set. ### Semantics The input integer expression `prng_id` specifies the active PRNG ID as `prng_id % N` where `N` is the total number of PRNGs for the given architecture. This builtin cannot be evaluated at comptime. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var prng_id: u16 = 2; @set_active_prng(prng_id); ``` ## @set\_color\_config, @set\_local\_color\_config Specify the color configuration for a specific color at a specific processing element (PE) from a layout block (`@set_color_config`) or from a processing element’s top-level comptime block (`@set_local_color_config`). A color configuration includes routing, switching and filter configurations. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @set_color_config(x_coord, y_coord, this_color, config); @set_local_color_config(this_color, config); ``` Where: * `x_coord` and `y_coord` are comptime-known integers indicating the PE coordinates. * `this_color` is a comptime-known expression yielding a color value. * `config` is a comptime-known anonymous struct with the following fields and sub-fields: * `routes` * `rx` * `tx` * `pop_mode` (deprecated, moved to `switches` field) * `color_swap_x` * `color_swap_y` * `switches` * `pos1` * `pos2` * `pos3` * `current_switch_pos` * `ring_mode` * `pop_mode` * `filter` * `teardown` ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const directions = @import_module(""); color main_color; color other_color; layout { // Struct format // Route wavelets of color main_color from west to ramp and east. const routes = .{ .rx = .{ WEST }, .tx = .{ RAMP, EAST } }; @set_color_config(0, 0, main_color, .{ .routes = routes }); // Bitvector format // RX_WEST (0x1) | TX_RAMP (0x200) | TX_EAST (0x40) = 0x241 @set_color_config(1, 0, other_color, .{ .routes = 0x241 }); } comptime { // Struct format // Route wavelets of color main_color from west to ramp and east. const main_route = .{ .rx = .{ WEST }, .tx = .{ RAMP, EAST } }; @set_local_color_config(main_color, .{ .routes = main_route }); // Bitvector format const dirs = directions.RX_WEST | directions.TX_RAMP | directions.TX_EAST; @set_local_color_config(other_color, .{ .routes = dirs }); } ``` ### Semantics Both `@set_color_config` and `@set_local_color_config` builtins will set the color configuration - provided by the `config` field - to the input color value of one or more PEs. Calls to `@set_color_config` are only allowed during the evaluation of a layout block. As a result they always refer to a specific PE that is specified by the coordinate fields `x_coord` and `y_coord`. Calls to `@set_local_color_config` are only allowed during the evaluation of a top-level `comptime` block that belongs to a specific PE’s code and thus explicit coordinates are not needed as in calls to `@set_color_config`. However, since one or more PEs may share the same code - and thus the same top-level `comptime` block - a call to `@set_local_color_config` may be associated with multiple PEs depending on the rectangle’s PE-to-code mapping defined by calls to the `@set_tile_code` builtin. Any two calls to `@set_color_config` and/or `set_local_color_config` that refer to the same combination of PE and color are not allowed. Finally, a color configuration without a `routes` field is not allowed and will result in an error. That’s because both the `switches` and `filter` configurations are not valid without `routes`. If needed (e.g., for testing), a user can specify an empty `routes` field as `.routes = .{}`. #### Routing Configuration Semantics ##### rx and tx The `rx` and `tx` fields specify the *receive* and *transmit* route configurations for the given color. `rx` specifies the single direction (`EAST`, `WEST`, `SOUTH`, `NORTH`, or `RAMP`) from which wavelets on this color are expected to arrive; `tx` specifies the direction or directions to which wavelets on this color should be transmitted. The example above demonstrates how these fields can be used in calls to the `@set_color_config` and `@set_local_color_config` builtins. The `rx` field accepts a single direction value (e.g., `.rx = RAMP`), or equivalently a single-element comptime-known struct (e.g., `.rx = .{ RAMP }`). The `tx` field accepts either a single direction value (e.g., `.tx = WEST`) or a comptime-known struct of unique direction values (e.g., `.tx = .{ WEST, EAST, NORTH }`). The single-direction restriction on `rx` applies regardless of whether the struct form above or the bitvector form below is used; setting more than one RX bit in the bitvector is a compile-time error. **Bitvector Format**: Alternatively, the entire `routes` field can be specified as a 10-bit integer bitvector that encodes both RX and TX directions. The format is as follows: * **RX directions** (bits 0-4): `WEST=0x1`, `EAST=0x2`, `SOUTH=0x4`, `NORTH=0x8`, `RAMP=0x10` * **TX directions** (bits 5-9): `WEST=0x20`, `EAST=0x40`, `SOUTH=0x80`, `NORTH=0x100`, `RAMP=0x200` For example, `.routes = 0x210` configures RX from RAMP (`0x10`) and TX to RAMP (`0x200`). The `` library provides helper constants (`RX_RAMP`, `TX_RAMP`, etc.) and conversion functions (`dirToRxBits`, `dirToTxBits`). ##### pop\_mode This field is deprecated as a route setting and therefore it is moved to the switch configuration semantics section. See [Switching Configuration Semantics](#switching-configuration-semantics). ##### color\_swap\_x and color\_swap\_y Both `color_swap_x` and `color_swap_y` fields expect a boolean value that indicates whether we want to enable color swapping for the horizontal and vertical direction respectively. More details about color swapping can be found in [Color Swapping](/csl/language/advanced-features#color-swapping). #### Switching Configuration Semantics ##### pos1, pos2, pos3 The `pos1`, `pos2` and `pos3` fields expect either an integer bitvector, as described in [rx and tx](#rx-and-tx), or an anonymous struct value with only one of the following fields: * `rx` * `tx` * `invalid` A route configuration for a given color can change dynamically through *control wavelets*; special wavelets that may carry route configuration instructions. If we consider the `rx` and `tx` fields as the initial configuration of the *receive* and *transmit* routes respectively, then `pos1`, `pos2` and `pos3` are additional configurations we can switch to in-sequence from `pos1` to `pos3` whenever we receive route-switching control wavelets on the given color. In particular, the first route-switching control wavelet will set the `pos1` configuration. The next one will cause an advance to the `pos2` configuration and the third will cause an advance to the `pos3` configuration. Any additional route-switching control wavelets will either have no effect or go back to the initial configuration (defined by the `rx` and `tx` fields) depending on whether the `ring_mode` field is specified (see next section about the [`ring_mode`](#ring_mode)). Unlike the top-level `rx` field, the one nested under the `pos1`, `pos2` and `pos3` fields can only have a single direction value (i.e., `EAST`, `WEST`, `NORTH`, `SOUTH` or `RAMP`). On the other hand, the `tx` field can accept a single direction value or more than one (if the target supports it) just like the top-level `tx` field. In the following example, the call to `@set_local_color config` builtin will configure routing for color `red` such that receiving a route-switching control wavelet will change the *receive* direction to `EAST`: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const route = .{ .rx = .{ WEST }, .tx = .{ RAMP }, .pos1 = .{ .rx = EAST } }; @set_local_color_config(red, route); ``` The `invalid` field expects a boolean that must always be `true` which will indicate that we can never advance to the corresponding switch position and we will either remain on the previous one (if `ring_mode` is not enabled) or advance back to the original switch position indicated by the top-level `rx` and `tx` fields (if `ring_mode` is enabled). All switch positions will default to `.{ .invalid = true }`. ##### pop\_mode The `pop_mode` field expects an anonymous struct value with only one of the following fields: * `no_pop` * `always_pop` * `pop_on_advance` * `pop_on_advance_nop` Each of these fields expects a boolean that must be `true`. In other words, the `pop_mode` field can be viewed as an enum value that can take one of the 4 possible values above. By specifying the `pop_mode` field in a color configuration we can effectively mutate the sequence of instructions carried by control wavelets as they pass through a PE on a given color. In particular, when we select the `no_pop` mode the instruction sequence of control wavelets remains as is. If we select the `always_pop` mode then the first instruction in the sequence is always popped every time a control wavelet arrives and the respective instruction executed. If we select `pop_on_advance` then the first instruction is popped only if the control wavelet has advanced the route configuration to a new switch position (see section about `pos1`, `pos2` and `pos3` [`fields`](#pos1-pos2-pos3)). If we select `pop_on_advance_nop` then the first instruction is popped if the control wavelet has advanced the route configuration to a new switch position or is a no-op. ##### ring\_mode The `ring_mode` field expects a boolean value. If `true` then route-switching control wavelets will cause the route configuration to loop-back to the original configuration (specified by the `tx` and `rx` fields) once all valid switch positions have been visited (see previous section about `pos1`, `pos2` and `pos3` fields). If `false` then route-switching control wavelets will have no effect on the routing configuration once we reach the last valid switch position. In the following example, if at a given point in time, the route configuration for color `red` is at switch position 3 (specified by the `pos3` field) then receiving a route-switching control wavelet will cause the route configuration for `red` to loop-back into the initial one specified by fields `rx` and `tx`: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const route = .{ .rx = .{ WEST }, .tx = .{ RAMP }, .pos3 = .{ .rx = EAST }, .ring_mode = true }; @set_local_color_config(red, route); ``` ##### current\_switch\_pos The `current_switch_pos` field expects a non-negative integer in the range \[0-3] with `0` representing the initial route configuration (specified by the `tx` and `rx` fields) and `1`, `2` and `3` representing switch positions `1`, `2` and `3` respectively specified by `pos1`, `pos2` and `pos3` fields. The switch position pointed to by the `current_switch_pos` field will specify the initial route configuration for the given color. On WSE-2, all colors support switch configuration. On WSE-3, only a subset of colors support switch configuration: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 12, 13, 16, 17, 20, and 21. Switches should only be configured for these colors on WSE-3. #### Filter Configuration Semantics The `filter` field expects an anonymous struct value with the following fields: * `kind` * `count_data` * `count_control` * `init_counter` * `limit1` * `limit2` * `max_counter` * `filter_control` * `max_idx` * `min_idx` The `kind` field specifies the kind of the filter (or *filter mode*) which is an anonymous struct value with a single boolean field that is always true. The name of the boolean field represents the filter kind mnemonic which is one of: * `counter` * `sparse_counter` * `range` The `kind` field can be viewed as an enum value that can take one of the 3 values above. The kind of the filter will determine the subset of filter fields that are legal for that kind. Let's look at each one of the three possible filter kinds separately. ##### Counter filter The legal filter fields for a counter filter are the following: * `count_data` * `count_control` * `init_counter` * `limit1` * `max_counter` * `filter_control` A counter filter consists of an active wavelet counter that gets incremented every time a wavelet arrives at the given color. We can configure the counter filter so that the active wavelet counter gets incremented for every data wavelet or control wavelet or both. This is done by setting the `count_data` and/or `count_control` fields that expect a boolean value where `true` means that we enable data and control wavelet counting respectively. The default is `false` for both fields meaning that neither data nor control wavelets will cause the counter to get incremented. We can initialize the active wavelet counter by specifying the `init_counter` field that expects a non-negative integer value (the default value is zero). The filter counter will get incremented up to a certain value (inclusive) and then get reset to zero. That value is specified by the `limit1` field that expects a non-negative integer value that defaults to zero. The counter filter will reject all wavelets whose active counter is greater than a maximum value (exclusive) specified by the `max_counter` field that expects a non-negative integer value that defaults to zero. Finally, the `filter_control` field expects a boolean value. If `true` then only control wavelets that arrive when the value of the active counter is equal to `max_counter` are allowed to pass. All other wavelets will be rejected by the filter. In the following example, we set a counter filter for color `red` so that every 5th data wavelet is rejected: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const filter = .{.kind = .{.counter = true}, .count_data = true, .limit1 = 5, .max_counter = 4}; @set_local_color_config(red, filter); ``` ##### Sparse counter filter The legal filter fields for a sparse counter filter are the following: * `count_data` * `count_control` * `init_counter` * `limit1` * `limit2` * `max_counter` Sparse counter filters are similar to counter filters in that they both rely on an active wavelet counter and the semantics of the `count_data`, `count_control` and `max_counter` fields are identical. The difference is that when the `limit2` field is specified then when the active counter reaches the `limit1` value (inclusive) then the `limit2` value is copied to `limit1` before the active counter gets reset back to zero. When this happens `limit2` is effectively disabled and won’t be copied to `limit1` again. If the wavelet counter reaches `limit1` a second time - after `limit2` was copied to `limit1` - then the filter will no longer filter any wavelets and thus it will be effectively disabled. Both `limit1` and `limit2` counters expect non-negative integer values that default to zero. However, it is important to note that when `limit2` is not specified and the active counter reaches `limit1` then no other wavelets are filtered which means that the sparse counter filter is effectively disabled. The same is true when no `limit1` value is provided, i.e., the filter is effectively disabled and thus no filtering is done. ##### Range filter The legal filter fields for a range filter are the following: * `max_idx` * `min_idx` When the filter’s kind is set to `range` then all control wavelets are accepted. However, data wavelets are filtered based on their index value. In particular, a data wavelet will be rejected iff its index value is not within the range \[`min_idx`, `max_idx`]. #### Teardown Configuration Semantics The `teardown` field expects a comptime-known boolean expression. If `true` then the color associated with the given configuration will be set to teardown mode when the program starts. This means that all traffic will be suspended on that color until the teardown mode is exited explicitly at runtime through the `` standard library API. While a color is in teardown mode, all the configuration settings can be re-set at runtime using a standard library API. For example, filters can be configured using the `` standard library API. ## @set\_config Write the value of a PE configuration register. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @set_config(addr, config_value); @set_config(addr, config_value, accessed_range); @set_config(addr, config_value, accessed_ranges); ``` Where: * `addr` is a machine-word-sized unsigned integer expression that represents the word-address of the configuration register. * `config_value` is a machine-word-sized unsigned integer expressions that represents the new configuration value. * `access_range`, if specified, is a comptime-known 2-element tuple of integers specifying an *inclusive* range of addresses that `addr` falls within. * `accessed_ranges`, if specified, is a tuple of comptime-known 2-element tuples of integers that each specify an *inclusive* range of addresses that `addr` may fall within. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} comptime { // The value '42' will be stored in the configuration address '0x7e00' // before the program begins execution. @set_config(0x7e00, 42); // This call will overwrite the previously set value in the configuration // address `0x7e00`. const new_value = old_value + 3; @set_config(0x7e00, new_value); } var addr: u16; task foo(value: u16) void { // Writes 'value' at the configuration address 'addr' at runtime. @set_config(addr, value); // Re-writes 'value'. The compiler will not optimize away the // first write because it is considered volatile. @set_config(addr, value); // Write 'value' at the configuration address 'addr', where 'addr' is known // to occur within the range [0x7e00, 0x7e0f]. @set_config(addr, value, .{0x7e00, 0x7e0f}); // Write 'value' at the configuration address 'addr', where 'addr' is known // to occur within one of the ranges [0x7e00, 0x7e0f], [0x7e20, 0x7e2f], or // [0x7e40, 0x7e4f]. @set_config(addr, value, .{.{0x7e00, 0x7e0f}, .{0x7e20, 0x7e2f}, .{0x7e40, 0x7e4f}}); } ``` ### Semantics The `@set_config` builtin can be called at runtime and during the evaluation of a top-level `comptime` block. It cannot be evaluated at comptime unless it is during the evaluation of a top-level `comptime` block. If `@set_config` is encountered during the evaluation of a top-level `comptime` block then it will store `config_value` to `addr` during link time and therefore the configuration value is guaranteed to be present before the program begins execution. A call to `@set_config` during top-level comptime evaluation will always overwrite any configuration value that was previously stored at `addr`. A call to `@set_config` at runtime will become a volatile runtime write operation that will store `config_value` to `addr`. In that scenario, both `config_value` and `addr` expressions don’t have to be comptime-known. If `addr` is comptime-known then it must be a comptime-known integer value that falls within the valid configuration address range for the selected target architecture. In cases where `addr` may be runtime but is known to occur within a specific range or set of ranges, a second argument can be provided to communicate this assumption. A single, contiguous range may be specified as a tuple of integers, `.{access_start, access_end}`. In this case, it is required that `access_start <= addr <= access_end`. Multiple ranges may be specified as a nested tuple, `.{.{access_start_1, access_end_1}, ..., .{access_start_N, access_end_N}}`. In this case, each inner tuple `.{access_start_i, access_end_i}` must satisfy `access_start_i <= access_end_i`, and `addr` must fall within one of the specified ranges. An error is emitted if the compiler is able to detect a violation of the above requirements. If a violation occurs at runtime that the compiler cannot detect, behavior is undefined. In addition, if `@set_config` is called during the evaluation of a top-level `comptime` block then it is not allowed to specify an address that falls within a configuration range that is reserved by the compiler. These ranges correspond to the following configurations: * All DSRs * Filters * Basic routing * Switches * Input queues * Task table Most of the configurations in the list above can be changed through other means (see `@set_color_config`, `@set_local_color_config` and `@initialize_queue` builtins) while the rest are managed automatically by the compiler (i.e., DSRs and task tables). Both `addr` and `config_value` must be coercible to a machine-word-sized unsigned integer expressions regardless of whether they are comptime-known or not. ## @set\_config\_unchecked Write the value of a PE configuration register unsafely. `@set_config_unchecked` is, by design, dangerous to use. `@set_config` (see [@set\_config](#@set_config)) should generally be preferred. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @set_config_unchecked(addr, config_value); ``` Where: * `addr` is a machine-word-sized unsigned integer expression that represents the word-address of the configuration register. * `config_value` is a machine-word-sized unsigned integer expressions that represents the new configuration value. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var addr: u16; task foo(value: u16) void { @set_config_unchecked(addr, value); } ``` ### Semantics `@set_config_unchecked` is identical to `@set_config` (see [@set\_config](#@set_config)) with two exceptions: * The compiler will not attempt to check if a reserved address is accessed by `@set_config_unchecked`. Accessing a reserved address may result in undefined behavior. * The code generated for `@set_config` may insert delays to guarantee that its effects are observed by subsequent writes to configuration space. In some cases, this may be overly conservative. `@set_config_unchecked` will not cause such delays to be inserted. It is the programmer’s responsibility to ensure that `@set_config_unchecked` does not cause indeterminate states of configuration space to be observed. ## @set\_control\_task\_table Available on WSE-3 only. Create a separate control task table. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @set_control_task_table(); @set_control_task_table(config); ``` Where: * `config` is a comptime-known (anonymous) struct with the following optional fields: * `instructions` * `stride` ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} task foo() void {} task bar() void {} comptime { @bind_control_task(foo, @get_control_task_id(10)); @bind_local_task(bar, @get_local_task_id(10)); // Even though 'foo' and 'bar' have the same IDs they do not // clash because this call to @set_control_task_table will // decouple control tasks from data and local tasks. Control // tasks now have their own separate task table. @set_control_task_table(.{.instructions = 8, .stride = 4}); } ``` ### Semantics The `@set_control_task_table` builtin will decouple control tasks from data and local tasks by creating a separate task table that is dedicated to control tasks. The builtin can only be called at most once during the evaluation of a top-level comptime block. It can have an optional argument that must be a comptime-known struct with two optional fields: `instructions` and `stride`. If `@set_control_task_table` is called without an argument then the default values for `instructions` and `stride` will be used (see below). The `instructions` field can be used to specify the number of instructions for each entry point in the new control task table. The number of instructions must be a comptime-known integer value within the valid set of options which are `2`, `4` and `8`. The default value is `4`. The `stride` field requires a comptime-known integer value that represents the stride - in number of entry points - per input queue’s local control table index (see [@initialize\_queue](#@initialize_queue)). Its value should be in the range `[1, 7]` and the default value is `1`. ## @set\_dsd\_base\_addr Set the base-address of a memory DSD value. See [Data Structure Descriptors](/csl/language/dsds) for details. ## @set\_dsd\_length Set the length of a 1D memory DSD value. See [Data Structure Descriptors](/csl/language/dsds) for details. ## @set\_dsd\_stride Set the stride of a 1D memory DSD value. See [Data Structure Descriptors](/csl/language/dsds) for details. ## @set\_empty\_queue\_handler Available on WSE-3 only. Set a function to be the empty queue handler for a given queue. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @set_empty_queue_handler(func, queue_id); ``` Where: * `func` is the name of a function with no input parameters and `void` return type. * `queue_id` is a comptime-known expression of type `input_queue` or `output_queue`. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const tile_config = @import_module(""); fn foo() void { // Reconfiguration of empty queue takes place here. ... // At the end, ensure that the queue flush status register is reset // to prevent re-entrancy. tile_config.queue_flush.exit(); } const in_q = @get_input_queue(4); comptime { // Specifies function 'foo' to be executed when 'in_q' is flushed // (i.e., becomes empty) after calling '@queue_flush(in_q)'. @set_empty_queue_handler(foo, in_q); } ``` ### Semantics The `@set_empty_queue_handler` builtin must appear in a top-level `comptime` block. When `@queue_flush` (see [@queue\_flush](#@queue_flush)) has been called for a given queue (input or output) and the teardown task is activated due to that queue becoming empty, then the function associated with that queue, through a call to `@set_empty_queue_handler`, will be executed. The user is responsible for resetting the status of the queue flush status register through the `queue_flush` submodule of the `` library (see [queue\_flush](/csl/language/libraries#queue_flush)). Calling `@set_empty_queue_handler` for the same queue more than once is not allowed and will result in an error. If there is at least 1 call to `@set_empty_queue_handler` in the program then no task is allowed to be bound to the *teardown task ID* and vice-versa. The teardown task ID is the value returned by the CSL standard library through the teardown API (see [teardown](/csl/language/libraries#teardown)). ## @set\_fifo\_read\_length Set the read length of a FIFO DSD. See [Data Structure Descriptors](/csl/language/dsds) for details. ## @set\_fifo\_write\_length Set the write length of a FIFO DSD. See [Data Structure Descriptors](/csl/language/dsds) for details. ## @set\_rectangle Specify the size of the rectangular region of processing element that will execute this code. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @set_rectangle(width, height); ``` Where: * `width` and `height` are `comptime` integers. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} layout { // Use just one processing element for running this kernel. @set_rectangle(1, 1); } ``` ### Semantics The `@set_rectangle` builtin must appear only in a `layout` block. Additionally, there must be exactly one call to `@set_rectangle` in a `layout` block. ## @set\_teardown\_handler Set a function to be the teardown handler for a given color. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @set_teardown_handler(this_func, this_color); ``` Where: * `this_func` is the name of a function with no input parameters and ‘void’ return type. * `this_color` is a comptime-known expression of type ‘color’. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn foo() void { ... } const blue = @get_color(0); comptime { @set_teardown_handler(foo, blue); } ``` ### Semantics The `@set_teardown_handler` builtin must appear in a top-level `comptime` block. When a color goes into *teardown* mode at runtime, then the function associated with that color, through a call to `@set_teardown_handler`, will be executed. Calling `@set_teardown_handler` for the same color more than once is not allowed and will result in an error. The color will not automatically exit from the teardown mode. The user is responsible for exiting teardown mode explicitly from within the respective teardown handler function. The color that is passed to a `@set_teardown_handler` call must be within the range of routable colors for the given target architecture. If there is at least 1 call to `@set_teardown_handler` in the program then no task is allowed to be bound to the *teardown task ID* and vice-versa. The teardown task ID is the value returned by the CSL standard library through the teardown API (see [teardown](/csl/language/libraries#teardown)). ## @set\_tile\_code Specify the file that contains instructions to execute on a specific processing element, while optionally initializing parameters defined in the file. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @set_tile_code(x_coord, y_coord); @set_tile_code(x_coord, y_coord, filename); @set_tile_code(x_coord, y_coord, filename, param_binding); ``` Where: * `x_coord` and `y_coord` are `comptime` integers. * `filename` is a `comptime` string. * `param_binding` is a `comptime` anonymous struct. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} layout { // Specify to the compiler that *this* file contains code for PE #0,0. @set_tile_code(0, 0); // Inform compiler that code for PE #0,1 is in file "program.csl", relative // to the file that contains this `layout` block. @set_tile_code(0, 1, "program.csl"); // Inform compiler about the location of the code using an absolute path. @set_tile_code(0, 2, "/var/lib/csl/modules/code.csl"); // Instruct the compiler to use the file "other.csl" for code for PE #0,3. // Also, initialize the parameters `foo` and `bar` in that file. @set_tile_code(0, 3, "other.csl", .{ .foo = 10, .bar = -1 }); } ``` ### Semantics The `@set_tile_code` builtin must appear only in a `layout` block. Additionally, there must be exactly one call to `@set_tile_code` for each coordinate contained in the dimensions specified in the call to `@set_rectangle`. Unless the specified file path is an absolute path, it is interpreted as relative to the path of the file that contains the `@set_tile_code()` builtin call. ## @strcat Concatenates compile-time strings. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @strcat(str1, str2, ..., strN); ``` where: * each argument is an expression of type `comptime_string`. ### Semantics The `@strcat` builtin returns a value of `comptime_string` that results from concatenating its arguments. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @strcat("abc", "123"); // returns "abc123" @strcat("hello", " ", "world!"); // returns "hello world!" @strcat("abc"); // returns "abc" @strcat(""); // returns "" @strcat(); // returns "" ``` ## @strlen Returns the length of a compile-time string. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @strlen(str); ``` where: * `str` is an expression of type `comptime_string`. ### Semantics The `@strlen` builtin returns a value of type `comptime_int` equal to the length of its argument, i.e., the number of characters in the string. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @strlen(""); // returns 0 @strlen("abc123"); // returns 6 @strlen(if (42 == 42) "abc" else "abc123"); // returns 3 ``` ## @type\_of Returns the type of an expression. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @type_of(any_expression); ``` where: * `any_expression` is any valid expression. ### Semantics The `@type_of` builtin returns a value of type `type` describing the evaluated type of the input expression. The builtin is always evaluated at compile-time, but the input expression does not need to be `comptime`. No code is generated for the input expression; as such, this expression will not have run-time effects on the program. ## @unblock Unblock the task associated with the input `color`, `data_task_id`, or `local_task_id` so that the task can be run when the task identifier is activated. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @unblock(id); ``` Where: * `id` is an expression of type * WSE-2: `color`, `data_task_id`, or `local_task_id`. * WSE-3: `input_queue`, `data_task_id`, `local_task_id`, or `ut_id`. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const task_id: local_task_id = @get_local_task_id(10); comptime { @bind_local_task(my_task, task_id); @block(task_id); } fn foo() void { // allow my_task to be run whenever my_task is activated @unblock(task_id); } ``` ## @zeros Initialize a tensor with zeros. ### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @zeros(tensor_type); ``` Where: * `tensor_type` is a comptime-known numeric tensor type. ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // Initialize a tensor of four rows and five columns with all zeros. const matrix = @zeros([4,5]f16); ``` ## Builtins for Remote Procedure Calls (RPC) This category includes builtins that enable users to advertise device symbols to the host so that the host can interact with them through wavelets akin to RPC. The advertised symbols could be data or functions forming a host-callable API. In addition, this builtin category includes builtins that allow users to interpret incoming wavelets by associating them with the respective advertised symbols. ### @export\_name Declare that a given symbolic name can be advertised from one or more processing elements with a specific type and mutability. #### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @export_name(name, type); @export_name(name, type, isMutable); ``` Where: * `name` is an expression of type `comptime_string` * `type` is an expression of type `type` * `isMutable` is a comptime-known expression of type `bool` #### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} layout { // Declare that symbolic name "A" can only be advertised // with type 'i16' by one or more PEs. The advertised // symbol must also be mutable (i.e., declared as 'var'). @export_name("A", i16, true); // Declare that symbol name "foo" can only be advertised // as a function with type 'fn(f32)void' by one or more // PEs. @export_name("foo", fn(f32)void); } ``` #### Semantics Calls to the `@export_name` builtin can only appear during the evaluation of a layout block. A given name can only be exported once with `@export_name`. The third `isMutable` parameter must be provided unless `type` is a function type. The `type` parameter cannot be a comptime-only type (e.g., `comptime_int`, `comptime_float` etc.) with the exception of function types. In addition, the `type` parameter cannot be an aggregate type like an array or struct. The `type` parameter cannot be an enum type as well. If `type` is a function type, then the same rules apply to the respective function parameter types and return type. If `type` is a function type, then it can have a maximum of 15 input parameters. ### @export\_symbol Advertise a device symbol to the host with a given name, if provided. #### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @export_symbol(symbol); @export_symbol(symbol, name); ``` Where: * `symbol` is a reference to a global device symbol. * `name` is an expression of type `comptime_string`. #### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var A: i16; fn bar(a: f32) void {...} comptime { // Advertise symbol 'A' to the host. Since no 'name' // is provided, the default name would be the symbol's // name, i.e., 'A'. @export_symbol(A); // Advertise function 'bar' as 'foo' to the host. @export_symbol(bar, "foo"); } ``` #### Semantics Calls to the `@export_symbol` builtin can only appear during the evaluation of a top-level comptime block. Its first argument must be a global symbol that is used at least once by code that is not comptime evaluated. A given symbol can be exported multiple times as long as each time the `name` argument is provided and it is unique. If `name` is not provided then the name of the symbol is used as the advertised name instead. The advertised name (whether it is explicitly provided or defaulted to the symbol’s actual name) must always correspond to a name that was exported during layout evaluation using the `@export_name` builtin. That is, there must be a name exported with `@export_name` during layout evaluation that has the same name, type and mutability. The compiler will collect all exported symbols and advertise them to the host by producing a JSON file containing meta-data for each one. The schema of the produced JSON file is as follows: ```json theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} { "rpc_symbols": [ { "id": "Unique integer identifier for the exported symbol." "comment1": "Boolean indicating whether the exported symbol is", "comment2": "immutable or not. Not specified for functions." "immutable": "True/False", "name": "The name that is advertised to the host.", "type": "Type of the advertised symbol. Return-type for functions.", "kind": "One of Var/Func/Stream.", "color": "Integer representing the color of a stream if kind=Stream", "inputs": [ "comment1": "The parameters of the function.", "comment2": "Not specified for data.", "name": "The name of the function parameter.", "type": "The type of the function parameter." ] } ] } ``` ### @get\_symbol\_id Returns the unique integer identifier for an advertised symbol. #### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @get_symbol_id(symbol); ``` Where: * `symbol` is a reference to an advertised global device symbol. #### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var A: i16; fn bar(a: f32) void {...} task main(idx: u16) void { // If the incoming wavelet corresponds to the unique integer // id for 'A' then use 'A'. // If the incoming wavelet corresponds to the unique integer // id for 'bar' the call 'bar'. if (@get_symbol_id(A) == idx) { A = 42; } else if (@get_symbol_id(bar) == idx) { bar(...); } } ``` #### Semantics The `@get_symbol_id` builtin can be called at comptime or runtime but it is not allowed to appear during layout evaluation. The input `symbol` must have been advertised using `@export_symbol`. ### @get\_symbol\_value Returns the value of an advertised global symbol given a runtime integer identifier value. #### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @get_symbol_value(type, id); ``` Where: * `type` an expression of type `type`. * `id` is a runtime-only integer identifier value. #### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var A: i16; var B: i16; task main(idx: u16) void { // Given an integer identifier passed with a wavelet, // return the value of 'A' or 'B' depending on the value // of 'idx'. var value = @get_symbol_value(i16, idx); ... } ``` #### Semantics The `@get_symbol_value` builtin can only be called at runtime. If no symbol was advertised with the given integer identifier then the behavior is undefined. If there is a symbol advertised with the given integer identifier, then the builtin will return a copy of its value. There has to be at least one global symbol advertised with type `type`. ### @get\_tensor\_ptr Returns the value of an exported tensor pointer given a runtime integer identifier value. #### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @get_tensor_ptr(id); ``` Where: * `id` is a runtime-determined integer identifier value. #### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var A_ptr: [*]f16 = &A; var B_ptr: *[size]i32 = &B; comptime { @export_symbol(A_ptr); @export_symbol(B_ptr); } task main(idx: u16) void { // Given an integer identifier passed with a wavelet, // return the address of 'A' or 'B' depending on the value // of 'idx'. var ptr = @get_tensor_ptr(idx); ... } ``` #### Semantics The `@get_tensor_ptr` builtin can only be called at runtime. If no tensor pointer has been advertised with the given integer identifier then the behavior is undefined. If there is a tensor pointer advertised with the given integer identifier, then the builtin will return a copy of the pointer bit-casted into a `[*]u16` type. ### @get\_xdsr Create a unique XDSR identifier value. This value will uniquely identify a physical XDSR. See [Data Structure Registers](/csl/language/dsrs) for details. ### @has\_exported\_tensors Returns `true` iff there is at least 1 tensor pointer exported and `false` otherwise. #### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @has_exported_tensors(); ``` #### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} task main(idx: u16) void { // If there are no exported tensors, the body // of the conditional branch will be removed. if (@has_exported_tensors()) { ... var ptr = @get_tensor_ptr(idx); ... } } ``` #### Semantics The `@has_exported_tensors` builtin cannot be called from a top-level comptime block or a layout block. It is guaranteed to be evaluated at comptime. It will return `true` iff there is at least 1 exported tensor pointer and `false` otherwise. ### @rpc Creates an RPC server listening to a given color. #### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @rpc(task_id); ``` Where: * `task_id` is an expression of type `data_task_id`. #### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const rpc_color = @get_color(22); const rpc_task_id = @get_data_task_id(rpc_color); fn foo(a: u16, b: f32) void {...} comptime { @export_symbol(foo); // Creates an RPC server listening to color '22' through // a wavelet-triggered task bound to data-task ID 'rpc_task_id'. // This RPC server can only dispatch calls to 'foo'. // Any other RPCs will be ignored. @rpc(rpc_task_id); // The user needs to ensure that traffic on color '22' // is directed into the RPC server. @set_local_color_config(rpc_color, .{.routes = .{.rx = .{WEST}, .tx = .{RAMP} }}); } ``` #### Semantics The `@rpc` builtin can only be called during the evaluation of a top-level comptime block. A call to `@rpc` will produce a wavelet-triggered task (WTT) that is bound to `task_id` and would receive data from the underlying routable color. Note that the user is responsible for routing the data through that color such that they are received by the produced WTT. No other task may be bound to `task_id` and vice-versa. The WTT-based RPC server is expected to receive sequences of wavelets. Each one of these sequences corresponds to a single RPC that consists of a unique integer identifier corresponding to an exported function along with its input arguments. If the input arguments of an RPC do not match the expected number of arguments for a given exported function, a runtime assertion is triggered. If the unique integer identifier of an RPC does not match any exported function, the call will be ignored and the server will be ready for the next RPC sequence. If the function called by the RPC server returns a value, then this value will be ignored. No more than 1 call to `@rpc` is allowed for a given tile code which means that we can always have up to 1 RPC server per PE. ## Builtins for DSD Operations These builtins perform bulk operations on a set of elements described by DSDs, by exploiting native hardware instructions. The destination operand is always the first argument, and the subsequent arguments are either DSDs, scalars, or pointers. Additionally, many of these builtins have a SIMD (single instruction, multiple data) mode. For more information, see [SIMD Mode](/csl/language/appendix#simd-mode). ### Syntax For the DSD operation builtins below, the arguments are labeled as follows: * `dest_dsd`, `src_dsd1`, and `src_dsd2` are constants or variables created using the `@get_dsd` builtin or the `get_dsr` builtin. * `dest_dsd` is the destination DSD or DSR. If this is a DSR value it must be of type `dsr_dest` or `dsr_src0`. * `src_dsd1` and `src_dsd2` are source DSDs or DSRs or any combination of them. If any of the source operands are DSRs then they cannot be of type `dsr_dest`. * `i16_value` is a value of type `i16`. * `i32_value` is a value of type `i32`. * `u16_value` is a value of type `u16`. * `u32_value` is a value of type `u32`. * `fp16_value` is a value of type `@fp16()`, where `@fp16()` gives the type of the selected runtime FP16 format (see [@fp16](#@fp16)). * `f32_value` is a value of type `f32`. * `i16_pointer` is a pointer to a value of type `i16`. * `i32_pointer` is a pointer to a value of type `i32`. * `u16_pointer` is a pointer to a value of type `u16`. * `u32_pointer` is a pointer to a value of type `u32`. * `fp16_pointer` is a pointer to a value of type `@fp16()`. * `f32_pointer` is a pointer to a value of type `f32`. ### @add16 Add two 16-bit integers. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @add16(dest_dsd, src_dsd1, src_dsd2); @add16(dest_dsd, i16_value, src_dsd1); @add16(dest_dsd, u16_value, src_dsd1); @add16(dest_dsd, src_dsd1, i16_value); @add16(dest_dsd, src_dsd1, u16_value); ``` ### @addc16 Add two 16-bit integers, with carry. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @addc16(dest_dsd, src_dsd1, src_dsd2); @addc16(dest_dsd, i16_value, src_dsd1); @addc16(dest_dsd, u16_value, src_dsd1); @addc16(dest_dsd, src_dsd1, i16_value); @addc16(dest_dsd, src_dsd1, u16_value); ``` ### @and16 Bitwise-and two 16-bit integers. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @and16(dest_dsd, src_dsd1, src_dsd2); @and16(dest_dsd, i16_value, src_dsd1); @and16(dest_dsd, u16_value, src_dsd1); @and16(dest_dsd, src_dsd1, i16_value); @and16(dest_dsd, src_dsd1, u16_value); ``` ### @clz Count leading zeros. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // count leading zeros @clz(dest_dsd, src_dsd1); @clz(dest_dsd, i16_value); @clz(dest_dsd, u16_value); ``` ### @ctz Count trailing zeros. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // count trailing zeros @ctz(dest_dsd, src_dsd1); @ctz(dest_dsd, i16_value); @ctz(dest_dsd, u16_value); ``` ### @fabsh Absolute value of a 16-bit floating point. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fabsh(dest_dsd, src_dsd1); @fabsh(dest_dsd, fp16_value); ``` ### @fabss Absolute value of a 32-bit floating point. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fabss(dest_dsd, src_dsd1); @fabss(dest_dsd, f32_value); ``` ### @faddh Add two 16-bit floating point values. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @faddh(dest_dsd, src_dsd1, src_dsd2); @faddh(dest_dsd, fp16_value, src_dsd1); @faddh(dest_dsd, src_dsd1, fp16_value); @faddh(fp16_pointer, fp16_value, src_dsd1); ``` ### @faddhs Add a 16-bit and 32-bit floating point value. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @faddhs(dest_dsd, src_dsd1, src_dsd2); @faddhs(dest_dsd, fp16_value, src_dsd1); @faddhs(dest_dsd, src_dsd1, fp16_value); @faddhs(f32_pointer, f32_value, src_dsd1); ``` ### @fadds Add two 32-bit floating point values. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fadds(dest_dsd, src_dsd1, src_dsd2); @fadds(dest_dsd, f32_value, src_dsd1); @fadds(dest_dsd, src_dsd1, f32_value); @fadds(f32_pointer, f32_value, src_dsd1); ``` ### @fh2s Convert a 16-bit floating point value to a 32-bit floating point value. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fh2s(dest_dsd, src_dsd1); @fh2s(dest_dsd, fp16_value); ``` ### @fh2xp16 Convert a 16-bit floating point value to a 16-bit integer. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fh2xp16(dest_dsd, src_dsd1); @fh2xp16(dest_dsd, fp16_value); @fh2xp16(i16_pointer, fp16_value); ``` ### @fmach 16-bit floating point multiply-add. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fmach(dest_dsd, src_dsd1, src_dsd2, fp16_value); ``` ### @fmachs 16-bit floating point multiply with 32-bit addition. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fmachs(dest_dsd, src_dsd1, src_dsd2, fp16_value); ``` ### @fmacs 32-bit floating point multiply-add. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fmacs(dest_dsd, src_dsd1, src_dsd2, f32_value); ``` ### @fmaxh 16-bit floating point max. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fmaxh(dest_dsd, src_dsd1, src_dsd2); @fmaxh(dest_dsd, fp16_value, src_dsd1); @fmaxh(dest_dsd, src_dsd1, fp16_value); @fmaxh(fp16_pointer, fp16_value, src_dsd1); ``` ### @fmaxs 32-bit floating point max. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fmaxs(dest_dsd, src_dsd1, src_dsd2); @fmaxs(dest_dsd, f32_value, src_dsd1); @fmaxs(dest_dsd, src_dsd1, f32_value); @fmaxs(f32_pointer, f32_value, src_dsd1); ``` ### @fmovh Move a 16-bit floating point value. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fmovh(dest_dsd, src_dsd1); @fmovh(fp16_pointer, src_dsd1); @fmovh(dest_dsd, fp16_value); ``` ### @fmovs Move a 32-bit floating point value. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fmovs(dest_dsd, src_dsd1); @fmovs(f32_pointer, src_dsd1) @fmovs(dest_dsd, f32_value); ``` ### @fmulh Multiply 16-bit floating point values. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fmulh(dest_dsd, src_dsd1, src_dsd2); @fmulh(dest_dsd, fp16_value, src_dsd1); @fmulh(dest_dsd, src_dsd1, fp16_value); @fmulh(fp16_pointer, fp16_value, src_dsd1); ``` ### @fmuls Multiply 32-bit floating point values. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fmuls(dest_dsd, src_dsd1, src_dsd2); @fmuls(dest_dsd, f32_value, src_dsd1); @fmuls(dest_dsd, src_dsd1, f32_value); @fmuls(f32_pointer, f32_value, src_dsd1); ``` ### @fnegh Negate a 16-bit floating point value. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fnegh(dest_dsd, src_dsd1); @fnegh(dest_dsd, fp16_value); ``` ### @fnegs Negate a 32-bit floating point value. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fnegs(dest_dsd, src_dsd1); @fnegs(dest_dsd, f32_value); ``` ### @fnormh Normalize a 16-bit floating point value. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fnormh(fp16_pointer, fp16_value); ``` ### @fnorms Normalize a 32-bit floating point value. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fnorms(f32_pointer, f32_value); ``` ### @fs2h Convert a 32-bit floating point value to a 16-bit floating point value. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fs2h(dest_dsd, src_dsd1); @fs2h(dest_dsd, f32_value); ``` ### @fs2xp16 Convert a 32-bit floating point value to a 16-bit integer. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fs2xp16(dest_dsd, src_dsd1); @fs2xp16(dest_dsd, f32_value); @fs2xp16(i16_pointer, f32_value); ``` ### @fscaleh 16-bit floating point multiplied by a constant. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fscaleh(fp16_pointer, fp16_value, i16_value); ``` ### @fscales 32-bit floating point multiplied by a constant. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fscales(f32_pointer, f32_value, i16_value); ``` ### @fsubh Subtract two 16-bit floating point values. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fsubh(dest_dsd, src_dsd1, src_dsd2); @fsubh(dest_dsd, fp16_value, src_dsd1); @fsubh(dest_dsd, src_dsd1, fp16_value); @fsubh(fp16_pointer, fp16_value, src_dsd1); ``` ### @fsubs Subtract two 32-bit floating point values. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @fsubs(dest_dsd, src_dsd1, src_dsd2); @fsubs(dest_dsd, f32_value, src_dsd1); @fsubs(dest_dsd, src_dsd1, f32_value); @fsubs(f32_pointer, f32_value, src_dsd1); ``` ### @mov16 Move a 16-bit integer. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @mov16(dest_dsd, src_dsd1); @mov16(i16_pointer, src_dsd1); @mov16(u16_pointer, src_dsd1); @mov16(dest_dsd, i16_value); @mov16(dest_dsd, u16_value); ``` ### @mov32 Move a 32-bit integer. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @mov32(dest_dsd, src_dsd1); @mov32(i32_pointer, src_dsd1); @mov32(u32_pointer, src_dsd1); @mov32(dest_dsd, i32_value); @mov32(dest_dsd, u32_value); ``` ### @or16 Bitwise-or on two 16-bit integers. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @or16(dest_dsd, src_dsd1, src_dsd2); @or16(dest_dsd, i16_value, src_dsd1); @or16(dest_dsd, u16_value, src_dsd1); @or16(dest_dsd, src_dsd1, i16_value); @or16(dest_dsd, src_dsd1, u16_value); ``` ### @popcnt Population count of an integer. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @popcnt(dest_dsd, src_dsd1); @popcnt(dest_dsd, i16_value); @popcnt(dest_dsd, u16_value); ``` ### @sar16 Arithmetic shift right of a 16-bit integer. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @sar16(dest_dsd, src_dsd1, src_dsd2); @sar16(dest_dsd, i16_value, src_dsd1); @sar16(dest_dsd, u16_value, src_dsd1); @sar16(dest_dsd, src_dsd1, i16_value); @sar16(dest_dsd, src_dsd1, u16_value); ``` ### @sll16 Logical shift left of a 16-bit integer. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @sll16(dest_dsd, src_dsd1, src_dsd2); @sll16(dest_dsd, i16_value, src_dsd1); @sll16(dest_dsd, u16_value, src_dsd1); @sll16(dest_dsd, src_dsd1, i16_value); @sll16(dest_dsd, src_dsd1, u16_value); ``` ### @slr16 Logical shift right of a 16-bit integer. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @slr16(dest_dsd, src_dsd1, src_dsd2); @slr16(dest_dsd, i16_value, src_dsd1); @slr16(dest_dsd, u16_value, src_dsd1); @slr16(dest_dsd, src_dsd1, i16_value); @slr16(dest_dsd, src_dsd1, u16_value); ``` ### @sub16 Subtract two 16-bit integers. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @sub16(dest_dsd, src_dsd1, src_dsd2); @sub16(dest_dsd, src_dsd1, i16_value); @sub16(dest_dsd, src_dsd1, u16_value); ``` ### @xor16 Xor two 16-bit integers. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @xor16(dest_dsd, src_dsd1, src_dsd2); @xor16(dest_dsd, i16_value, src_dsd1); @xor16(dest_dsd, u16_value, src_dsd1); @xor16(dest_dsd, src_dsd1, i16_value); @xor16(dest_dsd, src_dsd1, u16_value); ``` ### @xp162fh Convert a 16-bit integer into a 16-bit floating point value. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @xp162fh(dest_dsd, src_dsd1); @xp162fh(dest_dsd, i16_value); @xp162fh(dest_dsd, u16_value); ``` ### @xp162fs Convert a 16-bit integer into a 32-bit floating point value. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @xp162fs(dest_dsd, src_dsd1); @xp162fs(dest_dsd, i16_value); @xp162fs(dest_dsd, u16_value); ``` ### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var tensor = [5]i16 {1, 2, 3, 4, 5}; const dsd = @get_dsd(mem1d_dsd, .{ .tensor_access = |i|{5} -> tensor[i] }); fn foo() void { // Add the constant 10 to each element of `tensor`. // After executing this operation, `tensor` contains 11, 12, 13, 14, 15. @add16(dsd, dsd, 10); } ``` ### @dfilt Instructs an input queue to drop all data wavelets until a certain number of control wavelets are encountered. #### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @dfilt(dsd, configuration); ``` Where: * `dsd` is a `fabin_dsd` or DSR that contains a `fabin_DSD`. * If a DSR is used, it must have type `dsr_src1` and be loaded with the `async` configuration (see [Data Structure Registers](/csl/language/dsrs)). Behavior is undefined if `@dfilt` is used with a DSR that does not meet these conditions. * `configuration` is the configuration struct that is optionally provided to other DSD operations (see [Data Structure Descriptors](/csl/language/dsds)). #### Semantics The first argument to `@dfilt` must be a `fabin_dsd` or a DSR representing an ‘async’ `fabin_dsd`. A call to `@dfilt` will drop data wavelets arriving on the input queue associated with the input DSD. The `extent` of the DSD determines the number of control wavelets the operation expects. The input queue will drop all data wavelets until the specified number of control wavelets is encountered. Unlike other DSD operations, the configuration struct is required, and the `async` configuration must be `true`. `@dfilt` does not support the `on_control` or `index` configurations. #### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var dsd = @get_dsd(fabin_dsd, .{ .fabric_color = 2, .extent = 10, .input_queue = @get_input_queue(1) }); fn foo() void { // Executing this operation causes input queue 1 to drop data wavelets // until 10 control wavelets have been encountered. @dfilt(dsd, .{ .async = true }); } ``` # Comptime Source: https://sdk.cerebras.ai/csl/language/comptime Use the comptime keyword in CSL to guarantee compile-time evaluation of variables, expressions, and control flow with no runtime footprint. In CSL, it is possible to ensure that code is executed at compile-time by using the `comptime` keyword, which guarantees that the code will have no run-time footprint. An error is emitted if compile-time evaluation is not possible. ## Comptime Variables A `comptime` variable guarantees that all loads and stores to this variable happen at compile-time. As such, this variable has no run-time footprint and its address cannot be obtained. Unlike constants, `comptime` variables can be modified, as shown below. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} task foo() void { comptime var bitmap: u16 = 0xffff; bitmap &= 0x0400; ... } ``` `comptime` variables need to be declared inside a block or a function. For global variables, using `const` is enough: the initializer of global variables is always implicitly comptime, since CSL does not support run-time initialization of global variables. Since all loads and stores to a `comptime` variable must happen at compile-time, the stored value must be comptime-known (explained below) and any offsets (like array indices) must be comptime-known as well. Similarly, stores to `comptime` variables must not depend on run-time control flow. ## Comptime-Known Values All values that are known to the compiler at compile-time are comptime-known values. Formally, the compiler uses the following rules to determine whether a value is comptime-known: 1. All literals (e.g. `1.0`) are comptime-known values. 2. All `const` variables with a comptime-known initializer (e.g. `const x = 1.0`) are comptime-known values. 3. All uses of `comptime` or `param` variables are comptime-known values. 4. Expressions comprised of two or more comptime-known values (e.g. `1.0 + 2.0 * x`) are comptime-known values. However, function calls are an exception to this rule. The compiler ensures that, with the exception of function calls, all compositions of comptime-known values are comptime-known values as well. The next section describes how function calls can be explicitly marked as comptime-known. ## Comptime Expressions Expressions that depend on constants or other `comptime` variables can be explicitly marked for evaluation at compile time by prefixing the expression with the `comptime` keyword. For instance, the following snippet ensures that the call to `foo()` is replaced with its return value at compile time, so that we do not pay a run-time cost for the function call. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} param bar: u16; fn foo(arg: u16) u16 { return arg * 2; } task goo() void { var myVar = comptime foo(bar); ... } ``` When evaluating a function call at compile-time, all expressions and variables are implicitly comptime. The compiler will error out if such function attempts to read a non comptime-known global variable or attempts to store to a global variable. Note that the evaluation of binary logical connectives (i.e., `and` and `or` operators) will short-circuit, if possible, even at comptime. When short-circuiting applies, semantic checks like type-checking and checks for unbound identifiers will not be applied to the right-hand operand, as shown in the example below: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // The 'invalid' term of the following expression will // not be evaluated because comptime evaluation will short-circuit // on the 'false' term causing the whole expression to evaluate to // 'false'. const and_result = false and invalid; // Similarly, the following expression will evaluate to 'true' // without evaluating the 'invalid' term. const or_result = true or invalid; ``` ## Types Whose Values are Required to be Comptime Certain types are only allowed to exist in expressions and variables that are `comptime`. In general, these refer to types whose values: 1. have no possible memory layout associated with them, or 2. are required to be comptime to enable compiler analyses. If any of these types is used as the type of a function parameter or the type returned by a function, all calls to such function must be `comptime`. If any of these types is used as the type of a variable, these variables must be `comptime var` or `const` with a `comptime` initializer. Pointers to these types are not allowed. Values of the following types must always be comptime-known: * `comptime_int` * `comptime_float` * `type` * `comptime_string` * function type * `imported_module` If these types are used to create a new type, like an array, the new type is subject to the same constraints. See [Type System in CSL](/csl/language/types) for more information on each of those types. ## Evaluation of Comptime-Known Control Flow If the predicate of an `if` statement is a comptime-known value, the `if` statement is replaced with the block corresponding to the branch taken (if any), no run-time branches are created, and the block corresponding to the branch not taken is not semantically checked, as illustrated below: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} param mytype : type = f16; fn foo() mytype { // Since the predicate of the `if` statement is comptime-known to be // `true`, the compiler will prune the `else` branch, making the code // semantically correct. if (mytype == f16) { return 1.0; } else { return 1; } } ``` In `comptime` loops, all expressions and variables are implicitly `comptime`. This includes the induction variables of `for` loops and the continue expression of `while` loops. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn foo() void { comptime var sum_values: u16 = 0; comptime var sum_indices: u16 = 0; const three: u16 = 3; comptime { for ([3]u16 {1, 2, three}) |value, idx| { sum_values += value; sum_indices += idx; } }; @comptime_assert(sum_values == 6); @comptime_assert(sum_indices == 3); } ``` ## Typical Uses of the `comptime` Keyword `comptime` variables and operations enable powerful operations such as non-trivial memory initialization or routing rules, without paying a performance penalty at run-time. For instance, the following code initializes a global array as an identity matrix at compile time. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} param size: u16; // global initializers are implicitly comptime const identity = createIdentityMatrix(); fn createIdentityMatrix() [size, size]f16 { var result = @zeros([size, size]f16); var i: u16 = 0; while (i < size) : (i += 1) { result[i,i] = 1.0; } return result; } ``` The above program contains no run-time calls to `createIdentityMatrix()`, since that function is called on the host during program compilation. # Data Structure Descriptors Source: https://sdk.cerebras.ai/csl/language/dsds Use Data Structure Descriptors (DSDs) to efficiently express repeated operations on memory vectors, fabric streams, FIFOs, and circular buffers in CSL. Data Structure Descriptors (DSDs) are a compact representation of a (possibly non-contiguous) chunk of memory or a sequence of incoming or outgoing wavelets. Combined with DSD operations, DSDs enable various repeated operations to be expressed using just one hardware instruction. All kinds of DSDs share one key property, the `extent` or `length`. This property represents the number of repeated operations or number of iterations that the DSD represents. ## Basic Syntax DSDs are defined in CSL using the following syntax: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @get_dsd(dsd_type, properties); ``` Where: * `dsd_type` is one of `mem1d_dsd`, `mem4d_dsd`, `circbuf_dsd`, `fabin_dsd`, or `fabout_dsd`. * `properties` is a struct that specifies auxiliary properties of the particular DSD type. Different DSD types require different properties. ## One-Dimensional Memory Vectors The `mem1d_dsd` type is used to encode a memory vector using a single induction variable. Memory vectors are configured using the following fields: * `base_address`, any expression of pointer type. * `extent`, any expression of type `u16`. * `stride`, any expression of type `i8`. (optional, defaults to 1). * `offset`, any expression of type `i16`. (optional, defaults to zero). * `tensor_access`, a comptime-known tensor access expression (see [tensor\_access](#tensor_access)). * `wavelet_index_offset`, a comptime-known boolean expression (optional, defaults to `false`). ### tensor\_access The `tensor_access` field is a convenient grouping of properties that fully specifies a memory access pattern through an expression that is referred to as a **tensor access expression**. A tensor access expression has the following syntax: ``` || {} -> [] ``` Where: * `induction-variable`, a single identifier that represents the loop induction variable (its iteration variable). * `length`, a comptime-known non-negative integer expression that represents the number of times to iterate. * `base-address`, the name of a variable of tensor type or the name of a comptime-known variable of pointer-to-tensor type. * ``, a comma-separated list of affine expressions of the loop iteration variable and comptime-known values. There must be exactly as many affine expressions as the number of dimensions of the tensor that is specified by `base-address`. For example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const array = @zeros([10]u16); const tenWords = @get_dsd(mem1d_dsd, .{ .tensor_access = |i|{10} -> array[i] }); ``` In the above snippet, `tenWords` is a `mem1d_dsd` specifying accesses to `array` at indices 0 through 9. A tensor access expression can be used to refer to odd elements of an array: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const array = @zeros([10]u16); const oddElements = @get_dsd(mem1d_dsd, .{ .tensor_access = |i|{5} -> array[2 * i + 1] }); ``` It can also be used to refer to a single element of the array: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const array = @zeros([10]u16); const firstElement = @get_dsd(mem1d_dsd, .{ .tensor_access = |i|{10} -> array[0] }); ``` Such a pattern is often useful in reduction operations. Although `mem1d_dsd` allows only one induction variable, the underlying array can still be a multidimensional array. For instance, the following `mem1d` DSD refers to the diagonal elements of a 2D (20x20) array. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const array = @zeros([20,20]u16); const diagonal = @get_dsd(mem1d_dsd, .{ .tensor_access = |i|{20} -> array[i,i] }); ``` The tensor access expression is syntactic sugar to create an anonymous struct with the following comptime-known fields: * `base_address`, a pointer representing the base address of the underlying access pattern. * `offset`, an expression of type `i16` that is the access pattern’s offset from `base_address`. * `stride`, a tuple containing a single expression of type ‘i8’. * `extent`, a tuple containing a single expression of type ‘u16’. This also means that the type of a tensor access expression is that of the corresponding anonymous struct. For example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var A: [10]i16; // 'access_of_A' is exactly equivalent to: // .{.base_address = &A, .offset = 42, .stride = .{2}, .extent = .{10}} const access_of_A = |i|{10} -> A[2*i + 42]; // A 'mem1d_dsd' created through the 'tensor_access' property. const dsd1 = @get_dsd(mem1d_dsd, .{.tensor_access = access_of_A}); // A 'mem1d_dsd' created through explicitly specifying properties // individually. const dsd2 = @get_dsd(mem1d_dsd, .{.base_address = access_of_A.base_address, .offset = access_of_A.offset, .stride = access_of_A.stride, .extent = access_of_A.extent}); ``` In the example above, both ways of creating a 1D memory DSD are equivalent, which means that `dsd1` and `dsd2` are exactly the same. As a result, it is also possible to use an anonymous struct directly as the value of the `tensor_access` field, as follows: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var A: [10]i16; // The '.offset' field defaults to zero. const access_of_A = .{.base_address = &A, .stride = 2, .extent = 10}; const dsd = @get_dsd(mem1d_dsd, .{.tensor_access = access_of_A}); ``` It is not allowed to specify any DSD properties twice, which means that the `base_address`, `stride`, `extent` and `offset` fields cannot be specified in both the `tensor_access` value and as top-level properties at the same time. ### Runtime `mem1d_dsd` Tensor Access Properties By specifying memory access properties of 1D memory DSDs individually, we are able to use runtime values for them. This is not possible through the tensor access expression since they must be comptime-known. For example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var ptr: [*]i16; task foo(stride: i16, len: u16) void { // Definition of a 'mem1d_dsd' with runtime properties. var dsd = @get_dsd(mem1d_dsd, .{.base_address = ptr, .stride = @as(i8, stride), .extent = len}); } ``` ### wavelet\_index\_offset The `wavelet_index_offset` field expects a comptime-known boolean value that indicates whether the `wavelet_index_offset` mode is enabled. If the `wavelet_index_offset` mode is enabled, the address of the underlying memory buffer is incremented by the `index` specified in the DSD operation as explained in [Explicit Index Offset](#explicit-index-offset). If a DSD with `wavelet_index_offset` enabled is used in a DSD operation, the DSD operation must provide an index field. Otherwise, the behavior of the respective DSD operation is undefined. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const array = @zeros([size]u16); const memDSD = @get_dsd(mem1d_dsd, .{ .tensor_access = |i|{10} -> array[i], .wavelet_index_offset = true }); task my_task() void { // The addition will start at an offset specified // by 'my_index'. @add16(memDSD, memDSD, data, .{.index = my_index}); // The behavior of this builtin is undefined. @add16(memDSD, memDSD, data); } ``` ## Two-, Three-, or Four-Dimensional Memory Vectors The `mem4d_dsd` is a DSD type that is used to refer to multi-dimensional memory vectors, up to a maximum of four dimensions. Multi-dimensional memory vectors are configured using the following fields: * `base_address`, a comptime-known pointer to a tensor. * `offset`, any expression of type `i16` (optional, defaults to zero). * `stride`, a comptime-known tuple (i.e., anonymous struct with nameless fields) of expressions of type `i16`. (optional, defaults to a tuple with values of 1 that has the same size as the `extent` tuple). * `extent`, a comptime-known tuple (i.e., anonymous struct with nameless fields) of expressions of type `u16`. * `tensor_access`, a tensor access expression (see [tensor\_access](#tensor_access)). * `wavelet_index_offset`, a comptime-known boolean expression (optional, defaults to `false`). ### tensor\_access Like in 1D memory DSDs, the `tensor_access` field is a convenient grouping of properties through a **tensor access expression**. The only difference is that in multi-dimensional memory DSDs we can have up to 4 comma-separated induction variables and length expressions and the number of induction variables and length expressions must match. For example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const array = @zeros([4,3]u16); const subset = @get_dsd(mem4d_dsd, .{ .tensor_access = |i,j|{2,2} -> array[i, j] }); ``` The `subset` DSD will access four elements of `array` in the following order: `[0, 0]`, `[0, 1]`, `[1, 0]`, `[1, 1]`. The following, more complicated, example shows a DSD that uses all four dimensions with non-zero offsets. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const array = @zeros([1,2,3,4]u16); const subset = @get_dsd(mem4d_dsd, .{ .tensor_access = |i,j,k,l|{1,2,1,4} -> array[i, j, 1+k, l] }); ``` Here, the `subset` DSD will access 8 elements of `array` in the following order: ``` [0,0,1,0] [0,0,1,1] [0,0,1,2] [0,0,1,3] [0,1,1,0] [0,1,1,1] [0,1,1,2] [0,1,1,3] ``` `mem4d` DSDs can be used with single-dimensional vectors as well, like in the following, somewhat contrived, example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const subset = @get_dsd(mem4d_dsd, .{ .tensor_access = |i,j,k,l|{1,1,1,1} -> array[i + j + k + l] }); ``` In the above example, the `subset` will only access element `0` of `array`. Like 1D memory DSDs the tensor access expression is lowered into an anonymous struct with the same fields (see [tensor\_access](#tensor_access)). For example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var A: [20]i16; const dsd1 = @get_dsd(mem4d_dsd, .{ .tensor_access = |i,j|{5,5} -> A[2*i + j] }); // Equivalent to dsd1 const dsd2 = @get_dsd(mem4d_dsd, .{ .base_address = &A, .extent = .{5, 5}, .stride = .{1, -2} }); ``` The stride determines by how much the address changes when we increment the access. This must take into account the values that are reset when the access is incremented. To understand the mapping between tensor access expression and the strides, consider the behavior of the above example. Considering each dimension in turn: * Each time the inner dimension `j` is incremented, the accessed index of `A` increases by `1`, so stride 0 corresponding to the inner dimension is `1`. * Each time the outer dimension `i` is incremented, the inner dimension `j` resets from its upper limit `4` to `0`. The access at `i = 0, j = 4` is at index `2*0 + 4 = 4`, and the access at `i = 1, j = 0` is at index `2*1 + 0 = 2`. Thus, the accessed index of `A` changes by `2 - 4 = -2`, so stride 1 corresponding to the outer dimension is `-2`. For a more complex example, consider: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var A: [4, 5]i16; // 'access_of_A' is exactly equivalent to: // .{.base_address = &A, .offset = 2, // .stride = .{1,-3,-3,-23}, .extent = .{5,5,5,5}} const access_of_A = |i, j, k, l|{5, 5, 5, 5} -> A[i + j, k + l + 2]; // A 'mem4d_dsd' created through the 'tensor_access' property. const dsd1 = @get_dsd(mem4d_dsd, .{.tensor_access = access_of_A}); // A 'mem4d_dsd' created through explicitly specifying properties // individually. const dsd2 = @get_dsd(mem4d_dsd, .{.base_address = access_of_A.base_address, .offset = access_of_A.offset, .stride = access_of_A.stride, .extent = access_of_A.extent}); ``` The array `A` is laid out row-major in memory. Thus, we can rewrite the expression `A[i + j, k + l + 2]` as if it were a 1D array as `A[5 * (i + j) + k + l + 2]`. Considering each stride in turn: * To calculate the innermost stride 0, i.e. for `l`, each time `l` is incremented, the accessed index of `A` increases by `1`. * For stride 1, i.e. for `k`, each time `k` is incremented, the inner dimension `l` is reset from `4` to `0`, so the accessed index of `A` changes by `1 - 4 = -3`. * For stride 2, i.e. for `j`, incrementing `j` increases the accessed index of `A` by `5`, but both `k` and `l` are reset from `4` to `0`, so the accessed index of `A` changes by `5 - 4 - 4 = -3`. * For stride 3, i.e. for `i`, incrementing `i` increases the accessed index of `A` by `5`, but `j`, `k`, and `l` are reset from `4` to `0`, so the accessed index of `A` changes by `5 - 5*4 - 4 - 4 = -23`. The stride values are read left to right, but the access values are read right to left. For example, for a stride of `{1, -2}` and extent of `{2, 4}`, the innermost (fastest changing) dimension will have a stride of `1` and extent of `4`, while the outermost dimension will have a stride of `-2` and extent of `2`. ### wavelet\_index\_offset See [wavelet\_index\_offset](#wavelet_index_offset). ## Pointers To Scalars As Destinations Some DSD operations support pointers to scalars as destination arguments. These operations essentially behave as if the destination were a memory DSD with zero stride, whose destination is a one-element array whose data is stored at the pointer. For example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const src_array = [8]f16{ 0, 1, 2, 3, 4, 5, 6, 7 }; // src_dsd will access src_array at indices, 0, 2, 4, and 6. const src_dsd = @get_dsd(mem1d_dsd, .{ .tensor_access = |i|{4} -> src_array[2*i] }); const dst_array = @zeros([1]f16); // Because dst_dsd has zero stride (`i` is never mentioned to the right of // the arrow in the tensor access expression), the @fmovh below will first // move 0, then 2, then 4, then 6 into dst_array[0]. Thus afterwards, // dst_array[0] will have a value of 6. const dst_dsd = @get_dsd(mem1d_dsd, .{ .tensor_access = |i|{4} -> dst_array[0] }); @fmovh(dst_array, src_dsd); @assert(dst_array[0] == 6); // An @fmovh operation with a pointer to scalar as its destination behaves // similarly, so the value of dst_scalar after the following @fmovh will // also be 6. const dst_scalar: f16 = 0; @fmovh(&dst_scalar, src_dsd); @assert(dst_scalar == 6); ``` ## Circular Buffers The `circbuf_dsd` type is used to implement a typical circular buffer, i.e., a contiguous one-dimensional memory buffer that will wrap around to its base address (start) once computation reaches its wraparound position (end). Circular buffer DSDs are configured using the following fields: * `base_address`, a comptime-known pointer. * `extent`, a comptime-known non-negative integer expression. * `wraparound`, a comptime-known expression of type `u16`. The `base_address` field specifies the beginning (start) of the circular buffer as well as its current position (head). Once a DSD operation reaches the end of the circular buffer then the current position (head) will reset back to the start, i.e., `base_address`. The `extent` field specifies the number of iterations encapsulated by the DSD representation. When a `circbuf_dsd` is used as an operand to a DSD operation, the `extent` field determines how many elements are processed by that operation. The `wraparound` field represents the number of elements from the `base_address` to the exclusive end of the circular buffer, or in other words, the address at which the wraparound will occur. If `base_address` is a comptime-known pointer to a tensor then `wraparound` is optional and defaults to the size of the underlying tensor. If a `wraparound` is explicitly provided then it must be no larger than the size, in number of elements, of the underlying tensor. For example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var buffer: [10]f16; // In this example, 'wraparound' is automatically set to 'size'. // In this example, 'circbuf' will iterate over all the buffer // elements twice given that the 'extent' is twice the size of // the underlying buffer. const circbuf1 = @get_dsd(circbuf_dsd, .{ .base_address = &buffer, .extent = 20 }); // In this example, the wraparound will take place after 5 iterations, // which in this case, occur after `buffer[4]` is accessed. const circbuf2 = @get_dsd(circbuf_dsd, .{ .base_address = &buffer, .extent = 20, .wraparound = 5 }); // ERROR: wraparound exceeds the size of the underlying buffer. const circbuf3 = @get_dsd(circbuf_dsd, .{ .base_address = &buffer, .extent = 20, .wraparound = 15 }); // Wraparound is required since the base address is not a pointer // to a tensor but a pointer to a scalar. const circbuf4 = @get_dsd(circbuf_dsd, .{ .base_address = &buffer[offset], .extent = 20, .wraparound = 5 }); ``` Circular buffer DSDs cannot be used as operands to DSD operations directly. They must be loaded using the `@load_to_dsr_xdsr` builtin at comptime or runtime. See [@load\_to\_dsr\_xdsr](/csl/language/dsrs#@load_to_dsr_xdsr). ## Fabric Input Vectors The `fabin_dsd` DSD type is used to refer to wavelets arriving at the PE from the fabric. Fabric input vectors are configured using the following fields. * `input_queue`, which specifies the input queue supplying wavelets to associate with this vector * `extent`, which specifies the number of wavelets that this vector refers to On WSE-2, `fabric_color` can be used instead of `input_queue` to specify the color of the wavelets to associate with the vector. For instance, the following DSD refers to 5 wavelets expected to arrive on color `trigger`. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const inDsd = @get_dsd(fabin_dsd, .{ .extent = 5, .fabric_color = trigger }); ``` On WSE-3, it is illegal for a DSD operation to have multiple arguments that are fabric inputs. In addition, on WSE-3, the following optional field is available: * `priority`, an optional field that sets the priority of the microthread associated with the DSD. Possible values are `.{ .high = true }`, `.{ .medium = true }`, `.{ .low = true }`. ## Fabric Output Vectors Fabric output vectors, specified using the `fabout_dsd` type, are configured similarly to fabric input (`fabin_dsd`) vectors, with the exception that fabric output vectors may contain the following additional fields: * `control` * `wavelet_index_offset` ### control The `control` field expects a comptime-known boolean expression, which is used to signify control wavelets. For instance, the following DSD refers to 1024 non-control wavelets to be sent along the color `tx`. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const outDsd = @get_dsd(fabout_dsd, .{ .extent = 1024, .fabric_color = tx }); ``` Whereas the following DSD refers to a single control wavelet to be sent along the color `out`. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const dsd = @get_dsd(fabout_dsd, .{ .extent = 1, .control = true, .fabric_color = out, }); ``` ### wavelet\_index\_offset The `wavelet_index_offset` field expects a comptime-known boolean expression, which is used to enable the `wavelet_index_offset` mode. When this mode is enabled, the outgoing wavelets will carry a fixed index field specified by the user per-operation as explained in [Explicit Index Offset](#explicit-index-offset). Similar to the semantics of memory DSDs, if the operations that use fabric output DSDs with `wavelet_index_offset` enabled do not specify an `index` value, then the behavior is undefined. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const outDSD = @get_dsd(fabout_dsd, .{ .extent = 1, .fabric_color = out, .wavelet_index_offset = true, }); task my_task() void { // The outgoing wavelets will have 'my_index' stored in // their high 16-bits. @add16(outDSD, memDSD, 42, .{.index = my_index}); // The behavior of this builtin is undefined. @add16(outDSD, memDSD, 42); } ``` ## FIFOs A FIFO DSD is a kind of DSD that uses a memory region to create a First-In First-Out buffer. To create a FIFO DSD, the `@allocate_fifo` builtin is used: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var fifo_buffer = @zeros([32]i16); const fifo = @allocate_fifo(fifo_buffer); ``` The `@allocate_fifo` builtin must be associated with a `const` variable in the global scope. The argument to `@allocate_fifo` must be a global array or pointer to a global array. This array must be marked as `var` and its element type must be an ABI-compatible numeric type. If the fifo buffer (i.e., the argument to `@allocate_fifo`) was declared without an explicit alignment requirement (by using the `align`) directive (see [Variables](/csl/language/syntax#variables)) then the compiler will force its alignment to be the minimum alignment that is required for fifos on the target architecture. On the other hand, if the fifo buffer has been declared with an explicit alignment requirement that is less than the minimum alignment required for fifo buffers on the target architecture, an error will be raised. Note that if the fifo buffer is declared as `extern` (see [Variables](/csl/language/syntax#variables)) without an explicit `align` directive then a warning will be emitted indicating that proper alignment must be specified for the respective buffer definition. This warning can be suppressed by specifying an explicit alignment requirement to the `extern` fifo buffer declaration. Allocating a FIFO consumes hardware resources for the duration of the program, as such they should be used sparingly. The following restrictions apply when using a FIFO DSD in a DSD operation: * The FIFO DSD operand must be comptime-known. * A FIFO cannot be used as an operand to the `@map` builtin. * If a DSD operation uses more than one source operand: * at most one operand may be a FIFO DSD, and * the FIFO DSD operand must not be the first (left-most) source operand. The `@allocate_fifo` builtin takes an optional configuration struct which can optionally contain the fields described below. ### Full and Empty Actions When a DSD operation that reads from an empty FIFO terminates, the length of the FIFO will be updated to the remaining length after the FIFO became empty. If the destination operand is a pointer to a scalar, any data popped from the FIFO during the operation will be discarded, and the value stored at the pointer will remain unchanged. When a DSD operation that writes to a full FIFO terminates, the length of the FIFO will be updated to the remaining length after the FIFO became full. In addition, DSD operations that read from an empty FIFO or write to a full FIFO will execute actions specified by the `.empty_action` and `.full_action` configuration struct fields, respectively. Possible actions are: * `test_or_suspend`: If the DSD operation is synchronous, terminate the operation and return `false`. If the DSD operation is asynchronous, suspend the operation until the FIFO is no longer full or empty. * `terminate`: Terminate the operation and return `true`. * `suspend`: Suspend the operation until the FIFO is no longer full or empty. Not supported on WSE-2. * `fault`: Halt execution with an unrecoverable fault. Not supported on WSE-2. If `.empty_action` and/or `.full_action` are not specified, the corresponding action defaults to `test_or_suspend`. `.full_action` is not supported on WSE-2. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var fifo_buffer = @zeros([32]i16); const fifo = @allocate_fifo( fifo_buffer, .{ .empty_action = .{ .terminate = true }, .full_action = .{ .fault = true } } // Requires WSE-3 ); ``` FIFOs are typically used with a pair of DSD operations: one operation writing elements to the FIFO and one operation reading elements from the FIFO. For example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @mov16(fifo, ...); // Write to the FIFO @mov16(..., fifo); // Read from the FIFO ``` ### Task Activation on Pop and Push The `.activate_pop` configuration struct field specifies a `local_task_id` or comptime-known task name to be activated on pop from the FIFO. The `.activate_push` configuration struct field specifies a `local_task_id` or comptime-known task name to be activated on push to the FIFO. The associated task must be bound as a local task. Note that the specified `.activate_pop` task is only activated on pop if the FIFO has previously hit a FIFO full event, and if the pop causes the FIFO to transition from having insufficient free space to having sufficient free space, where “sufficient free space” means sufficient space for the push operation that originally triggered the FIFO full event to proceed. The amount of space required depends on the operand size and SIMD width of the push operation that previously triggered the FIFO full event. Similar rules apply in the other direction: the `.activate_push` task is only activated on push if the FIFO has previously hit a FIFO empty event, and if the push causes the FIFO to transition from having insufficient data to having sufficient data, where “sufficient data” means sufficient data in the queue for the pop operation that originally triggered the FIFO empty event to proceed. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} task on_push() void { ... } task on_pop() void { ... } var fifo_buffer = @zeros([32]i16); const fifo = @allocate_fifo( fifo_buffer, .{ .activate_pop = on_pop, .activate_push = on_push } ); ``` ### Change FIFO Properties The following builtins can be used to change the properties of a FIFO at runtime. Changing FIFO properties at comptime will be enabled in the future through the FIFO initialization builtin (i.e., `@allocate_fifo`). As was mentioned earlier, FIFOs acquire hardware resources for the duration of the program and therefore updating the properties of FIFOs happens in-place by directly accessing these hardware resources without creating new DSD values as is the case for the rest of the DSD kinds. #### @set\_fifo\_read\_length and @set\_fifo\_write\_length Update the length field of a FIFO that is associated with a read or write operation respectively. ##### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @set_fifo_read_length(fifo, length); @set_fifo_write_length(fifo, length); ``` Where: * `fifo` is a comptime-known FIFO DSD expression. * `length` is a 16-bit unsigned integer expression that specifies the length to be applied in number of FIFO elements. ##### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var fifo_buffer = @zeros([32]i16); // FIFOs are always initialized with read/write length zero. const fifo = @allocate_fifo(fifo_buffer); const fifo_length = 42; // Sets the FIFO write length before a write operation. @set_fifo_write_length(fifo, fifo_length); @mov16(fifo, ...); // Sets the FIFO read length before a read operation. @set_fifo_read_length(fifo, fifo_length); @move(..., fifo); ``` ##### Semantics The builtin will update the read or write length of the input FIFO in-place by modifying the underlying hardware resource directly. ## Change DSD Properties The following builtins can be used to change DSD properties at runtime or comptime. All of these builtins will always result in a new DSD value while the input value remains unchanged. ### @set\_dsd\_base\_addr Create a new memory DSD value based on the input memory DSD value and base address. #### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @set_dsd_base_addr(input_dsd, base_addr); ``` Where: * `input_dsd` is a memory DSD expression, i.e., a DSD expression with a type that is either `mem1d_dsd` or `mem4d_dsd`. * `base_addr` is a tensor identifier or a pointer expression whose base-type is a tensor. #### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var A = @zeros([10]i16); // Create a new DSD that is a clone of 'input_dsd' but has // 'A' as its base-address. var dsd1 = @set_dsd_base_addr(input_dsd, A); // Use a pointer expression as the new base-address parameter. var dsd2 = @set_dsd_base_addr(input_dsd, &A); ``` #### Semantics The builtin returns a new memory DSD value that is a clone of the input DSD value but with the provided `base_addr` parameter as the new base address. The new base address will replace both the base address and offset (if any) of the input DSD value. ### @increment\_dsd\_offset Create a new memory DSD value based on the input memory DSD value, offset and tensor element type. #### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @increment_dsd_offset(input_dsd, offset, elem_type); ``` Where: * `input_dsd` is a memory DSD expression, i.e., a DSD expression with a type that is either `mem1d_dsd` or `mem4d_dsd`. * `offset` is a 16-bit signed integer that specifies the offset to be applied as number of elements of `elem_type`. * `elem_type` is a type expression that is used to convert `offset` into number of words. It must be an ABI-compatible numeric type (`u16`, `i16`, `u32`, `i32`, `f16`, or `f32`). #### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const A = @zeros([10, 10]f32); // dsdA is defined as a 2x2 square that is at a diagonal // offset within A. const dsdA = @get_dsd(mem4d_dsd, .{ .tensor_access = |i, j|{2, 2} -> A[i + 1, j + 1]}); // Create a new DSD that is a clone of the 'dsdA' but its // base address is moved backwards by 10 f32 elements. In // practice, new_dsd will have moved the 2x2 square upwards // by one row. var new_dsd = @increment_dsd_offset(dsdA, -10, f32); ``` #### Semantics The builtin returns a new memory DSD value that is a clone of the input DSD value but with a new base address that is the result of adding the `offset` parameter to the base address of the input DSD. The `offset` parameter specifies the number of tensor elements to be added to the input DSD’s base address. The builtin performs no runtime or comptime checks for out-of-bounds accesses so the user needs to be aware of such risk. ### @set\_dsd\_length Create a new DSD value based on the input DSD and length. #### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @set_dsd_length(input_dsd, length); ``` Where: * `input_dsd` is a DSD expression with any DSD type except `mem4d_dsd`. * `length` is a 16-bit unsigned integer that specifies the length to be applied in number of tensor elements or wavelets. #### Semantics The builtin returns a new DSD value that is a clone of the input DSD value but with the new length applied. ### @set\_dsd\_stride Create a new 1D memory DSD value based on the input 1D memory DSD and stride. #### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @set_dsd_stride(input_dsd, stride); ``` Where: * `input_dsd` is a DSD expression that must be of type `mem1d_dsd`. * `stride` is an 8-bit signed integer that specifies the stride to be applied in number of tensor elements. #### Semantics The builtin returns a new DSD value that is a clone of the input DSD value but with the new stride applied. ## Asynchronous DSD Operations DSD operations **involving fabric operands** are allowed to happen asynchronously. This causes a new thread to start executing concurrently with any ongoing tasks and other asynchronous operations. A thread that starts executing as part of an asynchronous DSD operation is referred to as a **microthread** (see [Microthreads](#microthreads)). A DSD operation will happen asynchronously if either of these conditions are true: 1. At least one DSD operand has a fabric DSD type, that is, `fabin_dsd` or `fabout_dsd`, and the `async` configuration is used. 2. At least one DSR operand was loaded using the `async` configuration (see [@load\_to\_dsr](/csl/language/dsrs#@load_to_dsr)). For example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // The @mov16 operation will be asynchronous. @mov16(destination_dsd, source_dsd, .{.async = true}); // The @mov16 will also be asynchronous even though ``async`` is not // specified by the operation itself. const source_dsr = @get_dsr(dsr_src0, 0); @load_to_dsr(source_dsr, my_fabin_dsd, .{.async = true}); // Specifying async here is not strictly necessary, // but recommended to be explicit about the behavior of the operation. @mov16(destination_dsd, source_dsr, .{.async = true}); ``` For an operation on a DSR to occur asynchronously, the DSR *must* be marked as asynchronous when a DSD is loaded to it with `@load_to_dsr`. In this case, it is not necessary to specify `async` in the DSD operation. However, it is recommended to do so for clarity and explicitness. All of the following configuration settings are directly applicable to DSRs when using the `@load_to_dsr` builtin (see [@load\_to\_dsr](/csl/language/dsrs#@load_to_dsr)). ### Completion of Asynchronous DSD Operations When an asynchronous DSD operation completes, it may optionally activate or unblock a task. The task to be activated or unblocked is specified in the last argument of the DSD operation. For example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @mov16(destination_dsd, source_dsd, .{.async = true, .activate = mytask}); @mov16(destination_dsd, source_dsd, .{.async = true, .unblock = mytask}); ``` At most one of `activate` or `unblock` may be specified. The `activate` field can be a `local_task_id` or a comptime-known task name. The associated task must be bound as a local task. The `unblock` field can be a: * WSE-2: `color`, `data_task_id`, `local_task_id`, or comptime-known task name. * WSE-3: `input_queue`, `data_task_id`, `local_task_id`, or comptime-known task name. As with the `.async` field, if using a DSR in an asynchronous operation, the `.activate` and `.unblock` fields *must* be specified in the `@load_to_dsr` call that loads a fabric DSD to the DSR. For example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // The @mov16 will also be asynchronous, and my_task_id will // be activated upon completion. const source_dsr = @get_dsr(dsr_src0, 0); @load_to_dsr(source_dsr, my_fabin_dsd, .{.async = true, .activate = my_task_id}); // Specifying async and activate here is not strictly necessary, // but recommended to be explicit about the behavior of the operation. @mov16(destination_dsd, source_dsr, .{.async = true, .activate = my_task_id}); ``` In this case, as with `async`, it is not necessary to specify `activate` or `unblock` in the DSD operation. However, it is recommended to do so for clarity and explicitness. #### Dynamic Completion Based on Control Wavelets The completion of an asynchronous DSD operation can also be triggered by control wavelets. This capability must be explicitly enabled through the last argument of the DSD operation by specifying the `on_control` field. For example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // The asynchronous DSD operation will terminate. @mov16(destination_dsd, source_dsd, .{.async = true, .on_control = .{.terminate = true}}); // The asynchronous DSD operation will terminate and task 'mytask' will be // activated @mov16(destination_dsd, source_dsd, .{.async = true, .on_control = .{.activate = mytask}}); // The asynchronous DSD operation will terminate and task 'mytask' will be // unblocked @mov16(destination_dsd, source_dsd, .{.async = true, .on_control = .{.unblock = mytask}}); ``` The `terminate` action requires a boolean expression. The `activate` action requires a `local_task_id` or task name. For `activate`, the associated task must be bound as a local task. The `unblock` action requires a: * WSE-2: `color`, `data_task_id`, `local_task_id`, or task name. * WSE-3: `input_queue`, `data_task_id`, `local_task_id`, or task name. For `unblock`, the associated task must be bound as a data or local task. ### Hardware Resources and Asynchronous DSD Operations Asynchronous operations consume two kinds of hardware resources: queues and microthreads. It is the programmer’s responsibility to ensure that concurrent asynchronous DSD operations do not share the same resource (queue or microthread). #### Fabric Queues Fabric operands involved in asynchronous DSD operations must be associated with a queue. Input/Output queues are hardware buffers where data is temporarily stored before entering or leaving the compute engine (CE) of a PE. To specify a queue for fabric input DSDs (`fabin_dsd`), the `input_queue` attribute must be used, with a value of type `input_queue` as the queue identifier: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const my_fabin_dsd = @get_dsd(fabin_dsd, .{..., .input_queue = @get_input_queue(0), ...}); ``` To specify a queue for fabric output DSDs (`fabout_dsd`), the `output_queue` attribute must be used, with a value of type `output_queue` as the queue identifier: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const my_fabout_dsd = @get_dsd(fabout_dsd, .{..., .output_queue = @get_output_queue(0), ...}); ``` The hardware has a finite number of input and output queues, each with different buffering capabilities. | Queue Identifiers | WSE-2 Input Queue Length (words) | WSE-2 Output Queue Length (words) | WSE-3 Input Queue Length (words) | WSE-3 Output Queue Length (words) | | ----------------- | -------------------------------- | --------------------------------- | -------------------------------- | --------------------------------- | | 0, 1 | 6 | 2 | 8 | 8 | | 2, 3 | 4 | 6 | 4 | 8 | | 4, 5 | 2 | 2 | 4 | 8 | | 6, 7 | 2 | N/A | 4 | 8 | It is the programmer’s responsibility to ensure that no two concurrent DSD operations share an output or input queue: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} task t() void { @mov16(fabric_out_dsd, memory_dsd1, .{ .async = true }); @mov16(fabric_out_dsd, memory_dsd2, .{ .async = true }); // Bad: same // output queue } ``` In the example, two concurrent asynchronous operations are spawned using the same `fabout_dsd` as the destination operand. Therefore, they also use the same `output_queue`, which is invalid. It is the programmer’s responsibility to ensure that there are no elements in a queue before reusing it for a different operation. #### Microthreads Asynchronous DSD operations require a hardware microthread, which is a finite resource. A hardware microthread is identified by an integer identifier called a **microthread ID**. On WSE-2 the microthread ID is implicitly determined by one of the input or output queues involved in the operation: 1. If a `fabout_dsd` operand is used, the microthread identifier is the same as the `output_queue` identifier. 2. Otherwise, the microthread identifier is the same as the `input_queue` identifier of the first `fabin_dsd` operand. It is the programmer’s responsibility to ensure that no two concurrent DSD operations share a microthread. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const fabric_out_dsd = @get_dsd(fabout_dsd, .{.extent = 10, .output_queue = @get_output_queue(0)}); const fabric_in_dsd = @get_dsd(fabin_dsd, .{.extent = 10, .input_queue = @get_input_queue(0)}); const mem1_dsd = @get_dsd(mem1d_dsd, ...); const mem2_dsd = @get_dsd(mem1d_dsd, ...); task t() void { @mov16(fabric_out_dsd, mem1_dsd, .{ .async = true }); // Microthread ID 0 @mov16(mem2_dsd, fabric_in_dsd, .{ .async = true }); // Bad: same // Microthread ID! } ``` On WSE-3 the user has the option to explicitly specify the microthread ID of a given asynchronous DSD operation. This means that it can be different from the operands’ respective queue IDs (see [Microthread IDs](/csl/language/microthreads_wse3)). #### Microthread Priority The Cerebras hardware supports a priority setting for asynchronous operations. This is also called *microthread priority*. On WSE-2, an asynchronous DSD operation with a fabric input DSD as its destination may have priority specified as follows: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @mov16(destination_dsd, source_dsd, .{ .async = true, .priority = .{ .high = true } }); ``` Valid choices for priority are `high`, `medium`, and `low`. On WSE-2, the default when `.priority` is omitted is `low`. On WSE-3, the priority needs to be specified in the DSDs: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const source_dsd = @get_dsd(fabin_dsd, .{ .extent = 10, .input_queue = in_queue, .priority = .{ .high = true } }); @mov16(destination_dsd, source_dsd, .{ .async = true }); ``` or in the FIFOs ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var fifo_buffer = @zeros([32]i16); const fifo = @allocate_fifo(fifo_buffer, .{ .priority = .{ .low = true } }); ``` When `.priority` is omitted on WSE-3, the microthread priority defaults to `high`. In general, the hardware will favor the scheduling of higher-priority operations when multiple microthreads are running. Priority of the main thread, i.e., of non-async operations, may also be adjusted. By default, the main thread's synchronous operations sit between `medium` and `low` microthread priority — this level corresponds to the `MEDIUM_LOW` value of the [main\_thread\_priority](/csl/language/libraries#main_thread_priority) `level` enum. The main thread can be raised to `MEDIUM`, `MEDIUM_HIGH`, or `HIGH` at comptime or at runtime by calling `update_main_thread_priority` on the imported `.main_thread_priority` submodule. ## Explicit Index Offset A DSD operation may have an `index` configuration field, which is expected to be an unsigned 16-bit integer value. If this setting is combined with the `wavelet_index_offset` property of memory and/or fabout DSDs, it will have the following semantics: * **Memory DSDs**: the `index` value represents a word offset that is added to the base address of the underlying memory buffer. * **Fabric Output DSDs**: the `index` value represents the index that is set to all outgoing wavelets, i.e., all outgoing wavelets will have `index` set in their high 16-bits. For example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const array = @zeros([size]u16); const memDSD = @get_dsd(mem1d_dsd, .{ .tensor_access = |i|{10} -> array[i], .wavelet_index_offset = true }); const outDSD = @get_dsd(fabout_dsd, .{ .extent = 1, .fabric_color = out, .wavelet_index_offset = true, }); task my_task() void { // The addition will start at an offset specified // by 'my_index'. @add16(memDSD, memDSD, 42, .{.index = my_index}); // The outgoing wavelets will have 'my_index' stored in // their high 16-bits. @add16(outDSD, memDSD, 42, .{.index = my_index}); } ``` The `index` configuration will be ignored by DSDs that do not have the `wavelet_index_offset` property enabled. ## Advanced DSD Features ### SIMD Mode When using 16-bit values with fabric DSDs, it is possible to send or receive more than one value in a single wavelet using the so-called SIMD mode. The following code block shows how to use SIMD-32 mode with a fabric output DSD. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const out_dsd = @get_dsd(fabout_dsd, .{ .extent = 10, .fabric_color = out_color, .simd_mode = .{ .simd_32 = true }, }); ``` In `simd_32` mode, a single wavelet carries two 16-bit values. In `simd_64` mode, two wavelets must be ready, otherwise the DSD operation stalls. In `simd_32_or_64` mode, the operation proceeds (i.e. it doesn’t stall) as long as at least one wavelet is ready. On WSE-2, but not WSE-3, `simd_mode` may also be set on FIFOs: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const my_fifo = @allocate_fifo( some_buffer, if (@is_arch("wse2")) .{ .simd_mode = .{ .simd_64 = true } } else .{} ); ``` ### Reset a Source Operand When the destination operand is a fabric output DSD, once the DSD operation is complete, the architecture can clear the memory vector represented by the source operand of the DSD operation. For instance, the following block of code sets the fabric output DSD properties so the memory represented by the operation’s first source operand is reset to zero when the operation completes. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const out_dsd = @get_dsd(fabout_dsd, .{ .extent = 10, .fabric_color = out_color, .zero = .{ .first_source = true }, }); const in_first_dsd = @get_dsd(mem1d_dsd, .{ .tensor_access = |i|{10} -> first_source[i] }); const in_second_dsd = @get_dsd(mem1d_dsd, .{ .tensor_access = |i|{10} -> second_source[i] }); // Multiply vectors and send to fabric. Reset `first_source` when complete. @fmulh(out_dsd, in_first_dsd, out_first_dsd); ``` To reset the second source operand’s memory, use `.zero = .{ .second_source = true }`. When the DSD operation has just one source operand, use `.second_source = true`. At any time, only one of `first_source` or `second_source` can be used. ### Advance Switch Positions Fabric output DSDs can automatically advance switch positions when the last wavelet is sent. To use this feature, set the `advance_switch` field of the fabric output DSD to be true, like in the example below, which will cause the switch position for the color `out_color` to advance after all ten wavelets have been sent to the fabric. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const out_dsd = @get_dsd(fabout_dsd, .{ .extent = 10, .advance_switch = true, .fabric_color = out_color, }); ``` ### Control Wavelet Transform Control Wavelet Transform handles relaying control wavelets. Consider a scenario where there is a “buffering” PE which receives wavelets from the fabric and pushes them into a FIFO, using a microthread. There is also another microthread that pops data from the FIFO and sends them into the fabric. What if there is a requirement to relay control wavelets as well? By default, the approach described above cannot work since the task receives only the “index” and “data” bits of the wavelets, and the bit signifying that a wavelet is a control wavelet is outside of those bits. That means that if a control wavelet is pushed into the FIFO, the control bit is lost, so when it’s the time to pop it, it will be sent away as a regular wavelet, instead of a control wavelet. To get around this limitation, the `control_transform` field can be used. By specifying `control_transform` to be true for the fabric input DSD, when a control wavelet is received, the two most significant bits of the index portion of the wavelet are overwritten to signify that the wavelet stored in the FIFO is a control wavelet. Then, a fabric output DSD with `control_transform` set to true can be used to reconstruct control wavelets and send them to the fabric. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var in_dsd = @get_dsd(fabin_dsd, .{ .fabric_color = recv_channel, .extent = 100, .input_queue = @get_input_queue(0), .control_transform = true }); const out_dsd = @get_dsd(fabout_dsd, .{ .extent = 100, .fabric_color = send_channel, .output_queue = @get_output_queue(1), .control_transform = true }); var buf = @zeros([5]u32); const fifo = @allocate_fifo(buf); task buffer() void { @mov32(fifo, in_dsd, .{ .async = true }); @mov32(out_dsd, fifo, .{ .async = true }); } ``` Note since the two most significant bits of the index are overwritten, when `control_transform` is used, only the least significant 14 bits of the index can be utilized by the user. This property can only be used with fabric DSDs. # Data Structure Registers Source: https://sdk.cerebras.ai/csl/language/dsrs Use Data Structure Registers (DSRs) to load DSD values into physical registers and control asynchronous DSD operations in CSL. Data Structure Registers (DSRs) are physical registers that are used to store DSD values. Each DSR belongs to one of three DSR files, namely the `dest`, `src0` and `src1` DSR files. All DSD operations will actually operate on DSRs behind the scenes and therefore, all DSD operands to DSD operations must be loaded to DSRs before executing the respective DSD operation. ## Extended DSRs and Stride Registers Certain kinds of DSD values require additional registers called *Extended DSRs* (XDSRs) to be loaded as well. Specifically, FIFOs, circular buffers, and multi-dimensional vectors all require an XDSR. In addition, multi-dimensional vectors may also require an additional set of registers called *Stride Registers* (SRs) that are used to store strides of the underlying multi-dimensional access. ## DSR, XDSR and SR Allocation The allocation of DSRs, XDSRs and SRs, and the loading of DSDs to them, is typically done automatically by the compiler. However, it is possible to create and use DSRs, XDSRs and SRs directly. This chapter describes how users can allocate DSRs, XDSRs and SRs and then load DSDs to them without the compiler’s assistance. ## DSR Types There are 5 types of DSRs supported in CSL, each corresponding to one of the three DSR files. These are the following: * `dsr_dest` represents a DSR value that can only be used to store a destination operand to a DSD operation. * `dsr_src0` represents a DSR value that can be used to store a source as well as a destination operand to a DSD operation. * `dsr_src1` represents a DSR value that can be only be used to store a source operand to a DSD operation. * `dsr_fifo_dest` represents a `dsr_dest` DSR that is expected to store a FIFO (See [FIFO DSR types](#fifo-dsr-types)). * `dsr_fifo_src1` represents a `dsr_src1` DSR that is expected to store a FIFO. XDSR values are represented by the `xdsr` type while SR values are represented by the `sr` type. ### FIFO DSR Types The `dsr_fifo_dest` and `dsr_fifo_src1` types can be used instead of `dsr_dest` and `dsr_src1`, respectively, to represent DSRs that are known to store a FIFO if one does not have access to a FIFO object. Like FIFO objects, non-asynchronous DSD operations on FIFO DSRs will terminate and return `false` when reading from an empty FIFO or writing to a full FIFO. Otherwise, FIFO DSR-typed values have the same semantics as the corresponding non-FIFO DSR types. Behavior is undefined if a FIFO DSR-typed value is not initialized as part of a FIFO when it is used in a DSD operation. If a non-asynchronous DSD operation has a DSR operand that does **not** have FIFO DSR type, but that DSR holds a FIFO, behavior is undefined if that FIFO experiences a FIFO full or FIFO empty event. It is the programmer’s responsibility to avoid such FIFO full or FIFO empty events. ## DSR Builtins ### @get\_dsr Create a DSR identifier value. This value will identify a physical DSR along with the corresponding DSR file. #### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @get_dsr(dsr_type, dsr_id); @get_dsr(fifo_dsr_type, non_fifo_dsr); ``` Where: * `dsr_type` is an expression of type `type` and whose value must be one of the DSR types. * `dsr_id` is a comptime-known expression of integer type. * `fifo_dsr_type` is an expression of type `type` whose value is one of the FIFO DSR types (`dsr_fifo_dest` or `dsr_fifo_src1`). * `non_fifo_dsr` is a comptime-known expression of a non-FIFO DSR type. * Returns a value of `dsr_type`. #### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const dsr1 = @get_dsr(dsr_dest, 0); const dsr2 = @get_dsr(dsr_src0, 1); const dsr3 = @get_dsr(dsr_src1, 6); const dsr4 = @get_dsr(dsr_fifo_dest, 4); const dsr5 = @get_dsr(dsr_fifo_src1, dsr3); ``` #### Semantics Creates a DSR identifier value of `dsr_type` type using the specified integer identifier. This builtin must be evaluated at comptime. The provided integer identifier must be non-negative and smaller than the number of available DSRs for the given DSR file. Otherwise, an error will be emitted. The type of `non_fifo_dsr` must correspond to `fifo_dsr_type`. If `fifo_dsr_type` is `dsr_fifo_dest`, then `non_fifo_dsr` must have type `dsr_dest`, and if `fifo_dsr_type` is `dsr_fifo_src1`, then `non_fifo_dsr` must have type `dsr_src1`. ### @get\_xdsr Create an XDSR identifier value. This value will identify a physical XDSR. #### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @get_xdsr(xdsr_id); ``` Where: * `xdsr_id` is a comptime-known expression of integer type. * Returns a value of type `xdsr`. #### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const my_xdsr = @get_xdsr(4); ``` #### Semantics Creates an XDSR identifier value using the specified integer identifier. This builtin must be evaluated at comptime. The provided integer identifier must be non-negative and smaller than the number of available XDSRs. Otherwise, an error will be emitted. ### @get\_sr Create a *Stride Register* (SR) identifier value. This value will identify a physical SR. #### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @get_sr(sr_id); ``` Where: * `sr_id` is a comptime-known expression of integer type. * Returns a value of type `sr`. #### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const my_sr = @get_sr(4); ``` #### Semantics Creates an SR identifier value using the specified integer identifier. This builtin must be evaluated at comptime. The provided integer identifier must be non-negative and smaller than the number of available SRs. Otherwise, an error will be emitted. ### @load\_to\_dsr Load a DSD value into a DSR. #### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @load_to_dsr(dsr_value, dsd_value); @load_to_dsr(dsr_value, dsd_value, config_struct); ``` Where: * `dsr_value` a comptime-known expression of a DSR type. * `dsd_value` an expression of DSD type. * `config_struct` optional anonymous struct consisting of either of the following: * Asynchronous configuration setting fields as explained in [Asynchronous DSD Operations](/csl/language/dsds#asynchronous-dsd-operations). These are allowed only for fabric DSDs. The supported settings are: * `async` * `activate` * `unblock` * `on_control` * The `save_address` setting field. This is allowed only for `mem1d` and `mem4d` DSDs. See [save\_address](#save_address) for more details. * The `single_step` setting field. #### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const dsr1 = @get_dsr(dsr_dest, 0); const dsr2 = @get_dsr(dsr_src0, 1); fn foo(mem_dsd: mem1d_dsd, fab_dsd: fabin_dsd) void { // Loads a memory DSD to a DSR. @load_to_dsr(dsr1, mem_dsd); // Loads a fabric DSD to a DSR while specifying that the // input DSD is asynchronous with activation and on_control settings. @load_to_dsr(dsr2, fab_dsd, .{.async = true, .activate = callback, .on_control = .{.terminate = true}}); } const A = @zeros([10]f16); const mem_dsd = @get_dsd(mem1d_dsd, .{.tensor_access = |i|{10} -> A[i]}); const fab_dsd = @get_dsd(fabin_dsd, .{.extent = 10, .fabric_color = @get_color(1), .input_queue = @get_input_queue(1)}); comptime { // The DSD will be loaded to the DSR at comptime, i.e., before the // program begins its execution. @load_to_dsr(dsr1, mem_dsd); // A fabric DSD with asynchronous properties will be loaded at comptime. @load_to_dsr(dsr2, fab_dsd, .{.async = true, .activate = callback, .on_control = .{.terminate = true}}); // fab_dsd uses color 1 with input queue 1. When using explicit DSRs, // we must explicitly initialize the queue. @initialize_queue(@get_input_queue(1), .{ .color = @get_color(1) }); } ``` #### Semantics The `@load_to_dsr` builtin can be called at comptime or runtime. If it is called at runtime it will load the input DSD to the specified DSR at runtime. If it is called at comptime, the specified DSD will be loaded to the DSR before the program begins executing. A DSD of type `fabin_dsd` cannot be loaded to a `dsr_dest` DSR. A DSD of type `fabout_dsd` cannot be loaded to a `dsr_src0` or `dsr_src1` DSRs. A DSD of type `mem4d_dsd` cannot be loaded using `load_to_dsr`. It can only be loaded using `load_to_dsr_xdsr_sr`. FIFO DSRs are not permitted in `@load_to_dsr`. When using a `fabin_dsd` loaded to a DSR, the input queue used by the `fabin_dsd` must be explicitly initialized with the associated color via `@initialize_queue`. On WSE-3, when using a `fabout_dsd` loaded to a DSR, the output queue used by the `fabout_dsd` must be explicitly initialized with the associated color via `@initialize_queue`. ### @load\_to\_dsr\_xdsr Load a circular buffer DSD value into a DSR and XDSR. #### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @load_to_dsr_xdsr(dsr_value, xdsr_value, circbuf_dsd); @load_to_dsr_xdsr(dsr_value, xdsr_value, circbuf_dsd, config_struct); ``` Where: * `dsr_value` is a comptime-known expression of a DSR type. * `xdsr_value` is a comptime-known expression of XDSR type. * `circbuf_dsd` is an expression of `circbuf_dsd` type. * `config_struct` is an optional anonymous struct consisting of either of the following: * The `save_address` setting field. See [save\_address](#save_address) for more details. * The `single_step` setting field. #### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const dsr = @get_dsr(dsr_dest, 0); const xdsr = @get_xdsr(1); var circbuf: circbuf_dsd; task foo() void { // Loads a circular buffer DSD to a DSR and XDSR pair at runtime. @load_to_dsr_xdsr(dsr, xdsr, circbuf); // Runtime DSR/XDSR loading with additional configuration properties. @load_to_dsr_xdsr(dsr, xdsr, circbuf, .{.save_address = true}); @load_to_dsr_xdsr(dsr, xdsr, circbuf, .{.single_step = true}); } comptime { // Same DSR/XDSR loading calls but this time the loading takes // place at comptime. @load_to_dsr_xdsr(dsr, xdsr, circbuf); @load_to_dsr_xdsr(dsr, xdsr, circbuf, .{.save_address = true}); @load_to_dsr_xdsr(dsr, xdsr, circbuf, .{.single_step = true}); } ``` #### Semantics The `@load_to_dsr_xdsr` builtin can be called at runtime or during the evaluation of a top-level comptime block. The input DSD must be of type `circbuf_dsd` and it will be loaded to a pair of DSR and XDSR values. ### @load\_to\_dsr\_xdsr\_sr Load a 4D memory DSD value into a DSR, an XDSR and zero or more stride registers (SRs). #### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @load_to_dsr_xdsr_sr(dsr_value, xdsr_value, sr_tuple, dsd_value); @load_to_dsr_xdsr_sr(dsr_value, xdsr_value, sr_tuple, dsd_value, config_struct); ``` Where: * `dsr_value` is a comptime-known expression of a DSR type. * `xdsr_value` is a comptime-known expression of XDSR type. * `sr_tuple` is a comptime-known tuple expression with elements of SR type. * `config_struct` is an optional anonymous struct consisting of either of the following: * The `save_address` setting field. See [save\_address](#save_address) for more details. * The `single_step` setting field. See [single\_step](#single_step) for more details. #### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const dsr = @get_dsr(dsr_dest, 0); const xdsr = @get_xdsr(1); const sr1 = @get_sr(0); const sr2 = @get_sr(1); const sr3 = @get_sr(2); var dsd: mem4d_dsd; task foo() void { // Loads a 'mem4d_dsd' DSD to a DSR, an XDSR and three SRs at runtime. @load_to_dsr_xdsr_sr(dsr, xdsr, .{sr1, sr2, sr3}, dsd); // Runtime DSR/XDSR/SR loading with additional configuration properties. @load_to_dsr_xdsr_sr(dsr, xdsr, .{sr1, sr2, sr3}, dsd, .{.save_address = true}); @load_to_dsr_xdsr_sr(dsr, xdsr, .{sr1, sr2, sr3}, dsd, .{.single_step = true}); } const comptime_dsd = @get_dsd(mem4d_dsd, .{...}); comptime { // Load DSR/XDSR/SR at comptime. In this scenario, no stride registers // are needed, therefore the SR tuple remains empty. @load_to_dsr_xdsr_sr(dsr, xdsr, .{}, comptime_dsd); // Load DSR/XDSR/SR at comptime. In this scenario, only one stride // register is needed, therefore the SR tuple contains a single value. @load_to_dsr_xdsr_sr(dsr, xdsr, .{sr1}, comptime_dsd); // In this scenario, all three (maximum) stride registers are needed. // In addition, a configuration struct is also provided. @load_to_dsr_xdsr_sr(dsr, xdsr, .{sr1, sr2, sr3}, dsd, .{.single_step = true}); } ``` #### Semantics The `@load_to_dsr_xdsr_sr` builtin can be called at runtime or during the evaluation of a top-level comptime block. The input DSD value must be of type `mem4d_dsd` and it will be loaded to a pair of physical DSR and XDSR registers. In addition, some of the DSD's strides, if any, will also be loaded to the provided SRs. When the input DSD value is comptime-known then the number of SRs needed is determined by the access pattern. Specifically, a multi-dimensional vector can have up to four dimensions and therefore four strides, i.e., one for each dimension. However, the maximum number of SRs per multi-dimensional vector on all target architectures is currently three. This means that the first dimension (the fastest moving dimension) will never need an SR, only the other three, if they exist. In addition, if the stride of the first dimension (fastest moving dimension) is one, then the second dimension, if present, will also not need an SR. As a result, when the input DSD value is comptime-known, the user must provide the exact number of SRs needed or otherwise an error will be emitted. The error message will indicate the number of SRs that are needed. When the input DSD value is not comptime-known then `@load_to_dsr_xdsr_sr` will always need three SRs, which is the maximum amount. ### @set\_dsr\_base\_addr Update the base address of a previously loaded memory DSR to point at a new tensor without otherwise altering the DSR's loaded length, stride, or configuration. #### Syntax ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @set_dsr_base_addr(dsr, tensor); ``` Where: * `dsr` is a comptime-known expression of a DSR type. The DSR must have been initialized with a memory DSD (`mem1d_dsd` or `mem4d_dsd`) via [`@load_to_dsr`](#load_to_dsr). * `tensor` is either an expression of array type, in which case the new base address of the DSR is `&tensor[0]`, or a pointer to a tensor element, in which case the new base address is that pointer. Either form must refer to a tensor with a numeric element type. This builtin cannot be called from a top-level comptime block. #### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var A: [10]f16; var B: [10]f16; const dsr = @get_dsr(dsr_dest, 0); const dsd = @get_dsd(mem1d_dsd, .{ .base_address = &A, .extent = 10 }); comptime { @load_to_dsr(dsr, dsd); } task foo() void { // dsr now points at the start of A. @fmovh(dsr, ...); // Repoint dsr at the start of B without re-issuing @load_to_dsr. @set_dsr_base_addr(dsr, B); @fmovh(dsr, ...); // Repoint dsr at the third element of A. @set_dsr_base_addr(dsr, &A[2]); @fmovh(dsr, ...); } ``` This is the idiom shown implicitly in the [`save_address`](#save_address) example below — the comment block contrasts the manual base-address update against what `save_address` does automatically. ### save\_address The `save_address` option may be supplied to `@load_to_dsr` if the DSD is of the type `mem1d_dsd` or `mem4d_dsd`. This causes subsequent DSD operations on the DSR to update the DSR’s base address for the outermost (slowest-varying) dimension after termination to point one position past the end of the range covered by the DSD operation. The next operation on the DSR will effectively pick up where the previous one ended. #### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const CHUNK_LENGTH = 4; const N_CHUNKS = 3; var chunks_in = [CHUNK_LENGTH * N_CHUNKS]i16 { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 }; var chunks_out = @zeros( [CHUNK_LENGTH * N_CHUNKS]i16 ); const chunks_in_dsd = @get_dsd( mem1d_dsd, .{ .tensor_access = |i|{CHUNK_LENGTH} -> chunks_in[i] } ); const chunks_out_dsd = @get_dsd( mem1d_dsd, .{ .tensor_access = |i|{CHUNK_LENGTH} -> chunks_out[i] } ); comptime { @load_to_dsr(chunks_out_dsr, chunks_out_dsd, .{ .save_address = true }); @load_to_dsr(chunks_in_dsr, chunks_in_dsd, .{ .save_address = true }); } task main() void { // // Each call to @mov16 will copy a chunk of size CHUNK_LENGTH, as // specified in the .tensor_access expression. The base address for both // the source and target operations will be incremented by CHUNK_LENGTH // each time. // // Thus the following loop is semantically equivalent to: // // for (@range(i16, N_CHUNKS)) |i| { // @mov16(chunks_out_dsr, chunks_in_dsr); // @set_dsr_base_addr(chunks_out_dsr, // &chunks_out[CHUNK_LENGTH * (i+1)]); // @set_dsr_base_addr(chunks_in_dsr, // &chunks_in[CHUNK_LENGTH * (i+1)]); // } // for (@range(i16, N_CHUNKS)) |i| { @mov16(chunks_out_dsr, chunks_in_dsr); } } ``` ### single\_step The `single_step` option may be supplied to `@load_to_dsr` to support use with the `@map` builtin. When a DSR is used as an argument to `@map`, it should be loaded with a DSD value where `.single_step = true`, otherwise the behavior is unspecified. If a DSR loaded with a DSD value where `.single_step = true` is used as an argument to DSD builtins other than `@map`, the behavior is undefined. #### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const math_lib = @import_module(""); const memDSD = @get_dsd(mem1d_dsd, .{.tensor_access = |i|{size} -> A[i, i]); const faboutDSD = @get_dsd(fabout_dsd, .{.extent = size, .fabric_color = blue}); param inDSR: dsr_src1; param outDSR: dsr_dest; task foo() void { // Compute the square-root of each element of `memDSD` and // send it out to `faboutDSD`. @load_to_dsr(inDSR, memDSD, .{.single_step = true}); @load_to_dsr(outDSR, faboutDSD, .{.single_step = true}); @map(math_lib.sqrt_f16, inDSR, outDSR); } ``` ### @allocate\_fifo with DSRs By default, the DSRs and XDSR used by `@allocate_fifo` (see [FIFOs](/csl/language/dsds#fifos)) are allocated by the compiler. However, it supports the use of user-specified DSRs and XDSR as well, using the following syntax: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @allocate_fifo(fifo_buffer, config_struct); ``` Where, in order to allocate a FIFO with user-specified DSRs and XDSR, `config_struct` must contain the fields: * `dest`: a comptime-known expression of `dsr_dest` type. * `src`: a comptime-known expression of `dsr_src1` type. * `xdsr`: a comptime-known expression of `xdsr` type. The fields `dest`, `src`, and `xdsr` must all be specified together, or all absent, otherwise an error will be emitted. The integer identifiers of `dest` and `src` must match. If the provided DSR and XDSR identifiers have already been used for their respective types or exceed the valid range of values for the given target architecture, then an error will be emitted. Other fields of `config_struct` described in [Task Activation on Pop and Push](/csl/language/dsds#task-activation-on-pop-and-push) retain their same semantics when the DSRs and XDSR are specified. #### Example ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var buf = @zeros([240]u16); const my_fifo = @allocate_fifo(buf, .{ .dest = @get_dsr(dsr_dest, 4), .src = @get_dsr(dsr_src1, 4), .xdsr = @get_xdsr(1)}); ``` # Generics Source: https://sdk.cerebras.ai/csl/language/generics Learn how to write generic CSL code using comptime type parameters, anytype, and type computation to create reusable functions and structs. ## Generic Functions with `type` Parameters CSL’s type system does not have a notion of generic types like that of C++ or Java. Instead, generic programming is achieved through CSL’s comptime features. The basic idea is that `type` is a type, with values such as `i16`, `i32`, `bool`, and so on, and one can perform computation using `type`s at comptime. A very simple generic function looks something like: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn identity(comptime T: type, x: T) T { return x; } task t() void { var arg: i16 = 1; var one = identity(i16, arg); } ``` Since `type` is a comptime-only type just like `comptime_int`, `comptime_float`, or `comptime_string`, `T` must be marked `comptime` in order to appear as the type for `x` and as the function’s return type. CSL’s generics resemble C++ templates in some ways. Generic functions are monomorphized by the compiler. This means a generic function does not exist at runtime; instead, a copy of the function is compiled using each set of type arguments it is called with. The program will compile as long as each such copy is well-formed. An implication is that, for instance, a generic function that uses the unary `-` operator will not compile if it is called with a type like `comptime_string` that cannot be used with unary `-`: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn negate(comptime T: type, x: T) T { return -x; } task t() void { negate(f16, -3.14); // OK negate(i16, 42); // OK negate(comptime_string, "hello"); // error } ``` ### anytype While explicitly passing a parameter of type `type` to a generic function is a straightforward mechanism, it can be verbose. For the user of a library that provides an `abs` function, there is not much benefit to writing `abs(f32, 1.0 - x)` versus a non-generic equivalent of `abs_f32(1.0 - x)`. The `anytype` keyword provides a solution. `anytype` can only appear as the type of function parameters. It is another way to write a generic function and it has the same effect of creating a version of the function for each type that it is called with. When declaring parameters with `anytype`, the `@type_of` builtin is useful to relate the types of parameters and the return value to each other: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn ignore_x(x: anytype, y: @type_of(x)) @type_of(x) { return y; } task t() void { var arg: i16 = 1; var two = ignore_x(arg, arg + 1); } ``` Generic structs are also supported using the same notion of comptime computation with `type`s as generic functions: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn Point(comptime T: type) type { return struct { x: T, y: T, }; } const origin = Point(u16) { .x = 0, .y = 0 }; comptime { @comptime_print(origin); // {x = 0, y = 0} } ``` The generic `Point` above is a function that takes a `type` parameter and returns a struct parameterized by that type. ## Constrain Type Parameters If a generic function is called with an invalid type, an error occurs when the compiler discovers that the generic function’s body is trying to do something invalid with its argument. This is typically a lower-level error than the actual mistake of calling the function with an incorrect argument type: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn abs(x: anytype) @type_of(x) { if (x < 0.0) { return -x; } return x; } task t() void { abs("hello"); // error: invalid comparison operation for type: 'comptime_string' } ``` Here it is not so hard to piece together what went wrong, but if `abs` were a more complicated function, the mistake will be less obvious. Programmers who have used C++ templates may find this situation familiar. Instead, the function can test the provided type and fail a comptime assertion if it is invalid: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn abs(x: anytype) @type_of(x) { const T = @type_of(x); @comptime_assert(T == f16 or T == f32, "x is not a float"); if (x < 0.0) { return -x; } return x; } task t() void { abs("hello"); // error: comptime_assert failed: x is not a float } ``` ## Specialize Logic A related scenario is writing a generic function where a portion of the logic is only valid for some of the types over which one wants to define the function. Consider the example of `sign` from the `` library. This function returns `-1` if its argument is negative, `1` if it is positive, and `0` if it is zero. `sign` could naïvely be written like: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn sign(x : anytype) @type_of(x) { if (x < 0) { return -1; } else if (x > 0) { return 1; } return x; } comptime { var x: i16 = 12; @comptime_print(sign(x)); // 1 } ``` `math.sign` allows `x` to be an unsigned integer. While `sign()` is not quite as interesting as `sign` of a float or signed integer, it is perfectly valid to allow. However, the above code would not compile if passed a `u16` because `-1` is not a valid `u16`. To solve this problem, guard the `if (x < 0)` case with a check for the argument type: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn is_signed(comptime T: type) bool { return T == f16 or T == f32 or T == i8 or T == i16 or T == i32 or T == i64; } fn sign(x : anytype) @type_of(x) { if (comptime is_signed(@type_of(x))) { if (x < 0) { return -1; } } if (x > 0) { return 1; } return x; } comptime { var x: i16 = -12; var y: u16 = 25; @comptime_print(sign(x), sign(y)); // -1, 1 } ``` Evaluating the `if` condition at comptime ensures that the `if (x < 0)` case is only compiled at all if the type is correct. There is one final change that needs to be added to properly support floats: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // using same is_signed() as above fn sign(x : anytype) @type_of(x) { const T = @type_of(x); if (comptime is_signed(T)) { if (x < @as(T, 0)) { return @as(T, -1); } } if (x > @as(T, 0)) { return @as(T, 1); } return x; } comptime { var x: i16 = -12; var y: u16 = 25; var z: f16 = 0.0; @comptime_print(sign(x), sign(y), sign(z)); // -1, 1, 0 } ``` Since `1` and `0` are `comptime_int`s, they do not automatically convert to floats. ## Compute with Types As the previous use of `@type_of` alludes to, type specifiers can be any expression that has type `type`: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn Point(comptime T: type) type { return struct { x: T, y: T, }; } fn make_point(n: anytype) Point(@type_of(n)) { return Point(@type_of(n)) { .x = n, .y = n + 1, }; } comptime { @comptime_print(make_point(3)); // {x = 3, y = 4} } ``` A generic function can also abstract over properties of a type: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn size_of_int(comptime T: type) comptime_int { return if (T == i8) 1 else if (T == i16) 2 else if (T == i32) 4 else if (T == i64) 8 else @comptime_assert(false, "not an int"); } comptime { const word_type = i16; @comptime_print(size_of_int(word_type)); // 2 } ``` For a slightly more complex example, we can combine these two techniques to generically convert a float to its binary representation and extract the mantissa. The `@comptime_assert`s in helper functions also take care of validating that the type parameter is a float. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn bits_type(comptime T: type) type { return if (T == f16) u16 else if (T == f32) u32 else @comptime_assert(false, "not a float"); } fn mantissa_len(comptime T: type) comptime_int { return if (T == f16) 10 else if (T == f32) 23 else @comptime_assert(false, "not a float"); } fn mantissa_mask(comptime T: type) comptime_int { return comptime (1 << mantissa_len(T)) - 1; } fn get_mantissa(x: anytype) bits_type(@type_of(x)) { const T = @type_of(x); const bits = @bitcast(bits_type(T), x); return bits & mantissa_mask(T); } comptime { var x: f16 = 1.5; @comptime_print(get_mantissa(x)); // 512 (== 0x200) @comptime_print(get_mantissa(@as(f32, x))); // 4194304 (== 0x400000) } ``` The `` library internally uses this pattern to generically implement IEEE floating point functions like `isNaN`, `isInf`, and even `ceil` and `floor`. # Libraries Source: https://sdk.cerebras.ai/csl/language/libraries Explore the CSL standard library modules available for import, including math, complex numbers, data utilities, memory allocation, and collective operations. Libraries are imported by enclosing the name of the library with angled brackets. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // Filename: main.csl const math = @import_module(""); fn distance(x0 : f16, y0 : f16, x1 : f16, x1 : f16) f16 { return math.sqrt((x0-x1)*(x0-x1) + (x0-x1)*(x0-x1)); } ``` ## `` The `complex` library provides structs containing `real` and `imag` components and basic complex functions. `complex` is a generic struct parameterized by its field type. The `complex_32` and `complex_64` non-generic names are also provided; these define a complex number using two `f16` values and a complex number using two `f32` values, respectively. `get_complex` is a generic constructor that returns a complex struct based on the type of its inputs. The non-generic `get_complex_32` and `get_complex_64` constructor functions are provided as well: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // Returns struct {real: T, imag: T} where T can be f16 or f32 fn complex(comptime T: type) type const complex_32 = complex(f16); // struct {real: f16, imag: f16} const complex_64 = complex(f32); // struct {real: f32, imag: f32} // Can operate on f16 or f32 fn get_complex(r: anytype, i: @type_of(r)) complex(@type_of(r)) fn get_complex_32(r : f16, i : f16) complex_32 fn get_complex_64(r : f32, i : f32) complex_64 ``` The following functions are provided for operating on complex numbers. They are written as generic functions to facilitate use in other libraries or abstractions. In addition, non-generic `complex_32` and `complex_64` functions are provided. These functions have names suffixed with `_32` and `_64`, respectively. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // x, y can be complex_32 or complex_64 fn add_complex(x: anytype, y: @type_of(x)) @type_of(x) fn subtract_complex(x: anytype, y: @type_of(x)) @type_of(x) fn multiply_complex(x: anytype, y: @type_of(x)) @type_of(x) ``` ## `` The `control` library provides utilities for constructing control wavelets. The following functions and enums are provided by the library: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // Max commands that can be encoded in a control wavelet const MAX_CMDS = 8; // Struct for representing switching opcodes const opcode = enum(u32) { NOP = 0, SWITCH_ADV = 1, SWITCH_RST = 2, TEARDOWN = 3 }; // Encode payload that activates a control task with no argument fn encode_control_task_payload(entrypoint: control_task_id) u32; // Encode payload with one switch command, plus control task entrypoint. fn encode_single_payload(cmd: opcode, ce_ignore: bool, comptime entrypoint: control_task_id, data: u16) u32; // Encode general control wavelet payload fn encode_payload(comptime N: u16, comptime cmds: [N]opcode, comptime ce_ignore: [N]bool, ce_ignore_remaining: bool, comptime entrypoint: anytype) u32; ``` All functions construct a payload returned as a 32-bit unsigned integer which can be sent in a control wavelet. `encode_control_task_payload` returns a control wavelet payload which activates a control task on all receiving PEs. It has one argument: * `entrypoint`: a `control_task_id` which is bound to the control task activated on a CE by the receipt of this wavelet. `encode_single_payload` returns a control wavelet payload containing one switch command, along with an optional control task entrypoint with 16-bit data argument. The function has the following arguments: * `cmd`: a switching opcode to be consumed by the receiving PE router. This command will instruct the router to modify the configuration of the color on which the control wavelet is sent. This command can advance the switch position, reset the switch position, teardown the color, or do nothing. If the router of the PE on which the control wavelet is sent pops this command, then no additional receiving PEs will receive a switching opcode. * `ce_ignore`: a boolean which determines whether this control wavelet is to be ignored by the CE of PEs which receive it. If `true`, the control wavelet will not be forwarded to the CE. If `false`, and the receiving color is configured to transmit down the `RAMP`, the control wavelet will be forwarded to the CE. `ce_ignore` must be `false` for an `entrypoint` to be activated by a receiving PE. * `entrypoint`: a `control_task_id` will be activated on a CE by the receipt of this wavelet. Passing `{}` indicates that no control task activation on receiving PEs is desired. The control task will only be activated on a CE if `ce_ignore` is `false`, and the receiving color is configured to transmit down the `RAMP`. * `data`: The control task activated by `entrypoint` may take a single 16-bit argument. If the control task takes no argument, then this value will be ignored. `encode_payload` can encode a general control wavelet payload with up to eight switching commands. The function has the following arguments: * `N`: number of commands to encode in the control wavelet. Maximum number of commands is eight. * `cmd`: an array of switching opcodes to be consumed by PE routers. Each command will instruct the router to modify the configuration of the color on which the control wavelet is sent. Each command can advance the switch position, reset the switch position, teardown the color, or do nothing. If the router of the PE on which a command is executed pops the command, then the *next* command will be executed by the next receiving router. * `ce_ignore`: an array of booleans which determines whether this control wavelet is to be ignored by the CE of PEs which receive it. Each `ce_ignore` value is processed along with the associated `cmd`, i.e., the same rules for popping commands apply. If the processed value is `true`, the control wavelet will not be forwarded to the CE. If `false`, and the receiving color is configured to transmit down the `RAMP`, the control wavelet will be forwarded to the CE. `ce_ignore` must be `false` for an `entrypoint` to be activated by a receiving PE. * `ce_ignore_remaining`: a boolean which determines whether all other commands contained in this control wavelet are to be ignored by the CE of PEs receiving it. When `ce_ignore_remaining` is set to `false`, each unspecified command will travel down the `RAMP` and reach the CE (as a `NOP` command). * `entrypoint`: a `control_task_id` which is bound to the control task activated on a CE by the receipt of this wavelet. Passing `{}` indicates that no control task activation on receiving PEs is desired. The control task will only be activated on a CE if `ce_ignore` is `false`, and the receiving color is configured to transmit down the `RAMP`. Because this function can encode up to eight switching commands, no data payload can be provided for this control task. Unlike `encode_single_payload`, `encode_payload` does not take a `data` argument. If a control payload only contains a single switching command, then a 16-bit data argument can be supplied as an argument to the control task activated on receipt of the wavelet. `data` is not meaningful if there is more than one switching command in the control wavelet, because the bits that would encode `data` encode the additional switching commands instead. A control task that declares no arguments will ignore `data`, and furthermore, `data` is ignored if the wavelet is not forwarded to the CE (the current command’s `ce_ignore` value is `true`). ### Example The task `main_task` sends out a control wavelet along the color `comm`, which encodes a control task ID: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const ctrl = @import_module(""); const comm = @get_color(0); const comm_out_queue = @get_output_queue(2); const ctrl_entrypt_id = @get_control_task_id(40); task main_task() void { const comm_out_dsd = @get_dsd(fabout_dsd, .{ .extent = 1, .fabric_color = comm, .control = true, .output_queue = comm_out_queue, }); @mov32(comm_out_dsd, ctrl.encode_control_task_payload(ctrl_entrypt_id)); } ``` PEs which receive this wavelet along the color `comm` will activate a control task bound to this control task ID. For instance, if the receiving PE has the following code, then upon receipt of the control wavelet, it will activate a task which increments the value `my_global`: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const my_ctrl_id = @get_control_task_id(40); var my_global: u32 = 0; task my_ctrl_task() void { my_global += 1; } comptime { @bind_control_task(my_ctrl_task, my_ctrl_id); } ``` ## `` The `data_utils` library provides low-level data manipulation and bit extraction functions. The following functions return the lower or higher 16 bits of a 32-bit variable. The `lo16` function can also be called on a 16-bit data type. Similarly, variants for 64-bit data types are also available. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} inline fn lo16(src0: anytype) u16; inline fn hi16(src0: anytype) u16; inline fn lo32(src0: anytype) u32; inline fn hi32(src0: anytype) u32; ``` ## `` The `debug` library provides a tracing mechanism to record tagged values. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // Record values of the specified type fn trace_bool(x : bool) void fn trace_u8(x : u8) void fn trace_i8(x : i8) void fn trace_u16(x : u16) void fn trace_i16(x : i16) void fn trace_f16(x : @fp16()) void fn trace_u32(x : u32) void fn trace_i32(x : i32) void fn trace_f32(x : f32) void // Record a compile-time string fn trace_string(comptime str : comptime_string) void // Generic version fn trace(x : anytype) void // Record timestamp using the
Name Syntax Types Remarks Example
Addition a + b
a += b		 Integers, floats 2 + 5 == 7
Subtraction a - b
a -= b		 Integers, floats 2 - 5 == -3
Negation -a Integers, floats -1 == 0 - 1
Multiplication a \* b
a \*= b		 Integers, floats 2 \* 5 == 10
Division a / b
a /= b		 Integers, floats 10 / 5 == 2
Remainder of division a % b
a %= b		 Integers 10 % 3 == 1
Bit shift left a \<\< b
a \<\<= b		 Integers b must be unsigned. 0b1 \<\< 8 == 0b100000000
Bit shift right a >> b
a >>= b		 Integers Arithmetic shift right if a is signed, otherwise logical shift right. b must be unsigned. 0b1010 >> 1 == 0b101
Bitwise AND a & b
a &= b		 Integers 0b011 & 0b101 == 0b001
Bitwise OR a | b
a |= b		 Integers 0b010 | 0b100 == 0b110
Bitwise XOR a ^ b
a ^= b		 Integers 0b011 ^ 0b101 == 0b110
Bitwise NOT \~a Fixed-width integers \~@as(u8, 0b10101111) == 0b01010000
Logical AND a and b bool If a is false, returns false without evaluating b. Otherwise, returns b. (false and true) == false
Logical OR a or b bool If a is true, returns true without evaluating b. Otherwise, returns b. (false or true) == true
Logical NOT !a bool !false == true
Equality a == b Integers, floats, bool, enum, direction, comptime\_string, color, control\_task\_id, data\_task\_id, input\_queue, local\_task\_id, output\_queue, ut\_id, type (1 == 1) == true
Inequality a != b Integers, floats, bool, enum, direction, comptime\_string, color, control\_task\_id, data\_task\_id, input\_queue, local\_task\_id, output\_queue, ut\_id, type (1 != 1) == false
Greater than a > b Integers, floats (2 > 1) == true
Greater than or equal a >= b Integers, floats (2 >= 1) == true
Less than a \< b Integers, floats (1 \< 2) == true
Less than or equal a \<= b Integers, floats (1 \<= 2) == true
Except for logical AND and logical OR, the order in which expression operands are evaluated at runtime is undefined. For binary operations, both operands must have exactly the same type, unless one of them is a `comptime_int` (see [Comptime](/csl/language/comptime)). ## Comments `//` begins a single-line comment. Comments beginning with `///` or `//!` are doc comments, which are only allowed in certain positions. There are no multi-line comments in CSL. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // This function returns the value arg + 2 fn foo(arg : i16) i16 { var x : i16 = arg; // This and the next line are commented out: x will not be incremented by 1 // x += 1; x += 2; // Increment x by 2 return x; } ``` ### Doc Comments Doc comments come in two forms. A *regular* doc comment begins with exactly three `/` characters, so it must begin with `///` and cannot begin with `////`. A *top-level* doc comment begins with `//!`. Regular doc comments may occur immediately before top-level functions and variable declarations, and before members of a `struct`, `enum`, and `union` type definition. Top-level doc comments may occur at the very top of a source file, and at the very top of the body of a `struct`, `enum`, or `union` declaration, just after the opening curly brace. Doc comments are currently unused, but support for documentation generation is planned. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} //! This file contains some nice, well-documented functions and types. //! Please enjoy the example code. /// Given an integer `x` of type `i16`, return `x+x`. fn two_x(x: i16) i16 { return x+x; } /// Type for a pair of 16-bit floating point numbers. const two_floats = struct { //! You can put a top-level comment here if you wish. /// First element of the pair. x: @fp16(), /// Second element of the pair. y: @fp16() }; /// Type specifying a compass direction. const compass_direction = enum(u16) { //! Again, you can put a top-level comment here if you wish. /// Towards the north pole. NORTH, /// If "north" is considered to be "top" and we are looking down on the /// globe, this is the counter-clockwise direction from our point of view. EAST, /// Opposite direction from EAST. WEST, /// Opposite direction from NORTH. SOUTH, }; /// Union type containing either a 16-bit floating point number or a 16-bit /// signed integer. const int_or_float = union { //! Once again, top-level comments are allowed here. //! Top-level comments can span multiple lines. /// A 16-bit floating point number; the format of the number is determined /// by a compiler flag. float_val: @fp16(), /// A signed 16-bit integer. int_val: i16 }; ``` # Task Identifiers and Task Execution Source: https://sdk.cerebras.ai/csl/language/task-ids Learn how to bind, activate, and execute data, local, and control tasks in CSL using task identifiers and the CSL task model. The term “task identifier” or “task ID” is used to refer to a numerical value that can be associated with a task. A task ID is associated with a task through a process called “binding” that is carried out by special builtins depending on the type of task. There exist three types of tasks, each with an associated task ID handle type. These three types of tasks are: * Data Tasks * Local Tasks * Control Tasks The following three sections explain the semantics of each task type, how they can be bound to task IDs, and how they can be scheduled for execution. ## Data Tasks Data tasks are wavelet-triggered tasks (WTTs) that are associated with a `data_task_id`. On WSE-2, a `data_task_id` is constructed from a routable identifier known as a `color` (see [Routable Identifiers (Colors)](#routable-identifiers-colors)) using `@get_data_task_id` (see [@get\_data\_task\_id](/csl/language/builtins#@get_data_task_id)), and can be bound to a data task using `@bind_data_task` (see [@bind\_data\_task](/csl/language/builtins#@bind_data_task)). On WSE-3, a `data_task_id` is constructed from a hardware queue known as an `input_queue`, also using `@get_data_task_id`. We demonstrate constructing a data task ID and binding it to a task in the code block below: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const iq = @get_input_queue(2); // Input queue must be in range 0 to 7 const rc = @get_color(12); // Routable color must be in range 0 to 23 // On WSE-2, construct task ID from routable color ID // On WSE-3, construct task ID from input queue ID const task_id = if (@is_arch("wse3")) @get_data_task_id(iq) else @get_data_task_id(rc); // Bind data task to ID // On WSE-2, task ID will be 12; on WSE-3, task ID will be 2 @bind_data_task(my_task, task_id); // On WSE-3, associate input queue with a color if (@is_arch("wse3")) @initialize_queue(iq, .{ .color = rc }); ``` ### Routable Identifiers (Colors) On WSE-2 and WSE-3, the IDs 0 to 23 (inclusive) are recognized by the hardware as virtual communication channels for passing wavelets between PEs. These IDs are sometimes referred to as the routable “colors” and can be constructed using `@get_color` (see [@get\_color](/csl/language/builtins#@get_color)). Note that these IDs are the only ones that can be used with the `@set_color_config` and `@set_local_color_config` builtins (see [@set\_color\_config, @set\_local\_color\_config](/csl/language/builtins#@set_color_config-@set_local_color_config)). ### Execution Semantics A data task can be scheduled for execution if and only if it is bound to a data task ID that is activated and unblocked. A data task is activated by receiving a wavelet along a given color. On WSE-2, that color (or routable identifier) is the color that was used to create the `data_task_id` bound to the respective data task. On WSE-3, that color is the color to which the associated `input_queue` is bound. A data task must have at least one input argument representing the wavelet’s payload. If it has more than one input argument then the wavelet’s payload is equally split among those input arguments. All data task IDs that are bound to a data task are initially unblocked. They can be explicitly blocked using `@block` (see [@block](/csl/language/builtins#@block)) during the evaluation of a top-level comptime block or at runtime. Once blocked, a data task ID can be unblocked explicitly using `@unblock` (see [@unblock](/csl/language/builtins#@unblock)) or upon completion of asynchronous fabric DSD operations (see [Completion of Asynchronous DSD Operations](/csl/language/dsds#completion-of-asynchronous-dsd-operations)). ## Local Tasks Local tasks, or self-activated tasks, are associated with a `local_task_id`, which is constructed from an activatable identifier (see [Activatable Identifiers](#activatable-identifiers)) using `@get_local_task_id` (see [@get\_local\_task\_id](/csl/language/builtins#@get_local_task_id)) and then bound to a local task using `@bind_local_task` (see [@bind\_local\_task](/csl/language/builtins#@bind_local_task)). ### Activatable Identifiers The IDs 0 to 30 (inclusive) on WSE-2, or 8 to 30 (inclusive) on WSE-3, can be used to construct `local_task_id` values that can then be used to activate local tasks, which is why they are referred to as “activatable” identifiers. IDs 29 and 30 should generally be avoided in programs as they are used for system tasks. ID 29 is bound to a teardown task that runs when the tile is in teardown mode and ID 30 is bound to a timer task. ### Execution Semantics A local task can be scheduled for execution if and only if it is bound to a local task ID that is activated and unblocked. Local tasks accept no input arguments and can be activated explicitly using the `@activate` builtin (see [@activate](/csl/language/builtins#@activate)) or upon completion of an asynchronous fabric (see [Completion of Asynchronous DSD Operations](/csl/language/dsds#completion-of-asynchronous-dsd-operations)) or FIFO (see [Task Activation on Pop and Push](/csl/language/dsds#task-activation-on-pop-and-push)) DSD operation. All local task IDs that are bound to a local task are initially unblocked. They can be explicitly blocked using `@block` (see [@block](/csl/language/builtins#@block)) during the evaluation of a top-level comptime block or at runtime. Once blocked, a local task ID can be unblocked explicitly using `@unblock` (see [@unblock](/csl/language/builtins#@unblock)) or upon completion of asynchronous fabric DSD operations (see [Completion of Asynchronous DSD Operations](/csl/language/dsds#completion-of-asynchronous-dsd-operations)). ## Control Tasks Control tasks are control wavelet-triggered tasks that are associated with a `control_task_id`, which is constructed from a control identifier (see [Control Identifiers](#control-identifiers)) using `@get_control_task_id` (see [@get\_control\_task\_id](/csl/language/builtins#@get_control_task_id)) and then bound to a control task using `@bind_control_task` (see [@bind\_control\_task](/csl/language/builtins#@bind_control_task)). ### Control Identifiers On WSE-2 and WSE-3, the IDs 0 to 63 (inclusive) can be used to create `control_task_id` values. ### Execution Semantics A control task bound to a `control_task_id` value `CID` is scheduled for execution if and only if the following two conditions are met: * A control wavelet carrying the `CID` value in its payload is received. * The communication channel that is used to receive the aforementioned control wavelet is unblocked. The second condition needs to be explicitly satisfied using `@unblock` (see [@unblock](/csl/language/builtins#@unblock)) during the evaluation of a top-level comptime block or at runtime. The input to `@unblock` should be the routable identifier (value of type `color`) associated with the control wavelet’s input communication channel. Note that if the same communication channel is bound to a data task as well, then this unblocking is not necessary. The payload of a control wavelet consists of the `control_task_id` value and a data section. Both can be passed as input arguments to the control task. # Type System in CSL Source: https://sdk.cerebras.ai/csl/language/types Explore CSL's complete type system, including numeric types, structs, unions, enums, arrays, pointers, and comptime-only types. ## `void` type Expressions of type `void` have a single possible value. It describes constructs that do not produce a result. For example, blocks which do not break values have type `void` and `void` is the return type of functions and builtins that do not return anything. Using a block as an expression, the void value can be expressed with `{}`: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const void_val = {}; // type is void. const also_void_val : void = foo(); fn foo() void {} ``` `void` is allowed at runtime as a function return type. All other uses of `void` values must be comptime-known; see [Comptime](/csl/language/comptime) for more information on comptime. ## Numeric Types These types describe numbers: * signed integers (`iN` for arbitrary bit width `N`) * unsigned integers (`uN` for arbitrary bit width `N`) * arbitrary precision integers (`comptime_int`) * floating point numbers (`f16`, `f32`, `bf16`, `cb16`, `comptime_float`) ### Arbitrary-Width Integer Types CSL supports integer types with any bit width from 0 to 16777215. These are specified using `uN` for unsigned types and `iN` for signed types, where `N` is the bit width: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var small: u3 = 7; // 3-bit unsigned (range: 0 to 7) var tiny: i4 = -8; // 4-bit signed (range: -8 to 7) var flags: u1 = 1; // 1-bit unsigned (0 or 1) var byte: u8 = 255; // 8-bit unsigned var word: i16 = -100; // 16-bit signed var dword: u32 = 1000; // 32-bit unsigned ``` Integer types support arithmetic, comparisons, and bitwise operations: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var a: u5 = 10; var b: u5 = 5; var sum: u5 = a + b; // OK: result is 15, fits in u5 var product: u5 = 3 * 4; // OK: result is 12, fits in u5 ``` Arithmetic operations on integer literals produce `comptime_int` results at compile time, which are then coerced to the target type. An error occurs if the result cannot be represented in the target type: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var overflow: u5 = 31 + 1; // Error: 32 exceeds u5 max (31) var underflow: i4 = -8 - 1; // Error: -9 exceeds i4 min (-8) ``` Only integer types with bit widths of 16 or 32 are ABI-sized. Non-ABI-sized integer types cannot be used in `export` or `extern` declarations or as task parameters, and certain hardware-specific operations, such as DSD builtins, may require ABI-sized types. Non-ABI-sized types are primarily intended for compact data structures such as packed structs and unions. ### The `comptime_int` Type Values of `comptime_int` type can hold arbitrarily large (or small) integers. Integer literals have type `comptime_int`: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const ten = 10; // type is comptime_int. const also_ten : comptime_int = 10; ``` Character literals also have type `comptime_int`: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const some_char = 'a'; // type is comptime_int, value is 97 // (ASCII value of 'a') const some_other_char = '\x0a'; // type is comptime_int, value is 10 // (i.e., 0x0a, in hexadecimal) const yet_another_char = '\n'; // type is comptime_int, value is 10 // (ASCII value of newline character) ``` Arithmetic between `comptime_int` values happen at compile time, produce another `comptime_int` value, and never underflow or overflow: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const thousand = 1000; const trillion = thousand * thousand * thousand * thousand; const one = trillion / trillion; ``` Operations between a value of type `comptime_int` and a value of fixed-precision integer type cause the `comptime_int` value to be converted to the fixed-precision type. An error is emitted upon overflow or underflow: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const thousand = 1000; // comptime_int. const ten : i16 = 10; // 10 is converted from comptime_int to i16. const hundred : i16 = thousand / 10; // thousand is converted to i16. const overflow : i16 = 10000000000000; // error. ``` The builtin `@as` can be used to force literals to have a specific width, for example `@as(u16, 1) | 0xbeef`. All values of type `comptime_int` must be comptime-known, see [Comptime](/csl/language/comptime) for more information on comptime. ### The `comptime_float` Type Values of `comptime_float` type can hold any IEEE double precision floating point number. Float literals have type `comptime_float`: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const ten = 10.0; // type is comptime_float. const also_ten : comptime_float = 10.0; ``` Arithmetic between `comptime_float` values happen at compile time and produce another `comptime_float` value. If an operation performs division by zero or generates a `NaN` value, an error is emitted. Operations between a value of type `comptime_float` and a value of different float type cause the `comptime_float` value to be converted to other float type. An error is emitted if this is not possible. All values of type `comptime_float` must be comptime-known, see [Comptime](/csl/language/comptime) for more information on comptime. ### FP16 Types `f16`, `cb16`, and `bf16` are 16-bit floating point (“FP16”) types of different formats: | Type | Description | Exponent | Mantissa | | ------ | ------------------- | ----------------------- | -------- | | `f16` | IEEE half-precision | 5 bits | 10 bits | | `cb16` | Cerebras float16 | 6 bits, customized bias | 9 bits | | `bf16` | Brain float16 | 8 bits | 7 bits | ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const ieee_six: f16 = 6.0; const cb_six: cb16 = 6.0; const bf_six: bf16 = 6.0; comptime { @comptime_assert(@bitcast(u16, ieee_six) == 0b0100011000000000); @comptime_assert(@bitcast(u16, cb_six) == 0b0101100100000000); @comptime_assert(@bitcast(u16, bf_six) == 0b0100000011000000); } ``` CSL supports use of *a single FP16 type* within the runtime code of a program. This type is chosen by the value of the `--fp16-format` command line option, with a default of `f16`. All values of the other FP16 types must be comptime-known: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // With --fp16-format=f16 or --fp16-format absent, OK // Otherwise, error: variable of type 'f16' must be comptime-known var ieee_one: f16 = 1.0; // With --fp16-format=cb16, OK // Otherwise, error: variable of type 'cb16' must be comptime-known var cb_one: cb16 = 1.0; // With --fp16-format=bf16, OK // Otherwise, error: variable of type 'bf16' must be comptime-known var bf_one: bf16 = 1.0; ``` The `@fp16()` builtin (see [@fp16](/csl/language/builtins#@fp16)) facilitates programming with respect to the selected FP16 format. ## The `type` Type In CSL, the type `type` can be used to describe values that are themselves types: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const my_type = i16; const same_type : type = i16; ``` These values can be used anywhere a type is expected. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const my_type = i16; const array = @zeros([10]my_type); fn foo() my_type { ... } ``` All values of type `type` must be comptime-known, see [Comptime](/csl/language/comptime) for more information on comptime. ## Function Types Values of function type contain the name of a function and may be used anywhere a function is expected. The type is written as `fn() `. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn foo(arg1 : i16, arg2 : f16) void {...} const also_foo1 = foo; const also_foo2 : fn(i16, f16) void = foo; also_foo1(10, 10.0); also_foo2(20, 20.0); ``` Copying a function value does not create a new function, it copies the name of the function. All values of function type must be comptime-known, see [Comptime](/csl/language/comptime) for more information on comptime. ## Struct Types There are two kinds of struct types in CSL: anonymous structs and named structs. ### The Anonymous Struct Types Anonymous struct types are defined by an optional *list* of field names and a *list* of types. Two anonymous struct types are the same if they have the same list of field names (or both lack field names) and the same list of types. A value of an anonymous struct type with named fields is created with the syntax: `.{.field1 = value1, .field2 = value2, ...}`. A value of an anonymous struct type with unnamed fields is created with the syntax: `.{value1, value2, ...}`. Anonymous struct types with nameless fields are also known as *tuple* types. The elements of tuples may be accessed with the `[]` operator, as long as the index is known at compile time. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // Type: {a : comptime_int, b : comptime_float} var struct1 = .{.a = 10, .b = 1.0}; var struct2 = .{.a = 20, .b = 2.0}; var struct1 = struct2; // ok, same type! // Type: {a : comptime_float, b : comptime_float} var struct3 = .{.a = 10.0, .b = 1.0}; struct3 = struct1; // error: different types! var some_float = struct3[1]; // error: struct3 is not a tuple type! // Type: {comptime_float, comptime_float} var struct4 = .{10.0, 1.0}; var some_float = struct4[0]; // ok! var some_other_float = struct4[2]; // error: index 2 is out of bounds! task t(i: u16) void { var yet_another_float = struct4[i]; // error: i is not known // at comptime! } ``` Currently, it is not possible to spell out anonymous struct types using CSL syntax. ### The Named Struct Types Named struct types are similar to anonymous struct types, except that two named struct types defined at different places in the source code are considered to be different types, even if their field names and types are the same. A named struct type is expressed with the form `struct { field1: type1, field2: type2, ... }`. Once a named struct type has been defined, a value of that type can be created by giving the name of the type, followed by a field initializer list of the form `{ .field1 = value1, .field2 = value2, ... }`. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const complex = struct { real_part: f16, imag_part: f16 }; const one = complex { .real_part = 1.0, .imag_part = 0.0 }; const zero = complex { .real_part = 0.0, .imag_part = 0.0 }; ``` As mentioned above, two named struct types are considered equal if and only if they have identical field names and types *and* they were both defined at the same point in the program (i.e., by the same `struct` expression). ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const some_struct = struct { x: i16, y: i16 }; const some_other_struct = struct { x: i16, y: i16 }; comptime { @comptime_assert(some_struct == some_struct); @comptime_assert(some_other_struct == some_other_struct); // Although some_struct and some_other_struct have the same field names and // types, they are *not* the same type, because they were defined at // different source locations. @comptime_assert(some_struct != some_other_struct); } ``` Named struct types can also be returned from functions. When combined with comptime `type` arguments, this can be used to define parameterized struct types, whose field types can be customized by the user. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn pair(comptime T1: type, comptime T2: type) type { return struct { first: T1, second: T2 }; } const my_pair = pair(i32, comptime_string) { .first = 42, .second = "this is a struct" }; comptime { @comptime_assert(@type_of(my_pair) == pair(i32, comptime_string)); @comptime_assert(@type_of(my_pair.first) == i32); @comptime_assert(@type_of(my_pair.second) == comptime_string); } ``` The fields of a struct can be mutated by assigning to them via `.` syntax. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const s = struct { x: i32, y: i32 }; comptime { var my_s = s { .x = 15, .y = 99 }; @comptime_assert(my_s.x == 15); @comptime_assert(my_s.y == 99); my_s.x = 0; my_s.y = 33; @comptime_assert(my_s.x == 0); @comptime_assert(my_s.y == 33); } ``` ### `extern struct` By default, the memory layout of structs is not defined. Fields are guaranteed to be ABI-aligned, but no guarantees are provided about the ordering of fields or size of the struct. If a well-defined memory layout is required, a named struct type can be qualified with `extern`. This gives the struct in-memory layout matching the C ABI for the target, enabling `extern struct` types to be used in `export` and `extern` declarations. All fields of `extern struct` types must have an export-compatible type. See [Storage Classes](/csl/language/storage-classes) for details. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const s = extern struct { x: i16, y: f32, }; export var shared_s = s { .x = -1, .y = -1.1, }; ``` ### `packed struct` `packed` structs have a different kind of well-defined memory layout. All packed structs have a *backing integer*. The type of this integer is implicitly determined by the total bit count of fields, and the ABI of this integer is exactly the ABI of the `packed struct` type. This enables ABI-sized `packed struct` types to be used in `export` and `extern` declarations. See [Storage Classes](/csl/language/storage-classes) for details. Each field of a packed struct is interpreted as a logical sequence of bits, arranged from least to most significant. The following field types are allowed, with bit counts defined as follows: * A field of fixed-width integer or float type uses as many bits as its width. For example, a `u8` will use 8 bits of the backing integer. * A `bool` field uses exactly 1 bit. * An `enum` field uses exactly the bit width of its underlying integer type. * A `packed struct` field uses the bits of its backing integer. * A `packed union` field uses the bits of its backing integer. * A field of pointer type uses as many bits as the target architecture's word size. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const some_struct = packed struct { x: i16, y: f32, }; var s0 = some_struct { .x = 2, .y = 0.0, }; const some_other_struct = packed struct { x: i8, y: i8, p: [*]u16, }; // Size of some_other_struct is 32 bits, which is ABI-sized extern var s1: some_other_struct; ``` It is illegal to take the address of a `packed struct` field, since the field may be unaligned: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const some_struct = packed struct { x: i16, y: f32, }; var s0 = some_struct { .x = 2, .y = 0.0, }; // error: address of packed struct field is unsupported const ptr = &s0.y; ``` ## Union Types An untagged union type is similar to a named struct type, except that it represents a choice among the field types rather than a collection of field types. Only untagged union types are currently supported in CSL. An untagged union type is expressed with the form `union { field1: type1, field2: type2, ... }`. Once a union type has been defined, a value of that type can be created by giving the name of the type, followed by a field initializer of the form `{ .field = value }`. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const value = union { i: i16, f: f16 }; const i_value = value { .i = 57 }; const f_value = value { .f = 57.0 }; ``` The fields of a union type are also called its variants. The field that has been initialized is called the active variant. By default, only accesses to the active variant are allowed. Note that this is currently only enforced for comptime accesses and not for runtime accesses. At run time, it is therefore the programmer's responsibility to ensure only the active variant of a bare union is accessed. Otherwise, behavior is undefined. In extern and packed unions, accesses to the other variants are also allowed, in which case the memory is reinterpreted. See [`extern union`](#extern-union) and [`packed union`](#packed-union). Similarly to named struct types, two union types are considered equal if and only if they have identical field names and types *and* they were both defined at the same point in the program, i.e., by the same `union` expression. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const some_union = union { i: i16, f: f16 }; const some_other_union = union { i: i16, f: f16 }; comptime { @comptime_assert(some_union == some_union); @comptime_assert(some_other_union == some_other_union); // Although some_union and some_other_union have the same field names and // types, they are *not* the same type, because they were defined at // different source locations. @comptime_assert(some_union != some_other_union); } ``` Just like named struct types in the [parameterized struct types example](#parameterized-struct-types-example), it is possible to define parameterized union types, with field types that can be customized by the user. The active variant of a union can be mutated by assignment via `.` syntax. A new active variant can only be established by assigning a new value to the entire union object. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const u = union { i: i32, f: f32 }; comptime { var my_u = u { .i = 15 }; @comptime_assert(my_u.i == 15); my_u.i = 0; @comptime_assert(my_u.i == 0); my_u = u { .f = 33.0 }; @comptime_assert(my_u.f == 33.0); } ``` ### `extern union` Similarly to an [`extern struct`](#extern-struct), a union type qualified with `extern` has an in-memory layout matching the C ABI for the target, enabling `extern union` types to be used in `export` and `extern` declarations. All fields of `extern union` types must have an export-compatible type. See [Storage Classes](/csl/language/storage-classes) for details. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const u = extern union { i: i32, f: f32 }; export var shared_u = u { .i = -1, }; ``` Note that even though accessing a variant of an extern union other than the active variant is allowed, this is not yet fully supported at comptime. ### `packed union` Similarly to a [`packed struct`](#packed-struct), a union type qualified with `packed` has a *backing integer*. All fields of a packed union must have the same bit width, and the ABI of this integer is exactly the ABI of the `packed union` type. This enables ABI-sized `packed union` types to be used in `export` and `extern` declarations. See [Storage Classes](/csl/language/storage-classes) for details. The valid field types are the same as those that are valid for a [`packed struct`](#packed-struct). ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const some_struct = packed struct { x: i8, y: i8, }; const some_union = packed union { s: some_struct, i: i16, }; // The size of some_union is 16 bits, which is ABI-sized. extern var u: some_union; ``` Note that even though accessing a variant of a packed union other than the active variant is allowed, this is not yet fully supported at comptime. ## Enumeration Types An enumeration type is a set of named elements, each of which is represented by a unique integer value: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const colors = enum(u16) { red, white, blue }; const favorite = colors.red; ``` The underlying integer type is specified by the type argument (`u16` in the example above) and it can be any fixed-precision integer type (e.g. `i16` or `u32`). Any element can be assigned a comptime-known integer value: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const colors = enum(u16) { red, white = 1, blue }; ``` The values assigned must be unique within the type. Any element not assigned a value will be assigned a value by the compiler. The compiler assigns values from left to right, using consecutive integers starting with zero. In the example above, the compiler will assign the value `0` to `red` and the value `2` to `blue`. If `white` is instead assigned the value `4`, then the compiler will assign the value `0` to `red` and `1` to `blue`. Enumeration type values can be cast to and from their underlying numeric values using the `@as()` builtin, as the following assertions demonstrate: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @comptime_assert(@as(i16, colors.red) == 0); @comptime_assert(@as(u16, colors.white) == 1); @comptime_assert(@as(f32, colors.blue) == 2.0); @comptime_assert(@as(colors, 1) == colors.white); ``` An enumeration type can be cast to and from any numeric type regardless of its underlying integer type, subject only to the general compatibility rules of type casts. An enumeration value cannot be cast directly to a value of another enumeration type, but the same effect can be achieved by casting via a numeric type: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const game = enum(i16) {rock, paper = -3, scissors}; // @as(color, game.rock) <= Error! const myred = @as(colors, @as(u16, game.rock)); // OK @comptime_assert(@as(colors, @as(i16, colors.white) + 1) == colors.blue); ``` Two expressions of the same enumeration type can be compared using the `==` and `!=` operators. ### Enumeration Type Equality Two enumeration types are the same if and only if both of the following conditions are true: * they have the same structure, i.e., the same underlying integer type, and the same set of element values, each of which is assigned the same numeric value. * their definitions originate at the same source code location. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const e1 = enum(i16) {red, white}; const e2 = enum(i16) {red, white}; fn enum_type(base_type: type) type { return enum(base_type) {red, white}; }; const e3 = enum_type(i16); const e4 = enum_type(i32); const e5 = e1; const myred = e1.red; // different types, not originating from the same location: @comptime_assert(e1 != e2); // different types, same originating location, but not same structure @comptime_assert(e3 != e4); @comptime_assert(e5 == e1); // same type @comptime_assert(@type_of(myred) == e5); // same type ``` ## Array Types An array type is parameterized by an element type, describing a collection of elements of the base type. An array type whose element type is `T` can be written as `[size]T`. The element type must not be another array type. Multidimensional arrays are specified with a sequence of dimensions: `[size1, size2, size3]T`. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var 1d_array = @zeros([10]u16); var 2d_array = @zeros([10, 10]u16); var another_array = [2]i16 {1,2} ``` ## Pointer Types A value of pointer type contains the memory address where a variable is located. Pointer types are described by an element type and an optional `const` qualifier. In CSL, pointers are *only* created by taking the address of a variable. This provides the property that pointers always point to valid data when they are created. There are two kinds of pointers in CSL: pointers to a single element and pointers to an unknown number of elements. ### Pointers to a Single Element A pointer to a single value of type `T` is written as `*T`. For example: * a pointer to a single `i16` is written as `*i16` * a pointer to a single array of ten integers is written as `*[10]i16`. Pointers to a single element are created with the address-of operator (`&`): ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var array = @zeros([10]u16); const ptr = &array; const same_ptr:*[10]u16 = &array; ``` The only operation allowed on pointers to a single element is to dereference them with the dereference `.*` operator: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var array = @zeros([10]i32); const ptr = &array; ptr.* = @constants([10]i32, 42); ptr.*[2] = 1; ``` Pointer types may be `const` qualified, indicating that this pointer may not be used to modify the underlying memory: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var array = @zeros([10]i32); var ptr:*const[10]i32 = &array; ptr.* = @constants([10]i32, 42); // Error: pointer type is const qualified. ptr.*[2] = 1; // Error: pointer type is const qualified. ``` ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const array = @zeros([10]i32); var ptr = &array; // Type of ptr is *const[10]i32 ptr.* = @constants([10]i32, 42); // Error: pointer type is const qualified. ptr.*[2] = 1; // Error: pointer type is const qualified. ``` In the example above, `ptr` itself is mutable, but the memory it points to is not. ### Pointers to Unknown Number of Elements A pointer to an unknown number of elements of type `T` is written as `[*]T`. For example, a pointer to an unknown number of `f16` elements is written as `[*]f16`. Pointers to an unknown number of elements are created through coercion from pointers to a single element of array type (e.g. `*[2]i16`): ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn foo(ptr : [*]i32) void { // ... } var array10 = @zeros([10]i32); var array20 = @zeros([20]i32); foo(&array10); // ok! foo(&array20); // ok! var array_float = @zeros([20]f16); foo(&array_float); // Error: base type mismatch. var ptr : [*]i32 = &array10; ptr = &array20; ``` The original array must have rank one: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var array = @zeros([3,3,3]i32); var ptr : [*]i32 = &array; // Error. ``` Dereferencing a pointer to an unknown number of elements is not allowed. The access operator `[]` must be used instead: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn foo(ptr : [*]i32) void { ptr.* = 10; // Error: dereferencing is not allowed. ptr[0] = 10; } ``` It is illegal to access an element whose index is out-of-bounds on the original array: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn foo(ptr : [*]i32) void { ptr[1000] = 10; } var array10 = @zeros([10]i32); foo(&array10); // Bad: will create out-of-bounds access. ``` An error is emitted if the compiler is able to detect an out-of-bounds access. ### Pointers and Configuration Memory It is illegal to dereference or use the access operator `[]` on pointers occurring in the selected target’s configuration address range. An error is emitted if the compiler is able to detect such an access. Configuration memory should be accessed using the builtins `@get_config` and `@set_config` instead (see [@get\_config](/csl/language/builtins#@get_config), [@set\_config](/csl/language/builtins#@set_config)). ## The `anyopaque` Type The `anyopaque` type represents an opaque type whose size and alignment are unknown. It is primarily useful as the element type of a pointer (`*anyopaque`) to create type-erased pointers that can point to values of any type. The `anyopaque` type itself cannot be used directly as a value, as a function parameter, as a function return type, or in container types (structs, unions, arrays) because its size is not known. It can only be used behind a pointer. Pointers to any type can be implicitly coerced to `*anyopaque`: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var x: i32 = 42; var y: f16 = 3.14; var ptr_x: *anyopaque = &x; // Implicit coercion from *i32 var ptr_y: *anyopaque = &y; // Implicit coercion from *f16 ``` To recover the original type, use `@ptrcast`: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var x: i32 = 42; var opaque_ptr: *anyopaque = &x; var ptr_i32: *i32 = @ptrcast(*i32, opaque_ptr); ptr_i32.* = 100; // Modifies x ``` Casting a pointer to `*anyopaque` and then casting it back to an incorrect type results in undefined behavior. The programmer must ensure type safety when using `@ptrcast`. ## The `comptime_string` Type Support for compile-time strings is still experimental, and the set of operations available on the type `comptime_string` is very limited. Values of `comptime_string` type hold immutable strings that can be manipulated at compile time. All values of type `comptime_string` must be comptime-known. See [Comptime](/csl/language/comptime) for more information on comptime. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const hello = "abc"; // type is comptime_string fn bool_to_str(b: bool) comptime_string { if (b) { return "true"; } else { return "false"; } } comptime { const true_str = bool_to_str(true); @comptime_assert(true_str == "true"); var s = "hello"; @comptime_assert(s == "hello"); s = "goodbye"; @comptime_assert(s != "hello"); @comptime_assert(s == "goodbye"); } ``` As in C and C++, strings in CSL are sequences of bytes. A Unicode character may correspond to a sequence of more than one byte, and CSL does not have a “wide character” type. Like `std::string` in C++, but *unlike* `char *` strings in C, strings in CSL are *not* null-terminated. This means that the NUL character can occur anywhere in a string. For example, the following `@comptime_assert`s will succeed: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @comptime_assert("abc\x00xyz" != "abc") @comptime_assert("abc\x00xyz" == "abc\x00xyz") @comptime_assert(@strlen("abc\x00xyz") != 3) @comptime_assert(@strlen("abc\x00xyz") == 7) ``` UTF-8 encoded text is allowed in string literals, but if the text contains non-ASCII characters, the length of the string (as returned by `@strlen`) will not necessarily match the number of characters in the string. For example, the `@comptime_assert`s in the following code will succeed: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // Note: The UTF-8 encoding of the "thumbs up" emoji is F0 9F 91 8D. const thumbs_up = "👍"; @comptime_assert(@strlen(thumbs_up) == 4); var as_array: [4]u8 = @get_array(thumbs_up); @comptime_assert(as_array[0] == 0xf0); @comptime_assert(as_array[1] == 0x9f); @comptime_assert(as_array[2] == 0x91); @comptime_assert(as_array[3] == 0x8d); ``` ## The `imported_module` Type `imported_module` is the type returned by [`@import_module`](/csl/language/builtins#@import_module). A value of this type represents a collection of the symbols (constants, types, functions, tasks) exported by the imported module, accessed using dot notation on the value: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const memcpy = @import_module("", .{ .width = 4, .height = 1 }); // memcpy is of type imported_module; accessing its members: const params = memcpy.get_params(0); ``` `imported_module` values exist only at compile time and cannot be stored in a runtime variable, returned from a non-comptime function, or compared. ## The `direction` Type `direction` is the built-in enumeration type used to refer to one of the five ports of the tile's fabric router, and takes one of the values `EAST`, `WEST`, `NORTH`, `SOUTH`, or `RAMP`. It is used primarily to specify color routing in the `rx` and `tx` fields of a route definition: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const route = .{ .rx = .{ WEST }, .tx = .{ RAMP, EAST } }; ``` The `rx` field accepts a single `direction` value, and the `tx` field accepts either a single value or a comptime list of unique values. See [`@set_color_config`](/csl/language/builtins#@set_color_config-@set_local_color_config) for full routing semantics. ## Type Coercions ### Numeric Coercions CSL supports implicit numeric coercions that widen values to larger, compatible types. These coercions preserve all possible values from the source type. #### Integer Widening Integer types can be implicitly coerced to wider integer types when the destination type can represent all values that the source type can represent: * **Same signedness with wider width**: `i8` to `i16` to `i32` to `i64`, and `u8` to `u16` to `u32` to `u64` * **Unsigned to wider signed**: `u8` to `i16`, `u16` to `i32`, `u32` to `i64` ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var small: i8 = 42; var large: i32 = small; // OK: i8 can be widened to i32 var unsigned_val: u16 = 1000; var signed_val: i32 = unsigned_val; // OK: i32 can represent all u16 values ``` The following integer coercions are **not** allowed: * **Narrowing** (loses precision): This applies to any coercion where the destination integer type has a smaller bit width than the source integer type, such as `i32` to `i16`. * **Signed to unsigned** (unsigned types cannot represent negative values): This applies to all coercions from signed to unsigned types, regardless of bit width, such as `i8` to `u8` or `i32` to `u64`. #### Float Widening Float types can be implicitly coerced to wider float types: * **FP16 to f32**: `f16` to `f32`, `bf16` to `f32`, `cb16` to `f32` ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var half: f16 = 3.14; var single: f32 = half; // OK: f16 can be widened to f32 ``` The following float coercions are **not** allowed: * **Narrowing**: `f32` to `f16` (loses precision). This applies to any coercion where the destination float type has a smaller bit width than the source float type. * **Cross-format**: `f16` to `bf16` (different representations, not pure widening) #### Comptime Coercions Values of `comptime_int` and `comptime_float` can be coerced to compatible fixed-precision types as long as the target type can represent the source value. Specifically, `comptime_int` can be coerced to any integer type, and `comptime_float` can be coerced to any floating point type: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const int_val = 42; // comptime_int var x: i32 = int_val; // OK: 42 fits in i32 const big_val = 100000; // comptime_int var y: i16 = big_val; // Error: 100000 doesn't fit in i16 const float_val = 3.14; // comptime_float var z: f32 = float_val; // OK: comptime_float to f32 ``` ### Pointer Coercions CSL supports implicit coercions between certain pointer types. The following pointer coercions are supported, where `T` represents any specific element type, such as `f16` or `i32`, and `N` represents a specific array size. #### Const Qualification * `*T` to `*const T` * `[*]T` to `[*]const T` * `*[N]T` to `*const [N]T` ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var x: i32 = 42; var ptr: *i32 = &x; // OK: can add const qualifier var const_ptr: *const i32 = ptr; var arr: [10]i32; var arr_ptr: *[10]i32 = &arr; // OK: can add const qualifier var const_arr_ptr: *const [10]i32 = arr_ptr; // Cannot remove const qualifier // Error: cannot coerce '*const i32' to '*i32' var bad_ptr: *i32 = const_ptr; // Error: cannot coerce '*const [10]i32' to '*[10]i32' var bad_arr_ptr: *[10]i32 = const_arr_ptr; ``` #### Array Pointer to Many-Item Pointer * `*[N]T` to `[*]T` * `*[N]T` to `[*]const T` * `*const [N]T` to `[*]const T` ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var arr: [10]i32; var arr_ptr: *[10]i32 = &arr; // array pointer // OK: coerce to many-item pointer var many_ptr: [*]i32 = arr_ptr; // Cannot coerce many-item pointer back to array pointer // Error: cannot coerce '[*]i32' to '*[10]i32' var bad_arr_ptr: *[10]i32 = many_ptr; ``` #### Single-Element Pointer to Single-Element Array Pointer * `*T` to `*[1]T` * `*T` to `*const [1]T` * `*const T` to `*const [1]T` ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var x: i32 = 42; var ptr: *i32 = &x; // single-element pointer // OK: coerce to single-element array pointer var arr_ptr: *[1]i32 = ptr; // Cannot coerce array pointer back to single-element pointer // Error: cannot coerce '*[1]i32' to '*i32' var bad_ptr: *i32 = arr_ptr; ``` Note that the base element type must match for coercions to be valid. ### Struct Coercions Anonymous structs may be coerced to other struct types. For a coercion to be valid: * The destination struct type must have the same field names as the source value, but the order of the field names does not need to match. * Each source field value must be coercible to the corresponding destination field type. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const Point = struct { x: i32, y: i32, }; // OK: anonymous struct coerced to named struct var p0: Point = .{ .x = 10, .y = 20 }; // OK: anonymous struct coerced to anonymous struct var p1: struct { x: i32, y: i32 } = .{ .x = 10, .y = 20 }; // OK: field order does not need to match var p2: Point = .{ .y = 20, .x = 10 }; // Error: expected 2 fields, got 1 var p3: Point = .{ .x = 1 }; // Error: expected 2 fields, got 3 var p4: Point = .{ .x = 1, .y = 2, .z = 3 }; // Error: field 'y' not present in source var p5: Point = .{ .x = 1, .z = 2 }; // Error: cannot coerce field 'y' from type 'comptime_string' to type 'i32' var p6: Point = .{ .x = 1, .y = "str" }; ``` ### Peer Type Resolution Peer type resolution is used when CSL needs to find a common type for multiple expressions. CSL attempts to find a common type that all expressions can be coerced to. Peer type resolution occurs in the following contexts: * Conditional expressions (`if`/`else`), when the if and else branches have different but compatible types * Switch expressions, when different cases return different but compatible types * Block expressions, when multiple `break` statements with values have different but compatible types * Binary operations between compatible types, such as addition, subtraction, comparison, etc. The following coercions are supported in peer type resolution: #### Numeric Coercions All numeric coercions described in [Numeric Coercions](#numeric-coercions) are supported in peer type resolution. For example, `i16` can be widened to `i32`, and `f16` can be widened to `f32`. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn choose_int_or_comptime_int(condition: bool) i32 { var x: i32 = 10; // OK: 20 is comptime_int, coerced to i32; result has type i32 return if (condition) x else 20; } fn choose_float_or_comptime_float(condition: bool) f32 { var y: f32 = 1.0; // OK: 2.0 is comptime_float, coerced to f32; result has type f32 return if (condition) y else 2.0; } fn choose_narrow_or_wide_int(condition: bool) i32 { var small: i16 = 100; var large: i32 = 200; // OK: i16 widened to i32; result has type i32 return if (condition) small else large; } fn choose_narrow_or_wide_float(condition: bool) f32 { var half: f16 = 1.5; var single: f32 = 2.5; // OK: f16 widened to f32; result has type f32 return if (condition) half else single; } ``` #### Pointer Coercions All pointer coercions described in [Pointer Coercions](#pointer-coercions) are supported in peer type resolution. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn choose_ptr(condition: bool) *const i32 { var x: i32 = 1; const y: i32 = 2; // OK: both coerced to *const i32 return if (condition) &x else &y; } fn choose_array_ptr(condition: bool) [*]i32 { var arr1: [10]i32; var arr2: [20]i32; // OK: both coerced to [*]i32 return if (condition) &arr1 else &arr2; } fn choose_ptr_or_array_ptr(condition: bool) *const [1]i32 { var x: i32 = 1; var arr: [1]i32; // OK: &x coerced to *[1]i32, then both to *const [1]i32 return if (condition) &x else &arr; } ``` # CSL Language Reference Source: https://sdk.cerebras.ai/csl/language_index Browse the complete CSL language reference, covering syntax, builtins, types, modules, tasks, DSDs, DSRs, libraries, and advanced hardware features. * [Syntax of CSL](/csl/language/syntax) * [Type system overview](/csl/language/syntax#type-system-overview) * [Variables](/csl/language/syntax#variables) * [Pointers](/csl/language/syntax#pointers) * [Functions](/csl/language/syntax#functions) * [Statements](/csl/language/syntax#statements) * [Operators](/csl/language/syntax#operators) * [Comments](/csl/language/syntax#comments) * [Builtins](/csl/language/builtins) * [@activate](/csl/language/builtins#@activate) * [@allocate\_fifo](/csl/language/builtins#@allocate_fifo) * [@as](/csl/language/builtins#@as) * [@assert](/csl/language/builtins#@assert) * [@bitcast](/csl/language/builtins#@bitcast) * [@bind\_control\_task](/csl/language/builtins#@bind_control_task) * [@bind\_data\_task](/csl/language/builtins#@bind_data_task) * [@bind\_local\_task](/csl/language/builtins#@bind_local_task) * [@bind\_rotating\_tasks](/csl/language/builtins#@bind_rotating_tasks) * [@block](/csl/language/builtins#@block) * [@comptime\_assert](/csl/language/builtins#@comptime_assert) * [@comptime\_print](/csl/language/builtins#@comptime_print) * [@constants](/csl/language/builtins#@constants) * [@dimensions](/csl/language/builtins#@dimensions) * [@element\_count](/csl/language/builtins#@element_count) * [@element\_type](/csl/language/builtins#@element_type) * [@export](/csl/language/builtins#@export) * [@field](/csl/language/builtins#@field) * [@fp16](/csl/language/builtins#@fp16) * [@get\_array](/csl/language/builtins#@get_array) * [@get\_color](/csl/language/builtins#@get_color) * [@get\_config](/csl/language/builtins#@get_config) * [@get\_config\_unchecked](/csl/language/builtins#@get_config_unchecked) * [@get\_control\_task\_id](/csl/language/builtins#@get_control_task_id) * [@get\_data\_task\_id](/csl/language/builtins#@get_data_task_id) * [@get\_dsd](/csl/language/builtins#@get_dsd) * [@get\_dsr](/csl/language/builtins#@get_dsr) * [@get\_filter\_id](/csl/language/builtins#@get_filter_id) * [@get\_input\_queue](/csl/language/builtins#@get_input_queue) * [@get\_int](/csl/language/builtins#@get_int) * [@get\_local\_task\_id](/csl/language/builtins#@get_local_task_id) * [@get\_output\_queue](/csl/language/builtins#@get_output_queue) * [@get\_rectangle](/csl/language/builtins#@get_rectangle) * [@get\_string\_from\_byte](/csl/language/builtins#@get_string_from_byte) * [@get\_ut\_id](/csl/language/builtins#@get_ut_id) * [@has\_field](/csl/language/builtins#@has_field) * [@import\_module](/csl/language/builtins#@import_module) * [@increment\_dsd\_offset](/csl/language/builtins#@increment_dsd_offset) * [@initialize\_queue](/csl/language/builtins#@initialize_queue) * [@is\_arch](/csl/language/builtins#@is_arch) * [@is\_comptime](/csl/language/builtins#@is_comptime) * [@is\_same\_type](/csl/language/builtins#@is_same_type) * [@load\_to\_dsr](/csl/language/builtins#@load_to_dsr) * [@map](/csl/language/builtins#@map) * [@ptrcast](/csl/language/builtins#@ptrcast) * [@queue\_flush](/csl/language/builtins#@queue_flush) * [@random16](/csl/language/builtins#@random16) * [@range](/csl/language/builtins#@range) * [@range\_start, @range\_stop, @range\_step](/csl/language/builtins#@range_start-@range_stop-@range_step) * [@rank](/csl/language/builtins#@rank) * [@set\_active\_prng](/csl/language/builtins#@set_active_prng) * [@set\_color\_config, @set\_local\_color\_config](/csl/language/builtins#@set_color_config-@set_local_color_config) * [@set\_config](/csl/language/builtins#@set_config) * [@set\_config\_unchecked](/csl/language/builtins#@set_config_unchecked) * [@set\_control\_task\_table](/csl/language/builtins#@set_control_task_table) * [@set\_dsd\_base\_addr](/csl/language/builtins#@set_dsd_base_addr) * [@set\_dsd\_length](/csl/language/builtins#@set_dsd_length) * [@set\_dsd\_stride](/csl/language/builtins#@set_dsd_stride) * [@set\_empty\_queue\_handler](/csl/language/builtins#@set_empty_queue_handler) * [@set\_fifo\_read\_length](/csl/language/builtins#@set_fifo_read_length) * [@set\_fifo\_write\_length](/csl/language/builtins#@set_fifo_write_length) * [@set\_rectangle](/csl/language/builtins#@set_rectangle) * [@set\_teardown\_handler](/csl/language/builtins#@set_teardown_handler) * [@set\_tile\_code](/csl/language/builtins#@set_tile_code) * [@strcat](/csl/language/builtins#@strcat) * [@strlen](/csl/language/builtins#@strlen) * [@type\_of](/csl/language/builtins#@type_of) * [@unblock](/csl/language/builtins#@unblock) * [@zeros](/csl/language/builtins#@zeros) * [Builtins for Remote Procedure Calls (RPC)](/csl/language/builtins#builtins-for-remote-procedure-calls-rpc) * [Builtins for DSD Operations](/csl/language/builtins#builtins-for-dsd-operations) * [Comptime](/csl/language/comptime) * [Comptime Variables](/csl/language/comptime#comptime-variables) * [Comptime-Known Values](/csl/language/comptime#comptime-known-values) * [Comptime Expressions](/csl/language/comptime#comptime-expressions) * [Types Whose Values are Required to be Comptime](/csl/language/comptime#types-whose-values-are-required-to-be-comptime) * [Evaluation of Comptime-Known Control Flow](/csl/language/comptime#evaluation-of-comptime-known-control-flow) * [Typical Uses of the `comptime` Keyword](/csl/language/comptime#typical-uses-of-the-comptime-keyword) * [Data Structure Descriptors](/csl/language/dsds) * [Basic Syntax](/csl/language/dsds#basic-syntax) * [One-Dimensional Memory Vectors](/csl/language/dsds#one-dimensional-memory-vectors) * [Two-, Three-, or Four-Dimensional Memory Vectors](/csl/language/dsds#two-three-or-four-dimensional-memory-vectors) * [Pointers To Scalars As Destinations](/csl/language/dsds#pointers-to-scalars-as-destinations) * [Circular Buffers](/csl/language/dsds#circular-buffers) * [Fabric Input Vectors](/csl/language/dsds#fabric-input-vectors) * [Fabric Output Vectors](/csl/language/dsds#fabric-output-vectors) * [FIFOs](/csl/language/dsds#fifos) * [Change DSD Properties](/csl/language/dsds#change-dsd-properties) * [Asynchronous DSD Operations](/csl/language/dsds#asynchronous-dsd-operations) * [Explicit Index Offset](/csl/language/dsds#explicit-index-offset) * [Advanced DSD Features](/csl/language/dsds#advanced-dsd-features) * [Data Structure Registers](/csl/language/dsrs) * [DSR Types](/csl/language/dsrs#dsr-types) * [DSR Builtins](/csl/language/dsrs#dsr-builtins) * [Libraries](/csl/language/libraries) * [``](/csl/language/libraries#\) * [``](/csl/language/libraries#\) * [``](/csl/language/libraries#\) * [``](/csl/language/libraries#\) * [``](/csl/language/libraries#\) * [``](/csl/language/libraries#\) * [``](/csl/language/libraries#\) * [``](/csl/language/libraries#\) * [``](/csl/language/libraries#\) * [``](/csl/language/libraries#\) * [``](/csl/language/libraries#\) * [``](/csl/language/libraries#\) * [``](/csl/language/libraries#\) * [``](/csl/language/libraries#\) * [``](/csl/language/libraries#\) * [`Diagram of an 8 by 3 simulated fabric with the program's single PE placed at column 4, row 1 ### Problem Steps Visually, this program consists of the following steps: **1. Host launches function on PE.** Diagram of the host launching a function on the PE **2. Function initializes A, x, b, and computes y.** Diagram of the PE initializing A, x, and b, then computing y **3. Host copies result y from device.** Diagram of the host copying the result y from the device ## Write the CSL The previous tutorial declared arrays and wrote functions `initialize` and `gemv` to initialize and compute `y = Ax + b`. What else does the device code need to form a complete program? 1. A top-level “layout” file, which defines the program rectangle on which your kernel will run, and assigns a code file to the single PE in the rectangle. 2. Initialization of the memcpy library infrastructure, which allows the host to launch kernels and copy data to and from the device. This section first walks through `layout.csl`, which defines the program layout, included below. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // Import memcpy layout module for 1 x 1 grid of PEs // This module defines parameters passed to program on the single PE const memcpy = @import_module("", .{ .width = 1, .height = 1 }); layout { // Use just one 1 PE (columns=1, rows=1) @set_rectangle(1, 1); // The lone PE in this program should execute the code in "pe_program.csl" // This passes memcpy parameters to the program as a parameter. Note that // memcpy parameters are parameterized by the PE's column number. @set_tile_code(0, 0, "pe_program.csl", .{ .memcpy_params = memcpy.get_params(0) }); // Export device symbol for array "y" // Last argument is mutability: host can read y, but not write to it @export_name("y", [*]f32, false); // Export host-callable device function @export_name("init_and_compute", fn()void); } ``` ### Initialize Memcpy Infrastructure At the very top of this file is an `@import_module` call, which imports the top-level memcpy infrastructure. This module import requires width and height parameters which correspond to the dimensions of the program rectangle. This program only uses a single PE, so width and height are both 1. Module imports in CSL act like unique struct types. Thus, the code in the CSL standard library file `memcpy/get_params` can be used like a struct named `memcpy`. ### Define the Layout The layout block is evaluated at compile time. It defines the number of PEs used in the program and assigns code to each of those PEs. `@set_rectangle` defines the shape of the program. Because the program runs on a single PE, it's compiled for a 1x1 rectangle of PEs. The single PE has coordinate (0,0) and is assigned the code file `pe_program.csl`, explored later. The program also passes some memcpy-related parameters to it. The `memcpy` struct contains a function named `get_params`, which returns some parameters for the memcpy infrastructure that each PE’s code file must include. This function takes as an argument the column number of the PE; thus, for this program, the appropriate parameters are returned by `memcpy.get_params(0)`. ### Export Symbols The host program will directly launch a device kernel, and copy back the result `y`. The two `@export_name` calls make the symbols visible to the host program. The first `@export_name` call makes the symbol named `y` visible to the host, as a pointer to an array of type `f32`. Its mutability is set to `false`, meaning that the host can only read from and not write to the symbol. The second `@export_name` call makes the symbol `init_and_compute` visible to the host; this is the function you'll launch from the host to compute the GEMV. This function takes no arguments, so its type is `fn()void`. ### Add Memcpy to the PE Program Now, take a look at `pe_program.csl`, which defines the code assigned to the single PE. This program is largely the same as the preceding tutorial’s `code.csl` file, but with some additional infrastructure related to `memcpy`. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // Struct containing parameters for memcpy layout param memcpy_params; // memcpy module provides infrastructure for copying data // and launching functions from the host const sys_mod = @import_module("", memcpy_params); // Constants defining the matrix dimensions const M: i16 = 4; const N: i16 = 6; // 48 kB of global memory contain A, x, b, y var A: [M*N]f32; // A is stored row major var x: [N]f32; var b: [M]f32; var y: [M]f32; // Ptr to y will be exported as symbol to host // Ptr is const, so host can read but not write to y const y_ptr: [*]f32 = &y; // Initialize matrix and vectors fn initialize() void { // for loop with range syntax for (@range(i16, M*N)) |idx| { A[idx] = @as(f32, idx); } for (@range(i16, N)) |j| { x[j] = 1.0; } // while loop with iterator syntax var i: i16 = 0; while (i < M) : (i += 1) { b[i] = 2.0; y[i] = 0.0; } } // Compute gemv fn gemv() void { for (@range(i16, M)) |i| { var tmp: f32 = 0.0; for (@range(i16, N)) |j| { tmp += A[i*N + j] * x[j]; } y[i] = tmp + b[i]; } } // Call initialize and gemv functions fn init_and_compute() void { initialize(); gemv(); // After this function finishes, memcpy's cmd_stream must // be unblocked on all PEs for further memcpy commands // to execute sys_mod.unblock_cmd_stream(); } comptime { // Export symbol pointing to y so it is host-readable @export_symbol(y_ptr, "y"); // Export function so it is host-callable by RPC mechanism @export_symbol(init_and_compute); } ``` At the top, a parameter named `memcpy_params` is declared: this parameter’s value is set at compile time by `@set_tile_code` in `layout.csl`. Next is another memcpy-related `@import_module`, this time importing the PE-specific `` standard library file as a struct named `sys_mod`. The functions `initialize` and `gemv` are identical to the previous tutorial. However, note one addition to `init_and_compute`. After `gemv` finishes, the memcpy infrastructure must be notified that additional commands from the host can proceed. This is why the function calls `sys_mod.unblock_cmd_stream()` at its end. The control flow of every host-callable function in a CSL program must end with a call to `unblock_cmd_stream()`. Everything inside the `comptime` block is evaluated at compile time. This comptime block exports symbols so they can be advertised to the host. In particular, `y_ptr`, which is a pointer to the array `y`, is exported with the name `y`. The `init_and_compute` function is also exported. ## Compile CSL Code Compile this code for the CS-2 simulator using: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} $ cslc layout.csl --fabric-dims=8,3 --fabric-offsets=4,1 --memcpy --channels=1 -o out ``` This command produces multiple ELF files, in a directory named `out`. The following sections walk through several aspects of this command. First, specify the top-level file to be compiled, in this case `layout.csl`. `pe_program.csl` does not have to be specified in the compilation command, because it is included by `layout.csl`. You must also specify the fabric dimensions of the target device, and the fabric offset at which the program is placed. As specified above, this tutorial uses an 8 x 3 simulated fabric, with the program’s lone PE placed at column 4, row 1 of the fabric. Every program using memcpy **must** use a fabric offset of `4,1`, and if compiling for a simulated fabric, must use a fabric dimension of at least `width+7,height+1`, where `width` and `height` are the dimensions of the program. These additional PEs are used by memcpy to route data on and off the wafer. Last, note the flags specifying `memcpy` and `channels`. Every program using memcpy must include the `--memcpy` flag. When running on a real system, the `channels` flag determines the max throughput for transferring data on and off the wafer. Its value can be no larger than the height of the program rectangle (the number of rows), and maxes out at 16. Typically, performance improvements are minimal past 8 channels. This program is also compatible with the CS-3 architecture. Specify the `--arch` flag to determine which architecture to compile for. The default value is `--arch=wse2`, where WSE-2 is the processor architecture used in the CS-2. Specify the value `--arch=wse3` to compile for WSE-3, the processor architecture used in the CS-3. ## Write the Host Code What does the host code need to do? 1. Import needed libraries 2. Specify paths to compiled code and instantiate runner object 3. Run device kernel `init_and_compute` 4. Copy back `y` and check result The following sections explain some features of the `run.py` file containing the host code, shown below. ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} #!/usr/bin/env cs_python import argparse import numpy as np from cerebras.sdk.runtime.sdkruntimepybind import SdkRuntime, MemcpyDataType, MemcpyOrder # pylint: disable=no-name-in-module # Read arguments parser = argparse.ArgumentParser() parser.add_argument('--name', help="the test compile output dir") parser.add_argument('--cmaddr', help="IP:port for CS system") args = parser.parse_args() # Matrix dimensions M = 4 N = 6 # Construct A, x, b A = np.arange(M*N, dtype=np.float32).reshape(M, N) x = np.full(shape=N, fill_value=1.0, dtype=np.float32) b = np.full(shape=M, fill_value=2.0, dtype=np.float32) # Calculate expected y y_expected = A@x + b # Construct a runner using SdkRuntime runner = SdkRuntime(args.name, cmaddr=args.cmaddr) # Get symbol for copying y result off device y_symbol = runner.get_id('y') # Load and run the program runner.load() runner.run() # Launch the init_and_compute function on device runner.launch('init_and_compute', nonblock=False) # Copy y back from device # Arguments to memcpy_d2h: # - y_result is array on host which will store copied-back array # - y_symbol is symbol of device tensor to be copied # - 0, 0, 1, 1 are (starting x-coord, starting y-coord, width, height) # of rectangle of PEs whose data is to be copied # - M is number of elements to be copied from each PE y_result = np.zeros([1*1*M], dtype=np.float32) runner.memcpy_d2h(y_result, y_symbol, 0, 0, 1, 1, M, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) # Stop the program runner.stop() # Ensure that the result matches expectations np.testing.assert_allclose(y_result, y_expected, atol=0.01, rtol=0) print("SUCCESS!") ``` ### Imports `SdkRuntime` is the library containing the functionality necessary for loading and running the device code, as well as copying data on and off the wafer. Along with `SdkRuntime`, the code imports `MemcpyDataType` and `MemcpyOrder`, which are enums containing types for use with memcpy calls, explained in more detail below. ### Instantiate the Runner This script contains two arguments: `name` and `cmaddr`. Use `name` to specify the directory containing the compilation output. `cmaddr` is discussed later; for now, leave it unspecified. Instantiate a runner object using `SdkRuntime`’s constructor: ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} runner = SdkRuntime(args.name, cmaddr=args.cmaddr) ``` Before loading, grab a handle for later copying `y` off the device, with the call to `runner.get_id('y')`. Then load the program onto the device and begin running with `runner.load()` and `runner.run()`. ### Run the Device Kernel Next, launch the device kernel `init_and_compute`: ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} runner.launch('init_and_compute', nonblock=False) ``` The `nonblock=False` flag simply specifies that this call waits to return control to the host program until after the kernel has been launched. Otherwise, this call returns control to the host immediately. ### Copy Back the Result A call to `memcpy_d2h` copies the result `y` back from the device. First, allocate space on the host to hold the result: ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} y_result = np.zeros([1*1*M], dtype=np.float32) ``` Then, copy `y` from the device into this array: ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} runner.memcpy_d2h(y_result, y_symbol, 0, 0, 1, 1, M, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) ``` This call has quite a few arguments, so the following sections walk through them. The first argument is the array on the host to hold the result, allocated on the previous line. The next argument, `y_symbol`, is the symbol on device that points to the `y` array. The next four arguments specify the location of the rectangle of PEs from which to copy, referred to as the “region of interest” or ROI. The first two, `0, 0`, specify that the northwest corner of the ROI begins at PE (0, 0) within the program rectangle. Thus, it begins at the northwesternmost corner of the program rectangle. The next two specify the width and height of the ROI. This tutorial only copies the result back from a single PE, so the width and height of the ROI is simply `1, 1`. Note that the ROI is specified based on its position in the program rectangle, NOT its position in the device fabric. The next argument specifies how many elements to copy back from each PE in the ROI. In this case, the result `y` has `M` elements. The next four arguments are all keyword arguments specifying certain attributes of this copy operation. Discussion of the `streaming` keyword is deferred to a future tutorial. Note, however, that any copy between host-to-device which copies to or from a device symbol uses `streaming=False`. The `order` keyword specifies the layout of the data copied back to `y_result`. `memcpy_d2h` always copies into a 1D array on the host. `ROW_MAJOR` specifies that the data is ordered by (ROI height, ROI width, elements per PE). Thus, the data copied back from each PE is contiguous in the result array. `COL_MAJOR`, on the other hand, specifies that the data is ordered by (elements per PE, ROI width, ROI height). Thus, the result array will contain the 0th element from each PE, followed by the 1st element from each PE, and so on. For this tutorial, because the copy is from a single PE, `ROW_MAJOR` and `COL_MAJOR` are identical. In general, for copies over larger fabrics, `COL_MAJOR` is more performant than `ROW_MAJOR`. The `data_type` keyword specifies the width of the data copied back. This tutorial copies back single-precision floating point numbers, so the data width is 32 bit. `nonblock=False` specifies that this call will not return control to the host until the copy into `y_result` has finished. How does the program ensure that this copy does not happen until `init_and_compute` has finished? The memcpy infrastructure in the CSL program can only execute one command at a time. After a device kernel is launched, `unblock_cmd_stream` must be called before a `memcpy_d2h` can proceed. The call to `unblock_cmd_stream` at the end of the `init_and_compute` function in `pe_program.csl` guarantees that `init_and_compute` finishes before the `memcpy_d2h` occurs. ### Finish the Program and Check the Result The call to `runner.stop()` stops the execution of the program on device. The code then checks that the `y_result` copied back from the device matches the `y_expected` pre-computed on the host. If they match, it prints a `SUCCESS` message. ## Run the Program Run the program using `cs_python`, which wraps the Cerebras-provided Python instance for executing host code. ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} $ cs_python run.py --name out ``` You should see a `SUCCESS!` message at the end of execution. You have successfully run your first program! ## Move from Simulator to System So far, this program has been compiled and run using the fabric simulator, but with a few modest changes, you can also compile and run it on a real Cerebras system. First, modify the compile command to replace the `fabric-dims` with the actual dimensions of the target fabric. Most CS-3s have a fabric dimension of 762 x 1172, so the compile command becomes: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} $ cslc layout.csl --arch=wse3 --fabric-dims=762,1172 --fabric-offsets=4,1 --memcpy --channels=1 -o out ``` This program is also compatible with the CS-2, which has a fabric dimension of 757 x 996. Compiling for the CS-2 requires specifying the WSE-2 architecture: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} $ cslc layout.csl --arch=wse2 --fabric-dims=757,996 --fabric-offsets=4,1 --memcpy --channels=1 -o out ``` The Cerebras system is a network attached accelerator. When targeting a real system for running a program, you must know its IP address. This is the purpose of the `SdkRuntime` constructor’s `cmaddr` keyword argument. If the IP address is stored in an environment variable named `$CS_IP_ADDR`, then you can run on the system with: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} $ cs_python run.py --name out --cmaddr $CS_IP_ADDR:9000 ``` This example uses port 9000 to connect to the system and launch the program. The compile and run commands above are used when running the SDK directly from a host node connected to the CS system. If using a Wafer-Scale Cluster in appliance mode, see [Running SDK on a Wafer-Scale Cluster](/appliance-mode). ## Exercises This tutorial's host code initializes `A`, `x`, and `b` to the same values they're initialized to on the device, manually. Instead of initializing them like this, you could also use `memcpy_d2h` calls to copy them from the device just as with `y`. Create exported symbols for `A`, `x`, and `b`, and use them to copy these arrays back to the host and compute an expected result for `y`. Note that `A`, `x`, and `b` are not initialized until the `init_and_compute` device kernel executes. You can also break up `init_and_compute` into two device kernel calls. Create separate device kernel calls for `initialize` and `gemv` which are launched separately on the host, and copy back `A`, `x`, and `b` after you launch `initialize` but before you launch `gemv`. ## Next In the next tutorial, you'll expand this program to use data structure descriptors (DSDs), a core language feature of CSL. # 2. Memory DSDs Source: https://sdk.cerebras.ai/csl/tutorials/gemv-02-memory-dsds Use memory Data Structure Descriptors (DSDs) to perform efficient tensor operations in CSL without explicit loops. Now that you've written a complete program in the [previous tutorial](/csl/tutorials/gemv-01-complete-program), this tutorial introduces a central concept in CSL: memory Data Structure Descriptors (DSDs). Memory DSDs provide an efficient mechanism for performing operations on entire tensors. ## Learning Objectives After completing this tutorial, you should know how to: * Define memory DSDs for tensor accesses * Use memory DSDs in builtin operations on tensors * Use builtins to initialize tensors ## Example Overview Your program will run on a single processing element (PE). Like the previous tutorial, this tutorial demonstrates the program with a simulated fabric consisting of an 8 x 3 block of PEs. The problem steps are identical to the previous tutorial. The layout file, host code, and compile and run commands are also identical. Only `pe_program.csl` needs to change, and this tutorial takes a closer look at those changes. ## Modify the CSL The previous tutorial created a complete CSL program using a single PE to initialize and compute `y = Ax + b`. What needs to change in `pe_program.csl` to take advantage of memory DSDs and builtin operations on tensors? 1. Define DSDs for accessing the tensors 2. Rewrite the `gemv` function to operate on these DSDs The previous tutorial walked through `layout.csl`, which is the same for this tutorial. The new `pe_program.csl` is included below, with the changes highlighted. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} param memcpy_params; // memcpy module provides infrastructure for copying data // and launching functions from the host const sys_mod = @import_module("", memcpy_params); // Constants defining the matrix dimensions const M: i16 = 4; const N: i16 = 6; // 48 kB of global memory contain A, x, b, y var A: [M*N]f32; // A is stored row major // Initialize x, b, y using builtins var x = @constants([N]f32, 1.0); var b = @constants([M]f32, 2.0); var y = @zeros([M]f32); // DSDs for accessing A, b, y // b_dsd uses tensor access expression to specify access to M consecutive elements of b var b_dsd = @get_dsd(mem1d_dsd, .{ .tensor_access = |i|{M} -> b[i] }); // The above expression is equivalent to: // var b_dsd = @get_dsd(mem1d_dsd, .{ .base_address = &b, .extent = M }); // y_dsd uses base_address and extent fields to specify access to M consecutive elements of y var y_dsd = @get_dsd(mem1d_dsd, .{ .base_address = &y, .extent = M }); // The above expression is equivalent to: // var y_dsd = @get_dsd(mem1d_dsd, .{ .tensor_access = |i|{M} -> y[i] }); // A_dsd accesses column of A // A_dsd uses tensor access expression to specify access to every Nth element of A var A_dsd = @get_dsd(mem1d_dsd, .{ .tensor_access = |i|{M} -> A[i*N] }); // The above expression is equivalent to: // var A_dsd = @get_dsd(mem1d_dsd, .{ .base_address = &A, .extent = M, .stride = N }); // ptr to y will be advertised as symbol to host const y_ptr: [*]f32 = &y; // Initialize A matrix fn initialize() void { // for loop with range syntax for (@range(i16, M*N)) |idx| { A[idx] = @as(f32, idx); } } // Compute gemv fn gemv() void { // Loop over all columns of A for (@range(u16, N)) |i| { // Calculate contribution to A*x from ith column of A, ith elem of x @fmacs(y_dsd, y_dsd, A_dsd, x[i]); A_dsd = @increment_dsd_offset(A_dsd, 1, f32); } // Add b to A*x @fadds(y_dsd, y_dsd, b_dsd); } // Call initialize and gemv functions fn init_and_compute() void { initialize(); gemv(); sys_mod.unblock_cmd_stream(); } comptime { @export_symbol(y_ptr, "y"); @export_symbol(init_and_compute); } ``` ### Define the Memory DSDs First, take a look at the DSDs defined for accessing `b` and `y`: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var b_dsd = @get_dsd(mem1d_dsd, .{ .tensor_access = |i|{M} -> b[i] }); var y_dsd = @get_dsd(mem1d_dsd, .{ .tensor_access = |i|{M} -> y[i] }); ``` `b_dsd` and `y_dsd` are the memory DSDs for accessing `b`, and `y`, respectively. The `tensor_access` field defines the access pattern of these DSDs. `|i|` specifies the induction variable, and `{M}` specifies the loop bound; i.e., these DSDs access `M` elements. After `->`, an expression is given for accessing a memory location using the induction variable. This expression must be **affine**, or linear plus a constant. The access pattern for these DSDs is straightforward: these DSDs loop over all `M` elements, in order, of their respective tensors. Now take a look at the DSD for accessing `A`: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var A_dsd = @get_dsd(mem1d_dsd, .{ .tensor_access = |i|{M} -> A[i*N] }); ``` This DSD accesses `M` elements of `A`, but strided by `N` elements; i.e., `A_dsd` accesses elements `0, N, 2*N, ... (M-1)*N`. Because `A` is stored in row major format, this means that `A_dsd` as defined here accesses the 0th column of `A`. These memory DSDs are of type `mem1d_dsd`, which are one-dimensional memory DSDs. CSL also provides `mem4d_dsd`, multidimensional memory DSDs for up to four dimensions. You can learn more about memory DSDs in the language reference guide [Data Structure Descriptors](/csl/language/dsds). ### Use the DSDs to Compute GEMV Now that the DSDs are defined, take a look at how to use them to compute GEMV. Recall that the previous `gemv()` function was defined as follows: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn gemv() void { for (@range(i16, M)) |i| { var tmp: f32 = 0.0; for (@range(i16, N)) |j| { tmp += A[i*N + j] * x[j]; } y[i] = tmp + b[i]; } } ``` Now, `gemv()` looks like this: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn gemv() void { for (@range(u16, N)) |i| { @fmacs(y_dsd, y_dsd, A_dsd, x[i]); A_dsd = @increment_dsd_offset(A_dsd, 1, f32); } @fadds(y_dsd, y_dsd, b_dsd); } ``` Notice that there's now only one explicit loop over `N`, instead of two explicit loops. At each iteration, this `@fmacs` operation does the following: * performs a vector-scalar multiplication between the column of `A` referenced by `A_dsd` and the scalar `x[i]`, * performs an elementwise vector addition between this result and the vector `y`, * and stores this final result into `y`. Thus, each `@fmacs` operation increments the `M` elements of `y` by the vector-scalar product of column `i` of `A` and element `i` of `x`. The `@increment_dsd_offset` operation at each loop iteration increments `A_dsd` to reference the next column of `A`. This builtin operation takes `A_dsd` and creates a new DSD by offsetting its access by 1 `f32` element. For instance, the first time this operation occurs, `A_dsd` will now access elements `1, N+1, 2*N+1, ... (M-1)*N+1` of `A`. Again, because `A` is stored row major, this will access the 1st column of `A`. Once this loop over the `N` columns of `A` is complete, `y` contains the result of `A*x`. The `@fadds` operation performs an elementwise vector addition between `y` and `b`, storing the result back in `y`. Now `y` contains the result of `A*x + b`. ### Use Builtins to Initialize Tensors You may have noticed one other slight change to this code. Instead of initializing `x`, `b`, and `y`, in the `initialize` function, this code uses builtins to provide values for them at declaration: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var x = @constants([N]f32, 1.0); var b = @constants([M]f32, 2.0); var y = @zeros([M]f32); ``` The `@constants` builtin returns a tensor of the specified type, with all elements initialized to the specified value. Thus, `x` is initialized as an `N` element tensor of all ones, and `b` is initialized as an `M` element tensor of all twos. The `@zeros` builtin is rather obvious. `y` is initialized as an `M` element tensor of all zeros. ## Compile and Run the Program As with the previous tutorial, compile and run this code using: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} $ cslc layout.csl --fabric-dims=8,3 --fabric-offsets=4,1 --memcpy --channels=1 -o out $ cs_python run.py --name out ``` You should see a `SUCCESS!` message at the end of execution. ## Exercises `A` is stored row-major in the above code. How would you rewrite `A_dsd` and the `gemv` function if `A` were stored column major instead? ## Next In the next tutorial, you'll use host-to-device `memcpy`, and copy host-initialized values for `A`, `x`, and `b` onto the device. # 3. Memcpy Source: https://sdk.cerebras.ai/csl/tutorials/gemv-03-memcpy Copy tensor data between host and device using `SdkRuntime`'s `memcpy_h2d` and `memcpy_d2h` functions. The [previous tutorial](/csl/tutorials/gemv-02-memory-dsds) wrote a program that launches a kernel and copies the result back to the host. This tutorial extends that to copy the initial tensors from the host to the device. This program will now have three phases: 1. Host-to-device memcpy of `A`, `x`, and `b` 2. Kernel launch 3. Device-to-host memcpy of `y` ## Learning Objectives After completing this tutorial, you should know how to: * Copy data from host to device using `SdkRuntime`’s `memcpy_h2d` function ## Example Overview Your program will run on a single processing element (PE). Like the previous tutorials, this tutorial demonstrates the program with a simulated fabric consisting of an 8 x 3 block of PEs. The problem steps are nearly identical to the previous tutorials, except this program now copies `A`, `x`, and `b` to the device after initializing them on the host. `pe_program.csl` no longer needs to initialize `A`, `x`, and `b`, but both CSL files need to be updated to export symbols for these tensors. The host code needs to introduce three `memcpy_h2d` calls to copy the tensors to the device. ### Problem Steps Visually, this program consists of the following steps: **1. Host copies A, x, b to device.** Diagram of the host copying A, x, and b to the device **2. Host launches function to compute y.** Diagram of the host launching a function to compute y **3. Host copies result y from device.** Diagram of the host copying the result y from the device ## Modify the CSL The previous tutorials initialized `A`, `x`, and `b` on device before computing GEMV. What else does the device code need to support a host-to-device memcpy of `A`, `x`, and `b`, so that they only need to be initialized on the host? 1. The layout file needs to export the symbol names for `A`, `x`, and `b`. 2. The PE program needs to export pointers to `A`, `x`, and `b`. The PE program no longer needs to initialize these tensors. The new `layout.csl` is included below, with the changes highlighted. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const memcpy = @import_module("", .{ .width = 1, .height = 1 }); layout { @set_rectangle(1, 1); @set_tile_code(0, 0, "pe_program.csl", .{ .memcpy_params = memcpy.get_params(0) }); // export symbol names @export_name("A", [*]f32, true); @export_name("x", [*]f32, true); @export_name("b", [*]f32, true); @export_name("y", [*]f32, false); @export_name("init_and_compute", fn()void); } ``` As described previously, `@export_name` makes symbol names visible to the host program. Notice that there are now `@export_name` calls for `A`, `x`, and `b`. Unlike `y`, the mutability of these symbols is set to `true`, since the host will write to these symbols. Now take a look at `pe_program.csl`. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} param memcpy_params; // memcpy module provides infrastructure for copying data // and launching functions from the host const sys_mod = @import_module("", memcpy_params); // Constants defining the matrix dimensions const M: i16 = 4; const N: i16 = 6; // 48 kB of global memory contain A, x, b, y var A: [M*N]f32; // A is stored row major var x: [N]f32; var b: [M]f32; var y = @zeros([M]f32); // Initialize y to zero // DSDs for accessing A, b, y // A_dsd accesses column of A var A_dsd = @get_dsd(mem1d_dsd, .{ .tensor_access = |i|{M} -> A[i*N] }); var b_dsd = @get_dsd(mem1d_dsd, .{ .base_address = &b, .extent = M }); var y_dsd = @get_dsd(mem1d_dsd, .{ .base_address = &y, .extent = M }); // ptrs to A, x, b, y will be advertised as symbols to host var A_ptr: [*]f32 = &A; var x_ptr: [*]f32 = &x; var b_ptr: [*]f32 = &b; const y_ptr: [*]f32 = &y; // Compute gemv fn gemv() void { // Loop over all columns of A for (@range(i16, N)) |i| { // Calculate contribution to A*x from ith column of A, ith elem of x @fmacs(y_dsd, y_dsd, A_dsd, x[i]); // Move A_dsd to next column of A A_dsd = @increment_dsd_offset(A_dsd, 1, f32); } // Add b to A*x @fadds(y_dsd, y_dsd, b_dsd); } // Call initialize and gemv functions fn init_and_compute() void { gemv(); sys_mod.unblock_cmd_stream(); } comptime { @export_symbol(A_ptr, "A"); @export_symbol(x_ptr, "x"); @export_symbol(b_ptr, "b"); @export_symbol(y_ptr, "y"); @export_symbol(init_and_compute); } ``` Notice that an `initialize` function is no longer needed. Calling `init_and_compute` assumes `A`, `x`, and `b` have already been initialized. Pointers `A_ptr`, `x_ptr`, and `b_ptr` to `A`, `x`, and `b`, respectively, are also now defined. These pointers are exported with `@export_symbol`, so that they're visible to the host. ## Modify the Host Code The host code is largely similar to the previous tutorials, except `A`, `x`, and `b` now must be copied to the device after initializing them on the host. This uses `memcpy_h2d`, which has similar syntax to the previously introduced `memcpy_d2h`. The modified `run.py` is included below. ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} #!/usr/bin/env cs_python import argparse import numpy as np from cerebras.sdk.runtime.sdkruntimepybind import SdkRuntime, MemcpyDataType, MemcpyOrder # pylint: disable=no-name-in-module # Read arguments parser = argparse.ArgumentParser() parser.add_argument('--name', help="the test compile output dir") parser.add_argument('--cmaddr', help="IP:port for CS system") args = parser.parse_args() # Matrix dimensions M = 4 N = 6 # Construct A, x, b A = np.arange(M*N, dtype=np.float32) x = np.full(shape=N, fill_value=1.0, dtype=np.float32) b = np.full(shape=M, fill_value=2.0, dtype=np.float32) # Calculate expected y y_expected = A.reshape(M,N)@x + b # Construct a runner using SdkRuntime runner = SdkRuntime(args.name, cmaddr=args.cmaddr) # Get symbols for A, b, x, y on device A_symbol = runner.get_id('A') x_symbol = runner.get_id('x') b_symbol = runner.get_id('b') y_symbol = runner.get_id('y') # Load and run the program runner.load() runner.run() # Copy A, x, b to device runner.memcpy_h2d(A_symbol, A, 0, 0, 1, 1, M*N, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) runner.memcpy_h2d(x_symbol, x, 0, 0, 1, 1, N, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) runner.memcpy_h2d(b_symbol, b, 0, 0, 1, 1, M, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) # Launch the init_and_compute function on device runner.launch('init_and_compute', nonblock=False) # Copy y back from device y_result = np.zeros([M], dtype=np.float32) runner.memcpy_d2h(y_result, y_symbol, 0, 0, 1, 1, M, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) # Stop the program runner.stop() # Ensure that the result matches expectations np.testing.assert_allclose(y_result, y_expected, atol=0.01, rtol=0) print("SUCCESS!") ``` This code introduces three `memcpy_h2d` calls, one for each of `A`, `x`, and `b`: ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} runner.memcpy_h2d(A_symbol, A, 0, 0, 1, 1, M*N, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) runner.memcpy_h2d(x_symbol, x, 0, 0, 1, 1, N, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) runner.memcpy_h2d(b_symbol, b, 0, 0, 1, 1, M, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) ``` These calls have quite a few arguments, but they’re identical to those used by `memcpy_d2h`, other than the first two. For `memcpy_h2d`, the first argument is the symbol on device that points to the array to which you want to copy. The next argument is the `numpy` array from which you are copying. Note that the arrays passed to memcpy must be 1D. See [A Complete Program](/csl/tutorials/gemv-01-complete-program) for an explanation of the remaining arguments. ## Compile and Run the Program As with the previous tutorial, compile and run this code using: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} $ cslc layout.csl --fabric-dims=8,3 --fabric-offsets=4,1 --memcpy --channels=1 -o out $ cs_python run.py --name out ``` You should see a `SUCCESS!` message at the end of execution. ## Exercises Try initializing `A`, `x`, and `b` to other values. Modify the host code to do multiple matrix-vector products: Try using your output `y` from a matrix-vector product as your input `x` to another matrix-vector product. ## Next In the next tutorial, you'll use compile-time parameters so that the matrix dimensions `M` and `N` can be configured at compile time rather than hard-coded into the device kernel. # 4. Parameters Source: https://sdk.cerebras.ai/csl/tutorials/gemv-04-params Define and use compile-time parameters in CSL device code and read their values from the compile output in your Python host program. The [previous tutorial](/csl/tutorials/gemv-03-memcpy) wrote a complete program that copies data to and from the device, but both the host and device still need to define the dimensions `M` and `N`. This tutorial introduces compile time parameters to set `M` and `N` while compiling the code, and shows how your host code can read these values from the compile output. ## Learning Objectives After completing this tutorial, you should know how to: * Define compile time parameters for your device code * Set the value of compile time parameters when compiling * Read the value of compile time parameters from compile output in your host code ## Example Overview Your program will run on a single processing element (PE). Like the previous tutorials, this tutorial demonstrates the program with a simulated fabric consisting of an 8 x 3 block of PEs. The problem steps are identical to the previous tutorial. The device code needs to be modified to replace the constants `M` and `N` with parameters, and the compile command needs to be modified to set these parameter values. The host code must be modified to read these values from compile output. ## Modify the CSL How must the layout code be modified to support compile time parameters for `M` and `N`? 1. Define top-level parameters for `M` and `N` that will be set by the compile command 2. Pass these parameters along to the PE program Take a look at the modified `layout.csl`: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} param M: i16; param N: i16; const memcpy = @import_module("", .{ .width = 1, .height = 1 }); layout { @set_rectangle(1, 1); @set_tile_code(0, 0, "pe_program.csl", .{ .memcpy_params = memcpy.get_params(0), .M = M, .N = N }); // export symbol names @export_name("A", [*]f32, true); @export_name("x", [*]f32, true); @export_name("b", [*]f32, true); @export_name("y", [*]f32, false); @export_name("init_and_compute", fn()void); } ``` Notice that two parameters are defined at the top of the file: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} param M: i16; param N: i16; ``` These parameters are also passed along to the PE program inside of the `@set_tile_code` call: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @set_tile_code(0, 0, "pe_program.csl", .{ .memcpy_params = memcpy.get_params(0), .M = M, .N = N }); ``` Now take a look at the modified `pe_program.csl`: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} param memcpy_params; // Matrix dimensions param M: i16; param N: i16; // memcpy module provides infrastructure for copying data // and launching functions from the host const sys_mod = @import_module("", memcpy_params); // 48 kB of global memory contain A, x, b, y var A: [M*N]f32; // A is stored row major var x: [N]f32; var b: [M]f32; var y = @zeros([M]f32); // Initialize y to zero // DSDs for accessing A, b, y // A_dsd accesses column of A var A_dsd = @get_dsd(mem1d_dsd, .{ .tensor_access = |i|{M} -> A[i*N] }); var b_dsd = @get_dsd(mem1d_dsd, .{ .base_address = &b, .extent = M }); var y_dsd = @get_dsd(mem1d_dsd, .{ .base_address = &y, .extent = M }); // ptrs to A, x, b, y will be advertised as symbols to host var A_ptr: [*]f32 = &A; var x_ptr: [*]f32 = &x; var b_ptr: [*]f32 = &b; const y_ptr: [*]f32 = &y; // Compute gemv fn gemv() void { // Loop over all columns of A for (@range(i16, N)) |i| { // Calculate contribution to A*x from ith column of A, ith elem of x @fmacs(y_dsd, y_dsd, A_dsd, x[i]); // Move A_dsd to next column of A A_dsd = @increment_dsd_offset(A_dsd, 1, f32); } // Add b to A*x @fadds(y_dsd, y_dsd, b_dsd); } // Call initialize and gemv functions fn init_and_compute() void { gemv(); sys_mod.unblock_cmd_stream(); } comptime { @export_symbol(A_ptr, "A"); @export_symbol(x_ptr, "x"); @export_symbol(b_ptr, "b"); @export_symbol(y_ptr, "y"); @export_symbol(init_and_compute); } ``` `pe_program.csl` must also contain parameter declarations for `M` and `N`. When this file is compiled, it uses the values passed to it by `layout.csl`’s `@set_tile_code` call to bind them. `M` and `N` are no longer hard-coded in this file. ## Compile and Run the Program The device and host code now use the compile time parameters, but how are they set? The compile command now includes a `--params` flag, which specifies the values: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} $ cslc layout.csl --fabric-dims=8,3 --fabric-offsets=4,1 --params=M:4,N:6 --memcpy --channels=1 -o out $ cs_python run.py --name out ``` The run command stays the same. You should see a `SUCCESS!` message at the end of execution. ## Exercises `A` is stored row-major in the above code. How would you rewrite `A_dsd` and the `gemv` function if `A` were stored column major instead? ## Next This series has now gone over some basics for writing a complete program using a single PE. The next tutorial moves on to using multiple PEs. # 5. Multiple PEs Source: https://sdk.cerebras.ai/csl/tutorials/gemv-05-multiple-pes Scale your CSL program to run across multiple processing elements by configuring a multi-PE layout and copying data to and from each PE. The power of the Wafer-Scale Engine lies in its hundreds of thousands of processing elements. Now that the basics for writing a complete program using a single PE are covered, this tutorial creates your first program using multiple PEs. ## Learning Objectives After completing this tutorial, you should know how to: * Define a layout file that compiles code for multiple PEs * Copy data to and from multiple PEs on the device ## Example Overview Your program will now run on four processing elements (PE). This tutorial demonstrates the program with a simulated fabric consisting of an 11 x 3 block of PEs. For this program, each PE performs the exact same work; that is, `A`, `x`, and `b` are copied to each of the four PEs, the four PEs each perform a GEMV, and then the result `y` is copied back from each PE. `pe_program.csl` does not change. Only `layout.csl` needs to be modified to assign it to multiple PEs. The host code also needs to be modified to copy to and from multiple PEs instead of just one. ### Problem Steps Visually, this program consists of the following steps: **1. Host copies A, x, b to four PEs on device.** Diagram of the host copying A, x, and b to four PEs on the device **2. Host launches function on each PE to compute y.** Diagram of the host launching a function on each PE to compute y **3. Host copies result y from each PE.** Diagram of the host copying the result y from each PE ## Modify the CSL How does the layout file need to change to support running the program on multiple PEs? 1. Modify `@set_rectangle` to reflect the new program rectangle. 2. Modify the `memcpy` infrastructure to reflect the use of multiple PEs. 3. Call `@set_tile_code` for each coordinate inside this program rectangle. `pe_program.csl` remains largely the same; it's simply assigned to more PEs. The new `layout.csl` is included below, with the changes highlighted. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // matrix dimensions on each PE param M: i16; param N: i16; // number of PEs in program param width: i16; const memcpy = @import_module("", .{ .width = width, .height = 1 }); layout { // PE coordinates are (column, row) @set_rectangle(width, 1); for (@range(i16, width)) |x| { @set_tile_code(x, 0, "pe_program.csl", .{ .memcpy_params = memcpy.get_params(x), .M = M, .N = N }); } // export symbol names @export_name("A", [*]f32, true); @export_name("x", [*]f32, true); @export_name("b", [*]f32, true); @export_name("y", [*]f32, false); @export_name("compute", fn()void); } ``` Notice that a new compile time parameter `width` is defined, whose value is set in the compile command. This value sets the number of PEs in the row of PEs used by the program. When `` is imported, `width` specifies the width of the program rectangle for which memcpy infrastructure will be generated. The `height` is still 1. Inside the layout block, the program rectangle is now specified with `@set_rectangle(width, 1)`. For each of the PEs in this rectangle, `@set_tile_code` must be called, so this happens in a loop. The loop coordinate is the PE’s `x`-coordinate, or column number, which is needed to set the correct `memcpy_params` for each PE. ## Modify the Host Code The host code must now copy `A`, `x` and `b` to multiple PEs, and must copy back `y` from multiple PEs. Take a look at how the `memcpy_h2d` and `memcpy_d2h` calls in `run.py` need to change: ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} #!/usr/bin/env cs_python import argparse import json import numpy as np from cerebras.sdk.runtime.sdkruntimepybind import SdkRuntime, MemcpyDataType, MemcpyOrder # pylint: disable=no-name-in-module # Read arguments parser = argparse.ArgumentParser() parser.add_argument('--name', help="the test compile output dir") parser.add_argument('--cmaddr', help="IP:port for CS system") args = parser.parse_args() # Get matrix dimensions from compile metadata with open(f"{args.name}/out.json", encoding='utf-8') as json_file: compile_data = json.load(json_file) # Matrix dimensions N = int(compile_data['params']['N']) M = int(compile_data['params']['M']) # Number of PEs in program width = int(compile_data['params']['width']) # Construct A, x, b A = np.arange(M*N, dtype=np.float32) x = np.full(shape=N, fill_value=1.0, dtype=np.float32) b = np.full(shape=M, fill_value=2.0, dtype=np.float32) # Calculate expected y y_expected = A.reshape(M,N)@x + b # Construct a runner using SdkRuntime runner = SdkRuntime(args.name, cmaddr=args.cmaddr) # Get symbols for A, x, b, y on device A_symbol = runner.get_id('A') x_symbol = runner.get_id('x') b_symbol = runner.get_id('b') y_symbol = runner.get_id('y') # Load and run the program runner.load() runner.run() # Copy A, x, b to device runner.memcpy_h2d(A_symbol, np.tile(A, width), 0, 0, width, 1, M*N, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) runner.memcpy_h2d(x_symbol, np.tile(x, width), 0, 0, width, 1, N, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) runner.memcpy_h2d(b_symbol, np.tile(b, width), 0, 0, width, 1, M, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) # Launch the init_and_compute function on device runner.launch('compute', nonblock=False) # Copy y back from device y_result = np.zeros([M*width], dtype=np.float32) runner.memcpy_d2h(y_result, y_symbol, 0, 0, width, 1, M, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) # Stop the program runner.stop() # Ensure that the result matches expectations np.testing.assert_allclose(y_result, np.tile(y_expected, width), atol=0.01, rtol=0) print("SUCCESS!") ``` First, note that one more parameter is read from the compile output, `width`. The host code uses this to specify how many PEs it must copy tensors to and from. Now take a closer look at the `memcpy_h2d` calls: ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} runner.memcpy_h2d(A_symbol, np.tile(A, width), 0, 0, width, 1, M*N, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) runner.memcpy_h2d(x_symbol, np.tile(x, width), 0, 0, width, 1, N, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) runner.memcpy_h2d(b_symbol, np.tile(b, width), 0, 0, width, 1, M, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) ``` Each of `A`, `x`, and `b` needs to be copied to each PE in the program rectangle. But `memcpy_h2d` does not perform a broadcast; it takes its input array and distributes it within the region of interest (ROI) based on the `order` parameter. Here, `np.tile` duplicates each array `width` times. In the first `memcpy_h2d`, the input array `np.tile(A, width)` is a 1D array formed by duplicating `A` `width` times, so the full input array’s size is `M*N*width`. The ROI is specified by `0, 0, width, 1`, meaning the copy goes to a row of `width` PEs beginning at PE (0, 0). `M*N` elements are copied to each PE. Because the order is `ROW_MAJOR`, the result is that PE (0, 0) receives the first `M*N` elements of the tiled array, PE (1, 0) receives the next `M*N` elements, and so on. Thus, each PE receives an identical `M*N` elements corresponding to a copy of `A`. When `y` is copied back from the device, `memcpy_d2h` proceeds similarly: ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} y_result = np.zeros([M*width], dtype=np.float32) runner.memcpy_d2h(y_result, y_symbol, 0, 0, width, 1, M, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) ``` The output array `y_result` has size `M*width`, since each of the `width` PEs copies back the `M` elements of `y`. The copied-back result is tested for correctness across all PEs by comparing `y_result` to a tiled `y_expected`. See [A Complete Program](/csl/tutorials/gemv-01-complete-program) for an explanation of the remaining arguments. ## Compile and Run the Program This compile command adds one additional compile time parameter to specify the width of the program rectangle: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} $ cslc layout.csl --fabric-dims=11,3 --fabric-offsets=4,1 --params=M:4,N:6,width:4 --memcpy --channels=1 -o out $ cs_python run.py --name out ``` The run command stays the same. You should see a `SUCCESS!` message at the end of execution. ## Exercises In this program, each PE is computing an identical GEMV. Modify the program so that each PE receives different values for the input tensors `A`, `x`, and `b`, and check that the computed outputs `y` are correct. ## Next Now that multiple PEs have been introduced into the program, instead of duplicating the GEMV problem between them, the next tutorial distributes the work for computing a single GEMV. # 6. Routes and Fabric DSDs Source: https://sdk.cerebras.ai/csl/tutorials/gemv-06-routes-1 Use fabric DSDs and routes to distribute a GEMV computation across multiple PEs and pass data between them via the WSE fabric. Now that multiple PEs have been introduced into the program, instead of duplicating the GEMV problem between them, this tutorial distributes the work for computing a single GEMV. ## Learning Objectives After completing this tutorial, you should know how to: * Use fabric DSDs `fabout_dsd` and `fabin_dsd` to send and receive data between PEs * Utilize asynchronous builtin operations on fabric DSDs * Define a local task which is activated by a `local_task_id` ## Example Overview Your program will run on two processing elements (PE). This tutorial demonstrates the program with a simulated fabric consisting of a 9 x 3 block of PEs. The program will first copy `b` into the left PE’s `y` array. Then, it will copy the left half of `A`’s columns into the left PE, and the right half of `A`’s columns into the right PE. Similarly, it will copy the first `N/2` elements of `x` into the left PE, and the last `N/2` elements of `x` into the right PE. Each PE will then compute `A*x` for its local pieces of `A` and `x`. Thus, both PEs perform a matrix-vector product for an `M x N/2` matrix. The PEs will increment their local `y` arrays by this result. The left PE then sends its `y` array to the right PE, and the right PE increments its local `y` array by the received values. Because the left `y` array contained the contribution from `b`, the final summed `y` on the right PE is the GEMV result. The host then copies `y` off of the right PE. ### Problem Steps Visually, this program consists of the following steps: **1. Host copies b into y array of left PE.** Diagram of the host copying b into the y array of the left PE **2. Host copies left N/2 columns of A to left PE, right N/2 columns to right PE.** Diagram of the host copying the left N/2 columns of A to the left PE and the right N/2 columns to the right PE **3. Host copies first N/2 elements of x to left PE, last N/2 elements to right PE.** Diagram of the host copying the first N/2 elements of x to the left PE and the last N/2 elements to the right PE **4. Host launches function to compute GEMV.** Diagram of the host launching a function to compute the GEMV **5. Each PE increments local y by local portion of matrix-vector product Ax.** Diagram of each PE incrementing local y by its local portion of the matrix-vector product Ax **6. Left PE sends local y to right PE, and right PE increments y by received values.** Diagram of the left PE sending local y to the right PE, which increments y by the received values **7. Right PE now contains final result y. Host copies back y from right PE.** Diagram of the right PE holding the final result y, which the host then copies back ## Write the CSL What needs to change in the layout to distribute the GEMV between two PEs? 1. Define several new parameters for the two PE programs. This includes a `pe_id`, used to differentiate between the left and right PEs, and a color, which is used to route data between the PEs. 2. Set the color configuration on both PEs for the color used to send the left PE’s `y` array to the right PE. Take a look at the new `layout.csl`, included below. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // matrix dimensions on each PE param M: i16; param N: i16; // Colors const send_color: color = @get_color(0); // Color used to send/recv data between PEs // This example only uses 2 PEs const memcpy = @import_module("", .{ .width = 2, .height = 1, }); layout { // PE coordinates are (column, row) @set_rectangle(2, 1); // Left PE (0, 0) @set_tile_code(0, 0, "pe_program.csl", .{ .memcpy_params = memcpy.get_params(0), .M = M, .N_per_PE = N / 2, .pe_id = 0, .send_color = send_color }); // Left PE sends its result to the right @set_color_config(0, 0, send_color, .{.routes = .{ .rx = .{RAMP}, .tx = .{EAST} }}); // Right PE (1, 0) @set_tile_code(1, 0, "pe_program.csl", .{ .memcpy_params = memcpy.get_params(1), .M = M, .N_per_PE = N / 2, .pe_id = 1, .send_color = send_color }); // Right PE receives result of left PE @set_color_config(1, 0, send_color, .{.routes = .{ .rx = .{WEST}, .tx = .{RAMP} }}); // export symbol names @export_name("A", [*]f32, true); @export_name("x", [*]f32, true); @export_name("y", [*]f32, true); @export_name("compute", fn()void); } ``` There are two `@set_tile_code` calls, one for the left PE (0, 0), and one for the right PE (1, 0). Both PEs take a new parameter, `N_per_PE`, equal to `N / 2`. This is the number of columns of `A` that each PE will receive and operate on. Both PEs also receive as a parameter a `pe_id`: the left PE has `pe_id` 0, and the right PE has `pe_id` 1. `pe_program.csl`, covered next, shows how `pe_id` parameterizes the behavior of the program. There are also two `@set_color_config` calls, to set the configuration of `send_color` on each PE: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @set_color_config(0, 0, send_color, .{.routes = .{ .rx = .{RAMP}, .tx = .{EAST} }}); ... @set_color_config(1, 0, send_color, .{.routes = .{ .rx = .{WEST}, .tx = .{RAMP} }}); ``` The router of each PE has five directions: `RAMP`, `NORTH`, `SOUTH`, `EAST`, `WEST`. The cardinal directions refer to the routers of neighboring PEs: `NORTH` is the PE directly above the current PE, and so on. `RAMP` refers to the connection between a PE’s router and its compute element (CE). When setting a route for a color on a given PE, the receive `rx` and transmit `tx` fields are from the perspective of the router. Thus, receiving from the `RAMP` means that the compute element is sending data up to the fabric, where it can then be transmitted across the fabric. For the left PE (0, 0), `send_color` will send up the PE’s `RAMP` to the fabric, and then transmit data to the `EAST`. For the right PE (1, 0), `send_color` will receive data from the `WEST` on the fabric (i.e., from the left PE), and then transmit it down the `RAMP` to its compute element. Now take a look at the new `pe_program.csl`, included below. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} param memcpy_params; // Matrix dimensions param M: i16; param N_per_PE: i16; // ID of PE (0 is left, 1 is right) param pe_id: i16; // Colors param send_color: color; // Color used to send/recv data between PEs // Queue IDs const send_color_oq = @get_output_queue(2); const send_color_iq = @get_input_queue(2); // Task ID used by a local task to unblock cmd stream const exit_task_id: local_task_id = @get_local_task_id(9); // memcpy module provides infrastructure for copying data // and launching functions from the host const sys_mod = @import_module("", memcpy_params); // 48 kB of global memory contain A, x, y var A: [M*N_per_PE]f32; // A is stored column major var x: [N_per_PE]f32; var y: [M]f32; // DSDs for accessing A, b, y // A_dsd accesses column of A var A_dsd = @get_dsd(mem1d_dsd, .{ .base_address = &A, .extent = M }); var y_dsd = @get_dsd(mem1d_dsd, .{ .base_address = &y, .extent = M }); // ptrs to A, x, b, y will be advertised as symbols to host var A_ptr: [*]f32 = &A; var x_ptr: [*]f32 = &x; var y_ptr: [*]f32 = &y; // Compute gemv fn gemv() void { // Loop over all columns of A for (@range(i16, N_per_PE)) |i| { // Calculate contribution to A*x from ith column of A, ith elem of x @fmacs(y_dsd, y_dsd, A_dsd, x[i]); // Move A_dsd to next column of A A_dsd = @increment_dsd_offset(A_dsd, M, f32); } } fn send_right() void { const out_dsd = @get_dsd(fabout_dsd, if (@is_arch("wse3")) .{ .extent = M, .output_queue = send_color_oq } else .{ .fabric_color = send_color, .extent = M, .output_queue = send_color_oq }); // After fmovs is done, activate exit_task to unblock cmd_stream @fmovs(out_dsd, y_dsd, .{ .async = true, .activate = exit_task_id }); } fn recv_left() void { const in_dsd = @get_dsd(fabin_dsd, .{ .extent = M, .input_queue = send_color_iq }); // After fadds is done, activate exit_task to unblock cmd stream @fadds(y_dsd, y_dsd, in_dsd, .{ .async = true, .activate = exit_task_id }); } // Call gemv function and send/ receive partial result y fn compute() void { gemv(); if (pe_id == 0) { send_right(); } else { recv_left(); } } task exit_task() void { sys_mod.unblock_cmd_stream(); } comptime { // When exit_task_id is activated, exit_task will execute @bind_local_task(exit_task, exit_task_id); // On WSE-3, the output queue must be explicitly bound to a color. // Input queues must be bound to a color on both architectures. @initialize_queue(send_color_oq, if (@is_arch("wse3")) .{ .color = send_color } else .{}); @initialize_queue(send_color_iq, .{ .color = send_color }); @export_symbol(A_ptr, "A"); @export_symbol(x_ptr, "x"); @export_symbol(y_ptr, "y"); @export_symbol(compute); } ``` In addition to the new parameters `N_per_PE`, `pe_id`, and `send_color`, this code also introduces `exit_task_id`, the first value of type `local_task_id`, covered a bit later. The `A` array now has size `M*N_per_PE` instead of `M*N`, since each PE only stores half the columns. To make the data transfer easier, `A` is now stored column-major instead of row-major. Notice that `A_dsd` now accesses `M` contiguous elements, instead of `M` elements strided by the row size, since it's now stored column-major. The `gemv` function operates almost identically to before, except it only loops over `N_per_PE` columns instead of `N` columns. Since `A` is now column-major, `@increment_dsd_offset` must increment by the length of an entire column instead of by one element. Note that on the left PE, `y` already contains the elements of `b` before `gemv` executes. ### Fabric DSDs and Async Operations The `compute` function, which is called from the host, first calls `gemv` to compute the local contribution to `y` on each PE. Then, the left PE calls `send_right`, while the right PE calls `recv_left`. `send_right` defines a `fabout_dsd`, which is used to send wavelets to the fabric along the color `send_color`. Note that this `fabout_dsd` is given the extent `M`, since the goal is to send the `M` elements of `y` along the fabric. On WSE-3, a fabric DSD is bound to a color indirectly via its queue, so `.fabric_color` is omitted from the WSE-3 form; on WSE-2 the color is specified directly on the DSD. The `@fmovs` operation copies the `M` elements accessed by `y_dsd` into `out_dsd`. The `.async = true` field makes this operation asynchronous. The `.activate` field specifies a `local_task_id` to activate when this operation completes. When this operation completes, `exit_task_id` will be activated. `recv_left` defines a `fabin_dsd` to receive the wavelets sent along `send_color`. The `@fadds` operation here increments the right PE’s `y_dsd` by the elements received in `in_dsd`. Thus, after this operation, `y_dsd` contains the final GEMV result. This builtin also executes asynchronously, and activates `exit_task_id` when complete. Whenever using fabric DSDs in builtin operations, always make these operations execute asynchronously. Using fabric DSDs synchronously can result in poor performance or deadlocks. ### Tasks and Activatable Task IDs Now, what does activating `exit_task_id` do? In the comptime block, the `@bind_local_task` builtin binds `exit_task_id` to the task `exit_task`. When `exit_task_id` is activated, `exit_task`, which unblocks the `memcpy` command stream, executes. This task must execute on both PEs before control is returned to the host. ## Write the Host Code The host code must: 1. Copy `b` into the left PE’s `y` array 2. Copy the left halves of `A` and `x` to the left PE, and the right halves to the right PE 3. After the device kernel completes, copy `y` back from the right PE The following sections explain some features of the new `run.py`, shown below. ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} #!/usr/bin/env cs_python import argparse import json import numpy as np from cerebras.sdk.runtime.sdkruntimepybind import SdkRuntime, MemcpyDataType, MemcpyOrder # pylint: disable=no-name-in-module # Read arguments parser = argparse.ArgumentParser() parser.add_argument('--name', help="the test compile output dir") parser.add_argument('--cmaddr', help="IP:port for CS system") args = parser.parse_args() # Get matrix dimensions from compile metadata with open(f"{args.name}/out.json", encoding='utf-8') as json_file: compile_data = json.load(json_file) # Matrix dimensions N = int(compile_data['params']['N']) M = int(compile_data['params']['M']) # Construct A, x, b A = np.arange(M*N, dtype=np.float32).reshape(M,N) x = np.full(shape=N, fill_value=1.0, dtype=np.float32) b = np.full(shape=M, fill_value=2.0, dtype=np.float32) # Calculate expected y y_expected = A@x + b # Size of N dimension on each PE N_per_PE = N // 2 # Construct a runner using SdkRuntime runner = SdkRuntime(args.name, cmaddr=args.cmaddr) # Get symbols for A, x, y on device A_symbol = runner.get_id('A') x_symbol = runner.get_id('x') y_symbol = runner.get_id('y') # Load and run the program runner.load() runner.run() # Copy b into y of PE (0, 0) runner.memcpy_h2d(y_symbol, b, 0, 0, 1, 1, M, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) # Copy A in column major format # PE (0, 0) gets first N/2 columns; PE (1, 0) gets last N/2 columns runner.memcpy_h2d(A_symbol, A.transpose().ravel(), 0, 0, 2, 1, M*N_per_PE, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) # PE (0, 0) gets first N/2 elements; PE (1, 0) gets last N/2 elements runner.memcpy_h2d(x_symbol, x, 0, 0, 2, 1, N_per_PE, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) # Launch the compute function on device runner.launch('compute', nonblock=False) # Copy y back from PE (1, 0) y_result = np.zeros([M], dtype=np.float32) runner.memcpy_d2h(y_result, y_symbol, 1, 0, 1, 1, M, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) # Stop the program runner.stop() # Ensure that the result matches expectations np.testing.assert_allclose(y_result, y_expected, atol=0.01, rtol=0) print("SUCCESS!") ``` ### Copy `b` into `y` of the Left PE `b` is copied into `y` of the left PE here: ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} runner.memcpy_h2d(y_symbol, b, 0, 0, 1, 1, M, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) ``` Notice that the ROI is a single PE, located at (0, 0) in the program rectangle. The right PE (1, 0) is omitted from this `memcpy` call. ### Copy `A` and `x` `A` and `x` are copied to the device as follows: ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} runner.memcpy_h2d(A_symbol, A.transpose().ravel(), 0, 0, 2, 1, M*N_per_PE, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) runner.memcpy_h2d(x_symbol, x, 0, 0, 2, 1, N_per_PE, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) ``` Notice that the ROI is now both PEs, so the `memcpy` calls copy data into both the left and right PE. Because `A` is now stored column-major on the PEs, the `A` matrix is transposed, then flattened to a 1D array with `ravel()`. Each PE gets `M*N_per_PE` elements, so each PE gets `N_per_PE` columns of `A`. Similarly, each PE gets `N_per_PE` elements of `x`. ### Copy Back the Result `y` is copied back from the right PE as follows: ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} y_result = np.zeros([M], dtype=np.float32) runner.memcpy_d2h(y_result, y_symbol, 1, 0, 1, 1, M, streaming=False, order=MemcpyOrder.ROW_MAJOR, data_type=MemcpyDataType.MEMCPY_32BIT, nonblock=False) ``` Notice that the ROI now begins at (1, 0), and contains a single PE. Thus, this `memcpy` call copies back the `M` elements of `y` only from the right PE. Once this call is complete, as in the previous tutorials, the received result is checked for correctness. ## Compile and Run the Program Since this program only uses two PEs, the simulated fabric dimensions are adjusted accordingly: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} $ cslc layout.csl --fabric-dims=9,3 --fabric-offsets=4,1 --params=M:4,N:6 --memcpy --channels=1 -o out $ cs_python run.py --name out ``` The run command stays the same. You should see a `SUCCESS!` message at the end of execution. ## Exercises Instead of using two PEs along the same row to compute this GEMV, try using two PEs along the same column. ## Next Stay tuned for more tutorials! # Build a GEMV Kernel Source: https://sdk.cerebras.ai/csl/tutorials/index Build CSL programs step by step through a series of GEMV tutorials that progressively introduce language features, multiple PEs, and fabric communication. Each successive tutorial introduces additional language features, using a general matrix-vector product (GEMV) as its core computation. Learn preliminaries of CSL syntax. Write a complete CSL program. Use memory data structure descriptors (DSDs) for efficient operations on tensors. Copy tensors from device to host, and vice versa. Use parameters for compile-time specification of your program. Run your program on multiple PEs. Use routes and colors to distribute a single GEMV across multiple PEs. # Compiling and Running Examples Source: https://sdk.cerebras.ai/csl/working-with-code-samples Compile and run Cerebras SDK code examples using the provided command scripts and the CSL compiler and Python host runtime. The [SDK Code Examples](/csl/sdk-examples) section contains CSL programs, the `.csl` files, that each either demonstrate individual features of the language, or solve a larger application problem. A Python script, the `run.py` file, accompanies each program to run it with the simulator. The source for these code examples is hosted at the [SDK examples GitHub repository](https://github.com/Cerebras/sdk-examples). For the GEMV tutorial code examples, we additionally provide step-by-step explanations of the code in the [Tutorials](/csl/tutorials) section. If you’re just getting started, we recommend walking through the step-by-step tutorials in [Tutorials](/csl/tutorials/) to get a fuller explanation of these programs. ## Compile the Code Examples Each code example contains a CSL file as the top-level source file, typically named `layout.csl`. This file may reference additional CSL source files in that directory. Each code example also contains a `commands.sh` script, which contains the commands required to compile and run it. For example, the `tutorials/gemv-01-complete-program/commands.sh` in the [SDK examples repository](https://github.com/Cerebras/sdk-examples/tree/master/tutorials/gemv-01-complete-program) contains: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} cslc ./layout.csl --fabric-dims=8,3 \ --fabric-offsets=4,1 -o out --memcpy --channels 1 cs_python run.py --name out ``` See [CSL Compiler](/csl/csl-compiler) for the compiler options documentation. To compile the program: 1. First, cd into the directory that contains the CSL files. 2. Then run the `cslc` command shown in the `commands.sh` file to compile the program. Note, this command may span multiple lines and produces files with the `elf` extension. For example: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} $ cd tutorials/gemv-01-complete-program/ $ cslc ./layout.csl --fabric-dims=8,3 --fabric-offsets=4,1 -o out --memcpy --channels 1 $ ls out bin east generated out.json west $ ls out/bin out_0_0.elf out_rpc.json ``` ## Run the Program Use the `run.py` Python script that is in the code example directory to execute the compiled program. For example, to run the above compiled program, execute the following command in the `gemv-01-complete-program` directory: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} $ cs_python run.py --name out ``` If the program runs correctly, you will see the message `SUCCESS!` near the end of the output. ## Debug Your Program See [Debugging Guide](/debug/debugging) and [SDK GUI](/debug/sdk-gui). ## Move from Simulation to Hardware After successfully simulating your CSL program, you can run it on hardware by following the guidelines below when using `cslc`: ### Pass the `--arch` Flag Use the `--arch` flag with `cslc` to ensure the compiler targets the appropriate Cerebras system. Allowed values are `--arch=wse2` for WSE-2 architecture and `--arch=wse3` for WSE-3 architecture. The default value is `wse2`. For example: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} cslc --arch=wse2 ./layout.csl --fabric-dims=8,3 \ --fabric-offsets=4,1 -o out --memcpy --channels 1 ``` Note that `wse3` is not yet supported for all example programs. ### Provide `--fabric-dims` When compiling for simulation with `cslc`, the `--fabric-dims` flag can be any bounding box large enough to contain your program’s PEs. However, when compiling for hardware, these dimensions must match your Cerebras system’s fabric dimensions. For example: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} cslc --arch=wse2 ./layout.csl --fabric-dims=757,996 \ --fabric-offsets=4,1 -o out --memcpy --channels 1 ``` ### Provide an IP Address to SdkRuntime To run on the Cerebras system hardware, you must pass the IP and port address of the network-attached Cerebras system to the `cmaddr` argument of the `SdkRuntime` constructor in your `run.py`: ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} runner = SdkRuntime(compile_dir, cmaddr="1.2.3.4:9000") ``` # Debugging Guide Source: https://sdk.cerebras.ai/debug/debugging Debug CSL kernel code using the `csdb` interactive debugger, SDK GUI visualization, and simulator log files. This section describes how to debug your kernel code. See [Compiling and Running Examples](/csl/working-with-code-samples) for how to compile and simulate (run the program). To debug, you can use the following tools: * `csdb` debugger for interactive debugging on hardware. * `sdk_debug_shell visualize`, which launches the SDK GUI to look at all simulation results such as timeline and traces. See [SDK GUI](/debug/sdk-gui) for more information. * `sim.log` simulator log file, which records a cycle-by-cycle log of wavelets or instructions executed on each PE. * The [``](/csl/language/libraries#simprint) device-side library, which emits formatted runtime messages from each PE into `sim.log` (simulator only). Useful for printf-style debugging when a hang or wrong value cannot be diagnosed from static analysis or core dumps. ## `csdb` Debugger CSDB is the Cerebras Software Language Debugger for the Wafer-Scale Engine. CSDB can be run on the host machine for interactive debugging with the Wafer-Scale Engine on issues such as hangs and functional failures. CSDB can also be used to inspect and debug coredumps produced from a simulator run. For debugging on hardware, `csdb` is not supported on legacy CS-2 systems running Cerebras software version 1.6 or lower. Additionally, `csdb` cannot be run via appliance mode on Wafer-Scale Clusters. Below is a tutorial on how to use `csdb` to inspect a coredump from a simulator run. ### Inspect a Coredump with `csdb` This tutorial uses the [GEMV with Checkerboard Pattern](/csl/sdk-examples#gemv-checkerboard) example program. First, to produce corefiles, add the following line to `run.py` right before `runner.stop()` is called: ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} runner.dump_core("corefile.cs1") ``` Note that the specified filename for the coredump MUST be `corefile.cs1` to produce the correct file types for `csdb`. Run `commands.sh` to compile and execute the program and produce the corefiles. The run will produce four files: `corefile.cs1_0`, `corefile.cs1_1`, `corefile.cs1_2`, and `corefile.cs1_3`. Now you're ready to use `csdb`. Start it from the current working directory: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} $ csdb . INFO:csdb: . contains more than one CSL compile directory. Starting debug shell... ``` `csdb` reports multiple compile directories because the top-level compile directory, `out`, contains subdirectories containing compile output for the `memcpy` infrastructure. Select `out` as the compile context, and target the produced corefiles: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} (csdb) context select out (csdb) target create --core-file=corefile.cs1 ``` Run `settings` to see the current working directory, compile context, and target, along with the fabric rectangle dimensions: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} (csdb) settings INFO:csdb: Workdir: . INFO:csdb: Compile context: gemv-checkerboard-pattern/out/ INFO:csdb: Target (core file): corefile.cs1 INFO:csdb: Rectangle(s): INFO:csdb: Rect (x = 0, y = 0, width = 11, height = 6) selected INFO:csdb: Trace: no selected. ``` Run `help` to take a look at the available options: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} (csdb) help Documented commands (type help ): ======================================== context memory register target wavelet image rectangle settings trace workdir Undocumented commands: ====================== exit help quit ``` Select a new subrectangle of PEs, containing only a single PE, and deselect the default rectangle containing the whole fabric. Show the current rectangle(s) with `rectangle show`: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} (csdb) rectangle show INFO:csdb: Rectangle(s): INFO:csdb: Rect (x = 0, y = 0, width = 11, height = 6) selected (csdb) rectangle select 4,1,1,1 (csdb) rectangle show INFO:csdb: Rectangle(s): INFO:csdb: Rect (x = 0, y = 0, width = 11, height = 6) selected INFO:csdb: Rect (x = 4, y = 1, width = 1, height = 1) selected (csdb) rectangle deselect 0,0,11,6 INFO:csdb: Removing ('', Rect (x = 0, y = 0, width = 11, height = 6)) (csdb) rectangle show INFO:csdb: Rectangle(s): INFO:csdb: Rect (x = 4, y = 1, width = 1, height = 1) selected ``` Read memory values of the PE in the rectangle using the memory command: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} (csdb) memory read --address 0x9e0 --length 4 MSGS155 21:48:27 GMT Output will be directed to file 'memory-x4y1w1h1_09e0_09e4.log' MSGS155 21:48:27 GMT Log file: 'memory-x4y1w1h1_09e0_09e4.log' ``` The memory values will be written to the log file specified above. ### Terminology * Compilation context: The directory generated by `cslc`. By default, the name is `out`. * Trace: The directory generated after simulation is run. By default, the name is `simfab_traces`. * Working directory: Also known as workdir, this is the directory to which the debugger writes its output. ### Commands #### Context Command The context command is used to select or change the compile context created by CSL compiler. Once a context is selected, a debug session can be started by creating a target. ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} Usage: context [OPTIONS] COMMAND [ARGS]... Set compile context. Options: --help Show this message and exit. Commands: list List all the compile context in workdir. select Select the directory that contains the ELF binaries as compile... show Show the selected compile context. ``` **Example: list all the contexts and select one** The “.” after “\[2]” in the example below means current directory. ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} (csdb) context list INFO:csdb: [0] orig_hw2/out INFO:csdb: [1] orig_hw/out INFO:csdb: [2] . (csdb) context select orig_hw/out # is same as (csdb) context select [1] ``` #### Memory Command To read from the memory, you must first specify a rectangle and a target. When memory read is called, CSDB will read from a core file or a device. The output of the read is piped into a log file with name beginning with “memory”. All addresses and lengths are in units of 16 bits (2 bytes). ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} Usage: memory [OPTIONS] COMMAND [ARGS]... Read and write to memory locations in PEs. Options: --help Show this message and exit. Commands: read Read memory from a core file or a device. write Write memory to a device in units of 2-bytes. ``` **Example: output from reading tile (4,1) on address 0x09e0, length 4** ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} (4,1) 09e0: 06af 8af0 06af 8060 ``` #### Rectangle Command The purpose of the rectangle command is to allow you to select a rectangle within the fabric. By default, the selected rectangle is the entire fabric. The context must be selected before you can use the rectangle commands. ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} Usage: rectangle [OPTIONS] COMMAND [ARGS]... Select rectangle Options: --help Show this message and exit. Commands: reset Resets the current rectangle to fabric dimension. select Selects a rectangle. show Shows the current rectangle. ``` **Example: select a rectangle on (1,2) w3 h4** ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} (csdb) rectangle select 1,2,3,4 ``` #### Settings Command The settings command is used to see the work directory, compile context, target, rectangle and trace. #### Target Command The purpose of the target command is to create a debug session. It is similar to attaching `gdb` to a process. You can create an interactive debug session by connecting to a CM IP address, or perform a post-mortem debugging by examining a core file. During an interactive debug session, you can use `save-core` to save a core file for examination later. ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} (csdb) target Usage: target [OPTIONS] COMMAND [ARGS]... Connect to CM for interactive debugging or examine a core file. Options: --help Show this message and exit. Commands: create Connects to CM or read core file as target. list List all the core files. save-core Save a core file after connecting to a CM target show Show selected core file. ``` **Example: create an interactive debug session** ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} (csdb) target create --cmaddr 12.34.56.78:9000 ``` **Example: list and load the core file** ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} (csdb) target list INFO:csdb: [0] core-ckpt INFO:csdb: [1] my_try1-ckpt (csdb) target create --core-file core-ckpt ``` #### Trace Command The purpose of the trace command is to specify a directory in which a `simfab_traces` has been generated, so that the `simfab_traces` can be read for wavelet trace information. ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} (csdb) trace --help Usage: trace [OPTIONS] COMMAND [ARGS]... Select trace Options: --help Show this message and exit. Commands: list List all the valid post-run generated directory. select Select the directory that contains post-run traces. show Show the post-run directory that is set. ``` **Example: select current run directory with simfab\_traces** ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} (csdb) trace select . ``` At this point, you can use the `wavelet` command to inspect the wavelet traces of this run. #### Workdir Command The purpose of the workdir command is to specify a directory for output files to be written. ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} (csdb) workdir --help Usage: workdir [OPTIONS] COMMAND [ARGS]... workdir is the directory for debug session. Options: --help Show this message and exit. Commands: select Select a workdir. ``` **Example: select a workdir** ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} (csdb) workdir select path/to/workdir ``` ## sdk\_debug\_shell The `sdk_debug_shell` tool is used to run a smoke test or launch the SDK GUI visualizer. ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} $ sdk_debug_shell --help Usage: sdk_debug_shell [OPTIONS] COMMAND [ARGS] Debugger tool for the Cerebras WSE kernel code. Options: --help Show this message and exit. Commands: smoke Run the smoke tests. visualize Invokes the visualization tool csviz. ``` ### Smoke Test The `smoke` option runs the smoke tests in the specified directory. ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} Usage: sdk_debug_shell smoke [OPTIONS] [CSL_EXAMPLES_DIR]... Run the smoke tests. Options: --help Show this message and exit. ``` ### Visualizer When you use the `visualize` option, the debugger will invoke the [SDK GUI](/debug/sdk-gui) and you can visually inspect the debugging information in a web browser. The default `artifact_dir` is the current directory. ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} $ sdk_debug_shell visualize --help Usage: sdk_debug_shell visualize [OPTIONS] Visualize routing between PEs, post-simulation run results. For example, wavelet trace and instruction trace in a web browser. Options: --artifact_dir PATH --help Show this message and exit. ``` ## Simulator Logs When running in the simulator, you can produce a simulator log file `sim.log` with cycle-by-cycle information about wavelets or instructions. The `SINGULARITYENV_SIMFABRIC_DEBUG` environment variable is used to control the output of `sim.log`. ### Landing Logs `SINGULARITYENV_SIMFABRIC_DEBUG=landing` produces a log of wavelet landings on each PE’s router, giving the cycle and color on which the wavelet lands, the direction from which it landed, its data, and its identity. An example landing log looks like this: ``` @53 P6.1 (hwtile) landing C3 from link R, ctrl=0, idx=0000, data=0000 (+0.000(-15)), half=0, ident=00000E0300000000, lf=0 @55 P5.1 (hwtile) landing C3 from link E, ctrl=0, idx=0000, data=0000 (+0.000(-15)), half=0, ident=00000E0300000000, lf=0 ``` The first line says that on cycle 53, the router of the PE at X=6, Y=1 received a wavelet of color 3 (`C3`) from the ramp (i.e., sent by the CE). The second line says that on cycle 55, the router of the PE at X=5, Y=1, received a wavelet of color 3 from the EAST link (i.e., from the PE at X=6, Y=1). Take a closer look at each entry on the first line and its meaning: * `@53`: The cycle when the landing occurs. The first cycle of the simulation is zero. * `@P6.1`: The coordinates of the PE on which the landing occurs. The coordinates take the format `@PX.Y`, where X=0, Y=0 is the top left corner of the fabric rectangle. * `(hwtile)`: The type of tile implementation. `hwtile` is the full Cerebras microcode execution engine. `iotile` are the links by which data enters or exits the wafer, along the EAST or WEST edges. * `C3`: The color of the landing wavelet. * `link R`: The link from which the wavelet arrived. There are five links: EAST (`E`), WEST (`W`), NORTH (`N`), SOUTH (`S`), and RAMP (`R`). The four cardinal directions refer to the four neighboring PEs, while the RAMP refers to the CE of this PE. * `ctrl=0`: The control bit is not set. If 0, the wavelet is a data wavelet. If 1, a control wavelet. * `idx=0000, data=0000 (+0.000(-15))`: The wavelet interpreted as 16-bit index and data fields. The data field is shown again in `fp16` representation. * `half=0`: The wavelet is not a half-wavelet. On WSE-3, wavelets can be interpreted as 16-bit “half-wavelets.” This field is not present on WSE-2. * `ident=00000E0300000000`: Unique identifier for this wavelet. Notice that in the small example above, both lines in the landing log have the same identifier. The same wavelet that leaves the PE at X=6, Y=1 arrives on the EAST link of the PE at X=5, Y=1 two cycles later. Thus, the `ident` field allows you to trace the flow of wavelets across the fabric. * `lf=0`: The local flip bit is not set. The local flip bit is set by the CE to signal that the switch be advanced (from the RAMP direction to one of the cardinal directions). ### Instruction Trace Logs `SINGULARITYENV_SIMFABRIC_DEBUG=inst_trace` produces an instruction trace that shows which instruction a PE is executing at each cycle. An example instruction trace looks like this: ``` @497 P4.1: Id: 12, Instr: 225, Seq: 0, Pipe: 3, Msg: [IS OP] 0x021c: T01 FMACS Dest:[DDS1] Src0:[S0DS1] Src1:[S1DS1] Src2:R13,R12 @500 P4.1: Id: 12, Instr: 225, Seq: 0, Pipe: 6, Msg: [EX OP] 0x021c: T01 FMACS Dest:3f800000 Src0:00000000 Src1:3f800000 Src2:3f800000 [ ] U0 @501 P4.1: Id: 12, Instr: 225, Seq: 1, Pipe: 6, Msg: [EX OP] 0x021c: T01 FMACS Dest:41500000 Src0:40c00000 Src1:40e00000 Src2:3f800000 [ ] U0 @502 P4.1: Id: 12, Instr: 225, Seq: 2, Pipe: 6, Msg: [EX OP] 0x021c: T01 FMACS Dest:41c80000 Src0:41400000 Src1:41500000 Src2:3f800000 [ ] U0 @503 P4.1: Id: 12, Msg: [EX OP] IDLE @504 P4.1: Id: 12, Instr: 225, Seq: 3, Pipe: 6, Msg: [EX OP] 0x021c: T01 FMACS Dest:42140000 Src0:41900000 Src1:41980000 Src2:3f800000 [ ] U0 ``` The first line says that on cycle 497, an `FMACS` instruction was decoded `[IS OP]` by the PE at X=4, Y=1. The next four lines show this same instruction executing, save for an idle cycle at cycle 503. Take a closer look at each entry on the first line and its meaning: * `@497`: The cycle to which this line refers. The first cycle of the simulation is zero. * `@P4.1`: The coordinates of the PE on which the instruction executes. The coordinates take the format `@PX.Y`, where X=0, Y=0 is the top left corner of the fabric rectangle. * `Id: 12`: The position of the PE in a 1D array. This simulator log comes from a simulation of an 8 x 3 fabric, so the position X=4, Y=1 corresponds to PE 12. * `Instr: 225`: A unique instruction ID. The instruction ID stays with the instruction from beginning to end. Notice that the instruction ID is the same for all instructions in the above simulator log excerpt. * `Seq:` The sequence number of the instruction. For a vector instruction, the sequence number increases while stepping through the elements of the vector. Thus, for a vector instruction that is 100 elements long, the sequence number goes from 0 to 99. * `Pipe: 3`: The execution pipeline stage. On WSE-3, stage 3 is instruction decode, and stage 6 is instruction execution. On WSE-2, stage 2 is instruction decode, and stage 4 is instruction execution. * `Msg: [IS OP]`: The name of the pipeline stage. `[IS OP]` is instruction decode, and `[EX OP]` is instruction execution. * `0x021c`: The address of the instruction in memory. * `T01`: The task ID of the task in which the instruction is executing. On WSE-3, data tasks are bound to input queues, so `T00` to `T07` refer to data tasks and the ID of the input queue to which they are bound. On WSE-2, data tasks are bound to colors, so the ID of a data task can be in the range `T00` to `T23`, and refers to the color to which the task is bound. The task number can also be appended with a microthread ID. For example `T01.UT4` would mean this current instruction is running on microthread 4. * `FMACS`: The name of disassembled instruction. * `Dest:[DDS1] Src0:[S0DS1] Src1:[S1DS1] Src2:R13,R12`: The instruction operands. For the instruction decode pipeline stage, the operand registers are given. `FMACS` has one destination operand and three source operands. The destination operand is in `DDS1`, or destination data structure register (DSR) 1. The first two source operands are also DSR operands, in src0 DSR 1 and src1 DSR 1 respectively. The third source operand uses general purpose registers (GPR) 12 and 13\. This is a 32-bit operation, so the 32-bit scalar operand used for the third source operand uses two GPRs. For instruction execution `[EX OP]`, additional fields are present. The second line above shows: * `[ ]`: Error flags. In the above example, no error flags are set. There are five error cases, one for each position between the square brackets: * `u` = underflow * `o` = overflow * `x` = inexact * `i` = invalid op * `z` = divide by zero For example, `[ o ]` means the instruction encountered an overflow. * `U0`: The SIMD unit(s) involved in the instruction. In a single cycle, the CE can run up to SIMD-4 for WSE-2, and SIMD-8 for WSE-3, depending on the instruction. Because this instruction is a single precision `FMAC`, the instruction can only run in SIMD-1, and thus only one SIMD unit is used. The `[EX OP]` entry above with `IDLE` means that `P4.1` was idle on cycle 503. ### Router Logs `SINGULARITYENV_SIMFABRIC_DEBUG=router` produces a log of the router state and switch advances. An excerpt from an example router log looks like this: ``` @0 P5.2 (hwtile) router: C0 : input switch: 1=/ R/ -> 1=/ R/ (init) @0 P5.2 (hwtile) router: C0 : output switch 1=/ W / -> 1=/ W / (init) @376 P5.2 (hwtile) router: C0 : input switch: 1=/ R/ -> 2=/ R/ (advance) ctrl=1, idx=015F, data=0000 (+0.000(-15)), half=0, ident=0000190000000000 @376 P5.2 (hwtile) router: C0 : output switch 1=/ W / -> 2=/E / (advance) ctrl=1, idx=015F, data=0000 (+0.000(-15)), half=0, ident=0000190000000000 @386 P5.2 (hwtile) router: C0 : input switch: 2=/ R/ -> 3=/ R/ (advance) ctrl=1, idx=015F, data=0000 (+0.000(-15)), half=0, ident=0000190000000002 @386 P5.2 (hwtile) router: C0 : output switch 2=/E / -> 3=/ S / (advance) ctrl=1, idx=015F, data=0000 (+0.000(-15)), half=0, ident=0000190000000002 @396 P5.2 (hwtile) router: C0 : input switch: 3=/ R/ -> 0=/ R/ (advance) ctrl=1, idx=015F, data=0000 (+0.000(-15)), half=0, ident=0000190000000004 @396 P5.2 (hwtile) router: C0 : output switch 3=/ S / -> 0=/ N / (advance) ctrl=1, idx=015F, data=0000 (+0.000(-15)), half=0, ident=0000190000000004 @406 P5.2 (hwtile) router: C0 : input switch: 0=/ R/ -> 1=/ R/ (advance) ctrl=1, idx=015F, data=0000 (+0.000(-15)), half=0, ident=0000190000000006 @406 P5.2 (hwtile) router: C0 : output switch 0=/ N / -> 1=/ W / (advance) ctrl=1, idx=015F, data=0000 (+0.000(-15)), half=0, ident=0000190000000006 ``` The first two lines show that at the very beginning of the simulation, the router of the PE at X=5, Y=2 for color 0 (`C0`) has set its initial switch position to position 1, where `C0` receives from the RAMP and transmits to the WEST. The next two lines show that on cycle 376, the same router received a control wavelet which advanced the switch position from 1 to 2. While the input position did not change: `1=/ R/ -> 2=/ R/`, the output position changed from WEST to EAST: `1=/ W / -> 2=/E /`, so that `C0` now receives from the RAMP and transmits EAST. The rest of the entries describe the data contained in the received control wavelet, in the same format as the landing log. ### Interpret Logs for Programs with `memcpy` Most of the SDK example programs use the `memcpy` library, which, in conjunction with the host runtime `SdkRuntime`, can copy or stream data to and from PEs in your program rectangle, and launch functions in your program rectangle. When looking at the simulator logs, you may be surprised to see colors, resources, and PEs that your program does not explicitly use. These are `memcpy` resources. A few things to look out for: * Programs using `memcpy` are typically compiled with fabric offsets `4,1`, though additional East and West buffers can be introduced to reduce I/O latency. Without buffers, the top left-most PE of your program rectangle is at `P4.1`. * `memcpy` uses colors 21, 22, 23, local task IDs 27, 28, 30, and control task IDs 33, 34, 35, 36, and 37. It also uses microthread 0 (`UT0`), input queue 0, and output queue 0. On WSE-3, `memcpy` additionally uses input queue 1. * Color 21 is used for device-to-host data transfers, color 23 is used for host-to-device data transfers, and color 22 is used for the `memcpy` command sequence. * Functions launched via the `memcpy` kernel launch mechanism will execute within task 22 (`T22`) on WSE-2, since the task which calls these functions is bound to color 22. Functions will execute within task 1 (`T01`) on WSE-3, since the task is bound to input queue 1. # SDK GUI Source: https://sdk.cerebras.ai/debug/sdk-gui Use the `sdk_debug_shell visualize` tool to inspect simulation results including timelines and traces in a browser-based graphical interface. **WSE-3 compatibility**
Instruction traces in the SDK GUI are supported on WSE-3 starting with SDK 2.10. On earlier SDKs (1.4 and below) the GUI runs against a WSE-3 simulator but the instruction-trace pane will be empty; the timeline, routes, and landing logs still work. Other panes (timeline, routes, colors, landing logs) are supported on both WSE-2 and WSE-3 across all current SDK versions. Follow the documentation to launch `sdk_debug_shell visualize`, pointing to your test artifact directory. Open the URL that the command shows. The diagram below shows a summary of the functionality: Overview of the SDK GUI interface The artifacts from your directory will be loaded and shown visually for debugging. ## Change Directory If you wish to load a different directory, enter the new directory and click **Submit**. SDK GUI dialog for entering a new work directory and clicking Submit ## Dark and Light Modes To switch between dark and light modes, click the icons at the top right. Switch to dark mode: SDK GUI in dark mode Switch to light mode: SDK GUI in light mode ## Routes and Colors To view the routes on the fabric, select the color IDs. To view all the routes, click **Select All**. SDK GUI routes and colors panel with color ID selection Expanding/collapsing a color ID shows/hides a list of associated color names. Expanded color ID showing a list of associated color names ## Fabric View Move the fabric around by dragging it. To zoom in and out of the fabric, use the mouse scroll functionality. SDK GUI fabric view showing the PE grid Zoomed view of the fabric in the SDK GUI * To recenter the fabric, click the Re-center button. Re-center button for recentering the fabric view * To zoom into a particular process element (PE), enter a specific **PE Coordinate**. PE Coordinate field for zooming to a specific processing element For pre-execution artifacts, only routing information is available for viewing. For post-execution artifacts, you can view additional information as documented below. Double-click a PE to open the instruction trace, source code, wavelet trace and symbols panels. Panels opened by double-clicking a PE: instruction trace, source code, wavelet trace, and symbols To zoom in and center the selected PE, click the bottom-right of the page. Selected PE highlighted in the fabric view Zoomed and centered view of the selected PE When hovering over the fabric, the coordinates of the currently hovered PE are displayed on the bottom info pane: PE coordinates displayed in the bottom info pane on hover If your work directory uses a fabric size greater than 20 x 20, there is a scalable fabric view: Scalable fabric view for large fabric sizes The full dimensional view is shown on the right fabric. The left fabric reflects the area highlighted by the dark square on the right fabric. A dark square is drawn by mouse click on the right fabric surface. Left fabric pane reflecting the area highlighted on the right fabric Dark square marking the highlighted area on the right fabric Right fabric pane showing the full dimensional view Left, right, top, and bottom buttons allow you to navigate across the full dimensional view. Directional navigation buttons for the full dimensional fabric view Control moving the selected factory area to the top-left by hitting the **Re-corner** button. Re-corner button for moving the selected area to the top-left The location of the selected routes on the selected fabric area on the left is highlighted in red on the right / full view of the fabric. Full view of the fabric on the right pane Selected routes highlighted in red on the full fabric view To zoom in/out/reset the left fabric, use the zoom control buttons on the top-right. Zoom control buttons for the left fabric pane Mouse scroll is not supported to zoom in or out for a scalable fabric view. The fabric cannot be moved by dragging for a scalable fabric view. ## Instruction Traces You can view the instructions, tasks, statistics, uthreads or micro threads from the instruction trace panel. To view the instructions, from the dropdown list, select **Instructions**. Instruction trace panel showing the Instructions view To view the tasks, from the dropdown list, select **Tasks**. Instruction trace panel showing the Tasks view To view the statistics related to the instructions, from the dropdown list, select **Statistics**. Instruction trace panel showing the Statistics view To view the micro-threads, from the dropdown list, select **Uthreads**. Instruction trace panel showing the Uthreads view ## Source Code To view the source code for an instruction, follow the steps below: * To view all the instructions, in the instruction trace panel, select **Instructions**. * Click an instruction that has the source code file and line number. In the **Source Code** panel, you are navigated to the line number corresponding to the instruction. Source Code panel showing the line corresponding to a selected instruction ## Wavelet Traces Wavelet traces are displayed in the wavelet trace panel. * From the color dropdown, select a color. To select all colors, click **Select All**. * From the wavelet format dropdown, select a format. This formats the header and data fields of the wavelet. Wavelet trace panel showing traces for a selected color ## Timeline: Wavelet Traces To show the timeline for a wavelet, click a color and a PE. The timeline for wavelet appears in the bottom pane. Timeline pane showing wavelet traces for a selected color and PE The timeline has a list of PEs to the left and its corresponding timeline area with wavelet entries to the right. The axis on the top shows the cycle range of the view area. The rectangular bar represents a wavelet. The width of the rectangular bar represents how long the wavelet takes to travel from the current PE to the destination PE. The status of the wavelet is represented by the color of the bar. ### Legend Terminology * **Active**: Sender PE can send a wavelet without any backpressure or delay. Shown as green. * **Delayed**: The switch/router (represented by the circle) output is congested due to multiple input sources. Shown as yellow. * **Backpressure**: Destination PE is busy and cannot receive any wavelets from sender PE. The sender PE is back-pressured. Shown as red. * **Idle**: There is no wavelet in the router. Shown as empty spaces in the timeline for a PE. * **Wavelet Type**: : + **Control wavelet**: Wavelet that changes control, such as routing to a compute element (CE). Shown as blue. * **Data wavelet**: Wavelet that sends data. The active, delayed, and back-pressured wavelets are data wavelets. ## Timeline: Instruction Traces To view the timeline of instruction traces, click the instruction trace icon next to the header title. Timeline view of instruction traces The timeline area has rectangular bars depicting a timeline item. The timeline item depicts the number of cycles taken to execute a line of source code. There are tasks running in a PE that executes the source code. The color of the timeline item is linked to the task that is executing. The color legend below the timeline area shows the task numbers corresponding to the timeline color. ## Timeline: Combined View Combined view shows both the wavelet traces and the instruction traces associated with the PE and color. Whenever a new selection is made, two items are added to the timeline area: one corresponding to the timeline of wavelet traces, and the other corresponding to the timeline of instruction traces. Combined timeline view showing wavelet and instruction traces ## Timeline: Navigation Controls You can delete a PE from the timeline view. Hover over the PE and click the “trash” icon. To clear all the PEs from the timeline view, click the “trash” icon in the upper-right corner. By default, the timeline view shows only the first few hundred cycles. Not all cycles are shown in the initial view. To view the left or right, you can scroll the timeline. There is a red rectangular bar above the timeline area. The red bar represents the current view area. Dragging the red bar left or right moves the timeline view correspondingly. You can also zoom out to show more cycles in the view or zoom in to show fewer cycles in the view. Zoom controls for navigating the timeline view There are four buttons related to zoom in the top-right corner. The “**-**” button zooms out; the “**+**” button zooms in. To bring the zoom back to the default zoom, use the **Reset** button. The **Zoom to fit** button brings all the cycles into view. The green bars indicate Active wavelet traces that don’t have a delay. (See Legend terminology above). Because the rectangular bars representing the timeline are scaled, the green bars may not be visible in this view depending on the time range selected. Another way to zoom the timeline view is to resize the red bar that is at the top of the timeline. By resizing the red bar, you can zoom the timeline view below in and out. ## Debug Pane To show the details in the debug pane to the right, hover over a timeline item. For wavelets, the detail pane shows the cycles that the wavelet took to reach the next PE, a wavelet ID that is shown for debug purposes, the status of the wavelet, the direction (East, West, North, South), and whether it is a control wavelet. The pane also shows index and data fields. Debug pane showing details for a selected timeline item To keep a timeline item in the debug pane, click on it. Clicking anywhere on the timeline area unselects the timeline item and removes it from the debug pane. ## Settings Dialog You can enter a new work directory in the textbox at the top of the application. When you click the **Submit** button, the **Settings Dialog** appears. Settings dialog with compile directory, run directory, core file path, and source directory fields The Settings dialog has four settings: **Compile directory**, **Run directory**, **Core file path**, and **Source directory**. The application pre-populates these settings for you when you provide a work directory. Of these four settings, only the **Compile directory** is required. Three of the settings contain dropdown menus: the **Compile directory**, **Run directory**, and **Core file path**. Within the work directory, there may be multiple compile directories, run directories and core file paths. You can switch between multiple values for each in a dropdown. You can also clear the dropdown and enter a custom value for these settings. If there is no run directory provided, then the app cannot show the instruction trace, wavelet trace or the timeline view. If there is no core file path provided, the routes shown in the fabric are compile time routes, and not the routes of the actual simulation. To save the settings, click the **Save** button. To revert to the previous settings, click the **Close** button. ## Terminal To view the **Terminal** page, click the terminal icon on the top right. This page provides a UNIX command console for the root folder where the SDK-GUI web server is running. Terminal page providing a UNIX command console ## Panes You can toggle panes on and off. Pane toggle controls for showing and hiding panes Use the resize handle to resize panes. Resize handle for adjusting pane size You can expand panes to a separate page. To do this, click the **Expand** icon. Expand icon for opening a pane as a separate page When you click the expand icon, the pane appears as a full page, as shown below. Pane expanded to a full page view You can move from one pane to another. Click the icon on the left navigation bar. Left navigation bar for switching between panes * The first icon (Home) shows the original view with all the panes. * The second icon shows the instructions trace pane along with the source code pane. * The third icon shows the source code pane. * The fourth icon shows the wavelet trace pane. # Install the Cerebras SDK Source: https://sdk.cerebras.ai/installation-guide Install the Cerebras SDK Singularity container and configure your environment to compile and run CSL programs on the fabric simulator or a legacy Cerebras system installation. If you're using a Cerebras Wafer-Scale Cluster running in appliance mode, see [Running SDK on a Wafer-Scale Cluster](/appliance-mode). ## Request Access Request access to the Cerebras SDK Singularity container at [cerebras.ai/developers/sdk-request](https://www.cerebras.ai/developers/sdk-request). You can also find a repository of example programs [on GitHub](https://github.com/Cerebras/sdk-examples). Your download includes a tarball (`Cerebras-SDK-2.10.0-{build_id}.tar.gz`) containing the SDK software and a `sha256sum.txt` file for verifying the download integrity. ## Prerequisites Before you begin, make sure your system has the following: * **Apptainer or SingularityCE** — A container platform capable of running Singularity containers. See the [Apptainer Quick Start](https://apptainer.org/docs/user/main/quick_start.html) or [SingularityCE Quick Start](https://docs.sylabs.io/guides/latest/user-guide/quick_start.html) for setup instructions. * **Overlay filesystem** — Available by default on Linux kernel 3.18 and later. See the [kernel documentation](https://www.kernel.org/doc/html/latest/filesystems/overlayfs.html) for details. * **Bash** — The [Bash shell](https://www.gnu.org/software/bash/) must be available on your system. ## Install the SDK If you're on an Apple Silicon Mac, see [Apple Silicon Mac Installation](#apple-silicon-mac-installation) instead. In the directory where you downloaded the tarball, run the checksum verification: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} cd /my/install/location sha256sum --check sha256sum.txt ``` If the output shows `OK`, the file is intact. Set environment variables for your install location and tarball path. Both should be absolute paths: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} SDK_INSTALL_LOCATION=/my/install/location SDK_INSTALL_PATH=$SDK_INSTALL_LOCATION/cs_sdk SDK_TAR_PATH=/path/to/Cerebras-SDK-2.10.0-{build_id}.tar.gz ``` Create the install directory and extract the tarball: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} mkdir -p $SDK_INSTALL_PATH tar -C $SDK_INSTALL_PATH -xvf $SDK_TAR_PATH ``` After extraction, your `$SDK_INSTALL_PATH` directory should contain: | File | Description | | ------------------------------ | ----------------------------------------------------------------------------------------------- | | `sdk-cbcore-{build_id}.sif` | The SDK container image (\~3 GB) | | `cslc` | CSL compiler script | | `cs_python` | Script for running Python code within the container's Python environment | | `csdb` | Hardware debug tools script | | `cs_readelf` | Cerebras alternative to `readelf` | | `sdk_debug_shell` | Simulation debug tools, including the smoke test and SDK GUI | | `csl-extras-{build_id}.tar.gz` | Example programs, tutorials, and extras (including CSL syntax highlighters for Vim and VS Code) | | `cerebras-software-eula.pdf` | End User License Agreement | | `sdk-gui-LICENSE.txt` | License for the GUI tool | Add the SDK to your current session and persist it across future sessions: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} export PATH=$SDK_INSTALL_PATH:$PATH echo 'export PATH='$SDK_INSTALL_PATH':$PATH' >> ~/.bashrc ``` Extract the examples and run the smoke test to verify that everything compiles and runs correctly: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} cd $SDK_INSTALL_PATH tar -xzvf csl-extras-{build_id}.tar.gz sdk_debug_shell smoke csl-extras-{build_id} ``` A successful test ends with: ```text theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} SUCCESS! PASSED SMOKE CHECK COMPLETED SUCCESSFULLY ``` To speed up the smoke test and reduce disk usage, set `SINGULARITYENV_CSL_SUPPRESS_SIMFAB_TRACE=1` before running it. This skips generating `simfab_traces` used by the SDK GUI. Unset this variable before running examples you plan to visualize in the GUI. To confirm the GUI works, run the `gemm-collectives_2d` example: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} cd $SDK_INSTALL_PATH/csl-extras-{build_id}/csl_examples/benchmarks/gemm_collectives_2d ./commands.sh ``` Then launch the GUI: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} sdk_debug_shell visualize ``` This outputs one or more URLs. Open one in your browser to view the GUI: ```text theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} Click this link to open URL: http://:8000/sdk-gui?session_id=... ``` After selecting a PE, you should see a visualization like this: SDK Debug GUI showing a selected PE ## Apple Silicon Mac Installation On Apple Silicon Macs, the SDK runs inside a Linux virtual machine powered by [Lima](https://lima-vm.io/). You can use either Rosetta (recommended for performance) or QEMU for x86 emulation. Choose your approach below, then continue with the shared setup steps. Tested with Lima >= 2.0.0 and macOS >= 26.2. See the [Lima documentation](https://lima-vm.io/docs/config/multi-arch/#fast-mode-2) for details. ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} brew install lima ``` Save the following as `config.yml`. This creates an ARM virtual machine with Apptainer installed, using Rosetta to run the x86\_64 SDK container. Your Mac's home directory and `/tmp/lima` are mounted as writable: ```yaml theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} minimumLimaVersion: 2.0.0 containerd: system: false user: false provision: - mode: system script: | #!/bin/bash set -eux -o pipefail command -v apptainer >/dev/null 2>&1 && exit 0 # Workaround for https://github.com/apptainer/apptainer/issues/2027 echo "kernel.apparmor_restrict_unprivileged_userns = 0" >/etc/sysctl.d/99-userns.conf sysctl --system # add the "Official PPA for Apptainer" add-apt-repository -y ppa:apptainer/ppa apt-get update apt-get install -y apptainer probes: - script: | #!/bin/bash set -eux -o pipefail if ! timeout 30s bash -c "until command -v apptainer >/dev/null 2>&1; do sleep 3; done"; then echo >&2 "apptainer is not installed yet" exit 1 fi hint: See "/var/log/cloud-init-output.log" in the guest images: - location: "https://cloud-images.ubuntu.com/releases/noble/release-20251213/ubuntu-24.04-server-cloudimg-arm64.img" arch: "aarch64" digest: "sha256:a40713938d74aaec811f74cb1fa8bfcb535d22e26b2a0ca1cc90ad9db898feb9" - location: https://cloud-images.ubuntu.com/releases/noble/release/ubuntu-24.04-server-cloudimg-arm64.img arch: aarch64 mounts: - location: "~" writable: true - location: "/tmp/lima" writable: true mountType: "virtiofs" vmOpts: vz: rosetta: enabled: true binfmt: true ``` QEMU x86 emulation is noticeably slower than Rosetta, and emulation bugs are possible. Use Rosetta if your system supports it. ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} brew install lima qemu ``` Save the following as `config.yml`. This creates an x86\_64 virtual machine with SingularityCE installed. Your Mac's home directory and `/tmp/lima` are mounted as writable: ```yaml theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} vmType: "qemu" images: - location: "https://cloud-images.ubuntu.com/releases/22.04/release-20231130/ubuntu-22.04-server-cloudimg-amd64.img" arch: "x86_64" digest: "sha256:7edc2eccf1e34df23d9561b721b6fed381c3b6e8c916c91c71bbce7b8488b496" arch: "x86_64" cpuType: x86_64: "max" ssh: loadDotSSHPubKeys: false mounts: - location: "~" writable: true - location: "/tmp/lima" writable: true containerd: system: false user: false provision: - mode: system script: | #!/bin/bash set -eux -o pipefail export DEBIAN_FRONTEND=noninteractive apt-get update -y && apt install -y squashfs-tools-ng wget https://github.com/sylabs/singularity/releases/download/v4.0.2/singularity-ce_4.0.2-jammy_amd64.deb apt install -y ./singularity-ce_4.0.2-jammy_amd64.deb probes: - script: | #!/bin/bash set -eux -o pipefail if ! timeout 30s bash -c "until command -v singularity >/dev/null 2>&1; do sleep 3; done"; then echo >&2 "singularity is not installed yet" exit 1 fi hint: See "/var/log/cloud-init-output.log" in the guest ``` ### Create the VM and Run the SDK After completing either option above: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} limactl start ./config.yml --name cs_sdk ``` Extract the SDK tarball somewhere under your Mac's home directory, then start a shell in the VM: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} # Start a shell inside the VM limactl shell cs_sdk # Add the SDK to your PATH (use the absolute Mac path — the home directory # inside the VM is different from your Mac's home directory) export PATH=/Users/your-username/path/to/sdk:$PATH ``` From here, you can run SDK examples within the VM. Lima automatically forwards ports, so if you launch the SDK GUI inside the VM, you can access it in your Mac's browser at `127.0.0.1:8000/sdk-gui`. ## Add Python Packages The `cs_python` script runs host code from the Python environment inside the container. To add packages to this environment, save the following helper script as `sdk_install_python_package.sh`: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} #!/bin/bash py_path=$(realpath $1) package_name=$2 SINGULARITYENV_PYTHONPATH="$(realpath $py_path)" cs_python -c " import subprocess import sys package='$2' subprocess.check_call([sys.executable, '-m', 'pip', 'install', '--target=$py_path', package]) " echo -e "Please do\nexport SINGULARITYENV_PYTHONPATH=\"$SINGULARITYENV_PYTHONPATH\"\nbefore using cs_python" ``` Then install a package into a local directory: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} mkdir MY_LOCAL_PY_PATH bash /path/to/sdk_install_python_package.sh ./MY_LOCAL_PY_PATH ``` Before running `cs_python`, set the environment variable so it can find your installed packages: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} export SINGULARITYENV_PYTHONPATH=$(realpath $MY_LOCAL_PY_PATH) ``` # Documentation Changelog Source: https://sdk.cerebras.ai/sdk-release-notes/sdk-doc-updates Track the history of documentation changes for the Cerebras SDK, including new guides, API references, and updated language documentation. * Documentation for SDK 2.10.0 released, including the [Version 2.10.0](/sdk-release-notes/sdk-rel-notes-cumulative#version-2100) release notes. * Documentation for SDK 1.4.0 released, including the [Version 1.4.0](/sdk-release-notes/sdk-rel-notes-cumulative) release notes. * Added reference documentation for the SDK appliance mode API. See [SDK Appliance API Reference](/api-docs/appliance-api). * Added reference documentation for the new `SdkLayout` program layout specification API. See [SdkLayout API Reference](/api-docs/sdklayout-api). * Improved documentation for the `mem4d_dsd` type. See [Data Structure Descriptors](/csl/language/dsds). * Added documentation on controlling appliance logging level for appliance mode support. See [Running SDK on a Wafer-Scale Cluster](/appliance-mode). * Added documentation of router simulator logs. See [Debugging Guide](/debug/debugging). * Added documentation on installing additional Python packages into the container Python environment. See [Install the Cerebras SDK](/installation-guide). * Documentation for SDK 1.3.0 released, including the [Version 1.3.0](/sdk-release-notes/sdk-rel-notes-cumulative) release notes. * Added documentation for interpreting simulator logs. See [Debugging Guide](/debug/debugging). * Documentation for SDK 1.2.0 released, including the [Version 1.2.0](/sdk-release-notes/sdk-rel-notes-cumulative) release notes. * Documentation for SDK 1.1.0 released, including the [Version 1.1.0](/sdk-release-notes/sdk-rel-notes-cumulative) release notes. * Updated installation documentation to include information for using Lima and QEMU on Apple Silicon Macs. * Updated appliance mode documentation to reflect changes to compile artifacts output in Cerebras ML Software 2.1. * Documentation for SDK 1.0.0 released, including the [Version 1.0.0](/sdk-release-notes/sdk-rel-notes-cumulative) release notes. * Updated debugging documentation to reflect `csdb` functionality. See [Debugging Guide](/debug/debugging). * Documentation for SDK 0.9.0 released, including the [Version 0.9.0](/sdk-release-notes/sdk-rel-notes-cumulative) release notes. * Documentation for SDK 0.8.0 released, including the [Version 0.8.0](/sdk-release-notes/sdk-rel-notes-cumulative) release notes. * Documentation for SDK 0.7.0 released, including the [Version 0.7.0](/sdk-release-notes/sdk-rel-notes-cumulative) release notes. * Documentation for SDK 0.6.0 released, including the [Version 0.6.0](/sdk-release-notes/sdk-rel-notes-cumulative) release notes. * Cholesky decomposition example added. * Hadamard Product example added to demonstrate batched execution mode in the `CSELFRunner` runtime. * Information on CSL generics added to Language documentation. See [Generics](/csl/language/generics). * Details of the new runtime added to Tensor Streaming Implementations. See [Host Runtime and Tensor Streaming](/tensor-streaming). * Updated GEMV examples were added, including one demonstrating collective communications. * New tutorials section was added, with step-by-step instructions on CSL programs. See [Tutorials](/csl/tutorials). * Sample codes for collective communications and the debug library are added. * Documentation for SDK 0.5.1 released, including the [Version 0.5.1](/sdk-release-notes/sdk-rel-notes-cumulative) release notes. * Documentation for SDK 0.4.0 released, including the [Version 0.4.0](/sdk-release-notes/sdk-rel-notes-cumulative) release notes. * The GEMV example was added. * Documentation for SDK 0.3.1 released, including the [Version 0.3.1](/sdk-release-notes/sdk-rel-notes-cumulative) release notes. * Documentation for SDK 0.2.1 released, including the [Version 0.2.1](/sdk-release-notes/sdk-rel-notes-cumulative) release notes. * Updated [Version 0.2.0](/sdk-release-notes/sdk-rel-notes-cumulative) release notes with the note on FFT and wide-multiplication examples in the `cslang/benchmarks` directory. * Updated [Version 0.2.0](/sdk-release-notes/sdk-rel-notes-cumulative) release notes with Linux and image compatibility items. * Documentation for SDK 0.2.0 released. * Initial availability of the Pre-release 0.2.0 SDK developer documentation. # Release Notes Source: https://sdk.cerebras.ai/sdk-release-notes/sdk-rel-notes-cumulative Review the cumulative release notes for the Cerebras SDK, including new language features, bug fixes, and breaking changes for each version. ## Version 2.10.0 Released 15 March 2026 The SDK version numbering scheme has been updated to match the Cerebras ML Software release numbering scheme. SDK 2.10 is the SDK version following 1.4.0, and has been tested for functionality with the Cerebras Wafer-Scale Cluster appliance running Cerebras ML Software 2.10. ### New Features and Enhancements * CSL language and compiler enhancements: * Introduces `anyopaque` type, representing a value whose size and representation is unknown. This is mostly useful to describe type-erased pointers: `*anyopaque` is analogous to C's `void*`. * Introduces arbitrary-width integers. `iN` and `uN` describe a signed or unsigned integer of `N` bits, where `N` is any nonnegative integer, for example `u3`, `i4`, `u1`, `i0`, and `u128`. Only integer types with bit widths of 16 or 32 are ABI-sized. Non-ABI-sized integer types cannot be used as task parameters, and certain hardware-specific operations, such as DSD builtins, may require ABI-sized types. * Introduces `packed struct`. Fields are arranged as a sequence of bits with no gaps in between. All fields must have defined bit width. For example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const some_struct = packed struct { x: i16, y: f32, }; ``` * Introduces union types. Untagged unions, analogous to `union` in C, are supported. Unions may be `packed`, analogous to structs. For example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const my_union = packed union { full : u16, bits: packed struct { low: u8, high: u8, } }; ``` * Introduces integer and float widening coercions. Coercions among fixed-width integer types are now supported if the destination type can represent all possible values of the source type. Likewise for fixed-width float types. For example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var medium: i16 = 42; var large: i32 = medium; // OK: i16 can be widened to i32 var small: i8 = medium; // ERROR: i8 cannot represent all i16 values ``` * Introduces coercion of anonymous structs to named struct types. For example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const Point = struct { x: i32, y: i32, }; var p: Point = .{.x = 0, .y = 0}; ``` * Introduces `*T` to `*[1]T` coercion and `*[N]T` to `[*]T` in peer type resolution. The type `*T` will now automatically coerce to `*[1]T`, and the type `*[N]T` will now automatically coerce to `[*]T`. * Introduces `sr` type representing stride registers, and the `@get_sr` builtin. * Introduces `@load_to_dsr_xdsr_sr` builtin, used to load `mem4d_dsd` values to explicit DSRs, XDSRs, and SRs. `@load_to_dsr` with just a `mem4d_dsd` is no longer allowed; an XDSR and SR must also be allocated using `@load_to_dsr_xdsr_sr`. For example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const dsr = @get_dsr(dsr_dest, 0); const xdsr = @get_xdsr(1); const sr1 = @get_sr(0); const sr2 = @get_sr(1); const sr3 = @get_sr(2); var dsd: mem4d_dsd; task foo() void { @load_to_dsr_xdsr_sr(dsr, xdsr, .{sr1, sr2, sr3}, dsd); } ``` * Introduces FIFO full and empty actions via new `.full_action` (WSE-3 only) and `.empty_action` fields of the `@allocate_fifo` config struct. These are actions taken when a push occurs on a full FIFO or a pop occurs from an empty FIFO, respectively. Accepts options `test_or_suspend`, `terminate`, `suspend` (WSE-3 only), and `fault` (WSE-3 only). If omitted, `test_or_suspend` is the default. For example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} var fifo_buffer = @zeros([32]i16); const fifo = @allocate_fifo( fifo_buffer, .{ .empty_action = .{ .terminate = true }, .full_action = .{ .fault = true } } ); ``` * Introduces task rotation via `@bind_rotating_tasks`. For example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const iq = @get_input_queue(0); const main_id = @get_data_task_id(iq); task main(data: u32) void {} task alt() void {} comptime { @bind_rotating_tasks(main, alt, main_id, .{.limit = 10}); @initialize_queue(iq, .{.color = c}); @set_control_task_table(); } ``` * Introduces `src0` DSRs as operands of 1-source builtins on WSE-3. * Introduces support for additional DSD builtin operand combinations on WSE-3, including `dsr_src0, scalar, dsr_src0`. Additionally, removes the restriction that `dsr_dest` and `dsr_src0` operands must have the same number. * DSR, XDSR, and SR values now support `==` and `!=` comparison. `@get_int` now supports DSR, XDSR, and SR values. * `==` and `!=` can now compare `type`. The `@is_same_type` builtin is now deprecated. * DSD builtins are now restricted to no more than one `fabin_dsd` operand on WSE-3, properly reflecting the WSE-3 architectural restriction. * Introduces dense mode support for queues on WSE-3. See [@initialize\_queue](/csl/language/builtins#@initialize_queue). * Introduces support for `circbuf_dsd`. Currently only supported at comptime and must be manually loaded to DSR+XDSR using `@load_to_dsr_xdsr`. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const circbuf = @get_dsd(circbuf_dsd, .{.base_address = &B, .extent = A_size, .wraparound = 5}); const src1 = @get_dsr(dsr_src1, 2); const xdsr_id = @get_xdsr(3); comptime { @load_to_dsr_xdsr(src1, xdsr_id, circbuf); } ``` Wraparound can be inferred from size of `base_address` if it is an array. * Introduces support for `linksection` on functions. For example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} export fn has_special_section() linksection("dcache_sect") void { gv = 42; } ``` * Changed default SIMD setting on WSE-2 to `simd_32`. Changed behavior of SIMD settings on WSE-3: the valid options are now `simd_off` and `simd_max` (default). * Trailing comma is now allowed in enum members. * Change to export compatibility of integers and enums: only export compatible when ABI-sized (16 or 32 bits). Removes export compatibility for `i8`, `u8`, `i64`, `u64`. * Default `--max-inlined-iterations` limit is increased. * Improved pointer representation consistency: values of pointer type are now always encoded as byte addresses. * CSL library enhancements: * Introduces `` library, providing utility functions `lo16`, `hi16`, `lo32`, and `hi32`. * Introduces `` library, providing `comptime_int_to_string` and `fmt` functions. * Introduces `` library, providing type introspection functions such as `is_unsigned_int`, `is_signed_int`, `is_float`, `is_numeric`, `word_size_of`, `byte_size_of`, `bit_size_of`, `is_dsd`, `is_dsr`, and more. * Introduces `math.subsat` for saturating subtraction, with `f16`, `f32`, and generic variants. * Introduces `tile_config.filters` functions for half-wavelets. Only supported on WSE-3. * `SdkRuntime` host runtime enhancements: * Introduces `cslc_prefix` option in `SdkLayout`. * Introduces `f16_type` option in `SdkLayout.compile` to specify the 16-bit floating point format (`F16`, `BF16`, or `CB16`). * Introduces `libs` option in `SdkLayout.compile` to specify additional library search paths for the compiler. * Example programs: * Introduces a new tutorial example to demonstrate reusing output queues on WSE-3 with `@queue_flush`. See [topic-16-queue-flush](/csl/sdk-examples#topic-16-queue-flush). ### Resolved Issues * Instruction traces in the SDK GUI are now supported on WSE-3. * Fixes possible read-after-write hazard in `tile_config.teardown.exit()`. * Fixes bug in which a runtime call to a function could cause incorrect evaluation of comptime calls to the same function. * Fixes bug where a dereference expression passed to `@set_dsd_base_address` could crash the compiler. * Fixes bug where DSDs could not be passed as comptime function parameters. * Fixes `SdkLayout` crash with `simprint` library. * Fixes exception propagation in `SdkRuntime` when a simulator failure occurs. * Fixes potential segfault in `SdkLayout` caused by stale reference to `CodeRegion`. ### Known Issues * The `25-pt-stencil` and `histogram-torus` benchmark examples are not supported on WSE-3. * The bandwidth of memory transfers saturates at around 8 IO channels. ### Deprecations * `fabric_color` in `@get_dsd` is no longer required, and emits a deprecation warning for all `fabin_dsd` and `fabout_dsd` on WSE-3, and for `fabin_dsd` that specify `input_queue` on WSE-2. * Automatic input queue initialization for `fabin_dsd` operands of DSD/DSR builtins has been removed. * The `comptime_struct` type has been removed. See [Migrating from comptime\_struct and @concat\_structs](/csl/comptime-struct-migration) for a migration guide. * The `@concat_structs` builtin has been removed. See [Migrating from comptime\_struct and @concat\_structs](/csl/comptime-struct-migration) for a migration guide. ## Version 1.4.0 Released 26 May 2025 The Cerebras Wafer-Scale Cluster appliance running Cerebras ML Software 2.4 supports SDK 1.3. See the [SDK 1.3 documentation](https://cerebras-sdk-docs-130.netlify.app). The Cerebras Wafer-Scale Cluster appliance running Cerebras ML Software 2.5 supports SDK 1.4, the current version of SDK software. ### New Features and Enhancements * (beta) New `SdkLayout` program layout specification API: * Introduces a new `SdkLayout` Python API for specifying program layout. This API allows the user to define rectangular code regions, define color routing and switching, automatically allocate colors, and automatically route between code regions. * Introduces several example programs demonstrating the use of the `SdkLayout` API. See the list of new example programs below. * Introduces new documentation for this API. See [SdkLayout API Reference](/api-docs/sdklayout-api). * This API is in **beta**. The `memcpy` API for data transfers and remote kernel launches is not currently supported. CSL libraries with their own internal color routing are not currently supported. * CSL language and compiler enhancements: * `@map` now supports explicit DSR arguments. DSR input arguments must be `dsr_src1` and DSR output arguments must be `dsr_dest`. All DSR arguments should be loaded with the `single_step` property set. For example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} param inDSR: dsr_src1; param outDSR: dsr_dest; task foo() void { // Compute the square-root of each element of `memDSD` and // send it out to `faboutDSD`. @load_to_dsr(inDSR, memDSD, .{.single_step = true}); @load_to_dsr(outDSR, faboutDSD, .{.single_step = true}); @map(math_lib.sqrt_f16, inDSR, outDSR); } ``` * Introduces support for `cb16` (`cbfloat16`) and `bfloat16` (bfloat) 16-bit floating point types, and the associated `@fp16()` builtin. See [@fp16](/csl/language/builtins#@fp16) and [Type System in CSL](/csl/language/types). `cbfloat16` is a Cerebras-specific 16-bit floating point format with a 6-bit exponent and 9-bit explicit mantissa. * On WSE-3, introduces support for microthread priority via the `.priority` field in `@get_dsd` for `fabin_dsd` and `fabout_dsd`, and in `@allocate_fifo`. See [Data Structure Descriptors](/csl/language/dsds). * CSL library enhancements: * Introduces 3D FFT kernel library. See [``](/csl/language/libraries#\). * Introduces `tile_config.input_queue_status` and `tile_config.output_queue_status` to query input and output queue full/ empty status registers. See [input\_queue\_status](/csl/language/libraries#input_queue_status) and [output\_queue\_status](/csl/language/libraries#output_queue_status). * `SdkRuntime` host runtime enhancements: * Introduces the `SdkRuntime` direct link API functions `send` and `receive`, which are used to stream data into or out of the wafer via program input and output ports. This API can be used with `SdkLayout` as demonstrated in [SdkLayout 4: Host-to-device and device-to-host data streaming](/csl/sdk-examples#sdklayout-04-h2d-d2h). See [SdkRuntime API Reference](/api-docs/sdkruntime-api). * Example programs: * Introduces a series of example programs demonstrating the new `SdkLayout` API: * [SdkLayout 1: Introduction](/csl/sdk-examples#sdklayout-01-introduction) introduces the `SdkLayout` API with a single-PE program. * [SdkLayout 2: Basic routing](/csl/sdk-examples#sdklayout-02-routing) demonstrates color routing with the `SdkLayout` API and automatic color allocation. * [SdkLayout 3: Ports and connections](/csl/sdk-examples#sdklayout-03-ports-and-connections) demonstrates automatic routing between code regions. * [SdkLayout 4: Host-to-device and device-to-host data streaming](/csl/sdk-examples#sdklayout-04-h2d-d2h) demonstrates the use of the `SdkRuntime` direct link API with `SdkLayout` to create host-to-device and device-to-host streams. * [SdkLayout 5: Generalized matrix-vector multiplication (GEMV)](/csl/sdk-examples#sdklayout-05-gemv) implements a full GEMV program with the `SdkLayout` API. * Introduces an example using the 3D FFT kernel library. See [3D FFT](/csl/sdk-examples#fft-3d). ### Resolved Issues * Fixes incorrect parsing of CSL if statements whose body is an assignment without braces (e.g. `if (cond) lhs = rhs;`) * On WSE-2, fixes bug in which `@set_color_config` did not support all 6 available filters. Previously, only the first four were available. * Fixes potential stall caused by sending many small data transfers via `SdkRuntime`. * Appliance mode compilation via `SdkCompiler` no longer allocates a system while compiling. * Appliance mode SDK jobs launched via `SdkCompiler`, `SdkLauncher`, or `SdkRuntime` now exit gracefully. ### Known Issues * The `25-pt-stencil`, `histogram-torus`, and `spmv-hypersparse` benchmark examples are not supported on WSE-3. * Instruction traces in the SDK GUI are not supported on WSE-3. * The bandwidth of memory transfers saturates at around 8 IO channels. ### Deprecations * In CSL, calling a task is now an error. Only functions may be called. Tasks must be activated. * In CSL, dereference or access of pointers into config space is now illegal. The `@get_config` and `@set_config` builtins should be used instead. * WSE-1 is no longer supported. ## Version 1.3.0 Released 13 December 2024 ### New Features and Enhancements * CSL language and compiler enhancements: * For DSD definitions, a tensor access expression is now shorthand for a `comptime_struct` with `extent`, `stride`, and `base_address` fields. DSDs can now also be specified using these fields directly, for example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // These two definitions are equivalent: var my_dsd = @get_dsd(mem1d_dsd, .{ .extent = 10, .stride = 2, .base_address = &my_arr }); var my_dsd = @get_dsd(mem1d_dsd, .{ .tensor_access = |i|{10} -> my_arr[2*i] }); ``` `stride` is an optional parameter with default value 1. See [tensor\_access](/csl/language/dsds#tensor_access) for more information. * Memory DSD properties can now take runtime values when using the individual field specification format. However, `mem4d_dsd` extent and stride must still be comptime known. * Introduces inline functions, which are expanded during semantic analysis. See [Syntax of CSL](/csl/language/syntax) for more information. * Introduces labeled `break` and the ability to break values from blocks. See [Syntax of CSL](/csl/language/syntax) for more information. * Improves performance of CSL’s parser, potentially improving program compile times. * Improves DSR allocation diagnostics when using DSDs. Upon failure to allocate, diagnostics now contain information about operations that prevent a DSR from being allocated. * CSL library enhancements: * Introduces a `` library which provides wrappers around DSD op builtins that select an appropriate builtin depending on the underlying data types, enabling more concise and flexible code when supporting multiple data types. See [``](/csl/language/libraries#\) for more information. * `SdkRuntime` host runtime enhancements: * Introduces a strided version of `memcpy_h2d` for strided host-to-device data transfers. See `memcpy_h2d_stride` in [SdkRuntime API Reference](/api-docs/sdkruntime-api). * Introduces row and column broadcast variants of `memcpy_h2d` for host-to-device row and column broadcasts. See `memcpy_h2d_colbcast` and `memcpy_h2d_rowbcast` in [SdkRuntime API Reference](/api-docs/sdkruntime-api). Also see the example program [Host-to-Device Broadcast Test](/csl/sdk-examples#row-col-broadcast). * Example programs: * Introduces a new example program [Host-to-Device Broadcast Test](/csl/sdk-examples#row-col-broadcast) to demonstrate row and column broadcasts for host-to-device data transfers. ### Resolved Issues * Fixes an issue in the `` library where messages were limited to only 16 wavelets. The maximum message size is 32 wavelets. * Fixes bugs in the `` library in which `encode_payload()` could index out of bounds, and not set `NOCE` bit on unused commands. * Fixes a bug in which sequential `@map` operations within a function would not be able to reuse DSRs. ### Known Issues * The `25-pt-stencil`, `histogram-torus`, and `spmv-hypersparse` benchmark examples are not yet supported on WSE-3. * Instruction traces in the SDK GUI are not yet supported on WSE-3. * The bandwidth of memory transfers saturates at around 8 IO channels. ## Version 1.2.0 Released 28 June 2024 The Cerebras Wafer-Scale Cluster appliance running Cerebras ML Software 2.2 supports SDK 1.1. See the [SDK 1.1 documentation](https://cerebras-sdk-docs-110.netlify.app). The Cerebras Wafer-Scale Cluster appliance running Cerebras ML Software 2.3 supports SDK 1.2, the current version of SDK software. ### New Features and Enhancements * CSL language and compiler enhancements: * Introduces `inline` `for`-loops, which are unrolled at compile time. The body of an `inline` `for`-loop may assign to a `comptime` variable. For example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn length(comptime array: anytype) comptime_int { comptime var result = 0; // This loop will be inlined. inline for (array) |v| { result += 1; } return result; } ``` * Introduces the `@queue_flush` and `@set_empty_queue_handler` builtin for WSE-3. See [@queue\_flush](/csl/language/builtins#@queue_flush). * Runtime `on_control` values in DSD operations are now supported. For example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} fn f(out: fabout_dsd, in: fabin_dsd, act_id: local_task_id) void { @fmovh(out, in, .{ .async = true, .on_control = .{ .activate = act_id }}); } ``` * Improves `void` type semantics, enabling optionally specified module parameters and function arguments. * Significantly improves compile times for large programs. Compilation time for full-wafer programs may be improved as much as 10x. * CSL library enhancements: * Introduces a `` library for runtime debug printing to the simulator log. See [``](/csl/language/libraries#\). * Introduces a `` library for creating control wavelet payloads. See [``](/csl/language/libraries#\). * Introduces a `` library for WSE-3 point-to-point communication. See [``](/csl/language/libraries#\). * Introduces the `queue_flush` module within the `` library for WSE-3, which can be used for querying when a queue is flushed and to exit the flushed state. See [queue\_flush](/csl/language/libraries#queue_flush). * Adds WSE-3 support to the `collectives_2d` library. * `SdkRuntime` host runtime enhancements: * Adds WSE-3 support for `memcpy` streaming mode. * Example programs: * Reorganizes and updates all tutorial example programs with WSE-3 support. * Introduces two new tutorial examples for switches, demonstrating use of the `` library. See [Topic 6: Switches](/csl/sdk-examples#topic-06-switches) and [Topic 7: Switches and Control Entrypoints](/csl/sdk-examples#topic-07-switches-entrypt). * Introduces a new tutorial example to demonstrate the `` library. See [Topic 13: Simprint Library](/csl/sdk-examples#topic-13-simprint). * Introduces a new tutorial example to demonstrate color swapping on WSE-2. See [Topic 14: Color Swap](/csl/sdk-examples#topic-14-color-swap). * Adds WSE-3 support to the `wide-multiplication`, `residual`, `mandelbrot`, `gemv-collectives_2d`, `gemv-checkerboard-pattern`, `gemm-collectives_2d`, `7pt-stencil-spmv`, `bicgstab`, `conjugateGradient`, `preconditionedConjugateGradient`, and `powerMethod` benchmark example programs. ### Resolved Issues * Adds `memcpy` streaming support for WSE-3. * Adds WSE-3 support for the `` library. * Fixes potential bug in the `` library related to reconfiguring the library’s colors. * Fixes potential bug in the `` library related to reconfiguring the library’s colors. ### Known Issues * The `25-pt-stencil`, `histogram-torus`, and `spmv-hypersparse` benchmark examples are not yet supported on WSE-3. * The SDK GUI is not yet supported on WSE-3. * The bandwidth of memory transfers saturates at around 8 IO channels. ### Deprecations * The deprecated `@get_color_id` builtin to get the numerical value of a color is now removed. Use `@get_int` instead. * Use of `@get_color` on any ID other than a routable color ID is no longer supported. * `tile_config.reg_ptr` has been removed. Use `@get_config` and `@set_config` for direct manipulation of config space addresses. ## Version 1.1.0 Released 10 April 2024 This version of the Cerebras SDK is the first with experimental support for the WSE-3, the third generation Cerebras architecture. The WSE-3 is the wafer-scale processor powering the CS-3 Cerebras system. The Cerebras Wafer-Scale Cluster appliance running Cerebras ML Software 2.0 supports SDK 0.9. See the [SDK 0.9 documentation](https://cerebras-sdk-docs-090.netlify.app). The Cerebras Wafer-Scale Cluster appliance running Cerebras ML Software 2.1 supports SDK 1.0. See the [SDK 1.0 documentation](https://cerebras-sdk-docs-100.netlify.app). The Cerebras Wafer-Scale Cluster appliance running Cerebras ML Software 2.2 supports SDK 1.1, the current version of SDK software. ### New Features and Enhancements * CSL language and compiler enhancements: * Introduces initial support for WSE-3. * Introduces `ut_id` type and `@get_ut_id` builtin for representing microthread IDs. This feature is WSE-3 only. * Introduces runtime `@get_config` and `@set_config` support. * Introduces `i64` and `u64` types, and support in ``, ``, and `` libraries. Like `i8` and `u8`, these types are not allowed in memory DSD tensors or `@map`, nor as arguments to tasks. * CSL `memcpy` library enhancements: * `memcpy/get_params` no longer requires specifying a `LAUNCH` color for host kernel launch support. * The `@rpc` builtin is no longer necessary for host kernel launch support. The RPC server is now created internally. * Other CSL library enhancements: * Introduces `reset_tsc_counter()` function in `Diagram of I/O channels connecting the WSE's East and West edges to the host The SDK `memcpy` infrastructure uses additional PEs around your kernel to route tensor data and also adds a small executable component to the kernel PEs. In addition to a halo around the kernel, the additional support PEs consume three columns on the West of the kernel and two columns on the East. Diagram of the support PEs the memcpy infrastructure adds around a kernel `SdkRuntime` supports up to 16 I/O channels, and can further reduce the I/O latency by buffer insertion on either side of the core kernel. ## Set Up the memcpy Infrastructure The `memcpy` infrastructure of `SdkRuntime` moves data on and off the device in one of two modes: * `streaming` mode delivers data as a sequence of wavelets. Your kernel counts the wavelets as they arrive and acts once the full tensor has been received. * `copy` mode writes data directly into device memory without notifying your kernel — you copy the tensor in first, then launch a kernel function to act on it. For example, given a tensor `A` and a function `f` that transforms it, computing `f(A)` looks different depending on the mode: * In `streaming` mode, you'd define a wavelet-triggered data task that receives `A` and calls `f` once all of `A` has arrived; * In `copy` mode, you'd copy `A` onto the device first, then launch a kernel that calls `f`. To instantiate and use the `memcpy` infrastructure, you'll need to do the following: Pass `--memcpy` and `--channels=k` to `cslc` (the CSL compiler), where `k` is an integer between 1 and 16 specifying the number of I/O channels to use. Specify `--fabric-dims=dim_x,dim_y` and `--fabric-offsets=x,y` to `cslc`, where `width` and `height` are your program rectangle's dimensions, such that: ```text theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} dim_x >= 7 + width dim_y >= 2 + height x >= 4 y >= 1 ``` In your top-level layout CSL file, instantiate `memcpy` parameters by importing `` with an `@import_module()` statement. This specifies everything the infrastructure needs, including `width` and `height` of the kernel and any colors needed for `streaming` mode. Pass the `memcpy` params to the PE program in the `@set_tile_code` call. These params are parameterized by the PE's `x` coordinate in the program rectangle. In your PE program, instantiate the `memcpy` module by importing ``. Altogether, instantiating `memcpy` infrastructure in the top-level CSL file and the PE program will resemble the following example: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // in top-level CSL file const memcpy = @import_module("", .{ .width = width, .height = height }); layout { @set_rectangle(1, 1); @set_tile_code(0, 0, "pe_program.csl", .{ .memcpy_params = memcpy.get_params(0) }); } // in PE program CSL file pe_program.csl param memcpy_params: comptime_struct; const sys_mod = @import_module("", memcpy_params); ``` The `memcpy` infrastructure reserves the following resources. The compiler and runtime cannot detect all resource conflicts, so do not use these in your own program: | Resource | Reserved IDs | | ---------------- | -------------------------------------------- | | Colors | 21, 22, 23 | | Local task IDs | 27, 28, 30 | | Control task IDs | 33, 34, 35, 36, 37 | | Microthread | 0 | | Input queue | 0 (WSE-2 and WSE-3); additionally 1 on WSE-3 | | Output queue | 0 | Do not set or modify the routing of an input, output tensor color or kernel launch color. The compiler configures the routing pattern implicitly. If you modify those routing patterns, the behavior is undefined. ## Use Streaming Mode To use `streaming` mode, you must specify colors for host-to-device and device-to-host streaming. Input streaming parameters are prefixed with `MEMCPYH2D_DATA_` and output streaming parameters are prefixed with `MEMCPYD2H_DATA_`, followed by the tensor ID (an integer in the range 1-4) and an `_ID` suffix — for example, `MEMCPYH2D_DATA_1_ID`. Unused colors should be omitted, and only four colors per direction are allowed. You can block and unblock the input tensor colors to overlap computation and communication. Here's an example instantiation of a program in the top-level CSL file using colors for `memcpy` streaming: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // in top-level CSL file // Compile-time IDs for memcpy streaming colors param MEMCPYH2D_DATA_1_ID: i16; param MEMCPYH2D_DATA_2_ID: i16; param MEMCPYD2H_DATA_1_ID: i16; param MEMCPYD2H_DATA_2_ID: i16; // Generate colors from IDs const MEMCPYH2D_DATA_1: color = @get_color(MEMCPYH2D_DATA_1_ID); const MEMCPYH2D_DATA_2: color = @get_color(MEMCPYH2D_DATA_2_ID); const MEMCPYD2H_DATA_1: color = @get_color(MEMCPYD2H_DATA_1_ID); const MEMCPYD2H_DATA_2: color = @get_color(MEMCPYD2H_DATA_2_ID); const memcpy = @import_module("", .{ .width = width, .height = height, .MEMCPYH2D_1=MEMCPYH2D_DATA_1, .MEMCPYH2D_2=MEMCPYH2D_DATA_2, .MEMCPYD2H_1=MEMCPYD2H_DATA_1, .MEMCPYD2H_2=MEMCPYD2H_DATA_2 }); ``` You must also pass the input/output tensor ID and color value pairs to `cslc` as parameters. Here `` is the tensor index (1-4) and `` is the numeric ID of the color to use for that tensor: ```bash theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} --params=MEMCPYH2D_DATA__ID: --params=MEMCPYD2H_DATA__ID: ``` For example, to use color 1 for input tensor 1 and color 16 for output tensor 1: `--params=MEMCPYH2D_DATA_1_ID:1 --params=MEMCPYD2H_DATA_1_ID:16`. To stream the data into the device, you can either use a data task to read the data from the input tensor color or use a microthread (a lightweight hardware thread that can issue DSD operations without occupying the main compute engine) to read the data from a `fabin_dsd`. To bind a data task to an input color, call `@bind_data_task` at compile time: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} const MEMCPYH2D_1_TASK_ID = @get_data_task_id(MEMCPYH2D_DATA_1); comptime { // Task reads data on color MEMCPYH2D_DATA_1 @bind_data_task(memcpyh2d_data_1_task, MEMCPYH2D_DATA_1_TASK_ID); } ``` You can send data to an output tensor color using a `fabout_dsd`. For instance, assuming `my_fabout_dsd` and `my_mem_buf_dsd` are already defined: ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} @mov32(my_fabout_dsd, my_mem_buf_dsd, .{.async=true}); ``` ## Use Copy Mode To use `copy` mode to copy data to/from the device, you have to define the symbols for the tensors to be copied. For example, the following code defines a pointer `ptr_A` pointing to tensor `A`, and exports it. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // in top-level CSL file const memcpy = @import_module("", .{ .width = width, .height = height }); layout { @set_rectangle(1, 1); @set_tile_code(0, 0, "pe_program.csl", .{ .memcpy_params = memcpy.get_params(0) }); // export symbol names @export_name("A", [*]f32, true); } // in PE program CSL file pe_program.csl param memcpy_params: comptime_struct; const sys_mod = @import_module("", memcpy_params); var A = @zeros([4]f32); var ptr_A : [*]f32 = &A; comptime { @export_symbol(ptr_A, "A"); } ``` ## Launch Kernels We can additionally use `memcpy` to launch a kernel function. The following is an example of the kernel launching protocol. This program exports two functions to the host: `f1` and `f2`. ```csl theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} // in top-level CSL file const memcpy = @import_module("", .{ .width = width, .height = height }); layout { @set_rectangle(1, 1); @set_tile_code(0, 0, "pe_program.csl", .{ .memcpy_params = memcpy.get_params(0) }); // export symbol names @export_name("f1", fn()void); @export_name("f2", fn()void); } // in PE program CSL file pe_program.csl param memcpy_params: comptime_struct; const sys_mod = @import_module("", memcpy_params); fn f1() void { // do something } fn f2(my_arg: f32) void { // do something else } comptime { @export_symbol(f1); @export_symbol(f2); } ``` ## Use Buffers The compiler can insert buffers in the infrastructure to reduce the latency of the I/O. The buffer stores the wavelets from the I/O for one row of PEs while the core program rectangle is busy and cannot process the wavelets from the I/O. In other words, the buffer acts like a `prefetch` from the point of view of the computation. There are two kinds of buffers: one stores the data for host-to-device transfers, and the other stores the data for device-to-host transfers. The width of the former is configured by `--width-west-buf`, and the width of the latter is configured by `--width-east-buf`. By default, `--width-west-buf=0` and `--width-east-buf=0`, i.e., no buffers are inserted. `--width-west-buf=k` means `k` columns of PEs are inserted to the West of the core kernel, and each PE can buffer 46 KB of data. If you have 500 PEs in a row, then 46 KB can buffer 23 wavelets per PE (recall that each wavelet holds 32 bits of data). If you want to stream or copy a tensor of size 100 per PE, then `--width-west-buf=5` can buffer the whole tensor. When compiling with `--width-west-buf=k` and `--width-east-buf=p`, you must specify `--fabric-offsets=x,y` such that `x >= 4 + k` and `y >= 1`, and `--fabric-dims=dim_x,dim_y` such that `dim_x >= x + width + 3 + p` and `dim_y >= y + height + 1`, where `width` and `height` are the width and height of the program rectangle. ## SdkRuntime Host API See [SdkRuntime API Reference](/api-docs/sdkruntime-api) for full documentation of the `SdkRuntime` Python host API. The `SdkRuntime` Python host API supports memory transfers and kernel launches through the functions `memcpy_h2d()`, `memcpy_d2h()` and `launch()`: `memcpy_h2d()` is used for host-to-device data transfers, `memcpy_d2h()` is used for device-to-host data transfers, and `launch()` is used for kernel launches. Each function can be a blocking or nonblocking call, depending on the parameter `nonblock` of the API. If blocking mode (`nonblock=False`) is selected, the API waits until the operation is done. Otherwise, the function returns before the operation even starts. `SdkRuntime` can aggregate multiple nonblocking operations together to reduce the latency. However, you must take care to avoid race conditions in nonblocking mode. For example, if you have two `memcpy_d2h()` calls to the same destination, the content of the destination is undefined if both operations are nonblocking. ### Instantiate SdkRuntime You'll need to import the `SdkRuntime` module, as well as the `MemcpyDataType` and `MemcpyOrder` modules for specifying data type and ordering of tensors. To create an `SdkRuntime` object, pass the directory which contains the ELF files produced by the compiler, and the IP address of the WSE, if running on hardware, to `SdkRuntime()`. You can load the ELFs by `load()` and start the simulator or WSE with `run()`. After that, you can do any operation, either memory transfers or kernel launches. Finally, call `stop()` to shut down the simulator or WSE. ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} from cerebras.sdk.runtime.sdkruntimepybind import SdkRuntime from cerebras.sdk.runtime.sdkruntimepybind import MemcpyDataType from cerebras.sdk.runtime.sdkruntimepybind import MemcpyOrder simulator = SdkRuntime(args.name, cmaddr=args.cmaddr) simulator.load() simulator.run() # a sequence of operations simulator.stop() ``` Instantiating the SdkRuntime object uses slightly different syntax if you are compiling and running an SDK program on a Wafer-Scale Cluster in appliance mode. See [Running SDK on a Wafer-Scale Cluster](/appliance-mode). ### memcpy\_h2d() and memcpy\_d2h() The function `memcpy_h2d()` transfers a tensor from host to device using either `streaming` mode or `copy` mode. | Parameter | Description | | ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `streaming` | `True` for `streaming` mode, `False` for `copy` mode. | | ROI (`x, y, w, h`) | Region of interest: a subrectangle starting at `(x,y)` with size `(w, h)`. The origin `(0, 0)` is the top-left PE in your program rectangle — in absolute coordinates, the PE at the coordinates given by `--fabric-offsets`. The ROI must lie within the program rectangle. | | `l` | Number of elements (wavelets) per PE. | | `data_type` | For `copy` mode: `MemcpyDataType.MEMCPY_16BIT` or `MemcpyDataType.MEMCPY_32BIT`. | | `order` | `MemcpyOrder.ROW_MAJOR` or `MemcpyOrder.COL_MAJOR` for the input/output tensor of the form `A[h][w][l]`. | | `nonblock` | Whether the operation is blocking or nonblocking. | | `dest` | The color for this host-to-device transfer if `streaming=True`, or the symbol if `streaming=False`. | ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} memcpy_h2d(dest, src, x, y, w, h, l, streaming, data_type, order, nonblock) ``` Similarly, the function `memcpy_d2h()` transfers a tensor from device to host using either `streaming` mode or `copy` mode. The first parameter `dest` is the host tensor to receive the data from the device. The second parameter `src` is the color associated with this device-to-host transfer if `streaming=True` or the device symbol from which to copy if `streaming=False`. All other parameters are the same as `memcpy_h2d()`. ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} memcpy_d2h(dest, src, x, y, w, h, l, streaming, data_type, order, nonblock) ``` The parameter `order` of `memcpy_h2d()` and `memcpy_d2h()` specifies either row-major or column-major. In both cases, the host tensor from which or to which data is copied is a 1D array of length `w*h*l`, where `w` and `h` are the width and height of the region of interest and `l` is the number of elements per PE to copy. Mapping from 1D to `[w][h][l]`, `l` is the fastest varying dimension — elements contiguous on a PE will be contiguous on the host. Mapping from 1D to `[w][h][l]`, `w` is the fastest varying dimension. Column-major delivers better bandwidth than row-major. `memcpy_h2d()` and `memcpy_d2h()` support both 16-bit and 32-bit data transfer via `copy` mode or `streaming` mode. When using `memcpy_h2d()` for a 16-bit tensor, you must perform zero extension from 16-bit to 32-bit. When using `memcpy_d2h()` for a 16-bit tensor, the returned array will contain 32-bit data where the higher 16 bits are zero. You have to strip out the higher 16 bits. See the `sdk_utils` [module documentation](/api-docs/sdkruntime-api#sdk_utils-module) for utilities to help perform this data transformation. ### launch() The `launch()` function performs remote kernel launches of host-callable functions. | Parameter | Description | | -------------------- | --------------------------------------------------------------------------------- | | `sym` | The symbol of the host-callable function. | | positional arguments | Match the arguments of the host-callable function. | | `nonblock` | Keyword argument specifying whether the kernel launch is blocking or nonblocking. | For example, to launch a host-callable function `my_fun` with two arguments of type `f32` in blocking mode, the call would look as follows: ```python theme={"languages":{"custom":["/languages/csl-tmlanguage.json"]}} my_fun_symbol = runner.get_symbol('my_fun') runner.launch(my_fun_symbol, 1.0, 2.0, nonblock=False) ```