Skip to content
imaged

Prepare a model

Convert ONNX, check engine support, benchmark, and prepare smaller packs.

Use the slab-load tool in the macOS bundle to turn trained models into .slab packs. It runs on Apple silicon with macOS 15 or later. It is a model-preparation tool, not a component to ship inside your app.

A pack holds a graph with fixed input shapes, weights and metadata such as preprocessing, labels and a preferred engine. The same pack can be used on iOS, Android and macOS when the chosen engine supports it. You can also start with a ready-made model from the gallery.

1. Set up the tools

Get slab-macos-0.1.1.zip from SDK downloads. It extracts to a macos folder containing bin/slab-load, libslab.dylib, slab.h, the Python bindings and an Android runner in libexec/slab-load/android-arm64/. Keep the extracted folder structure intact. Put your downloaded license in that folder.

unzip slab-macos-0.1.1.zip
cd macos
bin/slab-load --version
export SLAB_RUNTIME_LICENSE_FILE="$PWD/license.slablic.txt"

Use the app-ID license downloaded from your customer console. The same file works with slab-load; you do not register the tool as another product. Running models and benchmarks requires a runtime license.

If macOS blocks the tools

The 0.1.1 macOS binaries are not notarized. If Gatekeeper blocks a download from this site, verify the archive's SHA-256 against SDK downloads. From the parent of the extracted macos folder, remove quarantine from that folder:

xattr -dr com.apple.quarantine macos/

2. Check and convert ONNX

Export your model to ONNX, using opsets 10–20. Opset 10's older forms of Clip, Pad, Resize and Upsample are refused. Language and vision-language models are converted from original checkpoints instead; follow their gallery instructions.

bin/slab-load convert model.onnx --dry-run
bin/slab-load convert model.onnx model.slab

The dry run lists inputs and every node the converter cannot lower, by name, without writing a pack. Fix unsupported operations before conversion.

Dynamic dimensions are fixed at conversion. Set an entire input shape with --input, or a named dimension with --dim:

bin/slab-load convert model.onnx model.slab --input pixel_values=1x3x384x384
# Alternatively, bind a named dimension:
bin/slab-load convert model.onnx model.slab --dim batch=1

With no option, an unset dynamic batch becomes 1 and the tool reports it.

3. Inspect and check engine support

bin/slab-load info model.slab
bin/slab-load check model.slab
bin/slab-load ops

info shows input/output names, shapes, weights and operators. check asks each engine on this Mac whether it supports the pack and estimates memory use. If an operator is missing, it names it. ops prints the operator table.

With SLAB_RUNTIME_LICENSE_FILE set, check also loads the pack on each engine with the required operators. It reports the load time or that engine's reason for refusing the pack. For native language and vision-language packs, it checks loading rather than an operator graph.

A Mac check covers that Mac's engines. Use the Android USB workflow below to test on your target phone. See Platforms & requirements for the platform-specific operator matrix and Qualcomm NPU requirements.

4. Benchmark and compare outputs

bin/slab-load bench model.slab all

The tool opens the pack on CPU, GPU and Neural Engine, runs 20 times on fixed pseudo-random inputs, and compares GPU and Neural Engine outputs with CPU. It reports load time, first-run time, median latency, per-engine session memory (mem_mb), process peak memory, maximum absolute difference and cosine similarity. Use cpu, gpu or ne to select one engine, or both for CPU and GPU.

For representative testing, provide your own raw little-endian input values. Replace input with the pack's actual input name:

bin/slab-load bench model.slab all input=in.bin
bin/slab-load run model.slab gpu out/ input=in.bin --runs 20

run writes outputs for comparison with your reference framework. Check accuracy on your own data as well as speed. Opening a model on GPU or Neural Engine includes preparing it for that engine, so separate load cost from steady-state inference.

Measure sustained performance

mem_mb measures the memory added by each engine's session: process footprint on Apple (including Metal buffers), resident memory on Android. Process peak memory is reported separately.

Add --duration SECONDS to keep each engine running and report frame rate in ten time windows. This helps you see GPU clock ramp-up and performance as the device warms up:

bin/slab-load bench model.slab gpu --duration 60

Test on an Android phone over USB

Install Android platform-tools (adb), enable USB debugging and authorize the connected phone. Keep the Android runner in the extracted macOS package. Then run:

bin/slab-load check model.slab --device android
bin/slab-load bench model.slab all --device android
bin/slab-load bench model.slab gpu --duration 60 --device android

check, info, ops, bench and precision accept --device android. Use --device android:SERIAL when several phones are connected. The tool copies its runner, pack, input files and license to the phone's /data/local/tmp/slab-load and prints the results on your Mac. The pack is copied again only when it changes. Your app-ID license also covers the USB runner.

For a Snapdragon NPU, set SLAB_QNN_LIBS to the folder containing Qualcomm's QNN libraries for that chip. See Android requirements. CPU and GPU testing does not require those libraries.

5. Reduce weight storage

Packs can store FP32, FP16, INT8 or 4-bit weights. Weight storage and computation precision are separate: CPU and GPU compute in FP32, while Neural Engine and Hexagon NPU compute in FP16.

For an FP32 pack intended for phones:

bin/slab-load quantize model.slab model-f16.slab --f16
bin/slab-load info model-f16.slab
bin/slab-load check model-f16.slab
bin/slab-load bench model-f16.slab all

FP16 weights use half the storage of FP32. The Android CPU and GPU keep those weights in FP16 memory too. The release reports identical CPU/GPU outputs for an FP16 pack versus the same pack with those weights widened to FP32. Converting original FP32 weights to FP16 can round values; compare against the original model on representative inputs.

quantize also supports INT8 and 4-bit weights. Follow the model's conversion instructions and measure accuracy after quantization. Use slab-load precision for per-engine precision settings supported by the tool.

Metadata and engine choice

The model author supplies preprocessing, labels and defaults once in the pack. Apps can override preprocessing when needed.

Setting Purpose
Task Enables results such as class rankings, detection boxes or embeddings
Labels Names used in classification or detection results
Resize, padding and color order Fits and orders image pixels for the model
Normalization Converts pixel values to model input values
Preferred engine Default engine selection for this model

With automatic selection, Slab tries the pack author's preference, then the available NPU, GPU and CPU. A model runs on one engine; Slab does not split its operations between engines. An explicitly requested engine must support the model.

Protected packs

slab-load protect encrypts a pack with a key controlled by the model owner. Apps supply an unlock key or model token separately from the runtime license. Protected packs run on CPU and GPU, not on Neural Engine or Hexagon NPU. Use 0.1.1 or later for protected FP16 packs on Apple devices: 0.1.0 could produce incorrect CPU outputs for these packs.

Ship the model

Put the pack in the app bundle or Documents directory on Apple platforms, or the app's private files/ directory on Android. Follow the language setup guides to attach the runtime license and open it.

Slab 0.1.1 uses the same pack format and API as 0.1.0, so this upgrade needs no reconversion or integration changes. The pack format may change before Slab 1.0; check the release notes when upgrading.