# Core ML conversion with coremltools

This file is about the Python side: exporting from the training framework,
converting, choosing a deployment target and input shapes, and preparing for
compression. Swift runtime wiring belongs to the `coreml` skill. In a
conversion review, describe what the app needs at runtime in prose unless the
user asked for Swift code.

## Contents

- [Installing coremltools](#installing-coremltools)
- [Where Core ML sits](#where-core-ml-sits)
- [ML program versus neural network](#ml-program-versus-neural-network)
- [ct.convert parameters](#ctconvert-parameters)
- [From PyTorch](#from-pytorch)
- [From TensorFlow, scikit-learn and XGBoost](#from-tensorflow-scikit-learn-and-xgboost)
- [ONNX](#onnx)
- [Describing inputs and outputs](#describing-inputs-and-outputs)
- [Flexible shapes](#flexible-shapes)
- [Deployment targets](#deployment-targets)
- [Float16 or Float32](#float16-or-float32)
- [Which processors to allow](#which-processors-to-allow)
- [Keeping state between predictions](#keeping-state-between-predictions)
- [Several functions in one package](#several-functions-in-one-package)
- [Utilities](#utilities)
- [Controlling graph passes](#controlling-graph-passes)
- [Unsupported PyTorch ops](#unsupported-pytorch-ops)
- [Conversion mistakes](#conversion-mistakes)

## Installing coremltools

Install into a fresh virtual environment and check the published wheels against
your Python version and the version of the source framework:

```bash
python3 -m venv .venv && source .venv/bin/activate
pip install coremltools
```

coremltools 9.0 has wheels up to CPython 3.13 and adds the iOS 26 and macOS 26
deployment targets.

## Where Core ML sits

Higher-level frameworks (SoundAnalysis, Natural Language, Vision and
Foundation Models) are built on Core ML. Core ML in turn runs each operation on
the CPU through Accelerate, on the GPU through MPS Graph, or on the Neural
Engine.

## ML program versus neural network

| Aspect | Neural network (`.mlmodel`) | ML program (`.mlpackage`) |
|---|---|---|
| Status | Frozen, no new features | Current format |
| GPU precision | Float16 only | Float16 or Float32 |
| Compression APIs | Limited | Quantize, palettize and prune |
| Where weights live | Inside the protobuf | Separate files, lighter on memory |
| Training on the device | Yes | No |
| State and multiple functions | No | Yes, from iOS 18 |

## ct.convert parameters

| Parameter | Values |
|---|---|
| `source` | `"auto"`, `"pytorch"`, `"tensorflow"` |
| `inputs`, `outputs` | Lists of `ct.TensorType` or `ct.ImageType` |
| `convert_to` | `"mlprogram"` (default) or `"neuralnetwork"` |
| `minimum_deployment_target` | any `ct.target` value from iOS15 up to iOS26 |
| `compute_precision` | `ct.precision.FLOAT16` (default) or `FLOAT32` |
| `compute_units` | `ct.ComputeUnit.ALL` and the other unit sets |
| `skip_model_load` | `True` when converting on Linux, where the model cannot be loaded |
| `states` | List of `ct.StateType` |
| `pass_pipeline` | A `ct.PassPipeline` |

## From PyTorch

Tracing is the recommended route:

```python
import torch
import coremltools as ct

model = load_keyword_spotter()
model.eval()
clip = torch.randn(1, 1, 16000)
traced = torch.jit.trace(model, clip)

spotter = ct.convert(
    traced,
    minimum_deployment_target=ct.target.iOS16,
    inputs=[ct.TensorType(name="audio", shape=clip.shape)],
)
spotter.save("KeywordSpotter.mlpackage")
```

`torch.export` is supported in beta. Dynamic dimensions are declared when you
export and become `RangeDim` automatically:

```python
from torch.export import Dim, export

length = Dim(name="length", min=4000, max=48000)
exported = export(model, (clip,), dynamic_shapes={"x": {2: length}})
spotter = ct.convert(exported, minimum_deployment_target=ct.target.iOS17)
```

With `export`, shapes are fixed up front; with `trace`, you express flexibility
inside `ct.convert()` using `RangeDim` or `EnumeratedShapes`.

## From TensorFlow, scikit-learn and XGBoost

TensorFlow: `ct.convert` accepts a Keras model object, a SavedModel directory,
an `.h5` file, or a frozen `.pb` graph (the last needs explicit `inputs`).

```python
keras_net = tf.keras.models.load_model("leaf_classifier.keras")
leaf = ct.convert(keras_net, minimum_deployment_target=ct.target.iOS17)
```

scikit-learn converts to the older `.mlmodel` format:

```python
price = ct.converters.sklearn.convert(forest, ["rooms", "area", "age"], "price")
price.save("HousePrice.mlmodel")
```

XGBoost:

```python
churn = ct.converters.xgboost.convert(booster)
```

## ONNX

coremltools stopped supporting direct ONNX input in version 6, so go back to
the framework the model was trained in. If ONNX is all you have, convert it back to PyTorch
with `onnx2torch` and trace that.

## Describing inputs and outputs

`ct.TensorType(name=, shape=, dtype=, default_value=)`:

- `name` must match the model's input name.
- `shape` takes fixed integers, `RangeDim` or `EnumeratedShapes`.
- `dtype`: `np.int32`, `np.float16` or `np.float32`, plus `np.int8` from iOS 26.
- `default_value` (a NumPy array) makes the input optional.

`ct.ImageType(name=, shape=, scale=, bias=, color_layout=, channel_first=)`:

```python
photo = ct.ImageType(
    name="photo",
    shape=(1, 3, 320, 320),
    scale=1 / 127.5,
    bias=[-1.0, -1.0, -1.0],             # maps pixels to the range -1...1
    color_layout=ct.colorlayout.RGB,
    channel_first=True,
)
```

- Colour layouts: `GRAYSCALE`, `GRAYSCALE_FLOAT16`, `RGB` or `BGR`.
- PyTorch models are NCHW, so pass `channel_first=True`; TensorFlow's NHWC
  needs `False`.

`ct.StateType(wrapped_type=ct.TensorType(...), name=)` (iOS 18+) declares
state, for example a float16 `kv_cache` tensor.

## Flexible shapes

```python
fixed = ct.TensorType(name="x", shape=(1, 3, 512, 512))

side = ct.RangeDim(lower_bound=256, upper_bound=1024, default=512)
ranged = ct.TensorType(name="x", shape=(1, 3, side, side))

sizes = [(1, 3, s, s) for s in (256, 512, 1024)]
listed = ct.TensorType(name="x", shape=ct.EnumeratedShapes(shapes=sizes, default=sizes[1]))
```

When the real sizes are known, list them with `EnumeratedShapes`: the Neural
Engine compiles an optimised plan for each listed shape, while with `RangeDim`
only the default shape gets that treatment. Until iOS 18,
`EnumeratedShapes` was limited to a single input; newer targets allow it on
several.

## Deployment targets

| iOS target | Matching macOS target | What it unlocks |
|---|---|---|
| `ct.target.iOS13` | `ct.target.macOS10_15` | Neural network models only |
| `ct.target.iOS15` | `ct.target.macOS12` | ML program, FP16, typed tensors |
| `ct.target.iOS16` | `ct.target.macOS13` | Palettized and sparse weights |
| `ct.target.iOS17` | `ct.target.macOS14` | W8A8 activation quantization (A17 Pro and later) |
| `ct.target.iOS18` | `ct.target.macOS15` | Stateful and multifunction models, block-wise quantization |
| `ct.target.iOS26` | `ct.target.macOS26` | INT8 input and output types, state read and write |

## Float16 or Float32

ML programs default to `ct.precision.FLOAT16`, which is smaller and faster. Use
`FLOAT32` only when half precision measurably hurts accuracy. To keep just the
sensitive ops in FP32:

```python
def keep_norms_fp32(op):
    return op.op_type != "layer_norm"

policy = ct.transform.FP16ComputePrecision(op_selector=keep_norms_fp32)
mixed = ct.convert(
    traced,
    compute_precision=policy,
    minimum_deployment_target=ct.target.iOS17,
    inputs=[ct.TensorType(name="tokens", shape=(1, 128))],
)
```

The selector returns `False` for an op that should stay in FP32.

## Which processors to allow

| Setting | When |
|---|---|
| `.all` | Default and recommended: CPU, GPU and Neural Engine |
| `.cpuOnly` | Debugging, or checking FP32 reference results |
| `.cpuAndGPU` | When the Neural Engine produces bad results |
| `.cpuAndNeuralEngine` | Lowest energy use; macOS 13 or later |

## Keeping state between predictions

From iOS 18 a model can keep values between predictions. The main use is the
key-value cache of a language model.

```python
cache = ct.StateType(
    name="kv_cache",
    wrapped_type=ct.TensorType(dtype=np.float16, shape=(2, 1, 16, 1024, 64)),
)
decoder = ct.convert(
    traced_decoder,
    minimum_deployment_target=ct.target.iOS18,
    states=[cache],
    inputs=[ct.TensorType(name="token", dtype=np.int32, shape=(1, 1))],
)
state = decoder.make_state()
step = decoder.predict({"token": np.array([[42]], dtype=np.int32)}, state=state)
```

On the Swift side the equivalent is `let state = model.makeState()` followed by
`try model.prediction(from: MLDictionaryFeatureProvider(dictionary: inputs), using: state)`.

Apple's published measurement for Llama 3.1: 16.26 tokens per second with a
stateful KV cache against 1.25 without, roughly thirteen times faster.

## Several functions in one package

Also from iOS 18, one `.mlpackage` can hold several functions that share
deduplicated weights, for example a base model and LoRA variants:

```python
bundle = ct.utils.MultiFunctionDescriptor()
for variant in ("plain", "legal"):
    bundle.add_function(f"Drafting-{variant}.mlpackage",
                        target_function_name=variant, src_function_name="main")
bundle.default_function_name = "plain"
ct.utils.save_multifunction(bundle, "Drafting.mlpackage")
```

In Swift, pick the function with `MLModelConfiguration.functionName`.

## Utilities

```python
spec = spotter.get_spec()
ct.utils.rename_feature(spec, "var_512", "probabilities")

spotter.author = "Audio team"
spotter.short_description = "Wake-word detector"
spotter.input_description["audio"] = "One second of 16 kHz mono audio"
```

- `ct.models.utils.bisect_model` splits a large model into pieces for
  debugging.
- `make_pipeline` chains models.
- `randomize_weights` produces a structurally identical model for tests.

## Controlling graph passes

```python
passes = ct.PassPipeline()
passes.remove_passes({"common::fuse_conv_batchnorm"})    # keep BN visible while debugging
debuggable = ct.convert(traced, minimum_deployment_target=ct.target.iOS17,
                        pass_pipeline=passes)
```

Predefined pipelines: `ct.PassPipeline.EMPTY`, `CLEANUP`, `DEFAULT_PRUNING`,
`DEFAULT_PALETTIZATION`.

## Unsupported PyTorch ops

Register a composite for an op the converter does not know, built from MIL
builder ops:

```python
from coremltools.converters.mil.frontend.torch.ops import _get_inputs
from coremltools.converters.mil import register_torch_op
from coremltools.converters.mil import Builder as mb

@register_torch_op
def hardswish(ctx, torch_node):
    (value,) = _get_inputs(ctx, torch_node, expected=1)
    gate = mb.clip(x=mb.add(x=value, y=3.0), alpha=0.0, beta=6.0)
    scaled = mb.mul(y=gate, x=value)
    ctx.add(mb.real_div(x=scaled, y=6.0, name=torch_node.name))
```

## Conversion mistakes

- Tracing or exporting without `model.eval()`.
- `RangeDim` where `EnumeratedShapes` would do.
- Targeting the neural network format for a new model.
- Leaving out `minimum_deployment_target`; always set it.
- Picking Float32 when Float16 is accurate enough.
- Converting straight from ONNX.
- Testing only in the Simulator, which has no Metal GPU or Neural Engine path;
  verify on physical devices.
