Run Meta's SAM 3 on macOS (Apple Silicon MPS or CPU). No triton, no CUDA, no fork.

SAM 3 is published as code + weights, but the code assumes CUDA throughout — hardcoded .cuda() calls, NVIDIA-only triton kernels in the import path, bfloat16-locked fused ops, and a couple of honest bugs. On a Mac you can't even import sam3 without errors. sam3-mac is a small shim that monkey-patches around those issues so image inference Just Works on M-series Macs (and any other non-CUDA host).

import sam3_mac
sam3_mac.install()                # auto-picks mps / cpu

model, processor = sam3_mac.build_image_model(confidence_threshold=0.5)

from PIL import Image
state = processor.set_image(Image.open("car.jpg"))
out   = processor.set_text_prompt("damage on car", state)

print(out["boxes"].shape, out["scores"], out["masks"].shape)

1. Prerequisites
2. Installation
3. Getting access to SAM 3 weights
4. Quick start (full working example)
5. API reference
6. Inputs and outputs
7. What this library patches and why
8. Performance
9. Troubleshooting
10. Limitations / what's not supported
11. License

• macOS (any recent version). Tested on macOS 15 on Apple M4 Pro.
• Python 3.10 or newer. SAM 3 itself officially requires 3.12, so 3.12 is the safe target. Homebrew install: brew install [email protected].
• About 5 GB free disk for the model weights (cached to ~/.cache/huggingface/ on first run).
• A HuggingFace account approved for access to the gated facebook/sam3 repo — see below.

This library does not require CUDA, Xcode command-line tools (beyond Python itself), or any GPU driver — Apple Silicon's MPS backend is built into PyTorch and works out of the box.

# 1. Create a virtual environment (recommended)
python3.12 -m venv .venv
source .venv/bin/activate

# 2. Install PyTorch (MPS-enabled wheels come from PyPI by default on macOS)
pip install "torch>=2.7" torchvision

# 3. Install SAM 3 from source + its runtime deps
pip install "git+https://github.com/facebookresearch/sam3.git"
pip install einops opencv-python-headless scikit-image pycocotools pandas psutil "numpy<2"

# 4. Install this library
pip install sam3-mac
#   or, from a clone:
#   pip install -e .

Why so many transitive deps? SAM 3's pyproject.toml lists some runtime dependencies (e.g. einops, pycocotools, psutil) under "notebook" / "training" extras even though import sam3 actually requires them. sam3-mac doesn't bundle these so you can pin them yourself, but the list above is what you need at minimum.

The model weights (~3.5 GB) live on HuggingFace at facebook/sam3 and are gated — Meta has to approve your account before download works. One-time setup:

1. Sign in (or create a free account) at https://huggingface.co.

2. Go to https://huggingface.co/facebook/sam3 and click Request access. Approval typically comes through within minutes.

3. Create a read-only access token: https://huggingface.co/settings/tokens.

4. Authenticate your machine:

   pip install -U huggingface_hub
   hf auth login
   # paste your token; answer "N" to the git-credential question

The first inference downloads the weights into ~/.cache/huggingface/ and reuses them after that.

A complete script that loads SAM 3 and runs damage detection on an image:

# basic.py
import time
import sam3_mac
from PIL import Image

# 1. Patch SAM 3 so it works on Mac. Call BEFORE `import sam3`.
device = sam3_mac.install(verbose=True)        # auto-picks mps / cpu
print(f"using device: {device}")

# 2. Build the model + processor. This downloads weights on first run.
t0 = time.perf_counter()
model, processor = sam3_mac.build_image_model(confidence_threshold=0.5)
print(f"model loaded in {time.perf_counter() - t0:.1f}s")

# 3. Run inference on an image with a text prompt.
image = Image.open("car.jpg").convert("RGB")
state = processor.set_image(image)
out   = processor.set_text_prompt("damage on car", state)

# 4. Inspect results.
boxes  = out["boxes"].detach().cpu().float().numpy()   # (N, 4) — [x1, y1, x2, y2] in pixels
scores = out["scores"].detach().cpu().float().numpy()  # (N,)
masks  = out["masks"].detach().cpu().bool().numpy()    # (N, H, W) — pixel mask per detection

print(f"{boxes.shape[0]} detections")
for i in range(boxes.shape[0]):
    x1, y1, x2, y2 = (int(round(v)) for v in boxes[i].tolist())
    print(f"  #{i}: score={scores[i]:.2f}  bbox=[{x1},{y1},{x2},{y2}]  mask_pixels={int(masks[i].sum())}")

Run it:

python basic.py

Expected output (on a damaged-car photo):

using device: cpu
model loaded in 6.2s
2 detections
  #0: score=0.61  bbox=[412,341,1383,1359]  mask_pixels=430877
  #1: score=0.52  bbox=[493,210,1356,820]   mask_pixels=181403

A working version is included as examples/basic.py — call it with python examples/basic.py path/to/image.jpg "damage on car".

sam3_mac.install(device: str | None = None, verbose: bool = False) -> str

Apply all patches. Call before import sam3 (or before build_image_model(), which imports sam3 internally).

• device — one of "mps", "cpu", "cuda", or None to auto-detect (cuda > mps > cpu).
• verbose — log each patch as it's applied.
• Returns the device that was set up. If the requested device isn't available, falls back to "cpu" with a warning. On a CUDA host this is a no-op (SAM 3 runs natively).
• Idempotent — repeated calls return the original device without re-patching.

sam3_mac.build_image_model(confidence_threshold: float = 0.5) -> (model, processor)

Convenience builder. Returns a (model, Sam3Processor) pair on the device install() set up, with the required float32 cast already applied (see Patches #5).

• Equivalent to model = build_sam3_image_model(device=...).float().to(device); processor = Sam3Processor(model, ...).
• Requires install() to have been called first.

sam3_mac.get_device() -> str | None

The device install() set up, or None if not called yet.

processor.set_image(image, state=None) -> dict

• image accepts:
  • PIL.Image.Image (recommended)
  • np.ndarray of shape (H, W, 3), dtype uint8, RGB. This library patches the upstream numpy-shape bug, so passing ndarray is safe.
• Returns an opaque state dict you pass to set_text_prompt.

processor.set_text_prompt(prompt: str, state: dict) -> dict

• prompt is a short noun phrase. Short, concrete phrases ("scratch", "dent", "broken headlight", "damage on car") work better than long sentences.
• Returns the same dict mutated with detection results:
  • "boxes" — torch.Tensor of shape (N, 4), [x1, y1, x2, y2] in pixel coords of the original image.
  • "scores" — torch.Tensor of shape (N,), confidence in [0, 1].
  • "masks" — torch.Tensor of shape (N, H, W), boolean per-pixel masks at original image resolution. (Without this library, masks would be (N, 1, H, W) — patch #5 squeezes the redundant dim.)
  • "masks_logits" — same shape as masks but float32 post-sigmoid probabilities, if you want to threshold differently.

Call set_text_prompt multiple times with the same state to run multiple prompts against the same image without re-encoding.

Six patches, each load-bearing. Without each one, SAM 3 fails on Mac with a specific error:

Plus one non-patch behavior: the SAM 3 checkpoint mixes bf16 and float32 parameters across submodules. CUDA hides this with torch.autocast; off CUDA you'd get matmul dtype mismatches. sam3_mac.build_image_model() applies a model.float() after construction so you don't have to remember.

On an M4 Pro, image-only inference (SAM 3 internally resizes to 1008×1008) is roughly:

MPS is currently slower than CPU because the MPS-fallback path ping-pongs unsupported ops to CPU and back. As PyTorch's MPS backend grows op coverage this should flip. For now, device="cpu" is the recommended default on Mac.

Model load + first inference takes ~10–15 s due to weight load and PyTorch kernel warmup; subsequent inferences are stable at the numbers above.

huggingface_hub.errors.GatedRepoError: Cannot access gated repo Your HuggingFace access request to facebook/sam3 hasn't been approved yet, OR your CLI isn't authenticated. Visit https://huggingface.co/facebook/sam3, click "Request access", and run hf auth login once approved.

ModuleNotFoundError: No module named 'einops' (or pycocotools, psutil, ...) SAM 3 lists these as "notebook/training" extras but actually needs them at runtime. Install:

pip install einops opencv-python-headless scikit-image pycocotools pandas psutil "numpy<2"

ModuleNotFoundError: No module named 'triton' You imported sam3 before calling sam3_mac.install(). Order matters: install first, then import.

import sam3_mac
sam3_mac.install()
import sam3   # only after install()

RuntimeError: failed assertion 'Destination NDArray and Accumulator NDArray cannot have different datatype' You're on MPS and the addmm_act patch didn't apply. Make sure sam3_mac.install(device="mps") was called before constructing the model. If it was, fall back to device="cpu" — that's tested and reliable.

Predicted boxes all have width 2–3 pixels at the left edge of the image The numpy-shape bug (patch #4) didn't apply. Either you bypassed sam3_mac.install(), or you called set_image with a torch.Tensor (not patched — pass PIL or numpy ndarray).

First inference takes 60+ seconds That's normal on first run — PyTorch is compiling MPS kernels and loading the 3.5 GB model from disk. Subsequent inferences are fast.

• Video tracking / Sam3VideoPredictor: pulls in the tracker code path that legitimately needs triton (Euclidean distance transform kernels). The triton stub raises a clear error if hit. Image inference is all this library targets.
• Training: not tested, not patched.
• Custom CUDA op kernels beyond what's in upstream SAM 3.
• Mixing CUDA and non-CUDA in the same process: install() modifies torch.* globally inside the process. Don't run this in a process that also expects native .cuda() behavior.
• Pinned to SAM 3 as of early 2026. If upstream restructures internals, patch #3 (which inspects sys.modules for consumers of addmm_act) may need updating.

MIT. See LICENSE.