Skip to main content

Apple Foundation Models 3 & the On-Device LLM Stack for Claude Users

Intermediate

At WWDC 2026 (announced 8 June 2026) Apple shipped the third generation of its Foundation Models — AFM 3 — and expanded the Foundation Models framework in a way that matters even if you never write Swift: the framework now takes third-party LLM providers (Anthropic Claude and Google Gemini are shipping as Swift packages), and the on-device path stops being a toy 3B model and adds a 20-billion-parameter sparse MoE that runs on Apple Silicon by keeping most experts in NAND and swapping only what a prompt needs into RAM. If you build Claude apps and have been treating "on-device" as a separate universe from your API workflows, that gap is closing this year. This page unpacks what actually shipped, the non-obvious mechanisms, and the concrete migration and compatibility choices you now have.

What you'll learn
  • Name the four AFM 3 models — Core (3B), Core Advanced (20B sparse), Cloud, Cloud Pro — and which one runs where
  • Understand the flash-plus-DRAM expert-routing trick that lets a 20B model fit on an iPhone at all
  • Use the Foundation Models framework's @Generable / @Guide macros for constrained decoding — the model literally cannot emit invalid JSON
  • Know what changed at WWDC26: the LanguageModel / LanguageModelExecutor protocols that let you plug Claude or any other LLM into the same Swift session API
  • Decide when to reach for the on-device 3B, when to jump to Private Cloud Compute, and when to keep the workload on the Claude API

The one-sentence version​

AFM 3 is Apple's second serious swing at on-device LLMs: a rebuilt 3B dense model (Core), a new 20B sparse MoE (Core Advanced) that runs on iPhone by keeping most experts in flash storage, a Cloud and Cloud Pro pair on Apple Silicon plus NVIDIA GPUs in Google Cloud, and a framework that now treats Anthropic Claude and Google Gemini as first-class swappable providers behind the same LanguageModel protocol. Free for developers to call on-device; free on Private Cloud Compute for App Store Small Business Program members under 2M first-time downloads.

Three non-obvious things about AFM 3​

Everyone is writing the "Apple has a 3B on-device model" headline. Here are the mechanisms most write-ups skip.

1. The 20B model fits on iPhone because most of it lives in NAND, not RAM​

The naive read of "20-billion-parameter model on iPhone" is impossible: even at 4-bit, 20B weights is ~10 GB, well past what an iPhone can hold in RAM. Apple's trick, per the WWDC 2026 announcement, is to store the full AFM 3 Core Advanced model in flash memory (NAND) and swap only the input-dependent routed experts into DRAM at inference time. The architecture has shared experts that stay resident and routed experts that a router picks per prompt (activating 1–4B parameters at a time, not the full 20B). Combined with Quantization Aware Training, this is what lets Apple ship a 20B-class model on a device that has ~8 GB of usable RAM. The consequence for you: iPhone-side latency is now dominated less by matrix multiply and more by NAND read bandwidth and the router's choices — a very different performance envelope than a 3B dense model.

2. Constrained decoding is baked into the Swift compiler, not the runtime​

The framework's @Generable macro is often described as "structured output," which understates what it does. @Generable runs at compile time: it generates a JSON schema and a parser for your Swift type, and the framework then uses that schema for constrained decoding — the on-device model is forced, at the token level, to produce output that parses back into your Swift struct. There is no "hope it's valid JSON, then retry." A @Guide annotation lets you add constraints (.anyOf(["PG", "PG-13", "R", "G"]), ranges, descriptions) that are also enforced at decode time. If you have spent time writing JSON-repair fallbacks around a Claude tool-use loop, the on-device 3B model side of your app can drop that entire retry layer.

3. The framework now takes any LLM — including Claude and Gemini — behind one Swift API​

The WWDC 2026 session "Bring an LLM provider to the Foundation Models framework" shipped two new protocols: LanguageModel (declares capabilities and executor configuration) and LanguageModelExecutor (handles the actual generation, KV cache, and streaming). Anyone can ship a Swift package that conforms to them — Apple explicitly named Anthropic Claude and Google Gemini as launch-time providers. The consequence: from the app's point of view, the same LanguageModelSession, the same @Generable types, the same tool-calling contract, and the same transcript model apply whether the underlying model is the on-device 3B, Private Cloud Compute, Claude on Anthropic's API, Gemini on Google Cloud, or an MLX weights file from Hugging Face. Model choice becomes a Swift package dependency rather than an SDK rewrite.

AFM 3 at a glance​

Press Enter or Space to flip the card. Use the left and right arrow keys to move between cards.Term shown.
1 / 6

What actually runs where​

Guided walkthrough1 of 4
  1. Runs on any Apple Intelligence device. Free to call from Swift via the Foundation Models framework — no API key, no network, no per-token bill. This is where you put classification, extraction, summarization, on-device rewrites, and anything that must work in airplane mode.

A minimal on-device call​

The framework's headline feature is that you can get typed, schema-validated output from the on-device 3B with almost no ceremony.

On-device 3B call with @Generable structured output (Swift)

import FoundationModels

@Generable
struct TripIdea {
  @Guide(description: "Short, evocative title")
  let title: String

  @Guide(description: "One-paragraph pitch, 40–80 words")
  let summary: String

  @Guide(.range(1...14))
  let estimatedDays: Int

  @Guide(.anyOf(["easy", "moderate", "hard"]))
  let difficulty: String
}

let session = LanguageModelSession()   // defaults to on-device Core
let idea = try await session.respond(
  to: "Suggest a shoulder-season trip in Puglia for a family with a 6-year-old.",
  generating: TripIdea.self          // constrained decoding
)

print(idea.title, idea.difficulty, idea.estimatedDays)

At runtime the model is forced, at the token level, to emit tokens that keep the running text parseable as a TripIdea. You never see a JSON string; you receive a TripIdea value. This is the same contract whether the session is talking to the on-device 3B, PCC, or a third-party provider you plumbed in as a Swift package.

Bringing Claude into the same Swift session​

The WWDC 2026 "Bring an LLM provider" session made this concrete: implement LanguageModel and LanguageModelExecutor, publish it as a git-tagged Swift package, and app code that already used the on-device 3B can call your provider by changing one initializer.

Sketch of a third-party provider conforming to LanguageModel

import FoundationModels

public struct ClaudeSonnet5: LanguageModel {
  public let capabilities = LanguageModelCapabilities(capabilities: [
      .toolCalling, .guidedGeneration, .reasoning
  ])
  public var executorConfiguration: ClaudeExecutor.Configuration
}

public final class ClaudeExecutor: LanguageModelExecutor {
  public struct Configuration: Hashable {   // hashable = KV-cache key
      public var modelID: String
      public var tokenProvider: TokenProvider   // NOT a plain API key string
  }

  public required init(configuration: Configuration) throws { /* … */ }

  public func respond(
      to request: LanguageModelExecutorGenerationRequest,
      model: Model,
      streamingInto channel: LanguageModelExecutorGenerationChannel
  ) async throws {
      // 1. Map Foundation Models transcript → Anthropic /messages payload
      // 2. Stream deltas back via channel.send(.response(action: .appendText(...)))
      // 3. Surface tool calls / reasoning via the same transcript entry types
  }
}

Two subtleties the API deliberately pushes you toward. First, Configuration is Hashable on purpose: the framework caches executors by configuration, so same config = same executor = preserved KV cache across turns. Second, the guidance explicitly discourages plain API-key strings in initializers — prefer a TokenProvider or a sign-in flow, keychain-persist tokens, and use App Attest for cloud calls to verify the device isn't a tampered build hitting your endpoint from the outside.

When to use what — a decision table for Claude builders​

Pro tip
  • Reach for on-device AFM 3 Core (3B): summarization, classification, entity extraction, on-device rewrites, keyboard suggestions — anything where a bounded prompt and typed output beats a network round-trip. Free.
  • Reach for AFM 3 Core Advanced (20B): private reasoning that cannot leave the device (health, finance, unreleased documents) and that the 3B fumbles. Expect variable latency because of NAND expert-swap.
  • Reach for Private Cloud Compute (Cloud / Cloud Pro): larger context, longer reasoning, or multimodal where the 20B on-device can't keep up but privacy still rules out third-party APIs.
  • Stay on Claude via the Anthropic API (or the new Swift-package provider): long agent runs, extended thinking, big-context codebase work, and cross-platform apps that need the same model on Android and web. See the Claude vs others foundations page.
  • Do NOT wrap the on-device 3B in a JSON-repair loop: it has constrained decoding. If your output isn't parsing, the schema is wrong or you skipped @Generable.

Quick check​

Check yourself

0/5
  1. How does AFM 3 Core Advanced fit a 20B-parameter model onto an iPhone?
  2. What does the @Generable macro give you that hand-rolled JSON prompting does not?
  3. You want to call Claude Sonnet 5 from an iOS app that already uses the Foundation Models framework for on-device summarization. What is the WWDC26-native path?
  4. Which developer group gets Private Cloud Compute calls for free from the framework?
  5. Why is the executor `Configuration` type required to be `Hashable`?

Sources & further reading​