WWDC26 AI features landed across two distinct on-device frameworks, a rebuilt Siri integration model, and a new agentic coding workflow in Xcode 27.
Foundation Models now supports third-party models including Claude and Gemini, multimodal prompts, and Private Cloud Compute at no cost for small developers. Core AI is a new lower-level framework for ML engineers who need full control over model loading and inference.
App Intents gets entity and intent schemas that let Siri understand your app's content and actions without example phrases.
If you're an iOS or macOS developer evaluating which AI API to adopt, here's what each framework does and how to choose between them.
What's new at WWDC26 for AI developers (quick reference)
Foundation Models framework
Available on iOS 27, iPadOS 27, macOS 27, watchOS 27
Foundation Models is Apple's high-level Swift API for building AI-powered app features. It manages model loading, memory, and inference for you, so your app works with sessions, prompts, and tools rather than hardware details. New in iOS 27, it can connect to any model provider via a protocol, including third-party options beyond Apple's own on-device model.
What's new this year:
- Third-party model support: any provider conforming to
LanguageModelExecutorplugs in alongside Apple's model; Claude and Gemini are named partner integrations (session #339) - Multimodal prompts: pass images alongside text for visual reasoning
- Vision framework tools: OCR and barcode readers available as on-device model tools, no cloud round-trip needed
- Dynamic Profiles: swap model, tools, and instructions mid-session without breaking continuity
- Private Cloud Compute: next-generation Apple Foundation Models on Apple Silicon servers, free for eligible developers
- Evaluations framework: define metrics, auto-grade outputs, aggregate statistics for AI feature quality
fmCLI: new command-line tool for scripting and testing Foundation Models locally- Foundation Models Python SDK: Python access to the framework for evaluation pipelines
Dynamic Profiles for agentic apps
Dynamic Profiles are the key primitive for building multi-step agentic workflows. Your app can start a session with a lightweight on-device model, then switch mid-conversation to a more capable provider for a complex subtask, all within the same session object with no break in context. Session #242 covers three orchestration patterns built on this: baton-pass (hand off between agents), phone-a-friend (call a specialist for a subtask), and skills (reusable agent capabilities you compose together).
Private Cloud Compute is free for small developers
PCC gives you access to more capable Apple Foundation Models running on Apple Silicon servers while keeping user data private. If you're enrolled in the App Store Small Business Program and your app has fewer than 2 million total first-time App Store downloads, you can use PCC at no cloud API cost. Session #319 covers availability checks and graceful fallback patterns for when PCC is unavailable.
The Evaluations framework and the fm CLI
Unit tests aren't enough for probabilistic AI outputs, because the same prompt can produce different results on different runs. The Evaluations framework gives you a structured way to define metrics, automatically grade outputs, and aggregate statistics across runs. It supports model judges for qualitative metrics and a hill-climbing workflow for iterating on prompt quality. The fm CLI and Python SDK let you run evaluations in scripts and CI pipelines without a full Xcode setup.
Core AI framework
Available on iOS 27, iPadOS 27, macOS 27
Core AI is Apple's low-level on-device inference framework for ML engineers who need to bring their own models. Where Foundation Models abstracts everything away, Core AI gives you direct control over memory, inference scheduling, and model conversion. There are no server dependencies and no token costs, because everything runs on-device.
What it includes:
- Model repository: curated open-source models optimized for Apple Silicon: Qwen, Mistral, SAM3, and more, with ready-to-run examples
- Python toolchain: Core AI Python (authoring), Core AI PyTorch Extensions (conversion), Core AI Optimization (quantization and tuning)
- Swift API: load → specialize → run pattern; specialization is automatic per hardware
- Ahead-of-time (AOT) compilation: faster first-time model load on repeat runs
- Fine-grained memory control: zero-copy data paths and stateful execution for tight latency budgets
- Core AI Debugger in Xcode 27: deep intrinsic analysis of on-device model performance
Core AI Debugger in Xcode 27
The Core AI Debugger is a new Xcode 27 tool for profiling on-device model performance at the layer level, covering memory usage, execution tracing, and bottleneck identification. It's part of an AI-assisted authoring workflow in Xcode that guides you from initial model concept through to optimized on-device execution. Session #325 covers the full authoring and optimization workflow.
Core AI vs Foundation Models: which should you use?
Most developers will only need one. The choice comes down to whether you control the model.
App Intents and Siri
Available on iOS 27, iPadOS 27, macOS 27, watchOS 27
The App Intents framework is how your app's content and actions become available to Siri. In iOS 27, Apple has simplified the integration model significantly. You define schemas describing your app's content and actions, and Siri's language understanding handles the matching.
New this year:
- Entity schemas: contribute your app's content to Spotlight's semantic index; Siri can surface it with attribution back to your app
- Intent schemas: define what actions your app can take with no phrases to write; Siri's understanding evolves without code changes on your side
- View Annotations API: map views to entities so Siri knows what's on screen; users can reference visible content conversationally
- App Intents Testing framework: validate your integration through real system pathways without UI automation
ValueRepresentation: share entities across apps (session #345)EntityCollection: improved performance for large entity setsSyncableEntity: sync entities across a user's devicesLongRunningIntent: handle long-running actions with graceful cancellation
Entity and intent schemas replace example phrases
The shift to schemas is the biggest change for existing App Intents adopters. Previously you'd write example phrases to teach Siri how users might invoke your intent. With intent schemas, you describe the action and its parameters, and Siri's language model handles matching. This means your integration improves as Siri's understanding improves, including expansion to new languages, without shipping an app update.
Agentic coding in Xcode 27
Available on macOS (Xcode 27)
Xcode 27 now has a four-phase agentic coding loop built in: it can explore a codebase, generate code, review it, and coordinate multi-step tasks. It works for solo developers and team environments alike.
The four phases:
- Explore: agent understands the codebase structure and relevant context
- Build: generates and iterates on code toward a goal
- Refine: reviews, tests, and polishes output
- Orchestrate: coordinates across multi-step tasks and multiple agents
Related sessions apply the same model to adjacent workflows: UI prototyping agents (#227), translation agents (#213), and game porting (#357).
Other AI-adjacent frameworks
LLM search with Core Spotlight
SpotlightSearchTool is a new API that connects LanguageModelSession to your app's Spotlight index, giving you RAG-style search without building a separate vector store or retrieval layer. What you get:
- Grounded answers from your app's indexed content
- Delegate-based hydration for customizing how results are returned
- Guidance profiles for tuning search behavior
- Custom
PipelineStages(for example, adding sentiment analysis to the retrieval pipeline)
Metadata quality directly affects result quality, so this is also an incentive to index your content properly.
Music Understanding framework
Music Understanding is a new framework for on-device audio analysis. It gives you structured musical data without sending audio off-device. The six analysis dimensions are:
- Key: detected musical key
- Rhythm: beat and tempo information
- Structure: section boundaries (verse, chorus, etc.)
- Pace: tempo feel and energy over time
- Instrument activity: which instruments are active
- Loudness: dynamic level over time
Apple provides a Music Understanding Lab sample app for visualizing results during development. Session #253 has the full walkthrough.
Security for agentic apps
Agentic apps that take real actions on behalf of users have a different threat model from passive AI features. Session #347 covers this for both Foundation Models and App Intents.
The three main risks:
- Indirect prompt injection: malicious content in retrieved data hijacks the model's behavior
- Data exfiltration: model is manipulated into leaking user data
- Unintended actions: agent takes actions the user didn't intend
Recommended mitigations:
- Require explicit user confirmation before consequential actions
- Design prompts that treat retrieved content as data, not instructions
- Use authentication for any actions that touch external systems
Which WWDC26 AI feature should you start with?
If you're an app developer who wants to add AI features quickly, start with Foundation Models. It handles model management for you and supports Claude and Gemini if you need a more capable model than Apple's on-device option. If you're an ML engineer with a custom or fine-tuned model, Core AI gives you the low-level control you need. If your app is already using App Intents, the new entity and intent schemas are a natural upgrade path to make your content and actions available through Siri.
For deeper dives, all the sessions referenced in this article are available at developer.apple.com/wwdc26.