2026 Apple Notarization on Mac Cloud CI: notarytool Credentials, Common Rejections, and a Headless SSH vs Short Desktop Decision Matrix
Many teams already ship reliable xcodebuild archive jobs on Mac cloud runners yet stall on notarization: API key mistakes, opaque rejections, or mysterious failures that disappear on a local GUI session. This article gives a runbook-friendly split: classify failures with a taxonomy table, decide when SSH-only automation is sufficient versus when you need a short interactive desktop path, and execute a five-step notarytool sequence with polling, stapler, log retention, and rollback hooks.
In this article
1. Pain points: mixing signing, notarization, and upload
On self-hosted Mac cloud nodes, engineers often bundle three different chains: code signing, Apple notarization, and App Store Connect / TestFlight upload. They interact, but credentials and failure semantics differ sharply.
- Signing answers “who shipped this binary.” Provisioning profiles, certificate chains, and Hardened Runtime flags decide whether
codesignpasses. Failures usually surface during archive or export, with logs anchored inxcodebuildandcodesign. - Notarization answers “does Apple accept this distribution shape.” You upload a zip, dmg, or signed bundle for remote scanning. Rejections appear in
notarytoolJSON ornotarytool logoutput, not in local compiler diagnostics. - Upload answers “did the channel receive the build.” Transporter, successor tooling, or Fastlane
upload_to_testflightuse App Store Connect API keys on different endpoints than notarization. Sharing one secret bucket destroys layered troubleshooting.
Separating the three is the prerequisite for VPS-like SSH automation on Mac hosts. Without that separation, you burn nightly windows chasing “works on my laptop, flakes in CI.”
2. Failure taxonomy for CI notarization
Use the table during postmortems to decide whether to inspect entitlements first or API keys and egress first.
| Symptom | Likely root cause | First action |
|---|---|---|
| Immediate authentication failure | Wrong issuer, expired key, or truncated secret injection | Verify key role, .p8 path, newline handling in CI secrets |
| Stuck in progress | Apple queue, oversized payload, or extra heuristics | Keep submission id; pull structured logs; lengthen backoff |
| Hardened runtime / library validation errors | Unsigned helpers, risky entitlement flags | Run deep codesign --verify and diff entitlements |
| Passes locally with GUI, fails headless | Keychain unlock or interactive consent | Apply section 3 matrix; isolate a human-in-the-loop step |
3. Headless SSH vs short desktop sessions
Mac cloud shines when you manage launchd, SSH, and monitoring like a Linux build farm. Notarization still occasionally drags in legacy keychain or GUI assumptions. The matrix below is an engineering split; your security team sets the final boundary.
| Dimension | SSH headless default | Short desktop / VNC window |
|---|---|---|
| Credentials | API key plus non-interactive notarytool flags | Keychain prompts or tools that insist on GUI |
| Auditability | Logs land under the build dir and ship with artifacts | Harder to prove every click without extra controls |
| Stability | Great for 24/7 repetition and many branches | Keep rare; queue separately from heavy compile jobs |
| Mac cloud fit | Pin PATH via launchd; lock CLI toolchain versions | Dedicate a notary queue to avoid CPU and disk contention |
4. Five-step runbook: submit, poll, staple, archive, rollback
This sequence stays portable across CI vendors in 2026; always confirm flags against your installed Command Line Tools.
- Pin the toolchain. Record
xcodebuild -versionon the Mac cloud image and select it explicitly in CI so silent upgrades do not change notary behavior mid-sprint. - Prepare the payload. Capture sha256 for zip/dmg/pkg, ensure the CI user can read paths under the workspace, and reserve scratch space for large bundles.
- Submit and capture ids. Call
notarytool submitwith API key parameters; persist stdout and JSON toartifacts/notary/submit.logfor later correlation with Apple-side records. - Poll to a terminal state. Use
notarytool info(or equivalent) with exponential backoff during queue spikes; on failure, fetch human-readable detail vianotarytool logand attach it to your artifact store. - Staple, archive, rollback. On success run
xcrun stapler stapleon the distributable carrier; archive both the stapled bundle and logs; if stapling fails, block the release train and retain the last known-good stapled build.
Illustrative commands (replace paths and ids). Wrap them in your CI’s secret loader so keys never echo to console logs, and export NOTARY_SUBMISSION_ID as a dedicated artifact field for support tickets.
5. Reference numbers: disk, timeouts, concurrency
Use these figures in capacity reviews and SRE runbooks; validate against Apple documentation and your internal compliance baseline.
When you migrate from a Linux VPS mindset, remember that macOS treats temporary directories, keychain access, and Xcode-derived caches differently. A notary job that looks “lightweight” can still spike IO because Apple’s service unpacks your submission server-side while the CLI stages local copies. Planning headroom therefore belongs in the same spreadsheet as compile concurrency, not as an afterthought.
- Scratch disk: Notarization may unpack and recompress artifacts. After archives exist, keep roughly 25 GB of headroom on the system volume; below about 10 GB, refuse new notary jobs or purge old
DerivedDataand staleipaartifacts first. - Polling budgets: Split metrics for “submitted” versus “accepted.” For a single submission, cap polling between 45 and 90 minutes depending on payload size so hung jobs surface alerts instead of silent stalls.
- Concurrency: Parallel
notarytoolruns on one Mac cloud host contend for CPU and disk IO. Limit notary concurrency to one or two lanes and schedule away from peakxcodebuildwindows. - Log retention: Store JSON and text logs for at least ninety days for security audits. Keep notary logs in a separate bucket from TestFlight upload logs so one mis-scoped key does not corrupt two triage streams.
- Key hygiene: Use a notary-scoped API key distinct from metadata-changing keys; rotate on a ninety-day cadence or whenever staffing changes.
6. FAQs and ordering with TestFlight upload
Should notarization run before TestFlight upload? For typical App Store flows you finish signing and required notarization or stapling before upload, but the exact order depends on export method and carrier. Write the order into the runbook to avoid Gatekeeper complaints while App Store Connect shows “ready.” Teams that distribute outside the store still benefit from the same layering: notarization artifacts and upload receipts should never share a single opaque log file.
Why intermittent TLS or proxy errors? Corporate egress often needs distinct HTTPS_PROXY and NO_PROXY rules for notary endpoints versus App Store Connect upload hosts.
Can we eliminate VNC entirely? Most modern pipelines can. If keychain prompts remain, prefer splitting secrets and a rare manual gate over permanently wide-open desktop sharing.
Laptop-based notarization breaks on sleep, travel, and local policy. Hosted per-minute macOS runners make large payloads and queue waits expensive and harder to reproduce. Teams that need predictable Apple tooling, auditable API keys, and less hardware toil usually fare better on dedicated Mac cloud capacity for the signing–notarization–upload chain; pair this article with the VPSMAC TestFlight guide to align jobs, secrets, and sequencing in one disciplined pass.