A voice-first fabric catalog for iPhone, iPad and Mac. Hold fabric up to the camera, describe it out loud, and on-device AI fills out the profile. Everything syncs through iCloud.
Everything runs on the device: Vision, Speech and Apple’s Foundation Models. No data leaves the phone except your own iCloud sync.
| Step | What does it | Where |
|---|---|---|
| Find the fabric, clip a square swatch | Vision foreground mask → largest inscribed square | FabricKit/.../SwatchExtractor.swift |
| Color tags | k-means on the swatch → plain color names | ColorAnalyzer.swift, RGB.swift |
| Pattern hints | Vision image classifier labels | PatternAnalyzer.swift |
| Speech → text, voice commands | SpeechAnalyzer + SpeechTranscriber |
FabricStash/Voice/SpeechListener.swift |
| Yards, width, fiber from what she said | Deterministic parsing | TranscriptParser.swift |
| Fill out the profile | Foundation Models (on-device, sees the swatch image) with guided generation | ProfileGenerator.swift |
| Hands-free flow | State machine: aim → describe → review → save | AddFabricCoordinator.swift |
| “Hey Siri, add fabric to Fabric Catalog” | App Intents / App Shortcuts | FabricIntents.swift |
| Sync and sharing | Core Data + NSPersistentCloudKitContainer (private + shared CloudKit databases) |
StashStore.swift, StashSharing.swift |
The profile streams in: the review screen appears right away and fields fill in live as the model writes them (Foundation Models streamResponse, about 50 updates over ~3 seconds).
Designer, collection, manufacturer, source and planned project are only kept if they were actually said or printed on the selvage, so the model can’t invent them. When the model and her words disagree, her words win: a spoken pattern, fiber or yardage always beats the model’s guess.
Each fabric can have a designer, collection and manufacturer (all optional; blank means “no designer”). They’re filled from what she says (“Heather Ross, Far Far Away, Windham”) and from text read off the selvage when it’s in the photo. The app knows about 175 designers and 85 fabric companies (FabricKit/Sources/FabricKit/Resources/fabric-brands.json, easy to add to) and learns every designer already in her stash. Those names also help speech recognition hear them correctly. Sort → Designer groups the catalog by designer.
Tap the microphone on the catalog and say what you want: “blue florals”, “anything for pajamas”, “maroon checked”. Search understands everyday color words (grey, ivory, burgundy, sage…) and pattern words (checked, polka dot, flowery).
Siri reads these answers aloud without opening the app:
Answers come from StashAnswer in FabricKit (deterministic and unit-tested), so amounts are always exact.
AimCheck needs a subject that isn’t just a face and, on phones with depth (TrueDepth on the front, LiDAR on the back of Pro models), a flat-ish surface big enough and 12–80 cm away. While she holds still and it isn’t fabric yet, the viewfinder says what would help (“Hold the fabric flat”, “Bring the fabric closer”). The flat surface also keeps the swatch crop off her face and clothes. “Capture” and the button take a photo of anything. Say “done” to finish. The preview is mirrored like a mirror, but the saved photo isn’t, so selvage text stays readable.VoiceTurn), so she always knows whether it’s listening: the orb glows and moves while it’s her turn and sits still while it isn’t. Outside her turn nothing it hears is used, which is what keeps a TV from changing anything. One rule to explain: when it glows, it’s listening; tap it to talk.CorrectionScope), never one she edited herself.SpeechFilter compares each sentence’s loudness with the room’s level over the previous 30 seconds (her own sentences and the app’s prompts left out). She’s much closer to the phone than a TV, so a sentence must be at least 6 dB above the room to be added (a quiet room accepts everything). When she has tapped to change something, it’s her, so there’s no loudness check.setVoiceProcessingEnabled, as in FaceTime): echo cancellation and noise suppression, with automatic gain off (it would erase the loudness gap SpeechFilter relies on) and no ducking of other audio. It also unlocks the phone’s Voice Isolation mic mode, which keeps the nearest voice and pushes down the TV and other talking. Apps can’t set the mic mode; she picks it once in Control Center and iOS remembers it for the app, so while adding fabric a small “Mic: Standard · Choose Voice Isolation” button under the orb opens the picker. Capture details record whether voice processing was on and the mic mode (device.voiceProcessing, device.micMode), and the export report counts them under “mic”. The app’s spoken prompts still go through AVSpeechSynthesizer outside the engine, so echo cancellation doesn’t cover them yet (the mic is still muted while it talks).VoiceGuide (FabricKit), and VoiceGuideTests runs every listed command through the real parser, so the guide can’t drift from what the app understands. Keep its Siri phrases in step with FabricShortcuts.CatalogSiriTip, iPhone and iPad): one small card at the top of the catalog, at the moment it’s useful. “Add fabric to Fabric Catalog” appears after her first fabric; “Ask Fabric Catalog” once the stash has 10. Each tip retires when she closes it, or by itself after 5 launches.FabricKit/ is a local Swift package with all the logic and its tests. It runs with swift test on the Mac, no simulator needed.FabricStash/ is the multiplatform SwiftUI app (iOS/iPadOS + native macOS). The camera code is iOS-only; the Mac adds fabric from photos.Config/ holds Info.plist additions and entitlements (iCloud/CloudKit, push for sync, Mac sandbox).T9RGDVM85M, same as Nerts) is already set on both targets with automatic signing, so Xcode creates the provisioning profiles on the first device build.com.remor.fabricstash is taken, change the bundle ID and update the container name iCloud.com.remor.fabricstash in both entitlements files.group.com.remor.fabricstash on both targets.Apple Intelligence must be on (Settings → Apple Intelligence & Siri) for the AI fill-in. Without it the app still works using measured colors and parsed facts.
She uses TestFlight builds only; development builds run on your own phone. The two use separate iCloud databases. Builds run from Xcode use CloudKit’s Development environment, and TestFlight uses Production, so fabric added in one never shows up in the other.
Once, before the first Production deploy: if any earlier build (the SwiftData versions, before stashes) ran signed into iCloud, the Development environment holds record shapes the app no longer uses (fabric with a photoData field, capture logs inside fabric). Deploying those would make them permanent. In CloudKit Console, Reset Development Environment first, then do the steps below.
Before her first build, and again after any change to the model (StashModel.swift):
iCloud.com.remor.fabricstash → Deploy Schema Changes…, deploy it to Production.Production schema changes are additive only: fields and entities can be added but never renamed or removed. Treat Stash and Fabric fields as permanent once deployed (CaptureRecord is local-only, so it isn’t in CloudKit). The model is defined in code (StashModel.swift), its header lists the rules, and DataModelTests.modelFollowsCloudKitRules fails the build’s tests if a change would break CloudKit.
Uploading: Product → Archive, then in the Organizer choose Distribute App → TestFlight & App Store. Leave “Manage Version and Build Number” on so each upload gets a new build number. The app declares no non-exempt encryption (ITSAppUsesNonExemptEncryption), so TestFlight won’t ask the export-compliance question. Add her as a tester in App Store Connect. An internal tester (a member of your team) gets builds without Beta App Review.
Capture logs record the app version and build number (e.g. 1.0 (3)). Tagging the commit you archived, e.g. git tag testflight-3, lets you match a Development Export to the exact code later.
With the phone plugged in (or on the same Wi-Fi with “Connect via network”), open Console.app, select the phone, click Start, and search subsystem:com.remor.fabricstash. Categories:
| Category | What it logs |
|---|---|
add-fabric |
photo analysis (fabric found, colors, pattern, selvage), AI timings, saves |
camera |
camera start (and depth source), auto-capture triggers and why it isn’t capturing (face, not flat, distance), photo timing, interruptions (calls, heat, other apps) |
speech |
language, audio route, speech-model download, recognition stopping |
ai |
model failures |
look-alike |
how many fabrics were compared and the nearest distance |
sync |
every iCloud setup/upload/download, with errors |
export |
exports and their failures |
Errors (problem) are also held in memory and attached to the next fabric’s capture log, so the same failures show up in a Development Export and in the importer’s Problems section.
Fabric lives in stashes. Everyone starts with “My Stash”; she can add more (e.g. “Quilting cottons”) from the stash button or by tapping the title. The catalog, Add Fabric, look-alike checks, Siri answers, the widget and backups all use the stash she’s in (StashSession, remembered in the App Group).
Sharing (Stashes and Sharing → ⋯ → Share…) uses the standard iCloud invite sheet: Messages, Mail or a link, with view-only or can-edit access. Everyone with edit access can add and edit fabric at the same time, which makes two people cataloging one stash practical. Tapping an invitation opens the app and switches to the stash once it syncs. People can leave a stash they were invited to; the owner manages people or stops sharing from the same sheet. In a shared stash, each fabric shows who added it.
How it’s stored (StashModel.swift, StashStore.swift):
CKShare.CaptureRecord is in the model’s Local configuration (not mirrored to CloudKit) and has no relationship to Fabric; tests enforce both.StashCleanup merges same-ID stashes and fabric within a store (never across stores).Testing sharing needs two Apple IDs on two devices (the simulator can’t). Check: share a stash from one, accept on the other, add fabric on both, confirm it appears on each, confirm “added by”, leave, and stop sharing.
Small and medium widgets (iPhone, iPad, Mac) show the fabric count, total yards, top colors and the newest swatches; tapping one opens Add Fabric. The app writes a small StashSnapshot to the shared App Group group.com.remor.fabricstash whenever the stash changes, and the widget reads it. Setup: in Xcode, add the App Groups capability with that group to both the FabricStash and FabricWidget targets (the entitlements files already list it). The layout lives in StashWidgetContent (FabricKit) and is rendered to Tests/Fixtures/report/widget-*.png by the tests.
Her data is never locked in: everything exports to ordinary files, and a backup restores fully.
Sort & filter menu → Export:
fabrics.csv (the swatch column names each image file), fabrics.json (every field), swatch-book.pdf, plus a swatches/ folder. A fabric keeps only the swatch she reviewed and saved; the full original photo is capture data on the device that took it (see Development Export).Settings → Your data has **Back Up “
Restoring: in the Files app (or Finder), tap the backup .zip once to unzip it, then choose the folder it makes, or the folder that contains it. The app never reads zip files itself; unzipping is left to Apple’s tools, so restore only reads plain files from a folder. It recognizes its own backups by a fabrics.json whose app is “Fabric Catalog”. It shows what’s inside and what it would add, then adds only the fabric she doesn’t already have in any stash. Fabric IDs survive the round trip, so restoring twice, or onto a device that already synced part of the stash, never makes duplicates. If she picks the still-zipped file, she gets the directions instead of an error. Restore also refuses image paths that point outside the backup folder, including through symlinks. Restored fabrics get their look-alike fingerprints rebuilt in the background. Capture logs aren’t part of a backup, so they’re never restored.
While she adds fabrics, the app keeps a small capture log with each one. It records what speech recognition heard, the voice commands she used, the model’s raw answer, what the app suggested, what she saved, the new names she confirmed or rejected, retakes, look-alike decisions, the crop box and the measured colors and pattern, the selvage text, and timings. Edits she makes later in the detail screen are noted too. The log, and the full original photo (the fabric itself keeps only the square swatch she reviewed), are development data, so they stay on the device that captured the fabric: it lives in a local-only store (CaptureRecord, linked to the fabric by ID), is never saved to iCloud and is never part of a shared stash. Since it isn’t in CloudKit, its format can keep changing freely. The trade-off: logs from captures on her iPad stay on the iPad, and a lost phone loses its logs.
Gear button → Settings:
diagnostics/ folder (one JSON per fabric) and a README.txt that explains each part. She decides whether to share it.On the Mac, turn an export into test material:
scripts/import-dev-export.sh "Fabric Catalog Development Export 2026-11-09.zip"
This does three things:
Tests/Fixtures/holding/ as real-*.jpg, with her saved colors and pattern as expectations in real-expectations.json. The real-photo tests and scripts/photo-report.sh then check them.Tests/Fixtures/real-cases.json. That file is the real set in PromptEval.Tests/Fixtures/report/dev-export-<date>.md. It shows how often each field was corrected, name answers, retakes, crop fallbacks, timings, problems (speech, camera, iCloud, saving), the devices and conditions (iOS version, TestFlight or development, heat, audio route), the nearest look-alike distance for each capture (for tuning the threshold), what the aim check saw at each photo (trigger, verdict, depth source, distance, how flat, crop on the surface) with the distance range of first-try captures (to tune AimCheck.Thresholds and find the distance to recommend), every sentence heard after the photo with whether it was used and its loudness against the room (to tune SpeechFilter), and every correction.Everything imported is git-ignored, so her photos and words never get committed.
Launch with -seedDemoData to fill an empty catalog with 10 generated sample fabrics, which is handy for the simulator and screenshots. In Xcode: Product → Scheme → Edit Scheme → Run → Arguments. From the terminal:
xcrun simctl launch booted com.remor.fabricstash -seedDemoData
It only seeds when the store is empty, and the code isn’t compiled into release builds.
Same setup as the Nerts project:
scripts/quality-check.sh lint # SwiftLint (also runs on every Xcode build)
scripts/quality-check.sh format-check # SwiftFormat (brew install swiftformat)
scripts/quality-check.sh test # FabricKit unit tests via SwiftPM
scripts/quality-check.sh all # everything, incl. xcodebuild analyze
Drop photos of fabric being held up into FabricKit/Tests/Fixtures/holding/ (optionally describe each one in expectations.json; see the README there) and run:
scripts/photo-report.sh
It runs every photo through the real crop, color and pattern pipeline on the Mac, checks the expectations, and opens a contact sheet showing each photo with the chosen crop drawn on it.
A designer, manufacturer or collection the app doesn’t already know (not in the built-in list, not already in her stash), or one she hedged about (“not sure if it’s Moda”), is never saved silently. In voice mode the app asks (“I don’t know that name yet. Is Juniper Hollow the designer?”); “yes” keeps it, “no” leaves the field blank. On screen a “New names: is this right?” row with Yes/No appears, and Save waits until it’s answered. Names she types herself count as confirmed. Once saved, a confirmed name is known from then on.
PromptEvalDesigner, collection, manufacturer and “where it came from” have to be trustworthy, so they’re measured against the real on-device model:
cd FabricKit && EVAL_PROMPT=1 swift test --filter PromptEval
It runs hand-written cases (what she says plus optional selvage text, with the correct answers) and reports, for the raw model and for the final result after the app’s rules, how often each field is correct and how often it is never wrong (blank is acceptable, invented is not). holdout2 holds cases the rules were never tuned on; that’s the honest number. Nov 2026, held-out: 90% correct, 96% never wrong, 100% never wrong without asking her first. Add transcripts of her real speech as new cases; that’s the test set that matters most. The rules live in FabricKit/Sources/FabricKit/AI/FieldRules.swift, and each one has a fast unit test in TrustRuleTests.swift.
ModelIntegrationTests calls the real on-device model and is skipped automatically where Apple Intelligence isn’t available.