The dawn of a new Siri (AI) looms in the air. When it ships, the promise of just saying what you want done, and having Siri just…do it, should be realistic. Apple, true to their roots of being masterful storytellers in terms of product, had many examples on tap. “Hey Siri, what was the water bottle I was thinking of getting a few months ago?”, and BAM — Siri went through a mountain of personal context, and found the answer.
That stuff is obvious. But, what about your app?
“Hey Siri, what team did I last make a play for in Elite Hoops?”
How do we do that, what’s involved, and how should you think about Siri and your app over the summer? I think I have a fairly good grasp on it, and I’ll summarize it here. I’m skipping deep dives, and just keeping it to a rundown.
let concatenatedThoughts = """
This is based off of having on seen this WWDC session, so I might be missing some context. Though, from what I've seen, this is where the goods are at.
"""
First off, the parts involved. The whole process is the same — it’s still app entities for your app’s models, schemas for a particular definition of those if applicable, and a few APIs to make those known to Siri when they are on screen.
App Entity
First, each of your app’s data models that should participate in Shortcuts, an App Intent, or general system-wide intelligence should hav a lightweight AppEntity version. This isn’t the same as the model itself, but a lightweight version of it:
struct TeamEntity: AppEntity {
static var defaultQuery = TeamEntityQuery()
static var typeDisplayRepresentation: TypeDisplayRepresentation = .init(stringLiteral: "Team")
var displayRepresentation: DisplayRepresentation {
DisplayRepresentation(title: "\(name)",
subtitle: "\(rosterSize) \(rosterSize == 1 ? "player" : "players")",
image: DisplayRepresentation.Image(data: logo))
}
let id: String
@Property(title: "Team Name")
var name: String
@Property(title: "Roster Size")
var rosterSize: Int
@Property(title: "Roster")
var roster: [String]
}
Surfacing App Entities
From there, Siri needs to be able to find them. There are two routes. First, if a network trip is required or otherwise surfacing content takes more work, then you should use EntityStringQuery:
extension TeamEntityQuery: EntityStringQuery {
func entities(matching string: String) async throws -> [TeamEntity] {
Logs.appIntents.info("TeamEntityQuery: String query for term \(string)")
guard let dao = DAO.readOnly else {
Logs.appIntents.info("There is no database available for the intent to read from.")
return []
}
let courts = try await dao.fetchCourtsBy(name: string)
let entities: [TeamEntity] = try await TeamEntity.fetchData(for: courts)
return entities
}
}
More commonly, IndexedEntity is the best path forward. The system automatically indexes this content for you. In fact, most cases just need you to declare conformance:
extension TeamEntity: IndexedEntity {}
In fact, you can simply conform to IndexedEntity directly as opposed to AppEntity. Once you do that, you can identify which parts should be index for search, too, with @Property(indexingKey:).
This is, however, where I’m not entirely sure of the relationships between Siri and your app. If you fill type @Property(indexingKey: \.) — you’ll get a list of autocompleted things you can use here. I believe all of these tie into a predefined App Schema, and the properties within them. So, what if your app doesn’t fit into those? I’m not sure, but I don’t think you can use indexing keys in that case.
App Schemas
Speaking of App Schemas, and their associated domains, are basically juiced up App Intents. Apple has already trained Siri to the moon and back over them, it’ll understand context better, and participate in follow up questions and everything else in-between. They’ve done the work for you. So, if you app fits into one of these schemas, absolutely use it.
You do so by using a property wrapper above your entity declaration:
@AppEntity(schema: .messages.message)
struct MessageEntity: IndexedEntity {
// The text content of the message
@Property(indexingKey: \.textContent)
var body: AttributedString?
}
…where messages is the overall schema, and the message is the domain within it. Xcode will also autocomplete the fields you need to implement a domain. Again, if your app’s data fits into a schema, it’s 100% what you should use. What’s less clear to me is what you can do, or should do, and how far your app can participate with Siri AI if you don’t fit into one of these predefined boxes.
On-screen Awareness
Once you’ve indexed data and exposed it to the system, next you wanna take part in that whole “Yo Siri, tell me about X or Y”, where X or Y is something currently on screen. Two different routes here:
-
NSUserActivity: The tried and true API around, since what — iOS 8?, can help with singular content. For example, a photo.
-
View annotations API: Use this when you’ve got more than one singular thing that’s presented, like a list of stuff.
Together, these tie back to an AppEntity, which means everything is structured in a way Siri can understand. Here’s how they look:
struct MyPractice: View {
let practice: Practice
var body: some View {
VStack {
StuffAndThings()
PracticeContainer()
.userActivity("com.example.elitehoops.practice",
element: practice.asEntity()) { entity, activity in
activity.title = practice.title
activity.appEntityIdentifier = .init(for: entity)
}
}
}
}
And for view annotations:
struct MyPractices: View {
let practices: [Practice]
var body: some View {
ForEach(practices) { p in
PracticeView(practice: p)
.appEntityIdentifier(
EntityIdentifier(
for: PracticeEntity.self,
identifier: p.id
)
)
}
}
}
Handle Data Coming In and Out
Finally, getting data out and in. When Siri wants to chain together stuff, like “Hey Siri, email this practice to Jansyn.” — we’ve now got a few moving parts. There’s the “practice”, I need to export my own data, and then there’s the recipient, Mail. To export or import, look no further than Transferable.
There is some nuance depending on the context. If you’re trying to handle an incoming request against some existing content, then you’ll need IntentValueQuery. But, if that request means your app should creating a new model or data, then implement the importing bit when wiring up your transferRepresentation within Transferable.
In Apple’s session, they use this example:
struct ContactEntityQuery: IntentValueQuery {
func values(for input: [IntentPerson]) async throws -> [ContactEntity] {
let names = input.map(\.displayName)
let descriptor = FetchDescriptor<Contact>()
let contacts = try model.mainContext.fetch(descriptor)
let matches = contacts.filter { contact in
names.contains(where: { name in
contact.name.localizedStandardContains(name)
})
}
return matches.map(\.entity)
}
}
However, this is the part that feels slightly limiting. It’s typed to handle only an IntentPerson — something the system already defines. As far as I can tell, your app would have to use some value-known intent already, like IntentPerson or something from the App Schemas. Otherwise, you also wouldn’t be able to use IntentValueRepresentation — which is exactly what Siri uses for this type of stuff.
Here’s another example I found in the docs, which handles things bidirectionally:
extension TeamEntity: Transferable {
static var transferRepresentation: some TransferRepresentation {
IntentValueRepresentation(
exporting: { entity in
IntentPerson(name: .displayName(entity.name))
},
importing: { person in
ContactEntity(name: person.name.displayString)
}
)
DataRepresentation(exportedContentType: .utf8PlainText) { entity in
entity.teamSummaryText().data(using: .utf8) ?? .init()
}
}
}
So, I’m unsure of how to handle this beyond that. If our apps don’t fit into schema, where does that leave us? I haven’t found the answer to that.
Final Thoughts
New Siri looks fantastic. No doubt, integrating it will finally bring us to that “This is what I wanted” phase of the tech. It should be an epochal event for our little digital assistant. For us? The APIs are easy to use, and there isn’t much guesswork. My only question mark is how to best integrate when our data models don’t fit into an App Schema, and if I find concrete answers — I’ll update this post.
Until next time ✌️
