Apple has introduced a new look and enhanced capabilities for Siri, their virtual assistant. The updated Siri features a more integrated design within the operating system, allowing for more natural and intuitive interactions.

One key improvement is Siri's enhanced language understanding. Users can now speak to Siri more naturally, even if they stumble over their words, and Siri will comprehend the intent behind the request. This is enabled by the advancements in large language models that power Siri's capabilities.

Additionally, Apple has introduced the ability to type to Siri instead of speaking out loud. With a double-tap at the bottom of the screen, users can quietly and discreetly communicate with Siri, which can be useful in public settings or when voice interaction is not preferred.

Furthermore, Apple has integrated gesture-based interactions with Siri through the AirPods. Users can now simply nod their head to answer "yes" or gently shake their head to answer "no" when interacting with Siri, providing a seamless hands-free experience.

These enhancements to Siri's interface and interaction methods demonstrate Apple's focus on making the virtual assistant more accessible, natural, and adaptable to users' needs and preferences. By leveraging the power of large language models and integrating Siri deeply into the Apple ecosystem, the company aims to provide a more personalized and efficient virtual assistant experience.

What's great is that now at anytime you have the option to type to Siri with just a double tap at the bottom of the screen. I can quickly and quietly ask Siri to set an alarm, and you can switch between text and voice communicating in whatever way feels right for the moment.

This year we're making it even easier to interact with Siri for a seamless hands-free experience. For those instances when you may not want to speak out loud in response to Siri, like on the bus to work or in those places that are a little too crowded, we've created the ability to simply nod your head yes or gently shake your head no to interact.

The ability to type to Siri and use simple gestures like nodding or shaking your head provides a more discreet way to interact with your virtual assistant, especially in public settings. This integration of text and gesture controls expands the ways users can engage with Siri, making it more accessible and adaptable to different situations. By offering these alternative input methods, Apple is enhancing the overall Siri experience and making it more seamless to use in daily life.

Siri will have the ability to take hundreds of new actions in and across apps, including some that leverage Apple's new writing and image generation capabilities. For example, you'll be able to say "Show me my photos of Stacy in New York wearing her pink coat" and Siri will bring those up. Then you might say "Make this photo pop" and Siri will enhance it.

Siri will also be able to take actions across apps. You could say "Add this to my note with Stacy's bio" and it will jump from the Photos app to the Notes app to make it happen. This is made possible through significant enhancements to the App Intents framework, which allows apps to define actions for Siri shortcuts and other system experiences.

This won't be limited to Apple's own apps - developers will be able to use the App Intents framework to expose actions in their apps that Siri can tap into as well.

Apple Intelligence also has a deep understanding of your personal context, with a semantic index of things like your photos, calendar events, files, messages, and more. Siri can leverage this context to be incredibly helpful, like finding that book recommendation your friend sent you months ago, or automatically filling out a form with your driver's license info.

The key is that this is all done while protecting your privacy - your data is never stored or accessible to Apple, and independent experts can verify the privacy claims of the system.

Apple intelligence offers a suite of powerful writing tools that can help you communicate more effectively across your everyday tasks. These tools leverage the language understanding capabilities of Apple's AI models to provide intelligent assistance.

Rewrite: This feature allows you to generate multiple variations of your written text, enabling you to choose the version that best fits your needs. Whether you're crafting a cover letter, revising an email, or fine-tuning the tone of a message, Rewrite gives you the flexibility to experiment and find the perfect phrasing.

Proofread: The proofread tool analyzes your writing for grammar, word choice, and sentence structure, providing suggestions to help you polish your work. This is particularly useful when communicating with important stakeholders, such as your professor or a potential employer, ensuring your message is polished and professional.

Smart Reply: Apple intelligence can now generate intelligent response suggestions based on the context of your emails. When you need to quickly RSVP to an event or provide a brief update, Smart Reply will offer tailored options, allowing you to respond efficiently without having to compose the entire message from scratch.

These writing tools are deeply integrated into your Apple ecosystem, making them accessible across a variety of apps, from Mail and Notes to Pages and third-party applications. With Apple intelligence powering these features, you can communicate with greater confidence and clarity, saving time and effort in your daily tasks.

With thousands of emoji to choose from, there are times when you can't quite find the right one to express how you feel. Apple is introducing a new feature called Gen Emoji that leverages the power of Apple Intelligence to create personalized emojis on-device.

To use Gen Emoji, you simply provide a description, and Apple Intelligence will generate a custom emoji that matches the moment perfectly. The process happens right in your keyboard, allowing you to experiment and create as many images as you want.

While this feature may seem fun and engaging, Apple acknowledges the potential for abuse, similar to what was seen with Facebook's AI stickers. They recognize that Gen Emoji could be used to create inappropriate or offensive content. As such, Apple has implemented stricter guardrails compared to open-ended generative models like Dall-E.

The Gen Emoji feature is designed to provide a more controlled and curated experience, limiting the types of concepts and styles that can be generated. This approach aims to balance the creativity and personalization that users desire, while mitigating the risks of unconstrained generative capabilities.

Overall, Gen Emoji represents Apple's attempt to bring the power of generative AI to emoji creation, offering users a new way to express themselves in a more personalized and visually engaging manner. However, the company's cautious approach reflects the need to navigate the complexities and potential pitfalls of such technologies.

Apple is introducing a new feature called "Image Playground" that allows users to create generative images directly on their iPhone, iPad, and Mac. Here are the key points:

• Image Playground leverages Apple Intelligence, the company's AI system, to generate images based on user descriptions and concepts.

• Users can choose from a range of themes, costumes, accessories, and places to get started, and Apple Intelligence will generate preview images. No need to craft the perfect prompt.

• The image generation happens on-device, allowing users to experiment and create multiple images without the need to send data to the cloud.

• Users can adjust the style of the generated images, choosing between realism, sketch, or illustration.

• Apple is also providing a dedicated Image Playground app to make it easy to explore and create generative images.

• For developers, Apple is providing an API to integrate the Image Playground experience directly into their own apps.

The focus here is on making generative image creation accessible and intuitive for everyday users, without the complexity of crafting precise prompts. Apple's on-device approach also emphasizes privacy by keeping the image generation process local to the user's device.

Suppose you want a better image for your architectural history course. With Image Wand, you can circle your sketch using Apple Pencil to open up an image playground within your note. Image Wand uses on-device intelligence to analyze your sketch and words, and creates an image for you.

What's really fun is that you can even circle empty space, and it will pull out context from the surrounding area to suggest the ideal image to go with your note. It has never been easier to make your notes more visual and engaging.

Still, there are other artificial intelligence tools available that can be useful for tasks that draw on broad world knowledge or offer specialized domain expertise. We want you to be able to use these external models without having to jump between different tools, so we're integrating them right into your experiences. And we're starting out with the best of these - the pioneer and market leader, ChatGPT, from OpenAI.

First, we've built support into Siri so Siri can tap into ChatGPT's expertise when it might be helpful for you. For example, if you need menu ideas for an elaborate meal to make for friends using some freshly caught fish and ingredients from your garden, you can just ask Siri. Siri determines that ChatGPT might have good ideas for this, asks your permission to share your question, and presents the answer directly.

You can also include photos with your questions. If you want some advice on decorating, you can take a picture and ask what kind of plants would go well on this deck. Siri confirms if it's okay to share your photo with ChatGPT and brings back relevant suggestions. It's a seamless integration.

In addition to photos, you can also ask questions related to your documents, presentations, or PDFs. We've also integrated ChatGPT into the system-wide writing tools. With Compose, you can create content with ChatGPT for whatever you're writing about. Suppose you want to create a custom bedtime story for your six-year-old who loves butterflies and solving riddles - put in your initial idea and send it to ChatGPT to get something back she'll love. Compose can also help you tap into ChatGPT's image capabilities to generate images in a wide variety of styles to illustrate your bedtime story.

You'll be able to access ChatGPT for free without creating an account. And for ChatGPT subscribers, you'll be able to connect your account and access paid features right within our experiences. Of course, you're in control over when ChatGPT is used, and we'll ask before any of your information is shared.