An AI loophole in your pocket: Why Apple’s Writing Tools require a second look

Generative AI technology introduces convenient new productivity tools left and right. Since this landscape is changing so rapidly, using AI tools thoughtfully and securely requires attentiveness, especially if you are working with any proprietary or sensitive information.

Case in point: recent Apple OS updates that enable some AI features without the usual guardrails we recommend for organizational use.

The feature: Quick and easy writing assistance

Apple’s new Writing Tools are designed to help users proofread, rewrite, and summarize text across apps. By default, some of these tools use a private AI model, but others may hook into the free version of Open AI’s ChatGPT. 

The core capability of the feature is its ability to perform advanced tasks like proofreading, rewriting and rephrasing, and summarizing. Writing Tools can be accessed directly from the text selection menu (the same menu where “Cut,” “Copy,” and “Paste” appear) within virtually any text field across various applications, including Mail, Notes, Messages, and third-party apps. A typical user, drafting an email to a client in the Mail app, can have Writing Tools suggest a reply based on the message thread or take a lengthy message and boil it down to a few bullet points. 

In order to use the ChatGPT features, you need to have set up your own ChatGPT account — which can be just a free account. But that’s where the question of privacy gets muddy.

The loophole: A gap in governance

Most users won’t find it easy to distinguish between the privacy Apple promises generally for “Apple Intelligence” and the terms and conditions of your free ChatGPT account. The problem is that when you use free genAI tools, there is no guarantee that your input data (proprietary emails, drafts, or internal notes) won’t be used to get added to and train the large language model (LLM), which makes that input essentially public. This seamless OS integration, activated by a single menu tap on selected text, facilitates behavior we strongly advise against: sending work-related data through a free genAI application.

The end result is that many people taking advantage of the ChatGPT-powered Writing Tools features may be exposing work-related data in an unprotected way to Open AI’s LLM.

(Google has released a fix to close this loophole in its own apps, allowing administrators to disable Writing Tools within Gmail, Docs, and other Workspace apps, but only for Workspace Enterprise clients who have implemented strict mobile device management (MDM). Most small to medium-sized businesses have operated quite effectively at the Business Standard or Business Plus tier, which does not offer this fix.)

What you should do

• Awareness is the first line of defense. Ensure your team understands that extending Apple’s Writing Tools to ChatGPT features is a gateway to a free LLM that is not recommended or permitted by your IT department.
• Review your organization’s use of generative AI. If your staff is already using free tools to improve their workflow (many people are!), it’s time to ensure they have access to a paid version that protects organizational security.
• Participate in a cybersecurity assessment. This is precisely the kind of nuance that we review as a part of our cybersecurity assessments. We look for these gaps where default behavior may conflict with your business interests.

If you’re concerned about how AI tools are interacting with your company data, let’s start a conversation. Security isn’t just about firewalls and passwords anymore. It’s about knowing where your data goes when you click “Rewrite.”