A team wants to use an AI assistant to summarize support tickets. The demonstration looks promising, and the subscription fits the budget. Then someone asks whether the tickets can be used to train the vendor’s models. The answer depends on the service, the agreement, and the settings on the account.
Before uploading a batch of customer records, map the information involved and ask the vendor to show how the proposed setup handles it. A short conversation with your privacy and security leads can save a much longer conversation with customers later.
Start with the task and the data
Write down what you want the tool to do. For the support team, that might be: “Draft a summary of each ticket for an employee to review.” List the information the tool will receive, including attachments, account details, and any notes pulled from connected systems. Include the output and logs in that inventory too.
Then ask what is necessary for the task. Could the team test with fictional tickets or remove fields the summary does not need? Removing names alone does not guarantee that a record is anonymous. Keep the first test small and use information your organization has approved for that purpose.
The NIST generative AI profile describes privacy and security risks associated with generative AI, including risks introduced by third-party integrations. It is a voluntary framework, and its guidance can help structure a review; it does not approve a particular vendor or make a deployment safe by itself.
Ask how the information will be used
Ask the vendor for answers that apply to the exact product, account, contract, and features you plan to use. A consumer account and an organization-managed account may have different terms.
- Are prompts, files, outputs, or feedback used to train or improve models? Are there exceptions, and are any opt-outs enabled by default?
- Can staff or contractors access the information for support, abuse monitoring, or other purposes? Under what conditions?
- Which other companies process the information to provide the service, and how will you learn if that list changes?
- Where is the information stored or processed, including backups and support systems? Are there cross-border transfers to assess?
Ask for the relevant contract terms, data-processing agreement, and product documentation. A sales answer is a starting point; make sure it matches the agreement and the settings your administrator can verify. If the answer differs by feature, document those differences before enabling the feature.
Check retention, deletion, and exit
“We do not train on your data” answers only one question. Find out how long prompts, uploaded files, outputs, logs, and backups are retained, and whether the periods change when you delete a conversation or close the account. Ask what the vendor can delete, what may remain for security or legal reasons, and when deletion occurs.
Also ask how you can export useful records in a usable format. If the service is discontinued or you decide to leave, who can retrieve your data, and for how long? Record the answers alongside the contract and give someone responsibility for revisiting them when the service changes.
Review access and connected tools
A chatbot that handles pasted text has a different exposure from an assistant connected to a shared drive. Ask which accounts can use the service, how access is granted and removed, and what administrators can see or control. Review any proposed connections before turning them on: which folders, mailboxes, or applications can the assistant access, and what actions can it take?
Start with the minimum access needed for the task. For example, give a pilot access to a selected collection of approved support articles rather than an entire company drive. Test the permissions with the people who administer those systems, and check how to revoke the connection. Have an employee review drafts before they are sent to customers.
Decide who owns the follow-up
Ask your privacy, security, and legal teams to review the answers in light of the data and jurisdictions involved. The UK Information Commissioner’s Office notes that buying an AI system from a supplier does not remove an organization’s data protection responsibilities, and recommends supplier due diligence and agreement on responsibilities. Roles can vary across the AI supply chain, so do not assume a contract label settles every question. See the ICO’s supplier guidance and its discussion of roles in the generative AI supply chain.
Before the pilot starts, name an owner for the service. Give employees a short explanation of which account to use, what information is permitted, and where to report a concern. Set a date to review the vendor’s terms, settings, access, and any incidents. Repeat the review when you add a new data source or change the way the tool is used.
If you would like help assessing a proposed use or preparing your team to use an approved tool, explore our AI strategy and AI training services.