Most AI features in business software work the same way. Your app sends text, and sometimes files, to an AI provider such as OpenAI, Anthropic or Google. The provider's model reads it and sends back an answer. If that text includes a customer's name, address, tenancy history or a complaint they emailed you, it is personal data, and you are still responsible for it.
This guide is for business owners and managers in the UK who are adding AI to an existing system, or buying software that already has it. It follows on from our checklist on what to check before using an AI-built app in your business. It covers:
where data goes when an app calls an AI model
what to check in an AI provider's terms
what UK GDPR expects from you
practical ways to send less data in the first place
the questions to ask your developer before any customer records are sent
Where your data goes when an app calls an AI model
A single AI request can leave copies of your data in more places than you might expect. Take a simple example: Real People Casting, the Glasgow casting agency whose new web application we are adding AI to. The agency manages thousands of adults and children on its books. Suppose the AI reads a production's brief and suggests members who might suit it.
If the feature is built the simple way, this is what happens:
Your app builds the request. It collects the brief, adds member profiles and instructions such as "suggest members who match this brief", and sends the whole package to the AI provider. Everything in that package, including names, ages, physical descriptions and children's details, leaves your system.
The provider processes it. The model reads the text and writes the suggestions. This usually happens in the provider's data centres, which may be in the US, the EU or elsewhere.
The provider may keep a copy. OpenAI generally keeps API abuse-monitoring logs for up to 30 days, while Anthropic normally deletes API inputs and outputs within 30 days. Different rules can apply to stored responses, files and other features. Both providers offer zero data retention options for eligible customers.
Your app stores the answer. The suggestions are saved in your own database. They now contain personal data too, so they need the same protection as the original profiles.
Logs keep their own copies. Developers often log requests and responses to help fix problems. Without care, full member details end up in error logs, monitoring tools and backups.
A better design sends far less. The AI reads only the production's brief and turns it into search criteria, such as age range, height and location. The app then searches the agency's own database itself, so member profiles never leave the system for that step. If the AI is used to rank or explain a shortlist, it receives only the few details that task needs, not names or contact details.
Good AI integration is not about giving the model more information. It is about designing the application so the model receives only what it needs. Because this example involves children's data and AI-assisted decisions about access to work, a data protection impact assessment (DPIA) would normally be required before the feature goes live. A person should also remain responsible for the final selection.
Deleting data gets harder too. When someone asks you to delete their details, removing the original record is not enough. Copies may sit in AI-generated summaries, stored suggestions, application logs, backups and the provider's own storage. Your retention and deletion rules need to cover all of these, not just the main database.
Two other routes are easy to miss. If your app uses a tool built on top of an AI model, such as a chatbot or document service, there may be two companies handling the data, not one. And if staff paste customer details into a free AI chat website, that information may be processed under terms your business has never reviewed or approved. We looked at how these separate tools add up for estate agencies in why separate AI tools can create more work.
What to check in an AI provider's terms
One of the first things to check is which terms govern your use of the AI service. The same provider can handle your data differently depending on whether you use a consumer account, a business subscription or an API.
Google's Gemini API terms show why the exact wording matters. For its free services, Google generally reserves the right to use what you send to improve its products, and human reviewers may read it. But for users in the UK, the EEA and Switzerland, the paid-service data terms apply even to the free tier. Developers offering apps to UK users must use the paid service. So check which terms apply to your location and use, rather than assuming every free plan works the same way.
These are the questions to ask. Providers change their terms, so check the current version rather than relying on what was true last year.
Will our data be used to train the model? If it is, parts of it could influence answers given to other customers. A good answer: your data is not used for training by default, and this is stated in writing.
How long are our requests kept? Every day a copy exists is a day it could be exposed. A good answer: a short, stated period, with an option for zero retention if you need it.
Where is our data processed? Data sent outside the UK needs a legal route and safeguards. A good answer: a named region, and a clear explanation of any transfers.
Who else handles our data? The provider may pass data to other companies, known as sub-processors, such as cloud hosts. A good answer: a published list, with notice before it changes.
Is there a data processing agreement? UK GDPR requires an appropriate written agreement with organisations acting as processors for you. A good answer: a standard agreement you can accept or sign.
How is the provider’s security checked? Independent checks show the provider’s security is more than a promise. A good answer: recognised certifications or reports, such as ISO 27001 certification or a SOC 2 report.
Can the provider’s staff read our requests? Some providers let staff review requests that have been flagged. A good answer: clear rules on when this happens and who can do it.
If a provider cannot answer these questions clearly, that is useful information in itself. It usually means the product is not ready for customer data, however good it looks in a demonstration.
What UK GDPR expects from you
UK GDPR does not have special rules that ban AI. The usual data protection rules apply, and adding AI makes some of them more important. In a typical business application, your organisation is the controller and the AI provider processes data on your behalf. The exact roles depend on how the service works and how the provider uses the data, so check rather than assume.
A lawful reason for the new use. You need a lawful basis for using personal data in this way. If customers gave you their details to manage a tenancy, using them for a related AI summary may fit. Using them for something new, such as profiling or marketing, may not.
A written agreement with the provider. You need a data processing agreement that sets out what the provider can and cannot do with your data. For most large providers this is a standard document attached to their business terms.
Safeguards for data leaving the UK. If the provider processes data abroad, the transfer needs a legal route. This is usually UK adequacy regulations for the destination country, or a safeguard such as the international data transfer agreement (IDTA). For the US, the UK-US data bridge covers companies certified under it. A good provider will tell you which route applies.
A risk assessment where it is needed. A data protection impact assessment (DPIA) is required when processing is likely to be high risk. The ICO's list says using innovative technology, including AI, needs a DPIA when combined with another risk factor, such as large-scale processing or vulnerable people. It also needs one for decisions about access to a service that rely on automated decision-making to any extent. The ICO explains this in when do we need to do a DPIA. Even when it is not required, a short written assessment helps you think the risks through.
An honest privacy notice. Your privacy notice should say that you use AI services to process personal data, what the data is used for, and which providers or types of provider receive it. Customers should not find out by accident.
Care with automated decisions. If AI makes a significant decision about someone on its own, such as turning down a member for work, extra rules apply. The Data (Use and Access) Act 2025 widened the lawful bases you can rely on for these decisions, but only with safeguards in place. People must be told about the decision, be able to challenge it and be able to ask for a person to review it. Special category data, such as health information, stays more tightly protected.
The simplest approach is to use AI to help staff decide, not to decide for them. That oversight must be real: staff need enough information and authority to question, reject or change what the AI suggests. Someone who approves every suggestion without looking is not providing oversight.
The ICO publishes guidance on AI and data protection that goes into more detail. If your use involves sensitive data such as health or financial records, take proper legal advice.
Ways to send less data
The simplest way to reduce risk is to send the AI provider less personal data. Personal data you never send to an AI provider cannot be kept or exposed by that provider. A well-built AI feature often needs far less than people assume.
Send only the fields the task needs. To summarise a maintenance request, the model needs the description of the problem. It does not need the tenant's date of birth, bank details or full tenancy history. Your developer can choose exactly which fields go into each request.
Remove or mask details before sending. Names, email addresses and phone numbers can be stripped out or replaced automatically before the request is sent. "Jane Smith at 14 Park Road" becomes "Person A at Address 1".
Swap details back in afterwards. Your app can keep a private list that matches "Person A" to the real person. When the answer comes back, the app puts the real names back in.
Masking reduces the risk, but it does not remove it. This is pseudonymisation, not anonymisation, and pseudonymised data can still be personal data under UK GDPR. The rest of the text may still identify someone, and automatic removal can miss details hidden in free text.
Keep some data out entirely. Decide in advance which kinds of data are never sent, such as health information, criminal records or payment details. Build that rule into the app, rather than relying on staff to remember it.
Keep your logs clean. Make sure error logs and monitoring tools record that a request happened, not the full customer details inside it.
These protections are much easier to build in from the start than to add after launch. Adding them later usually means reworking every place the app talks to the AI provider.
This is how we approach AI integration at Niletech. Connecting an app to an AI service is the easy part. The real work is deciding which tasks need an AI model at all, which should stay in the application's normal logic, which searches should run in your own database, what the model needs to see, and where a person must approve the result.
Before we build anything, we agree with you what data the AI can access and assess where it will be stored and processed. We choose providers and settings that do not use your data for training by default, and we put safeguards in place for any transfer abroad. Activity logs, with access and retention agreed for your project, are part of what we deliver.
Questions to ask your developer
Before any customer records are sent to an AI service, your developer or software supplier should be able to answer these questions clearly. You do not need to understand the technical detail. You need answers you could repeat to a customer or to the ICO.
Which AI provider will we use, and are we on a business plan or API rather than a consumer account?
Exactly which customer data will be sent in each request, and why is each piece needed?
Is any personal data removed or masked before it is sent?
Will the provider use our data to train its models?
How long does the provider keep our requests, and can we reduce that?
In which country is the data processed, and what legal route covers the transfer?
Is there a data processing agreement in place, and who has accepted it?
Where are the AI's answers stored in our system, and who can see them?
What do our logs record, and do they contain customer details?
Will the AI make any decisions on its own, or will a person always check the result?
If we stop using this provider, how do we make sure our data is deleted?
Do we need to update our privacy notice or carry out a DPIA?
A good developer will welcome these questions. If the answers are vague, or the feature is already live and nobody knows, it is worth pausing to find out before going further.
If you are still at the planning stage, our three-week AI discovery sprint answers many of these questions before you commit to a build. It checks whether your data is good enough and can be used lawfully and safely. You get a written recommendation either way.
Summary
Adding AI to a business app can be done safely when you know where the data goes and have chosen what to send. No system is free of risk, but most avoidable problems come from not asking the right questions.
In short:
check which terms govern your use, whether consumer, business or API, and which apply in the UK
check training, retention, location and sub-processors in the provider's terms
have a data processing agreement and a legal route for any transfer abroad
send only the data each task needs, and remember masked data is still personal data
make sure deletion covers AI outputs, logs and backups, not just the original record
keep a person meaningfully in charge of decisions that affect people
update your privacy notice, and carry out a DPIA where the risk is high
If you are planning an AI feature, or already have one and are not sure what it sends, we are happy to talk it through. Book a 30-minute call or send us a short brief. You'll speak to the person who would do the work.
Sources
All checked on 16 October 2026. Provider terms change often, so check the current versions before you rely on them.
