Your team is almost certainly using AI already. Someone is asking ChatGPT to tidy up a client email, someone else is pasting a spreadsheet into a chatbot to explain the numbers, and a third person is leaning on Copilot to draft a quote. Used well, these tools save real time. The problem is that most of this is happening quietly, with no rules, and the information going into these tools does not always stay private. For an Australian business, that is where a helpful shortcut can turn into a data breach.
This guide explains, in plain English, what actually happens to the data you type into AI tools, what the Australian regulators expect of you, and how to give your team the benefits of AI without handing your customers’ information to a system you do not control.
The AI tools your team is already using
Generative AI is the technology behind chatbots and writing assistants such as ChatGPT, Microsoft Copilot, Google Gemini and Claude. You give them a prompt, which is just a request in ordinary language, and they produce text, code, summaries or images in seconds. They are genuinely useful for drafting, summarising long documents, brainstorming and cleaning up messy notes.
The catch is how easily business information ends up inside them. To get a good summary of a contract, someone pastes the whole contract in. To fix a payroll spreadsheet, someone uploads the file with every employee’s pay in it. To reply to an unhappy customer, someone drops the entire email thread, name and all, into a chatbot. Each of these feels harmless in the moment. Together they add up to a steady leak of sensitive data into tools your business has not vetted. This is a close cousin of shadow IT, the apps staff sign up for without telling anyone.
What actually happens to the data you paste in
Here is the part most people never stop to check. With many free and consumer versions of these tools, the information you enter can be stored on the provider’s servers and used to help train or improve their models. The Australian Cyber Security Centre warns that some providers use customer submitted data to refine their systems, which can lead to that data being reused or disclosed in ways you never intended.
Once information leaves your business and enters one of these platforms, it becomes very hard to track, control or delete. You cannot easily pull it back. If the provider suffers a breach, or if a setting changes, your data is caught up in it. The Office of the Australian Information Commissioner puts it bluntly: businesses should refrain from entering personal information, and especially sensitive information, into publicly available AI tools, because once it is in, it is extremely difficult to remove.
There is a second risk that has nothing to do with privacy. AI tools can be confidently wrong. They invent facts, a habit known as hallucination, and they can be tricked by malicious instructions hidden in a document or web page. So the output needs a human check before it goes anywhere near a customer, a contract or a financial decision.
Information that should never go into a public AI tool
A simple rule helps here: if you would not post it publicly, do not paste it into a consumer AI tool. In practice, the following should stay out of any public chatbot unless you are using a properly configured business version with the right privacy controls:
- Customer personal details such as names, addresses, phone numbers, dates of birth and account numbers.
- Employee records, including pay, tax file numbers, health information and performance notes.
- Financial data such as bank details, card numbers, invoices and full accounting exports.
- Login credentials, API keys, passwords and anything else that grants access to a system.
- Confidential business material such as contracts, pricing, source code, strategy documents and anything covered by a non disclosure agreement.
- Health, legal or other sensitive information, which carries extra obligations under the Privacy Act.
If you genuinely need AI help with a document that contains these details, remove or anonymise the sensitive parts first. Replace real names with placeholders, strip out account numbers, and give the tool only the piece it needs to do the job.
What Australian regulators expect of you
You do not get a free pass on privacy just because a third party AI tool did the typing. If your business is covered by the Privacy Act, you remain responsible for the personal information you handle, including anything staff feed into an AI system. The OAIC’s guidance is clear that using these tools does not reduce your obligations under the Australian Privacy Principles, and that a staff member misusing a chatbot can expose customer information even when the business meant well.
The regulators point in the same direction. Before rolling out an AI tool, think through the privacy risks, be honest with customers about how their information is used, and put controls in place to keep personal data inside systems you actually manage. If a leak through an AI tool exposes personal information, it can trigger duties under the Notifiable Data Breaches scheme, so this is a compliance question, not just an IT one. For the bigger picture on where the rules are heading, see our guide to the Privacy Act changes for Australian SMBs.
A simple AI use policy for small business
You do not need a twenty page document. Most small businesses can get the protection they need from a one page policy that everyone actually reads. Cover these points:
- Which tools are approved for work, and which are off limits. Naming the specific tools removes the guesswork.
- What data must never be entered, using the list above as your starting point.
- A rule that a person always reviews AI output before it is sent, published or acted on.
- Who to ask when someone is unsure, so people do not just guess and paste.
- A reminder to turn off training or data sharing settings where the tool allows it, and to use business accounts rather than personal logins.
A policy only works if people understand the why behind it, so pair it with a short conversation rather than just an email. Most staff are not trying to be careless. They simply have not been told what happens to the data, and once they know, they usually make good choices.
Safer ways to get the benefits of AI
Saying no to AI entirely is neither realistic nor necessary. The goal is to get the productivity without the exposure. A few practical moves make a big difference.
Choose business or enterprise versions of the tools where you can. Paid business tiers of ChatGPT, Copilot and similar products often commit in writing not to train on your data, and they give you admin controls that consumer accounts do not. Before you commit, read the vendor’s privacy terms, confirm who owns the data and whether it is used for training, and favour providers that can show recognised security certifications such as ISO 27001. This is exactly the kind of check covered in our guide to running a vendor security review.
Where the information is genuinely sensitive, keep it in tools you control. Microsoft 365 Copilot working over your own tenant, or a private setup, keeps data within your environment rather than sending it to a public chatbot. And whatever you use, keep a human in the loop for anything that matters. Treat AI output as a helpful first draft from a fast but occasionally unreliable assistant, not as the final word.
Find out where you stand
AI is just one of the places modern business data can leak. Passwords, backups, email, supplier access and staff habits all matter too, and it is hard to know where your real gaps are until you look. That is what our free self assessment is for. Answer 27 short questions and you will get a personalised report mapped to the ACSC Essential Eight, ISO 27001 and SOC 2, with clear next steps for your business. It takes about 5 to 10 minutes and the PDF report is instant. Take the free security assessment and see exactly where you stand.