I have been working with data for over 10 years, and I've been dabbling with AI for a couple of years. I still think of myself as a beginner with AI, which I think the majority of small business owners are too. So, I wanted to round up some issues that I have found using AI generally, through creating dashboards without AI and when I asked it to help with some code/make up some data for dashboard*.
*Before I found Kaggle (a website database of free data to use in projects), I tried asking claude to make some data, I had mixed results, I'll refer back to this process a few times.
AI will present anything as fact
It will confidently tell you the answer, even if it is not right. When I asked Claude to help write a DAX formula for a rolling average, it gave me a function that doesn't exist in Power BI. It didn't say 'I'm not sure', it wrote the formula out like it was standard syntax. I only caught it because it threw an error when I pasted it in.
Even if your input is broken/incorrect (which AI won't be able to point out, so you need to be able too), AI can hallucinate and can give an explanation that fits the data, but isnt true.
Someone still needs to get the data
With certain platforms, you can absolutelty automate data exporting, however this does need setting up, inputting logins, the columns required and the repetition. No AI is bright enough to gather the data from multiple sources and then investigate it, because it takes alot of experience and training for a human to do it.
Ai doesnt use its brain
It doesnt really have a brain, only what you train it on, and what is on the internet. It won't know if you have a one off sale, unless you tell it about it, and some don't even ask you claryfying questions about what you have given it.
Say you had a flash sale in March. If you don't tell AI that, it'll treat that spike as a normal trend and build its whole explanation of your data around a pattern that isn't real.
Before I found Kaggle, I asked Claude to generate some sample sales data for a dashboard I was building. It gave me numbers, but they didn't hang together — trends that made no sense, totals that didn't add up across tabs. AI can invent data that looks plausible at a glance, but it's not the same as real, messy, human-generated business data.
Ai doesnt know business context
Just like my above point, AI has no clue unless you tell it, and even then it will only have a basic understanding. It can't read your mind and you shouldnt really let it read emails, if customer information is in there.
- A supplier issue caused a stock-out → AI reads it as falling demand
- You put prices up → AI reads it as customers dropping off
- You paused ads for a week → AI reads it as organic decline
It will always find an explanation. It won't tell you it's guessing.
Ai will describe what happened but doesnt know why
It can tell you that your revenue, might have increased month on month (any beginner analyst and most business owners can too), but explaining why takes someone who knows the business: was it the new product launch, the weather, a competitor going quiet? That's not something AI can pull out of a spreadsheet.
It won't probe your data more, unless you ask it to
AI answers exactly the question you asked, and stops there. A human analyst would look at your revenue dip and ask 'has this happened before, same time last year?' AI won't volunteer that unless you specifically ask it to check.
Can be risky pasting in sensitive data into a generic Ai tool
If you're pasting customer names, emails, or supplier pricing into a free AI chatbot to 'just get a quick answer,' that data may be stored or used to train the model, depending on the tool and your settings. As a small business owner, you're also responsible for that data under UK GDPR — so it's worth checking a tool's data policy before you paste anything identifiable into it
There is no accountability
The amount of times, I have had to tell Ai that it's wrong is unbelieveable and it just replies with 'oops, sorry my mistake' and carries on.ruse at. Had I not been able to point this out, it would have just carried on telling me more absolute bull.... Sometimes, you will have spent hours trying to implement what it suggests, only to find out it is wrong, and the time has just gone, no way to claim it back.
There's no flag, no correction log, no way to know how many other answers it's given me with the same confidence and the same problem. If I hadn't caught it, I'd have made a decision based on something wrong.
If you decide to use Ai, you need to understand your data and be able to spot, when it goes wrong.
After all this do you still trust AI with your data? If you are an AI expert, and you have found an AI platform that works for you, maybe it's an option, if you know what you are looking at.
If you've had AI confidently tell you something wrong, I'd love to hear it — drop it in the comments.