Data quality before your first AI project: 7 questions before the budget

Most AI projects fail on data, not on the model. Seven questions you can ask and check in a week, before the budget starts.

At the kickoff of an AI project, the conversation is usually about the model: which one to pick, what it will cost, when the demo will be ready. The question about data comes last, often as "well, we have it in our systems". Three months later it turns out that "we have it" means five Excel exports, two different definitions of a customer and a table nobody has refreshed since last autumn.

This article is a list of questions worth asking before you sign a contract with anyone, us included. Each one can be checked within a week, without a big team and without new tools.

Why this matters

Numbers tend to get exaggerated, so let's stick to those from credible sources. Gartner predicts that by the end of 2026 organisations will abandon 60% of AI projects that are not supported by AI-ready data. In the same publication it reports that 63% of organisations either do not have the right data management practices for AI or are not sure they do (a survey of 1,203 data leaders from July 2024).

RAND, in an August 2024 report based on interviews with 65 experienced data scientists and engineers, lists five main root causes of AI project failure. Inadequate data is one of them, next to a misunderstood problem, chasing technology instead of the problem, weak infrastructure and tasks too hard for AI. The report also mentions the often-quoted estimate that over 80% of AI projects fail, but notes that these are "estimates", not the result of its own measurements.

You can buy a model. You have to own the data. That is why the questions below are about data, not algorithms.

Seven diagnostic questions

1. Do we know which data this one use case needs?

Not "what data do we have", but "what data does this specific decision require". Demand forecasting needs sales, prices, promotions, stock levels and a calendar. It does not need the whole warehouse.

How to check in a week: take one use case and sketch on a sheet of paper which fields must go into the model, which system they come from and who enters them. Do it with the person who makes this decision every day, not just with the IT department.

Bad answer: the list starts with "everything we have". That usually means the business problem is not defined yet, and according to RAND that is the most common source of trouble.

2. Who owns this data?

Not the system, the data. A named person who can answer "is this field correct" and can decide to change its definition. A data owner does not have to be technical. In retail it is often the head of sales, in logistics the operations manager.

How to check: for each key field from question one, write down the owner. Call three of them and ask when someone last asked them about the quality of that field.

Bad answer: "it's in the ERP, so IT". IT manages the database, but does not know whether the order status "completed" means shipment, payment or the customer's signature.

3. Do the same terms mean the same thing in every system?

This is a consistency problem, and the most common one in companies that grew through acquisitions or several generations of systems. An "active customer" in the CRM is someone who talked to a salesperson last quarter. In invoicing it is someone who bought in the last 12 months. The model will learn one of these meanings, and the board report will show the other.

How to check: pick five terms that appear in management reports (customer, order, revenue, active user, margin). Ask for the definition from each system and from finance. Compare them on one slide.

Bad answer: three different definitions and nobody knows which one is binding. Before any model starts, someone has to decide and write it down.

4. How fresh is the data, and who will notice when it stops refreshing?

Freshness is the dimension that breaks quietly. A pipeline stops on Friday, and the first signal arrives on Tuesday, when the model recommends something based on week-old data.

How to check: for each table from question one, ask for the date of the last update and the expected frequency. The gap between them is the lag. Then ask a second question: is there an alert that catches this, or do we hear about it from users?

Bad answer: "it should refresh daily", with no way to check whether it does.

5. How many gaps, duplicates and impossible values are there?

Three of the classic quality dimensions come in here: completeness (how many fields are empty), uniqueness (how many records are the same entity entered twice) and validity (how many values defy common sense, like an order dated 1970 or a negative parcel weight).

How to check: you do not need a special tool. A dozen SQL queries on selected tables will give you a picture in one afternoon. For key columns, count the share of empty values, the number of duplicated identifiers and the number of values outside the expected range. If you already use dbt, the same checks are ready-made tests: unique, not_null, accepted_values and relationships (key consistency between tables). Each one is a query that returns the faulty rows, and the test passes when it returns none.

Bad answer: a key column has 30% gaps, or a customer ID repeats across thousands of records. It does not mean the project is doomed, only that cleaning has to go into the schedule and the budget.

6. Do we know where the data comes from and what happens to it on the way?

This is data lineage. The field the model receives may have gone through three transformations, one spreadsheet corrected by hand by an assistant, and a script written by someone who left the company two years ago.

How to check: pick one key field and trace it back from the report to the source. Count the steps and the people who could have changed it. If lineage is documented in a tool, it takes an hour. If not, it takes a day and teaches you more than any survey.

Bad answer: there is a step in the chain that nobody can explain. That step is the first suspect when the model starts producing strange results.

7. Are we allowed to use this data for this purpose?

This question comes up latest in Polish companies, and costs the most. If the data includes personal data (customers, employees, patients), the GDPR applies (RODO in Poland), including the data minimisation principle in Art. 5(1)(c): data must be adequate, relevant and limited to what is necessary for the purpose.

In practice, this means we do not feed the whole customer database into training because "it might come in handy". The European Data Protection Board, in Opinion 28/2024 of December 2024, stated that an AI model trained on personal data is not always anonymous, and that this is assessed case by case. Legitimate interest as a legal basis requires a three-step test: the interest, necessity, and balancing against the rights of individuals. The opinion also points to mitigating measures such as pseudonymisation or synthetic data.

How to check: among the fields from question one, mark the ones that are personal data. Ask the data protection officer (or a lawyer) whether the training purpose fits the purpose for which the data was collected, and whether every marked field is really needed. It is an hour-long conversation if you come with a list.

Bad answer: "the data is in the system, so we can". Or "we don't know whether those free-text fields contain personal data". In salespeople's notes fields, they usually do.

What to do if the answers are bad

Bad answers are not a reason to give up on AI. They tell you where to put the money first. A few rules.

Start with one use case, not the whole company. Bring the data it needs to a decent state instead of chasing perfect quality everywhere. The first results will show which gaps really hurt.

Appoint owners for the key fields and write the definitions on one page. It is the cheapest intervention and usually delivers the most. Without it, every further cleanup slides back within a quarter.

Add automated tests where data feeds the model. The same set of rules (uniqueness, gaps, ranges, freshness) can be written in dbt, Great Expectations or Soda, depending on what you already have. The tool matters less than the fact that a test actually blocks a bad batch of data before it reaches the model.

For data that comes from other teams, agree a simple contract: which fields, in what format, how fresh, and who gets told about changes. A data contract does not have to be a separate product. Sometimes it is a single YAML file in a repository, signed off by both sides.

Finally, count the cost. If nobody looked after the data before, cleaning and tests will eat a large share of the first project's time. Better to see that in the plan than on the second invoice.

What a quick data audit looks like in practice

A week is enough for a solid initial assessment, if someone leads it and the business people have time for conversations.

Day one: a workshop with the decision-maker, choosing one use case and listing the fields it needs (question 1). Day two: interviews with owners and an inventory of definitions (questions 2 and 3). Days three and four: profiling queries on key tables, a freshness check and tracing one field to its source (questions 4, 5 and 6). Day five: a conversation with the data protection officer and a summary.

The result is one table: for each field a rating (ok, needs work, blocker), an owner and an estimated cost of repair. Plus a decision: we start, we start after fixing X, or we pick a different use case. In the projects we run, such a table changes the order of work, and sometimes saves you from ordering a model that would have had nothing to learn from anyway.

If you want to go through these questions on your own data, write to us. We are happy to do it together, even before you decide on any project.