Assess analytics data quality against a real decision: set an error tolerance, check completeness, validity, consistency and timeliness, then reach a verdict.
7 min readBranislav Mateas
Copy post as Markdown
On this page
"Is our data good?" has no useful answer. The same report can be fine for spotting a trend and too weak for moving a budget. So ask a narrower question: is this data good enough for this decision, and what would have to be wrong for the decision to change?
That view has a long history. In their 1996 study, Richard Wang and Diane Strong defined data quality as data that are fit for use by data consumers, and argued that quality must be considered within the context of the task at hand. This article turns that idea into a practical assessment, with one worked example.
The example is hypothetical. The campaigns, leads and costs are invented to show the method, and none of it comes from a real account.
Start with the decision and an error tolerance
Imagine a marketing lead who wants to move budget from campaign A to campaign B. Both ran for 28 days, and the GA4 report shows this:
B looks 20% cheaper. Before checking anything, work out what would erase that gap. The two costs match if A really had 50 leads (25% more) or B really had 20 (20% fewer). So the tolerance is clear: an error of around 20% in one campaign, but not the other, changes the decision.
That one step saves a lot of work. An error that hits both campaigns equally matters much less here than a small error that hits only one. It also gives every check below a test to answer: does this finding move the gap?
Completeness: is anything missing, and is it missing evenly?
Compare the report with a source closer to the event. Here, that is the CRM, which stores a lead ID and the campaign from the form's UTM fields.
Campaign
GA4 leads
CRM leads
GA4 share
A
40
46
87%
B
25
30
83%
GA4 misses some leads, which is common. Declined consent and blocked scripts are two possible reasons, though this comparison alone cannot prove the cause. The useful part is that the shortfall is similar for both. On CRM counts, A costs about €43 per lead and B about €33, so B is still ahead, by about 23%.
The CRM is a better reference, not a perfect one. Check how it can fail too. Here, count the leads with an empty campaign field, because they cannot be assigned to either side.
The ISO/IEC 25012 summary treats completeness as having values for the expected attributes and records in a given context. The words "in a given context" matter. A missing city would not affect this decision. A missing campaign would.
Validity: are the values allowed and real?
Next, check that the records are what they claim to be. A short query shows duplicates and gaps:
SELECT
campaign,
COUNT(*) AS row_count,
COUNT(DISTINCT lead_id) AS unique_leads,
COUNTIF(lead_id IS NULL) AS missing_ids,
COUNTIF(email LIKE '%@yourcompany.example') AS internal_tests
FROM crm.leads
WHERE created_date BETWEEN '2026-09-07' AND '2026-10-04'
GROUP BY campaign;
The table, fields and dates are hypothetical, and the query has not been run. Adapt the rules to your own data: allowed campaign names, plausible values, and dates inside the period.
Suppose it finds one internal test in A and three in B. That leaves 45 and 27 real leads, or about €44 and €37 per lead. B's lead shrinks from 23% to about 17%. Nothing is broken, but the margin has moved closer to the tolerance. If you had only asked "are there any test leads?", you would have missed that they fell unevenly.
Consistency: does the number mean the same thing throughout?
Consistency has two sides: across sources and across time. Across sources, write down the definition of each number. GA4 may count a form submission event, while the CRM counts a stored record. The earlier comparison is only fair because both try to measure the same action.
Across time, split the period and look for breaks. This is the check most likely to change the outcome, because a total can hide a change in the middle. Suppose that B's landing page gained a required phone field on day 15. Assume spend was even across the period:
Campaign
Days 1 to 14
Days 15 to 28
A
23 leads, about €43 each
22 leads, about €45 each
B
18 leads, about €28 each
9 leads, about €56 each
Now the answer looks different. B's overall advantage comes entirely from its first two weeks. Since the form changed, B has been more expensive than A.
Nine leads is a small number, so this does not prove that the field caused the fall. Other things could have changed as well, and the drop could partly be chance. But the old total can no longer support a claim that B is cheaper today.
Timeliness and usability: can it support the decision now?
Timeliness asks whether the data has settled and whether it is recent enough. For GA4, Google says processing can take 24 to 48 hours and report data may change during that time. Key event attribution credit can also change for up to 12 days after the event. So a report that ends yesterday may still move. In this example, the CRM counts and the GA4 share give a cross-check, but a final GA4 comparison should wait until those windows have passed.
Recency is a separate question. A 28-day average may be settled and still out of date, as the form change shows.
Usability asks whether the person deciding can read the number correctly. Wang and Strong's framework also covers whether data are clearly represented and accessible to the people using them. Ask the decision maker what "lead" means. If sales only value qualified leads, and this report counts every form, then the report may be accurate and still answer the wrong question. No query on this table can fix that.
Turn the findings into a verdict
Avoid a single quality score. An average would blend a harmless gap with the one break that reverses the decision. Record each finding and its effect on the decision instead:
Check
Finding
Effect on the decision
Completeness
GA4 misses 13% and 17%
Small, as both are similar
Validity
1 and 3 test leads
Narrows B's lead to 17%
Consistency
B falls after a form change
Reverses the recent comparison
Timeliness
Recent GA4 credit may change
Recheck before final reporting
Usability
"Lead" is not "qualified lead"
Open question for sales
Then give one of four verdicts: use it as it is, use it with stated limits, fix it first, or do not use it for this decision.
Here, the data does not support moving the budget. It does support a smaller next step: review the form change, get qualified lead counts from sales, and compare the campaigns again over a period with no known break. That is a useful result, even without the answer the marketing lead hoped for.
If the numbers come from an e-commerce setup rather than lead forms, validating the tracking layer by layer is the natural next step when a check points to a technical fault.
Keep the assessment short and attach it to the report: the decision, the tolerance, the checks, what you found and the verdict. Whenever the decision changes, repeat the assessment, because the same data may now pass or fail for different reasons.
