How To Debug Reporting Anomalies

Have you ever tried to check the numbers for a campaign your team ran and the campaign code is nowhere to be found? Or check the KPIs for a website only to realize that not even basic visits and page views data has been recorded for the past two months. Still worse, you can pull some data, but right away you can tell that there is no way this data can be correct?

Having seen our share of data anomalies, we have put together a cheat sheet classifying the most common issues into five categories. The cheat sheet also provides debugging techniques, explanation of the possible causes, and recommendations on the type of automated audit steps you can take to prevent such anomalies from occurring in the future.

Download a print-friendly PDF

REPORTS CONTAIN NO DATA (NOT EVEN VISITS AND PAGE VIEWS)
ANOMALY TYPE	DESCRIPTION AND POSSIBLE CAUSES	RECOMMENDED AUDIT
Generic page tag missing	The tracking library is not loading on page view.	Scan the website to identify any pages where the generic tag is not firing.
Generic page tag not firing	Other errors are preventing proper execution of the tracking JavaScript.	Perform additional Scans to catch any fatal JavaScript errors.
Wrong reporting bucket	Generic page tag is firing and data is getting collected, but it is getting sent to a different reporting property or collection server.	Scan the website with checks configured to look for the reporting bucket identifier. Most common checks include verifying the report suite id in Adobe Analytics and the property id in Google Analytics.
Browser-specific issues	While your site might be tagged and sending data to the correct reporting bucket, browser-specific errors may be causing data to be missing from your reports.	Configure Scans to be ran from different user agents.
Vendor-specific requirements	Analytics vendors might have rules for acceptable values for certain built-in parameters. A good example is Adobe Analytics, where the currency parameter needs to follow a particular format, i.e. "USD" vs. "$". Passing a wrongly-formatted value will result in data getting discarded from your reports.	When setting up Scans for such parameters, enable Regular Expression-based rules for the list of acceptable values.
Environment-specific issues	In some government or other highly-controlled environments, network or anti-virus, anti-spam restrictions might result in data not reaching the collection servers. Depending on the type of web property you are responsible for, environment-specific issues can be negligible or really extreme.	Your only recourse in such cases is to lobby for lifting or making exceptions for some of these restrictions.
KPIS / METRICS / MEASURES / SUCCESS EVENTS ARE TOO HIGH
ANOMALY TYPE	DESCRIPTION AND POSSIBLE CAUSES	RECOMMENDED AUDIT
Double-tagging	The tracking library is loaded twice on the page resulting in more than one tracking request. Another explanation might include tracking functions being called more than once when the users interact with elements on your web property.	Use occurrence checks inside Scan or Flow tasks to make sure that a specific action is only tracked once.
Parameter is set at the wrong time	The intended tracking tag has been moved to a place where it no longer correctly tracks the intended user interaction. A common example for this category is tracking form submissions on form load. In some more difficult to debug scenarios the tag gets hijacked altogether to track unrelated interactions.	Configure Flow tasks to automate the validation of such scenarios.
Parameter has wrong value	In addition to tracking counts, some vendors allow you to assign numeric values associated with KPIs or various transactions. Watch out for commas, dots and other punctuation that may wreak havoc.	Configure Flow tasks to automate the validation of such scenarios along with rules for acceptable values of specific parameters.
Bots	For easily accessible pages (pages that do not require user input) beware of bots hitting these pages and inflating counts. In most cases bots will not be able to executive JavaScript, but as technology keeps evolving, keep an eye out for unexpected increases in traffic that can be attributed to bots. Reports based on IP or User Agent-related information can be a life-saver.	Perform investigation of this type of traffic using segmentation based on IP, Geography, or User-Agent related info.
KPIS / METRICS / MEASURES / SUCCESS EVENTS ARE TOO LOW
ANOMALY TYPE	DESCRIPTION AND POSSIBLE CAUSES	RECOMMENDED AUDIT
Generic tag is firing, but specific parameters are not set	The generic tag is set successfully, but the payload does not contain the specific parameter that is responsible for the KPI in question.	Configure Flow tasks.
Specific Parameter is set, but at the wrong place	The tag responsible for counting the KPI in question is set at a different place in the flow.	Configure Flow tasks.
Specific Parameter is set late on the page	The tag is firing too late on the page - visitors may move on to the next step of their journey and the tag never gets issued.	Configure Flow tasks.
Specific Parameter is set, but has the wrong value	In addition to tracking counts, some vendors allow you to assign numeric values associated with KPIs or various transactions. Watch out for commas, dots and other punctuation that may wreak havoc.	Configure Flow tasks to automate the validation of such scenarios along with rules for acceptable values of specific parameters.
Tag requires other tag	In the case of some campaign or e-commerce reports, tags need to fire in tandem with other tags.	Automated Flow tasks can verify multiple parameters at a time, ensuring that the payload contains all required values.
DIMENSION ANOMALIES
ANOMALY TYPE	DESCRIPTION AND POSSIBLE CAUSES	RECOMMENDED AUDIT
Generic tag is firing, but specific parameters are not set	The generic tag is set successfully, but the payload does not contain the specific parameter that is responsible for the dimension in question.	Configure Flow tasks.
Specific Parameter is set, but at the wrong time	The tag responsible for counting the KPI in question is set at a different place in the flow.	Configure Flow tasks.
Specific Parameter is set late on the page	The tag is firing too late on the page - visitors may move on to the next step of their journey and the tag never gets issued.	Configure Flow tasks.
Parameters are firing OK, but the tracking cookie responsible for visitorization is different	Most commonly occurs when the interactions span different domains, resulting in different tracking cookies getting set for the same visitor.	Configure Flow tasks.
Specific Parameter is set, but has the wrong value	The parameter is set, but it reflects a value that is not expected.	Configure Flow tasks to automate the validation of such scenarios along with rules for acceptable values of specific parameters.
CAMPAIGN ANOMALIES
ANOMALY TYPE	DESCRIPTION AND POSSIBLE CAUSES	RECOMMENDED AUDIT
Generic tag is firing but campaign query parameter is missing or misspelled	This occurs quite often with campaign tags missing or the campaign query parameter name having some sort of a spelling error.	Configure Flow tasks to check transitions from third party site to landing page.
Campaign query parameter is set but has the wrong value	Manually managing long lists of campaign codes can often lead to issues with the campaign code values.	Configure Flow tasks to enable checks for a range of acceptable values.
Redirects result in stripping of the campaign parameters	The query string parameters are composed correctly, but once you go to the landing URL, a redirect results in the overwriting of the query parameters.	Configure Flow tasks to check transitions from third party site to landing page
Wrong or non-existing URL	When the landing URL is successfully composed but it has not been promoted to production or simply does not exist.	Configure Flow tasks to check transitions from third party site to landing page.
Campaign parameters are correct, but exceed character limits	Some vendors have limitations on the number of chars contained in a particular parameter value. Similarly, parts of the HTTP tracking request may be getting stripped due to browser-related limitations.	Configure Flow tasks together with Regular Expressions to check the length of parameters.

Tags: Data Quality Whitepapers

How To Debug Reporting Anomalies

Try it real quick: