fbpx
Learn to build large language model applications: vector databases, langchain, fine tuning and prompt engineering. Learn more

Ad Hoc Analysis trap – Here’s how to avoid the 3 most common mistakes

Data Science Dojo
Matthew Gierc

November 27

Maybe your boss isn’t Bill Lumberg, but if his understanding of analytics is limited to green and red (ad hoc), chances are you’ve rolled your eyes more than once.

Hello Peter, what’s happening? Ummm, I’m going to need you to go ahead and come in tomorrow to build that report. So, if you could be here around 9 that would be great, mmmk… oh oh! and I almost forgot ahh, I’m also going to need you to go ahead and come in on Sunday too, kay? We ahh don’t understand why our sales dropped this week and ah, we need to play catch up and analyze it.

Honestly how many times has this happened to you? Maybe your boss isn’t a Bill Lumberg, but if his understanding of analytics is limited to green = good and red = bad, chances are you’ve rolled your eyes in disgust more than once.

It happens a lot with Ad hoc analysis

And you’re not alone. Ad hoc analytics requests can make up 50% of an analytics team’s time. So what is a pragmatic analyst to do? According to Phil Kemelor, one strategy is to adopt a “don’t say yes” approach. Before committing to a request, ask yourself: • What is the business reason behind the request? • Will this kind of analysis answer the business question? • Do we know how long this will take? • Can we fit this into our funnel?

In other words, having an intake process to prioritize analytics requests can save teams a lot of weekend work.

When deciding whether to commit or pushback on a request, it’s also important to remember that your efforts will be wasted if the analysis cannot be acted upon. In other words, what will the business unit be able to do when they get access to your analysis?

Will the organization be able to make changes to improve a bad situation? Nothing is more frustrating than spending a weekend building a report that subsequently gets printed out and put on a shelf to collect dust.

1. Knowing the difference between driving the car and fixing the engine

Brent Dykes also emphasizes the importance of understanding the difference between reporting and analysis. Reporting is the process of organizing data to monitor performance; analysis is the process of exploring data and reports to extract insights.

The former helps a company ensure that everything is running well, the latter is an investigative tool used to figure out what’s going on “underneath the hood.” Organizations that don’t understand the difference between the two are more susceptible to ask for more ad hoc requests.

2. Doesn’t self-serve solve this problem?

What about self-service tools? After all, if any employee has the potential to become a citizen data scientist, then the demand for ad hoc requests should drop, right? Perhaps, but the costs to the organization might outweigh the benefits. Literally.

The ad hoc reporting promise fails when ad hoc reports: • Are treated like official reports shared broadly across the organization• Perform shallow analysis that lacks real insight• Are subject to the author’s own confirmation bias

3. Take a stand, for the right reason

Not everyone has the luxury of saying no to their Bill Lumberg. But, as a recognized data expert in your organization you do have something much more powerful – credibility.

In the long run, this means that you have the ability to shape your company’s data strategy, and ultimately wean the business off random ad hoc analysis. Start flexing those muscles today.

DSD Sign
Written by Matthew Gierc
Have a similar idea? Submit your guest post with us
Newsletters | Data Science Dojo
Up for a Weekly Dose of Data Science?

Subscribe to our weekly newsletter & stay up-to-date with current data science news, blogs, and resources.

Data Science Dojo | data science for everyone

Discover more from Data Science Dojo

Subscribe to get the latest updates on AI, Data Science, LLMs, and Machine Learning.