Thursday, August 15, 2024

The Fallacy of Baselines

In the past, I’ve recommended “establishing a baseline” as the starting point for tracking metrics. “Track a few quarters to establish a baseline,” I said. I now realize that’s wrong!

 

There’s really no such thing as a baseline, when looking at presales or sales metrics. Every data point is meaningful. Calling the first few quarters a “baseline” artificially biases ones perceptions of what “good” or “bad” is!

 

Example: If you start measuring daily outdoor temperature in June (Northern Hemisphere), then your first several months of “baseline” will force the fall and winter months to look particularly cold (or bad). 

 

What you are really interested in is the change from point to point, and then the aggregate change over time and slices of time. Is the first derivative (the velocity) positive or negative (and are things getting better or worse)? Is the second derivative (acceleration) positive or negative (and are things getting better or worse more and more rapidly?).

 

Examining your data longitudinally (over time) provides the ability to compare current numbers with their historical counterparts. Perhaps another way to perceive this is that every prior datapoint is a part of your baseline!

 

The takeaway is to collect data. Lots of data, and don’t wait start now! Because without data, your decisions are based on guesses, impressions, opinions, and assumptions. And you know what happens when you make assumptions!

 

No comments: