This is a problem I am sure almost anyone who works with data runs into from time to time. It is also something we need to teach our students. Which I am sure we all do. This post is simply elegant example of the problem.
We all know that definitions matter because sometimes the phenomenon we want to look at is very dependent on how it is defined. Sometimes a small change in the underlying definition is not just sufficient to change the level but also the trend in the time series. Take the US and the incidence of part time (PT) employment. If you use the BLS definition of PT employment as anything below 35 hours then since the 1970s there has been an increasing trend in the incidence of part time employment. If you use the OECD definition you will arrive at exactly the opposite conclusion.
What is interesting is that for the other national cases that I am presently looking at the definition only changes the level not the trend. That is to say, in the case of Canada the long term trend is towards the increasing incidence of part time employment even if on different definitions the prevalence is greater or lesser. So in the case of the US you can literally come to an opposing conclusion just by a subtle shift in the underlying definition of the phenomenon being investigated. Econometricians regularly face this problem and it will not do to simply choose one of the two definitions. Say you want to “test” the proposition that unemployment rates are correlated with the incidence of PT employment. Based on the US case you have good reason to believe that the results are definition dependent and you know the definition is arbitrary as the underlying phenomenon is a relative concept. I would be interested to know what econometricians council when they face this sensitivity problem.