The Sensitivity Problem and the Social Sciences: Warn your students

This is a problem I am sure almost anyone who works with data runs into from time to time.  It is also something we need to teach our students.  Which I am sure we all do.  This post is simply elegant example of the problem.

We all know that definitions matter because sometimes the phenomenon we want to look at is very dependent on how it is defined.  Sometimes a small change in the underlying definition is not just sufficient to change the level but also the trend in the time series.  Take the US and the incidence of part time (PT) employment.   If you use the BLS definition of PT employment as anything below 35 hours then since the 1970s there has been an increasing trend in the incidence of part time employment.  If you use the OECD  definition you will arrive at exactly the opposite conclusion.

*Please note the labels on the graph are inverted: fix to come.

What is interesting is that for the other national cases that I am presently looking at the definition only changes the level not the trend.  That is to say, in the case of Canada the long term trend is towards the increasing incidence of part time employment even if on different definitions the prevalence is greater or lesser.  So in the case of the US you can literally come to an opposing conclusion just by a subtle shift in the underlying definition of the phenomenon being investigated.   Econometricians regularly face this problem and it will not do to simply choose one of the two definitions.  Say you want to “test” the proposition that unemployment rates are correlated with the incidence of  PT employment.  Based on the US case you have good reason to believe that the results are definition dependent and you know the definition is arbitrary as the underlying phenomenon is a relative concept.  I would be interested to know what econometricians council when they face this sensitivity problem.


5 thoughts on “The Sensitivity Problem and the Social Sciences: Warn your students

  1. National data can be even more confusing and absurd. According to the United Kingdom’s Department of Work and Pensions a full-time job is sixteen hours or more. Only below that is defined as part-time.

  2. The second paragraph states that the national (BLS) definition produces an increasing trend and the OECD definition produces the opposite. But the graph shows “OECD Def.” increasing and “National Def.” decreasing.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s