Blog of Sara Jakša

Using Information - Case of Reinhart's Report

There is a lot of talk about a student finding an error in the Reinhart and Rogoff report about correlation between public debt and economic growth.

There were three types of mistakes found.

The first mistake was an error in a spreadsheet.

Let’s be honest, that can happened to everybody. That is why we are having a peer-review process, to make sure mistakes a found and corrected. But even so, simply having another person look over the spreadsheet would have solved the problem.

While a lot of people found this the biggest problem, I don’t think it was. Out of the three, it was the smallest.

The second mistake was, that not all the data was included.

They did mention, that some data was not available at the time of the calculations being made, but that does not mean, that things like that should go unmentioned.

I heard about a lot of stories from years ago, when the pharmaceutical enterprise would have limited the data used, so the results would appear more favorable. That could mean, that a defective drug could have come on the market. Or that allergic reactions and unwanted side-effects could have become widespread with using.

These days, the methods to discovering them are widely used, so data manipulation like that have a lot lower chances on not getting discovered.

I am wondering, why do not apply similar methods to other disciplines. Maybe then the data exclusions like that wouldn't have happened.

The third mistake was the calculation method.

The economist have a bad habit. They tend to overcomplicate things. That is why we had some many different financial instruments before the recession, that nobody understood. It was one of the reason, there even was a recession.

The method used in this case was inappropriate, if they wanted to prove what they did. If the peer-review process would have worked as intended, thing like that could be prevented.

I am more wondering, why nobody wanted to go over the calculation, before the report’s finding were used to dictate the policies in the countries. I would have thought, that before the governments would use a research, they would want to check it first.

But that clearly didn’t happened.

On the end, the basic findings were the same, but the correlation was smaller. The underlying truth is still similar before and after.

But what if it wasn’t? What if the next time, there are going to be the quality of lives or in some disciplines the lives themselves on the line?

Which just means, that if the research is truly important, we need to make sure, there are people double checking it. Also, having some scepticism can help.