In reading through some of my old research, I stumbled upon an important idea, that of cause and effect.  In academia we call that correlation and causation.  There are a lot of analytical tools to prove relationships, and while we don’t need to be experts on the different types of analysis, it is important to understand that there ARE different types of math that can be done to show different relationships.  One type of analysis is regression analysis.  This is used to predict outcomes, while correlation is used to look at the relationship between variables.  There is also something called Structural Equation Modeling which allows researchers to run simulations on data to answer related questions about that data.  What that means to the practitioner, or leader out in industry, is that there are a LOT of ways to examine data to find answers.  Is that a good thing or a bad thing?

That all depends.  If I have a topic that I think makes sense, perhaps the cause and effect seem plausible.  Take for instance a German study that found people who have the most sex make the most money.  In a narrative way this could make some sense, especially when you consider the study also found these more sexually active specimen are healthier and happier than (and probably sexier) than those who strike out more often.  “Wages are higher for those with extraversion and openness traits who are sexually active,” according to this study which surveyed 7,500 people in Germany (ref 1,2).  By this logic, we should all strive to have more sex if we want a raise.

But context matters, so let’s consider the impact factor (basically, the academic reputation of the journal based on how many citations it gets).  The publishing journal is sitting at an impact factor of 1.64 which ranks as 5,799 for this type of journal.  If other academics are not citing the work contained in the journal, does that mean it’s bad?  It’s hard to say a journal is bad simply because of a low impact score (cause doesn’t always equal effect), it looks like the journal just started in 2013, so maybe the newness has something to do with a low score?  Or maybe it is a bad journal, I can’t say without doing a large amount of work to investigate it, and that, I am unwilling to do with so many other things on my plate.  I’ll just say it has a low impact score and leave it at that.

I do not doubt that the results the author received indicated that having more sex leads to positive outcomes in earning, I am not accusing the author of manipulating the data.  But does that mean that having more sex CAUSED a higher outcome in earning?  Do you believe that the number of films Nicholas Cage appeared in can cause people to drown by falling into a swimming pool?  Or that per capita cheese consumption leads to people dying by becoming tangled in their bed sheets?  Or that the divorce rate in Maine determines the per capita consumption of Margarine?  Of course not, because those things have nothing to do with each other, but the numbers do correlate (ref 4).  Tyler Vigen talks about “spurious correlations” which are examples of how cause does not always mean effect (correlation does not equal causation).  This is easy to see in an extreme case such as Nicholas Cage acting roles and death by drowning in a swimming pool, but what if the proposed model is not so outlandish?  If we look at widely accepted (socially, if not academically) relationships, such as the idea that tall people make better leaders or extroverts are better at leadership than introverts?

All the tools I mentioned earlier are great, but they are not infallible.  Especially when you consider the arbitrary rules journals put in place to judge research.  Currently, a golden measure of statistical analysis is the p-value.  This is a value that gives probability that a hypothesis is more likely to be true than the alternatives presented (null hypothesis).  So, a p-value of less than 0.05 is required by most journals (that means the null hypothesis has less than 5% chance of being true).  What happens if I am over that 0.05 p-value?  Well, I can massage the data a bit and look at what null hypotheses would fall below that number and use that.  Its not exactly unethical, but it is at best self-serving and at worst goes against the scientific method.  Another trick of the trade is to report other numbers that support your claims that are over their “required” reporting levels.  You see, not all researchers present all their data with transparency, or even present the same format for outcomes.  That’s because there is no consensus in academia for what makes a study accurate.  We simply don’t know.  Other reported metrics are effect size, quantification of differences, confidence intervals, median difference with confidence intervals, the list goes on.

I am going to include a little research I wrote about statistical analysis to show the disconnect between research and practical application.  I know the information so far may have been a bit tricky to follow, but contrast it with the following excerpt from one of my papers:

The method of statistical analysis depends on the requirements of the application, that is to say that it is imperative to understand what kind of data the researcher will be using and what they are trying to accomplish with the data.  First generation multivariate methods, like multiple regression, are appropriate for evaluating constructs and relationships between constructs.  SEM provides advantages over traditional multivariate techniques by facilitating the analysis of “path diagrams when these involve latent variables with multiple indicators…latent variables are theoretical constructs that, prior to neuroscience techniques, could not be measured directly (such as beliefs, intentions, and feelings)” (Gefen et al., 2011). This is of particular importance in studying the perception of leadership and modeling its impacts.  “(T)he success of operations management tools and techniques, and the accuracy of its theories, relies heavily on our understanding of human behavior” (Bendoly et. al, 2006).  SEM is particularly useful for application in marketing and management research for determining cause and effect relationships, with “more than 100 published studies featuring partial least squares (PLS) SEM in the top 20 marketing journals” in 2009 (Hair et al. 2011).  In the context of behavior research, Iacobucci et al. (2007) show that SEM is superior to traditional multivariate techniques in assessing mediation questions and for enabling researchers to extend beyond basic inquiries (ref 8, 9, 10, 11).

Hopefully, my blog post is a little clearer than that excerpt, but the takeaway is that this is a complicated topic.  Just look at Lindsay, Pluckrose, and Boghosssian or the Sokal hoax as an example (ref 5).  In 2018, three authors wrote 20 fake papers and tried to publish in top tier journals.  They used jargon to argue for conclusions that didn’t make any sense.  7 of their articles had been accepted for publication in peer-reviewed journals, while only 6 had been rejected and the final 7 were still in review.  Ill get into the peer-review process in another blog, but just understand this shot a big hole in the credibility of academia.  In the late 90’s, Alan Sokal did something similar in getting a phony article published in the leading forum for famous scholars.  These ruses were in an effort to showcase low standards to serve progressive goals, but I want to take the politics out of it.  People published paper that supported their own beliefs, or what they wanted to hear, because it was a narrative the publishers could get behind.  Its new and provocative and would likely increase notoriety of the publishing journal.  If researchers can knowingly get bad research published on a lark, how many researchers accidentally publish bad results.  Even more nefariously, how many authors knowingly publish bad results because they are incentivized to get publications?

In addition to journals having made up measures of how they judge good and bad research, there are pay to publish options as well.  These won’t get an author credit toward tenure, but it does confuse the field and leads to some strange outcomes.  Look at Johannes Bohannon and his study on the slimming effects of Chocolate.  Dr. Oz and Oprah even bought into this idea.  Dr. Bohannon (a PhD in molecular biology of bacteria, nothing to do with his topic) is a reporter.  He has an amazing write up (ref 6) talking about how he fooled everyone.  How did he do it?  The underlying idea is that there was a narrative idea that people WANTED to believe.  The placebo effect is a very real thing, but this wasn’t even a placebo, they just measured an insane number of variables for a large group of people and viola, they found the correlation they needed!  There are countless examples of researchers and publishers not understanding their topic, but with enough letters after your name and a convincing narrative (that references enough of the folks who will be “peer-reviewing” your article) you obviously can get published.

Am I saying that we can’t trust any research?  Absolutely not but take it all with a grain of salt.  Just because a book or an article presents an idea (no matter how compelling) does not make it so.  We must be excellent judges of what works for us and take our own personal bias out of the equation.  We are our own worst enemies in evaluating our own performance, so how can we be expected to accurately observe our outcomes?  Even success in a field is not an indicator of being good.  I’m sure most everyone has had a terrible boss.  One who we wonder how they got into that position.  Maybe that is a case of positive reinforcement for negative behaviors, like the ideas presented in Marshall Goldsmiths book What Got You Here Won’t Get You There (ref 7).  Regardless, it is imperative that we take one simple thing away from this blog:  The leading authorities cannot define, measure, or prove their own hypothesis (regardless of how fancy the research sounds), so we cannot blindly accept it either.  We need to think critically about how we can apply the things that work within our own personal contexts, especially when it comes to leadership…or chocolate.  Also, Nicholas Cage has a new movie out, be careful around swimming pools! In the meantime, let’s work to be humble, work to be kind, and work to be a little better today than we were yesterday. Thank you for your time, and please keep an eye out for my upcoming podcast!

1. https://www.gawker.com/more-bang-for-your-buck-people-who-have-more-sex-make-1159315115
2. https://ftp.iza.org/dp7529.pdf
3. https://www.resurchify.com/impact/details/21100775634#:~:text=The%20impact%20score%20(IS)%202020,which%20shows%20a%20rising%20trend.
4. http://www.tylervigen.com/spurious-correlations
5. https://www.theatlantic.com/ideas/archive/2018/10/new-sokal-hoax/572212/
6. https://gizmodo.com/i-fooled-millions-into-thinking-chocolate-helps-weight-1707251800
7. https://marshallgoldsmith.com/
8. Gefen, D., Straub, D. W., & Rigdon, E. E. (2011). An update and extension to SEM guidelines for administrative and social science research. Manag. Inf. Syst. Q, 35(2).
9. Bendoly, E., Donohue, K., & Schultz, K. L. (2006). Behavior in operations management: Assessing recent findings and revisiting old assumptions. Journal of operations management, 24(6), 737-752.
10. Hair, J.F., Sarstedt, M., Hopkins, L., & G. Kuppelwieser, V. (2014). Partial least squares structural equation modeling (PLS-SEM) An emerging tool in business research. European Business Review, 26(2), 106-121.
11. Iacobucci, D., Saldanha, N., & Deng, X. (2007). A meditation on mediation: Evidence that structural equations models perform better than regressions. Journal of Consumer Psychology, 17(2), 139-153.