You gain knowledge from information coming from many different sources including books, articles, blogs, conferences and all the discussions you have with other professionals. Being able to interpret, evaluate, assimilate, synthesize and apply the data you collect is called critical thinking and is an essential skill for anyone (including the developer).

“A persistent effort to examine any belief or supposed form of knowledge in the light of the evidence that supports it and the further conclusions to which it tends” Edward M. Glaser

The word critical derives from the Greek word kriticos, which means discerning judgment. The roots of critical thinking come from analytic philosophy (Greek Socratic tradition) and pragmatist constructivism (Buddhist teachings).

In this article, I’ll try to isolate the 3 most common steps to practice critical thinking which is similar to scientific skepticism.

1. Identify potential cognitive bias

Cognitive bias, such as the confirmation bias, is a pattern of deviation in judgment that occurs in particular situations. Everybody is more or less affected by it. The more you know about those biases, the less likely they are to affect your judgment negatively. Here are some well-known cognitive biases:

  • Confirmation bias: the tendency to search for or interpret information in a way that confirms one’s preconceptions.
  • Mental filter: inability to view positive or negative features of an experience, for example, noticing only a tiny imperfection in a piece of otherwise useful clothing.
  • Gambler’s fallacy: the tendency to think that future probabilities are altered by past events, when in reality they are unchanged.  This results from an erroneous conceptualization of the Law of large numbers. For example, “I’ve flipped heads with this coin five times consecutively, so the chance of tails coming out on the sixth flip is much greater than heads.”
  • Overgeneralization: Extrapolating limited experiences and evidence to broad generalizations.

Sometimes journalists, politicians and even experts are affected by the overgeneralization bias and write like this: “The scientists confirmed global warming”. Try to replace words like “the scientists” or “the experts” respectively by “some scientists” and “some experts” which usually reflects the reality. It will give a very different meaning to the text. Be aware that identifying another’s bias is easier than identifying your own and don’t forget some people will use tricks to consciously manipulate opinion.

2. Separate facts from opinions

Anyone can post anything online and this is a great opening for narcissist leaders and other fake experts with extrovert personality.  The internet is full of information coming from these sources and a lot is based on opinions rather than facts. A critical thinker is able to separate the two.

You will prefer references to recent scientifical studies. Serious papers will reference multiple sources. But as you will see in next point, mentioning references is not a guarantee that the information and its interpretation are correct.

Always verify the credentials of the experts.  Has the business expert only created one or two businesses or has (s)he created several ones facing difficulties? Is it easy to find information about them or does any data about their past seem hidden or difficult to reach?

3. Analyze the data

To be reliable, the source must be based on empirical data that is produced by observation or experiment. The theory based on the experiment must be refutable.

“A theory which is not refutable by any conceivable event is non-scientific”. Karl Popper

In human sciences, any experiment which aims to define a theory or methodology should be reproducible in at least 95% of the attempts. A good example of non scientific theory is the Freud’s Oedipus complex in psychology. There is no way to refute the theory because Freud states that if the behavior doesn’t appear, it’s because it is repressed.  There is no way to validate or invalidate the theory. Even if there is a possibility than the theory is true, there is no way to verify it and so it should treated with care.

Here are the research methods commonly used in human sciences:

  • Observation: usually the first step of research to attempt to identify potential causes of a behavior.
  • Surveys & tests: if you can’t observe thoughts, we can ask people to describe them. The problem with surveys is that you can’t be sure that the answers are correct.  Social desirability bias, demand characteristics, memory errors are some of the problems you will encounter in addition to the sampling bias. After that, when you interpret the results using correlational approach, it’s impossible to prove that changes in variable A causes changes in variable B. At best, this method can be used effectively to describe or predict a behavior (what), but not to explain it (why).
  • Case study: this is the most popular research method in software as it is easy to do. Observe a few persons and try to determine a pattern. You can’t really prove a causal effect,  just like with surveys. But like observation, this is a good first step for the experimental approach.
  • Experimental: the experimental approach is the only type of methodology that, if well conducted, is able to make causal statements.  They are very difficult to carry out, especially in the field of software development. A well conducted experiment will include the following:

Have you ever read a book written by successful entrepreneur or software developer that converted his own and unique experience into a methodology? Your critical thinking would force you to evaluate the methodology by calculating how many successes have been made out of the millions readers. How many of these would have been successes anyway even without applying the methodology? As a reader with critical thinking you will be able to take what is useful in the book and leave the rest for what it is: a case study at best, observation in most cases. This applies, of course, to any source of information.

But even the results of well conducted studies can be wrongly interpreted, consciously or not, by the person that mentions it. Sometimes it is even conscious: a great example is how some people are caught lying with statistics. Politicians against the decriminalization of marijuana claimed that studies showed 87% of heroin addicts started by using cannabis. Cannabis would  therefore lead inevitably to hard drugs. What they forgot to mention are the millions of people smoking cannabis that never use heroin. The information is true, but manipulated.  In fact, we could present a study that demonstrates that 100% of heroin addicts used coca-cola! Should we prohibit coca-cola?

The 3 steps

Developing a critical mind is not easy, and we must be prepared to accept that a certain number of our current beliefs are wrong. To summarize, here are the three steps to follow to ensure you won’t be intoxicated by the information you gather:

  1. Step 1: identify potential cognitive bias.
  2. Step 2: separate facts from opinions.
  3. Step 3: analyze the data.
To learn more

How much of your current knowledge and beliefs are opinions rather than facts?

Check the cognitive bias list on Wikipedia to learn more about them. You can read more about the different methods summarized above on this page.

Book recommendations:

            

Related posts:

  1. How to convince your boss
  2. The actor–observer asymmetry
  3. Are all software developers introverts?
  4. The problem of inductive business hypothesis generation