Cognitive bias in QA: the curse of knowledge

 According to Wiki, a cognitive bias is "a systematic pattern of deviation from norm or rationality in judgment. Individuals create their own "subjective social reality" from their perception of the input. An individual's construction of social reality, not the objective input, may dictate their behavior in the social world. Thus, cognitive biases may sometimes lead to perceptual distortion, inaccurate judgment, illogical interpretation, or what is broadly called irrationality".

What does it mean? In general, we are not the logical creatures that we think we are, sometimes even when we are at the crossroad of decisions and we think we decide in a balanced manner. The problem is that we are often not aware of these biases, and therefore can not "fix" our behavior.

An example of this is the curse of knowledge. The Curse of Knowledge, again from Wiki, "occurs when an individual, communicating with other individuals, unknowingly assumes that the others have the background to understand."


It is easy to see this bias in action when you get to a new job, and the person who is supposed to guide you uses "internal" concepts and initials on the unconscious assumption that everyone knows the interpretation. For example, I recently came across a place where, for historical reasons, the term "Front End" (which is usually accepted to be the side the client interfaces with) and the term "Back End" (which is usually the server side) are used in the opposite way. It took me a while to understand this because, for people who have used these terms for years, it seems natural and does not require explanations.

The connection to the software testing is in two different ways:

1. We all know that when new software testing engineers are recruited to an organization, they tend to find bugs that have been in the system for a long time but have never been reported. Since the testers are new, these are probably not very complex bugs, but things that were just lying there, begging for a little attention. We saw them all the time but didn't realize that there was a bug here.

The reason is not that we test poorly, but we are not aware of this weakness.

What do we do to minimize the phenomenon? First of all, try to improve ourselves: awareness of this during testing will already reduce the problem. In addition, imitation of specific made-up users (we try to imitate the use of the product of various predefined people, such as a girl who is very attached to technology, or an older man, etc.) and exploratory testing. These may refresh our point of view. The second thing (and it is worth combining) is to ask people in the organization who are not exposed to the product on a daily basis to work on it and give their impressions, the same thing about the testers of another product in the organization,

2. Someone who is very knowledgeable about a particular field assumes that people who recently have been exposed to it know more than they actually do. A person in the project for a year does not think to tell the tester who is new to the project that we do not even support Firefox specifically in this product (and then he tests a whole day, and reports 7 bugs, all in vain). This is also true of a new tester in the field who comes to a strong team and they assume that he knows more than he actually does, even if they know he is new in the field, which causes his adjustment period to grow.

Here awareness also can help, an atmosphere of openness to the questions of the new guy, and try to explain everything from the very beginning. Sometimes we are uncomfortable, but you can always ask whether the subject is familiar or not.

Comments