Fighting life's distractions

Looking at the Open Organization Definition principles of Transparency, Inclusivity, Adaptability, Collaboration and Community, I was wondering if there are times when it is not good to collaborate and discuss issues.

According to at least two experts, the answer is yes. This is because a discussion may be counterproductive to helpful information gathering. Based on the book Influential Mind by Tali Sharot, our minds are always being influenced by our surroundings. It impacts our recall at any given time. Therefore, interactions with people too early might bias and block information from being obtained. In a decision-making meeting, the person that first presents a suggestion will influence everyone, and the decision-making process could be biased. She calls it "group think." Yet that first comment may be based on poor data from a very outspoken person.

I just read the book Noise: A Flaw in Human Judgment by Daniel Kahneman, Olivier Sibony and Cass R. Sunstein. I noticed that Sharot was consulted for this book, which introduces two collaboration concerns, namely biases and "noise."

To explain the difference, the book takes the bias problem one step further. They offer four outcomes to a discussion, through the imagining of a "shooting gallery target" and hits on the target. Consider these hits as human judgment or decisions made correctly or wrongly. Simply, was it a successful, accurate decision or an unsuccessful, decision error? If unsuccessful, by how much was the whole team off what they wanted to achieve?

Everyone on Team A (no bias and no noise) hits the bullseye. This means they gathered, used, collaborated on and agreed on good information to reach the ideal result. They are not particularly biased or impacted by what Kahneman calls "noise." 

Team B (bias but no noise) was off target, but all in the same area, in the lower left. This means the team's judgment was universally biased by something or someone specific, as all the members hit the same area on the target. They believed in the same false information. They didn't have any "noise" to speak of though.

Team C (no bias but noise) judgment results were somewhat centered, but scattered around the bull's-eye except one. This means that many things had negatively impacted their judgment accuracy, except for one member. There was no single, interesting but false factor that influenced accuracy. This is what Kahneman calls "noise," those random unknown factors. It is different for each individual and very hard to identify, but it exists. It adversely impacts judgment though. If ignored, poor judgments can result, even on extremely important issues. Imagine being sick and going to five doctors to be cured, but all five doctors give different diagnoses (judgment). The book gives countless examples like this variance. This problem is far greater than we usually think, according to Kahneman. The book offers suggestions which could be collaboration methods in open organization communities. Hopefully, this will result in better community judgments and decisions.

Team D (both biased and noisy). This team is both biased and influenced by noise. This means someone, or something, has biased the group's judgment to push it off target toward the lower left direction. Also, other things impact each member's judgment which results in the spread between them. They may be ignoring important information or considering unrelated information.

So, why are they off target? It is because of human error due to bias (systematic deviation) or noise (random error). Once we learn the reason why, then bias or noise reduction measures can be put in place to improve judgment outcomes. Bear in mind that both are errors and should be reduced. Depending on the actual case, one error might be more important than another, but Kahneman's book mostly concentrates on random noise.

What is this hidden noise?

This "noise" can come from a variety of sources. It could be a person's stress level or mood while making a judgment. It could be unrelated intimidation or outside opinions. Furthermore, it could be distractions or dramatic events. It could be room temperature, or unnecessary, unrelated, irrelevant, useless information that distracts the judgment process. Furthermore, it could be the time of day which impacts everyone differently. 

In groups, does a person give a strong opinion that moves the discussion in the wrong direction? Also, who speaks last, who speaks with confidence, who is wearing black, who is seated next to you, who smiles, frowns or gives odd gestures? All these could have an impact on the group. This is called "social influence," a form of noise.

The importance of finding and reducing noise

If a team always has vastly different judgments, there will be poor and unjust outcomes. This is particularly true for extremely important decisions or judgments. To address this, first, the noise must be identified. Then, an appropriate strategy to address it must be collaborated on and adopted. This will result in teams or communities making better decisions.

Imagine that your Open Organization team is making a decision on the price of some software you jointly developed, and everyone offers their best quotation. If they all are around 10% more or less from the average, that would be normal, and an agreement can be easily reached. But, what if many are 50% to 90% more or less than the average, some doubling others? In that case, there is probably excess noise. 

Start with mediating assessment protocol

Before reducing noise, teams need to evaluate what noise that is present and determine how impactful that noise is on collaboration and decision-making within the team. Once identified, then the appropriate noise reduction protocols can be introduced, installed and supervised. Noise will always exist and cannot be completely eliminated, but it can be reduced.

In these types of "noise audits," many independent individuals should evaluate the situation. They observe the differing situations in which it is present. After confirming that noise is present, strategies can be put in place to reduce it, which tightens the wide-ranging judgments between team members.

Audits are conducted by observing how people evaluate certain things, like my software pricing case above. A team of qualified auditors are selected. They develop a jointly decided on, structured questionnaire of relevant questions, making sure to avoid irrelevant information. 

Auditors then ask each member for a quotation individually and get the reasoning behind that quotation. Each member being questioned should not know that all members are asked the same questions. With the answers given to the auditors, first they look at the variation between quotations (the average quotation and standard deviation determined). Then, they list all the perspectives side-by-side and compare reasoning between them by looking for similarities and contradictions. Finally, they evaluate each of the reasons. There could be important, informative perspectives by only one person's reasons, but other's reasons could be based on incorrect information. That all has to be exposed.

Sometimes, the observations of multiple independent judges are helpful. After the full independent evaluation, an aggregate, concise, understandable finding is formulated. The finding is presented to all the participants which leads to thought triggers, and stimulates quality discussion. This should take place only after all perspectives and components are assembled, explained and understood. If time permits, possibly all the perspectives should be evaluated separately by the judges only before any overall discussion takes place. Evaluate each perspective or factor by talking about them and evaluating each perspective jointly. After that, evaluate each perspective against all the other perspectives. With all the perspectives visible, a quality judgment can be made.

Judgments, not personal opinions

This refers to matters of accurate judgment, not matters of individual opinions or tastes, where differences are entirely understandable. For judgment calls, everyone should not be exactly the same, but in pretty close agreement. If there is massive disagreement, then noise is probably present. 

Let's look at the extremes, where noise is everywhere and where it doesn't exist at all. On the one hand, you have computer-generated decisions or hard rules that have no noise, as no human judgment is required. On the other, you have wildly different tastes or opinions on the issue at hand. Between them is judgment and a certain noise level when addressing an issue. Our goal is to reduce the major noise within that judgment environment (some call this the "expectation of bounded disagreement").

What you want to study in an audit is not just what each person's judgment is. You want to learn exactly what factors impact each person when making a judgment. Then, by exposing those factors, noise issues can be addressed, and a tighter agreement should result.

You can audit the noise in your team's judgments by performing a simple review. To examine the noise surrounding your team's decisions, follow these audit steps:

1. Break the decision into components to assess data.
2. Use outside views and comparisons for evaluation.
3. Keep all judgments independent and isolated from others.
4. Review each assessment separately.
5. Judgments should be individually made initially.
6. Delay all final decisions until a mass amount of components and perspectives are exposed. In some cases, each component's judgment might be far more important than any final decision. The final decision could at best be a continual work-in-process.
7. Expose all information in a carefully decided sequence and with common frames of reference.
8. Aggregate the findings from many independent judgments.

In a followup article, I'll go into more detail about how to reduce noise and improve collaboration.