Reduce noise to make better decisions

Noise reduction strategies are like washing your hands before eating a sandwich. You may not know what germs you're killing, but you know it improves your health. This decision-making error prevention is often thankless, as it can't be observed. Noise reduction are measures that prevent unwanted things or events from happening, but you don't know if it was the preventive measures, or something else, that caused the judgment impacting noise. Noise is an invisible enemy, and therefore a victory against it is also invisible.

What you have to do is a cost-benefit analysis. If washing your hands is extremely easy with soap and clean water, and not washing your hands will get you killed, the benefit of washing your hands is greater than the cost. For all decision hygiene projects, this analysis must be done to confirm its need and convince every one of its importance. In other cases, where the effort is great and the benefits little, it might be best to just tolerate a certain level of noise.

Principles of decision hygiene to improve collaboration

According to the book Noise: A Flaw in Human Judgment by Daniel Kahneman, Olivier Sibony and Cass R. Sunstein, there are six decision hygiene principles:

1. The goal of judgment is accuracy, not individual expression or tastes.
2. You must think statistically, make comparisons and take an outside, comparing view.
3. Structure and break down judgments into several independent components.
4. Resist premature overall intuitions. Kahneman says judgments should be a 2-step process. First, individually and independently make judgments on all components. Then, second, combine all components for the final overall judgment, not a quick 1-step process.
5. Obtain independent judgments from multiple judges, then consider aggregating those judgments.
6. Find comparisons to evaluate against (one thing over another thing) and develop relationship scales (framing range).

One way to improve decision hygiene and reduce the chance of noise and bias is to sequence the introduction of information. While collaborating, information should be provided when it is immediately needed. If it is delivered too early, it could create noise/biases, be distracting or be irrelevant. This is called "linear sequential unmasking."

Irrelevant information could be considered "noise" if it adversely impacts good judgment." Kahneman writes that "bias" pushes a judgment in only one direction away from the ideal, not in any direction like noise. Rules, standards, guidelines, formulas, and algorithms can reduce noise and judgment errors. These rules, formulas, and algorithms are not noisy and could be superior in accuracy. Here, unbiased software developers can be very helpful.

Think of the error chance difference between paying a machine cashier or a human when buying something in the store. Which has the higher chance of not giving you the correct change? Wherever there is human judgment, there is noise. If machines make the judgment, there is no noise, but totally relying on heartless computers may not be appropriate or welcomed. Furthermore, machines can obtain more information faster than humans. Considering this, we need to find a balance.

Sometimes, using a detailed checklist of factors impacting any judgment is helpful. After going through the list and considering those factors, better judgments can be made.

In other cases, general guidelines (which are more flexible than firm rules) can help reduce noise. Measuring scales, and comparisons, can also improve judgment. For example, structured, scripted job interview guidelines are always more reliable than unstructured, free speaking ones.

Complexity breakdown

In complex judgments, breaking the issues down into component parts, tasks and assignments can reduce wild variation in judgment. Those components can be evaluated separately, weighted and finally combined.

First word impact

The book Noise: A Flaw in Human Judgment calls this "anchoring" which reduces the chance of highly diverse judgments. The first word in a list becomes the anchor, which we have to fight to avoid putting extra weight on. We must apply critical thinking to this sequencing impact.

Consider this example list of words, "unprincipled," "cunning," "persistent," and "intelligent." Jumping to conclusions, we tend to anchor to the first word and give an overall negative judgment. If the words were instead ordered "intelligent," "persistent," "cunning," and "unprincipled," we might give a more favorable judgment.

Also, we should find quality comparisons (or anchors) to value against. Words like "best," "quality," and "good" could all carry different meanings to team members, but those words are more understandable when compared to something specific.

Making the right decision easy

Critical thinking requires effort. Therefore, to reach agreement easily, use justifiable information that is readily available and easy to understand.

Let's consider an example of a team making a decision. Imagine that your Open Organization team must agree on the price of a new product. To start the conversation, everyone suggests a price. If everyone's suggestions are around the same value, an agreement can be easily reached. But if the proposed prices differ greatly from each other, the group needs to find a comfortable middle ground. Here are some steps you might use to come to agreement:

1. Assign a facilitator to govern the judgment process. The facilitator asks each team member to individually consider what they think the ideal price should be and include reasons explaining why. They should then prepare a document of that price with those reasons and store the document somewhere that is not visible.
2. The members should wait for some time before reviewing the document. It could be overnight ("sleep on it"), a week later, two weeks later, or a month later depending on time available.
3. After some time has passed, and without looking at the concealed quotation, the facilitator asks the team members again for the ideal price and to give their reasoning. Then compare the two quotations and compare reason. Hopefully, different perspectives will come out, and they come up with a more accurate price.
4. The members might try to argue against their own reasons, and that's okay. Hopefully, each member will be in a different mood, under different time pressures and have a different energy/emotional level the second time, so different insights are generated.
5. Then, the team can collectively make a second document with these new insights after reviewing the original quotation (the original "anchor" that Kahneman and his team writes about).
6. After all the members have completed their revised document of the quotation and reasons for it, they submit their quotations to the team facilitator. The facilitator calculates the average quotation submitted and the standard deviation among them.
7. The facilitator prepares visual slides like a “target” illustration (below) of each document submitted, with each's deviation from the average. Then, the facilitator invites all the submitting members to attend a discussion meeting. He asks them to be open-minded to new information, perspectives and reasoning.

   

   If you consider "above the bull's eye" to be overpriced, "below the bull's eye" underpriced, "to the left of the bull's eye" one major reason and "to the right" another major reason, you can visually make difference comparisons. In Target A, all four prices and reasons are about the same (all close to each other). In Target B, all are underpriced to what they could sell the software for in the market, and for similar reasons. In Target C, one quotation is very high but has similar reasons for another very low quotation. Also, the other two have similar quotations but for very different reasons, one having very accurate reasoning. In Target D, two have similar accurate pricing but for similar inaccurate reasons. The other two have similar low prices, but their reasoning is not that accurate.
8. Looking at the visuals of each, one-by-one, invite the members to explain their quotation and reasoning. The order of the presentations are randomly selected. During each member's initial presentation, don't allow comments or evaluations until all presentations have been given.
9. Once all the presentations have been explained in detail, line up all the visuals side-by-side for comparison. Side-by-side, everyone can identify the quotations with the largest deviation from others, and their reasoning. Was it based on something no one considered? Did it rely on false information? Was it ego driven? This now can be discussed and explored while they notice similarities and contrasts of all the documents.
10. After discussion, the team comes up with an agreed on most accurate, important reasoning and determine a jointly decided quotation. This process could be repeated after the software has been on the market for a period of time to make pricing adjustments if need be.

This method will result in a more reasonable quotation with both less noise and less bias. With what I learned from reading Noise: A Flaw in Human Judgment and Influential Mind, the skills of collaborating should improve greatly.

For more information about noise in the decision-making process, and how to reduce noise, read Fighting life's distractions.