The Idea

Imagine you have a difficult problem and ask an expert for advice. Why not ask several experts? One expert could tell you something wrong, but you do not realize that he or she is wrong, because you aren’t an expert. If we have a list of experts at our disposal, what can we do with their advice?

• We could ask all experts and then find a ‘meta-expert’ who knows how to combine all the answers to a final answer. This is known as stacking.

• We could ask them independently and apply some simple aggregation function to their answers to obtain a final answer. If we ask for numerical values, we could take the mean. If we ask for categories, we could take the one appearing most often in the list of answers. That’s the basic idea of bagging.

• We could ask one expert, think about his answer and identify possible weak points in her or his answer. Then we go to the second expert, ask the same question but tell her or him to look at certain aspects more closely. Then we go to the third expert and add information about weak points in the second expert’s answer. We do this as long as we feel uncomfortable with the answers or as long as our list of experts isn’t exhausted. This strategy is known as boosting. Each expert boosts the answer of the previous one.

All three strategies have in common that each expert could be rather weak (not really an expert), but the final answer will be quite accurate.