・Authors: Masaru Shirasuna & Hidehito Honda

・Title: Can individual subjective confidence in training questions predict group performance in test questions?

・Journal(書誌情報）: PLOS ONE

・doi: 10.1371/journal.pone.0280984

・論文URL: https://doi.org/10.1371/journal.pone.0280984

・Abstract:
When people have to solve many tasks, they can aggregate diverse individuals’ judgments using the majority rule, which often improves the accuracy of judgments (wisdom of crowds). When aggregating judgments, individuals’ subjective confidence is a useful cue for deciding which judgments to accept. However, can confidence in one task set predict performance not only in the same task set, but also in another? We examined this issue through computer simulations using behavioral data obtained from binary-choice experimental tasks. In our simulations, we developed a “training-test” approach: We split the questions used in the behavioral experiments into “training questions” (as questions to identify individuals’ confidence levels) and “test questions” (as questions to be solved), similar to the cross-validation method in machine learning. We found that (i) through analyses of behavioral data, confidence in a certain question could predict accuracy in the same question, but not always well in another question. (ii) Through a computer simulation for the accordance of two individuals’ judgments, individuals with high confidence in one training question tended to make less diverse judgments in other test questions. (iii) Through a computer simulation of group judgments, the groups constructed from individuals with high confidence in the training question(s) generally performed well; however, their performance sometimes largely decreased in the test questions especially when only one training question was available. These results suggest that when situations are highly uncertain, an effective strategy is to aggregate various individuals regardless of confidence levels in the training questions to avoid decreasing the group accuracy in test questions. We believe that our simulations, which follow a “training-test” approach, provide practical implications in terms of retaining groups’ ability to solve many tasks.

・著者Contact先の email: m.shirasuna1392[at]gmail.com（[at]を@に変更してください。） (白砂大)

・日本語によるコメント:
物事を判断する際、目の前の問題に高い確信度を持つ人々からなる集団が、将来出くわす(現時点では未知の)問題においても正確な判断を行うとは限りません。我々はこの点を、行動実験と計算機シミュレーションから理論的に検証しました。特にシミュレーションでは、「行動実験の課題セットを二分し、一方を『目の前の問題』、他方を『将来出くわす問題』と見なす」という手法を考案して検討しています。本稿の最後では、本研究結果の実用的示唆についても考察しました。

投稿タグ