Process tracing
Process tracing is a method used to evaluate and develop theories in psychology,political science, or usability studies.
In process tracing studies multiple data points are collected in comparison to simple input-output methods, where only one measurement per task is available.
Thinking aloud protocols are a type of verbal protocols, used for eliciting and analyzing verbal data. Once transcribed, the verbalizations can be categorized into a defined scheme. This makes the data manageable.
A thinking aloud protocol is one way to assess cognitive processes by letting people verbalize aloud what they currently think (concurrent verbal protocol) or what they were thinking (retrospective verbal protocol) while performing a task. Furthermore, depending on the type of the question asked, it can be distinguished structured verbal protocols and unstructured verbal protocols.
It is assumed that the report of this verbal stream can be used as a proxy for the content of working memory and hence reflects the cognitive processes while performing a task. Verbal protocols can be used in any research that focusses on understanding cognitive processes. Specifically, thinking aloud protocols can be helpful to study human decision making.
The use of thinking aloud protocols was introduced into decision process analysis in the 1970s by Montgomery and Svenson. Since then, the method has continually developed and has made valuable contributions to decision research.
1) development of the task, based upon a hypothesis
2) instruction and training of the participant
3) recording audio of the performance
4) transcription of the verbalizations
5) definition of units of analysis (components, sequences, or complete models)
6) definition of exclusive and exhaustive encoding categories
7) encoding of the units of analysis into this scheme
8) quantification of the prevalence of components
9) interpretation, in terms of the theory
The quality of thinking aloud protocols depends on characteristics of the decision problem, as complexity, familiarity, importance, or whether singular or repeated. It is recommended to construct a decision problem with a medium level of complexity and importance and a low level of familiarity and little repetition.
One general problem with thinking aloud protocols is their validity (Russo, Johnson & Stephens, 1989). We cannot prove whether the participant really says what he is thinking. Particularly the validity of retrospective verbal protocols is debatable. Verbal protocols are vulnerable to non-veridicality. A verbal protocol lacks of veridicality if it fails to adequately represent the process that should be described. Errors of omission and errors of commission are typical reasons for a non-veridical measurement. Errors of omission happen if thoughts that are part of the process are not mentioned. Errors of commission happen if the verbal protocol contains statements about thoughts that were not actually part of the process.
...
Wikipedia