Process tracing

Process tracing is a method used to evaluate and develop theories in psychology,political scienceor usability studies.

In process tracing studies multiple data points are collected in comparison to simple input-output methods, where only one measurement per task is available.

Contents

  

• 1 Thinking aloud
  • 1.1 Historical background
  • 1.2 Procedure
  • 1.3 The quality of verbal protocols in decision research
  • 1.4 Measurement criteria using thinking aloud protocols
  • 1.5 Advantages and disadvantages of verbal protocols 
• 2 Eye tracking
  • 2.1 Objects of regard
  • 2.2 Time Measures
• 3 Mouselab
  • 3.1 Designing
  • 3.2 Process Data

    Thinking aloud

    Thinking aloud protocols are a type of verbal protocols, used for eliciting and analyzing verbal data. Once transcribed, the verbalizations can be categorized into a defined scheme. This makes the data manageable.
    A thinking aloud protocol is one way to assess cognitive processes by letting people verbalize aloud what they currently think (concurrent verbal protocol) or what they were thinking (retrospective verbal protocol) while performing a task. Furthermore, depending on the type of the question asked, it can be distinguished structured verbal protocols and unstructured verbal protocols.
    It is assumed that the report of this verbal stream can be used as a proxy for the content of working memory and hence reflects the cognitive processes whilegnitive processes. Specifically, thinking aloud protocols can be helpful to study human decision making .

    Historical background

    Into decision process analysis, the use of thinking aloud protocols was introduced in the 1970s by Montgomery and Svenson. Since then the method has continually developed and has made valuable contributions to decision research (Schulte-Mecklenbeck, Kühberger, & Ranyard, 2011).

    Procedure (Walker, 2004)

    • data collection
    1) development of the task, based upon a hypothesis
    2) instruction and training of the participant
    3) recording audio of the performance
    • data reduction/content analysis
    4) transcription of the verbalizations
    5) definition of units of analysis (components, sequences, or complete models)
    6) definition of exclusive and exhaustive encoding categories
    7) encoding of the units of analysis into this scheme
    • analysis and interpretation
    8) quantification of the prevalence of components
    9) interpretation, in terms of the theory

    The quality of verbal protocols in decision research

    The quality of thinking aloud protocols depends on characteristics of the decision problem, as complexity, familiarity, importance, or whether singular or repeated. It is recommended to construct a decision problem with a medium level of complexity and importance and a low level of familiarity and little repetition (Schulte-Mecklenbeck, Kühberger, & Ranyard, 2011).

    Measurement criteria using thinking aloud protocols

    One general problem with thinking aloud protocols is their validity (Russo, Johnson & Stephens, 1989). We cannot prove whether the participant really says what he is thinking. Particularly the validity of retrospective verbal protocols is debatable. Verbal protocols are vulnerable to non-veridicality. A verbal protocol lacks of veridicality if it fails to adequately represent the process that should be described. Errors of omission and errors of commission are typical reasons for a non-veridical measurement. Errors of omission happen if thoughts that are part of the process are not mentioned. Errors of commission happen if the verbal protocol contains statements about thoughts that were not actually part of the process.
    Another important problem is the potential reactivity of concurrent verbal protocols . A verbal protocol is reactive if the verbalization process changes the cognitive process (Russo, Johnson & Stephens, 1989). This is particularly problematical when asking the participant why he’s doing something. performing a task.