Mode effect is a broad term referring to a phenomenon where a particular survey administration mode causes different data to be collected. For example, when asking a question using two different modes (e.g. paper and telephone), responses to one mode may be significantly and substantially different from responses given in the other mode. Mode effects are a methodological artifact, limiting the ability to compare results from different modes of collection.
Particular survey modes put respondents into different frames of mind, referred to as a mental "script". This can affect the results they give. For example:
Mode effects are likely to be larger when the differences between modes are larger. Face-to-face interviews are substantially different from self-completed pen-and-paper forms. By contrast, web-surveys, pen-and-paper and other self-completed forms are quite similar (each requiring respondents to read and privately respond to a question) and therefore mode effects may be minimised.
Users of surveys must consider the potential for mode effects when comparing results from studies in different modes. However, this is difficult as mode effects can be complex and subject to interactions between respondent demographics, subject matter and mode. Unless the mode effects are formally investigated for the survey instrument, it is difficult to quantify their size and qualitative judgments by experts familiar with the subject matter and respective modes are required instead.
Studies of mode effects are sometimes contradictory but some general patterns do emerge. For example, Social desirability bias tends to be highest for telephone surveys and lowest for web surveys:
Therefore as the data collected on sensitive topics (such as sexual behavior or illicit activities) will change depending on the administration mode, researchers should be cautious of combining data or comparing results from different modes.
Some modes require different question wording from others, in order to suit the features of the mode. For example, self-complete forms can use lists of examples or extensive instructions to help respondents answer relatively complex questions. By contrast, in telephone interviews, respondents are often limited by their working memory and are unlikely to understand a long question with multiple sub-clauses. Another example is a 'matrix' of questions, commonly found on self-complete forms, cannot be read out easily in a verbal interview; rather a matrix would generally need to be scripted as a series of individual questions.