Design-focused evaluation

Design Focused Evaluation (DFE) is an approach to the evaluation of educational quality. In this approach the evaluation specifically focuses on the effectiveness of the constructive alignment in the course (subject, paper, unit) design. It was developed for use in the higher education sector but has relevance in training environments also, and may be broadly applicable in any educational environment.

DFE was developed by Calvin Smith, an educational researcher in Australia, to address the fact that most evaluation systems involve a focus on either inputs (the quality of the teaching, the quality of the resources, etc.) or outputs/outcomes (e.g. learning outcomes). Where both inputs and outcomes are included in an approach the items are separate (e.g. The Course Experience Questionnaire (Ramsden 1991) which has scales on inputs and a scale on generic skill development as outcomes). There was at the time no approach that focused on the alignment of the learning activities and the learning objectives.

DFE is primarily to be used to gather students' perceptions of the efficacy of the alignment (of teaching and learning activities with learning objectives) in the design of a course or unit of study (or part thereof). The reason for doing this is that it is considered a high-quality design practice that learning activities and learning objectives are aligned (Biggs and Tang 2007). Therefore, 'efficacy' here means 'the degree to which the teaching and learning activities supported the development of the learning outcomes'.

DFE is a survey-based approach to gathering evaluative data, therefore it generates quantitative data. The approach relies heavily on the way items are written for the surveys used. There is a clear articulation of how items can be written for the DFE approach in Smith (2008). In short, each question is composed in two parts conjoined by a grammatical structure that does the work of '...helped me to learn...'. The first part of the question indexes the teaching and learning activity (e.g. 'the lab session on dissection...') and the second part indexes the learning objective ('...how to dissect the human torso.').

One sophistication in this process is that on both sides of the item structure there can be varying degrees of specificity-generality (or granularity). Compare the following three items all of which are stable on the right hand side, but vary on the left hand side:

These items become increasingly granular, or more specific, moving from reference to all the lectures right down to a specific lecture session. The same issue of degrees of granularity/specificity can apply on the learning objective side of the item construction. Consider the following three items:

Again, the degree of specificity increases with each step.

This ability means the DFE approach is extremely flexible and allows the course designer to target specific aspects of the course design. This allows a more strategic approach to evaluation design, and allows for the selective and high-leverage use of the limited 'real estate' in any survey instrument.

...
Wikipedia