Subjective video quality

Subjective video quality is video quality as experienced by humans. It is concerned with how video is perceived by a viewer (also called "observer" or "subject") and designates their opinion on a particular video sequence and therefore related to the field of Quality of Experience. The measurement of subjective video quality is necessary since objective quality assessment algorithms such as PSNR have been shown to correlate badly with ratings. Subjective ratings may also be used as ground truth to develop new algorithms.

Subjective video quality tests are psychophysical experiments in which a number of viewers rate a given set of stimuli. These tests are quite expensive in terms of time (preparation and running) and human resources and must therefore be carefully designed.

In subjective video quality tests, typically, SRCs ("Sources", i.e. original video sequences) are treated with various conditions (HRCs for "Hypothetical Reference Circuits") to generate PVSs ("Processed Video Sequences").

The main idea of measuring subjective video quality is similar to the Mean Opinion Score (MOS) evaluation for audio. To evaluate the subjective video quality of a video processing system, the following steps are typically taken:

Many parameters of the viewing conditions may influence the results, such as room illumination, display type, brightness, contrast, resolution, viewing distance, and the age and educational level of viewers. It is therefore advised to report this information along with the obtained ratings.

Typically, a system should be tested with a representative number of different contents and content characteristics. For example, one may select excerpts from contents of different genres, such as action movies, news shows, and cartoons. The length of the source video depends on the purpose of the test, but typically, sequences of no less than 10 seconds are used.

The amount of motion and spatial detail should also cover a broad range. This ensures that the test contains sequences which are of different complexity.

Sources should be of pristine quality. There should be no visible coding artifacts or other properties that would lower the quality of the original sequence.

The design of the HRCs depends on the system under study. Typically, multiple independent variables are introduced at this stage, and they are varied with a number of levels. For example, to test the quality of a video codec, independent variables may be the video encoding software, a target bitrate, and the target resolution of the processed sequence.

...
Wikipedia