Spectral mask

In telecommunications, a spectral mask, also known as a channel mask or transmission mask, is a mathematically-defined set of lines applied to the levels of radio (or optical) transmissions. The spectral mask is generally intended to reduce adjacent-channel interference by limiting excessive radiation at frequencies beyond the necessary bandwidth. Attenuation of these spurious emissions is usually done with a band-pass filter, tuned to allow through the correct center frequency of the carrier wave, as well as all necessary sidebands.

The spectral mask is usually one of the things defined in a bandplan for each particular band. It is essential in assuring that a transmission stays within its channel. An FM radio station, for example, must attenuate everything beyond ±75kHz from the center frequency by a few decibels, and anything beyond ±100 kHz (the channel boundary) by much more. Emissions on further adjacent channels must be reduced to almost zero.

FM broadcast subcarriers are normally required to stay under 75 kHz (and up to 100 kHz if reduced) to comply with the mask. The introduction of in-band on-channel (IBOC) digital radio in the United States has been slowed down by issues concerning the subcarriers it uses – and the corresponding increase in the amount of energy in the sidebands – overstepping the bounds of the spectral mask set forth for FM by the NRSC and enforced by the FCC.

...
Wikipedia