VEX prefix

The VEX prefix (from "vector extensions") and VEX coding scheme are comprising an extension to the x86 and x86-64 instruction set architecture for microprocessors from Intel, AMD and others.

The VEX coding scheme allows the definition of new instructions and the extension or modification of previously existing instruction codes. This serves the following purposes:

The VEX prefix replaces the most commonly used instruction prefix bytes and escape codes. In many cases, the number of prefix bytes and escape bytes that are replaced is the same as the number of bytes in the VEX prefix, so that the total length of the VEX-encoded instruction is the same as the length of the legacy instruction code. In other cases, the VEX-encoded version is longer or shorter than the legacy code. In 32-bit mode VEX encoded instructions can only access the first 8 YMM/XMM registers; the encodings for the other registers would be interpreted as the legacy LDS and LES instructions that are not supported in 64-bit mode.

The two-byte VEX prefix contains the following components:

The three-byte VEX prefix additionally contains:

The VEX coding scheme uses a code prefix consisting of two or three bytes, which is added to existing or new instruction codes.

In x86 architecture, instructions with a memory operand may use the ModR/M byte which specifies the addressing mode. This byte has three bit fields:

The base-plus-index and scale-plus-index forms of 32-bit addressing (encoded with r/m=100 and mod <>11) require another addressing byte, the SIB byte. It has the following fields:

To use 64-bit addressing and additional registers present in the x86-64 architecture, the REX prefix has been introduced which provides additional space for encoding addressing modes. Bit-field W expands the operand size to 64 bits, R expands reg, B expands r/m or reg (depending on the opcode format used), and X and B expand index and base in the SIB byte. However REX prefix is encoded quite inefficiently, wasting half of its 8 bits.

...
Wikipedia