The title Lord Duffus was created by Charles II in the Peerage of Scotland on 8 December 1650 for Alexander Sutherland. He was a descendant of the 4th Earl of Sutherland, who fell in battle in 1333. The title is now extinct, although there may be male-line Sutherlands descended from earlier lairds of Duffus.
In 1734, the 3rd Lord was attainted and the lordship was forfeited. His son Eric tried but failed to get a reverse of the attainder. His son James Sutherland of Duffus got the attainder reversed, and was restored to the lordship as 4th (titular 5th) Lord Duffus on 25 May 1826. The last two Lords Duffus were also baronets, of Hempriggs in the County of Caithness (3rd and 4th). The lordship became extinct on the death of the 6th (titular 7th) Lord Duffus on 28 August 1875.
The Sutherland family of Duffus descended from Nicholas Sutherland, only brother of William de Moravia, 5th Earl of Sutherland (d. 1370), and younger son of Kenneth de Moravia, 4th Earl of Sutherland. The village of Duffus is outside of the county of Sutherland. Some of the Sutherland of Duffus family may have assumed the surname Duffus, which was the family title instead of the surname Sutherland.
These are the early ancestors of the Lords Duffus.
Information on the Lords Duffus was obtained in part from the Sutherland history (below), and from Elinor Glyn (1955), a memoir by her grandson Anthony Glyn. Glyn and her sister Lucy, Lady Duff-Gordon were born Sutherlands, descended from David Sutherland, Laird of Cambusavie, allegedly a son of Alexander Sutherland, a younger brother of the Jacobite 3rd Lord Duffus, who is described in The Scots Peerage as having died without issue. The fact that the 6th Lord Duffus inherited in 1827 over the now Canadian Sutherlands, who sold their estates in the 1770s to the Earl of Sutherland, probably means that the relationship was more distant, or if the same, that the Laid of Cambusavie was illegitimate.