Six-year itch

The six-year itch, according to political scientists, is the pattern which takes place during a US president's sixth year in office. This year is characterized by the nation's disgruntled attitude towards the president and his political party. During this time, there is a midterm election and the party in power usually loses a significant number of seats in Congress.

Prior to Reconstruction, the six-year itch saw the President's party gain seats in one house, while losing seats in the other house. Presidents before Reconstruction whose party had this occur:

1814 - Democratic-Republican James Madison: Gained 5 seats in the House, but lost 2 seats in the Senate.
1822 - Democratic-Republican James Monroe: Gained 34 seats in the House, while the Senate was unchanged.
1834 - Democrat Andrew Jackson: Lost 0 seats in the House (the smallest swing in the history of the House), but gained 1 seat in the Senate.

...
Wikipedia