The song is said to have been written on a mattress in the front yard of the Oconee Street church in Athens, Georgia. At that point in the band's career, Peter Buck has stated that their musical modus operandi was "three chords and a six-pack of beer."

In the booklet of the 2006 And I Feel Fine... The Best of the I.R.S Years 1982–1987, Bill Berry wrote the following of "Gardening at Night":

The song's title would serve as the inspiration for the name of the band's publishing company, Night Garden Music.

Four different studio recordings of the song have been officially released. The original Chronic Town version can be found on the CD edition of Dead Letter Office, on the 2006 R.E.M. compilation And I Feel Fine: The Best of the I.R.S. Years 1982-1987 and more recently in the band's 2011 career-spanning greatest hits compilation Part Lies, Part Heart, Part Truth, Part Garbage 1982–2011. A version featuring an earlier vocal take appears on the 1988 compilation Eponymous. An acoustic version of the song appears as a bonus track on the European "I.R.S. Vintage Years" reissue of Dead Letter Office. Finally, a substantially slower electric version of the song appears on the bonus disc accompanying the special collector's edition of And I Feel Fine.

Five live versions have been officially released–the first, recorded on July 13, 1983, appeared on the Dutch "I.R.S. Years" reissue of the band's debut album, Murmur. The second appeared as a B-side of the 12-inch "(Don't Go Back To) Rockville" single in 1984. The third appeared on a promotional CD called The Alternative Radio Sampler. A performance at Larry's Hideaway, Toronto, Ontario, Canada, from July 9, 1983, was released on the 2008 Deluxe Edition reissue of Murmur. Finally, a performance from Chicago's Aragon Ballroom from July 7, 1984 was released on the 2009 Deluxe Edition reissue of Reckoning.

...
Wikipedia