The Milk Carton Kids' fifth album, All The Things That I Did And All The Things That I Didn't Do, marks the first time that acoustic duo Joey Ryan and Kenneth Pattengale have brought a band into the studio with them. "We had been going around the country yet another time to do the duo show, going to the places we'd been before. There arose some sort of need for change. It was liberating to know we didn't have to be able to carry every song with just our two guitars.