Formed in Brooklyn in 2002, Dirty on Purpose originally consisted of keyboardist Erika Forster, guitarist Joseph Jurewicz, and drummer Doug Marvin (all of whom shared lead vocal duties), plus non-singing guitarist George Wilson and bassist D.J. Boudreau. Following a self-titled and self-released four-track EP released in 2003, Dirty on Purpose made a bigger splash with 2004's Sleep Late for a Better Tomorrow.
Following the release of that debut effort, Forster left the band to start her own group, the electronically oriented Au Revoir Simone, who released their debut album, Verses of Comfort, Assurance & Salvation, in early 2006. Adding Wilson to fill the third lead singer spot, with occasional help from guest female vocalists, the slimmed-down Dirty on Purpose released their second album, Hallelujah Sirens, in May 2006.