Upon arriving in San Francisco (via Amarillo, Texas), Owens met White, a Bay Area native who was raised in Santa Cruz, California. White began recording bands in his parents' garage. Upon meeting, the two developed a strong bond and interest in similar music, as well as a shared background in Punk/Hardcore music. Shortly thereafter Owens and White started recording what was to later be their debut album, titled Album.
A former member of the band Holy Shit with Ariel Pink and Matt Fishbeck, in an interview with Exclaim! magazine, Owens related much of how being part of the Children of God cult informed his childhood and his music. While there, he was not allowed to listen to music from outside the group, but was allowed to watch movies from which he absorbed music like Queen and Guns N' Roses. Girls' Album is an "overflow of expression", as Owens could do whatever he liked on the album without worry for boundaries.
The band released their second full length album Father, Son, Holy Ghost on September 9, 2011 in UK/Europe and on September 13 in the USA. On July 20, 2011, they released "Vomit" from their upcoming album as a free download. This was followed by a video of the same song in August. "Honey Bunny" is the second cut off the record and will receive a proper single release in vinyl and other formats.
Besides its two official members the band has recorded and/or toured with: Ryan William Lynch, Garett Godard, Matthew Kallman, Darren Weiss, Dan Eisenberg, John Anderson.