Little Bushman grew out of members of several bands in New Zealand's sometimes incestuous musical scene. The original members were Warren Maxwell, of Fat Freddys Drop, Trinity Roots, and the Village Jazz Quartet; Rick Cranson, also of the Quartet; and Tom Callwood. Maxwell abandoned the saxophone he'd been playing in Fat Freddys Drop and the guitar he'd used in Trinity Roots, switching to a Rhodes piano instead, with Cranson on drums and Callwood on double bass. Feeling the need for a guitarist, they brought in Tom's brother Joe Callwood.

Other members of Maxwell's former band helped out with the recording of their first album, The Onus of Sand, with Lisa Tomlins and Deva Mahal, both of Fat Freddys Drop, contributing their talents to the well-received album. The songs were recorded live to replicate the energy of their concerts.

In 2006 Little Bushman released their follow-up, Pendulum. Each of the four members took on more instrumental duties, with Maxwell playing harmonica, piano, saxophone, guitar, and tambourine, Joe Callwood playing acoustic guitar as well as his usual electric, Tom Callwood on violin, cello, and electric bass, and Cranson adding additional percussion and vocals. ~ Jody Macgregor