After releasing the EP Will Wallow and Roam After the Ruin in 1980, Gelb fired everyone but bassist Scott Gerber (although founding guitarist Rainer Ptacek returned to the fold many times in the future) and started over as simply Giant Sand, essentially a one-man band backed by a revolving cast of players.

The first Giant Sand LP, 1985's Valley of Rain, earned Gelb comparisons to Neil Young for his reedy vocals and country-flavored, grungy guitar aesthetic; like Young, Gelb also proved to be a restless creative spirit, a notice served by 1986's Ballad of a Thin Line Man, an acoustic effort that featured the harmony vocals of ex-Go-Go (and Gelb's then-girlfriend) Paula Jean Brown. In 1988, Giant Sand issued a pair of new LPs, the equally diffuse Storm and The Love Songs.

By 1989's raw, improvisational Long Stem Rant, the group consisted only of Gelb and drummer John Convertino, while 1990's Swerve featured guests like Juliana Hatfield and Poi Dog Pondering. Ramp (1991) and Center of the Universe (1992) returned to the ragged desert rock of their earliest material, but with 1994's Glum (the band's first and only effort for major-label Imago), Giant Sand's music turned unexpectedly moody and restrained. Backyard Barbecue Broadcast, released in 1995, culled material from a pair of live radio sets.

In addition to Giant Sand, Gelb occasionally recorded under the guise of the Band of Blacky Ranchette, an outlet for his country leanings; in 1991, he also issued a solo album, Dreaded Brown Recluse. In addition, longtime drummer Convertino moonlighted in the lounge revival group Friends of Dean Martinez, and frequent collaborator Ptacek often performed as a solo artist prior to his cancer-related death on November 12, 1997. Chore of Enchantment, the first Giant Sand release on noted indie Thrill Jockey, followed in early 2000. That was followed by the all-covers Cover Magazine in 2002 and a new studio album called All Over the Map in 2004. They made the switch to Yep Roc for 2008's Provisions, and marked their 25th anniversary two years later with the album Blurry Blue Mountain.

Gelb expanded his vision of the band and paid tribute to his adopted home with Tucson, a self-described country-rock opera under the moniker Giant Giant Sand, which arrived in July 2012. As Fire Records began an extensive reissue campaign of Giant Sand's back catalog and Gelb released a pair of solo efforts (2013's The Coincidentalist and 2014's Radian Verses Howe Gelb), Giant Sand returned in 2015 with Heartbreak Pass, which included guest appearances by Grant-Lee Phillips, Steve Shelley of Sonic Youth, and Ilse DeLange of the Common Linnets.

In February 2016, Howe Gelb issued a press release in which he announced the upcoming release of a new album from the group, The Sun Sets, and a tour of Europe, as well as the surprising news that Giant Sand was breaking up. "30 years seems an adequate number to aptly utter 'I kinda quit,'" Gelb said in his statement. "There’s plenty enough here, more than imaginable." The release did suggest that Gelb would continue to make music as a solo artist, stating, "Piano for now. Songs forever." ~ Jason Ankeny