He is a top-tier sideman chosen not only for jazz dates but as a guest on rock recordings. His work has consistently challenged not only the role his chosen instrument plays in jazz, but also in new music, klezmer, classical, and pop.

He was born and raised in Denver, Colorado, later earning an undergraduate music degree from the University of California, Santa Cruz and a subsequent Master of Arts in Composition from Mills College. A student of Rosario Mazzeo, Steve Lacy, and Joe Lovano, Goldberg initially won acclaim as a member of the New Klezmer Trio, debuting in 1991 with Masks and Faces; two years later he won a grant from the National Endowment for the Arts to mount a retrospective series spotlighting the music of key American jazz composers including Thelonious Monk and Herbie Nichols, a project on which he collaborated with the likes of Andrew Hill and Bobby Bradford. After teaming with Kenny Wollesen for the 1993 LP The Relative Value of Things, Goldberg resurfaced two years later with another New Klezmer Trio record, Melt Zonk Rewire; with the group Snorkel, he also contributed to 1996's Live at the Elbo Room. In 1998, Goldberg headlined no less than four new recordings: Eight Phrases for Jefferson Rubin, Twelve Minor, Here by Now, and What Comes Before. He ended the century with Ghost of Electricity as part of the Junk Genius band with Kenny Wollesen, Trevor Dunn, and John Schott.

Goldberg kicked off the new century with Short for Something, the third release by his New Klezmer Trio on Tzadik. He also released Almost Never, a new collection of his own songs with Dunn and Schott. He appeared on Schott's Elegies for the Recording Angel, as well as recording Helium with the Tin Hat Trio.

In 2003, Goldberg arranged Voices in the Wilderness, the tenth anniversary celebration of John Zorn's Masada compositions.

When Steve Lacy fell ill with cancer in 2004, Goldberg began considering the soprano saxophonist's massive influence on him as a composer, bandleader, and soloist. He began writing and performing compositions for quintet (Carla Kihlstedt, violin; Rob Sudduth, saxophone; Devin Hoff, bass; Ches Smith, drums). Recorded just after Lacy's death, they were released in 2006 as The Door, The Hat, The Chair, The Fact on Cryptogramophone. (He received a grant from the Aaron Copland Fund for Music the following year to write and record more compositions for this group.) Goldberg was also part of Nels Cline's group on New Monastery: The Music of Andrew Hill.

While Goldberg remained busy composing and playing on others' records, in the intervening two years, a trio recording, Plays Monk with Scott Amendola & Devin Hoff, was released on Cryptogramophone. He became prolific again in 2009, recording in a variety of settings: the trio offering Speech Communication on Tzadik with Wollesen and Greg Cohen, The Sad Machinery of Spring with Tin Hat, and Cry, Want with Beth Custer's Clarinet Thing, on Myra Melford's Be Bread album The Whole Tree Gone; he also guested on Xiu Xiu's La Forêt.

After being nominated previously, Goldberg was the recipient of the Alpert Award in the Arts in 2010. He kicked off his own label, BAG Productions, with Go Home, a quartet date Charlie Hunter, Ron Miles, and Amendola, led a quartet for Baal: The Book of Angels, Vol. 15 from Zorn's second Masada book, and recorded as part of Tin Hat for Foreign Legion. In 2011, he topped the Downbeat Critic's Poll as Rising Star Clarinetist.

Goldberg guested on Aaron Novik's Secrets of Secrets on Tzadik in 2012, and Tin Hat released The Rain Is a Handsome Animal, a 17-movement song cycle based on the work of e.e. cummings.

Two releases from BAG appeared in 2013: a quintet date entitled Unfold Ordinary Mind, and Subatomic Particle Homesick Blues, a set recorded back in 2008 with a group that lasted only a couple of days and included saxophonist Joshua Redman. Goldberg also began writing the music for Orphic Machine, a series of jazz art songs based on poet and philosopher Allen Grossman's Summa Lyrica: A Primer of the Commonplaces in Speculative Poetics.

Back in 2010, Goldberg recorded with Deerhoof guitarist John Dieterich and Amendola. An album from those sessions, Short-Sighted Dream Colossus, was finally released in 2014. The clarinetist was also honored with a residency at The Stone in New York.

In early 2015, Goldberg released his nonet project Orphic Machine via BAG. ~ Jason Ankeny & Thom Jurek