He played locally in his native Pittsburgh in his early days.

Arriving in New York in 1945, on his first day in town Brown met and played with Dizzy Gillespie, Charlie Parker, and Bud Powell. He was hired by Gillespie for his small groups and his big band; "One Bass Hit" and "Two Bass Hit" were early features, and he can be seen with Dizzy Gillespie in the 1947 film Jiving in Bebop. Although not a soloist on the level of an Oscar Pettiford, Brown's quick reflexes and ability to accompany soloists in a swinging fashion put him near the top of his field. After playing with Jazz at the Philharmonic, he married Ella Fitzgerald (their marriage only lasted during 1948-1952), and for a time led his own trio to back the singer. Brown recorded with an early version of the Modern Jazz Quartet (under Milt Jackson's leadership), and then became a permanent member of the Oscar Peterson Trio (1951-1966).

With Peterson, the bassist traveled the world, guested with other top jazz artists, was featured on JATP tours, became famous, and recorded constantly. He began playing cello in the late '50s, and used it on a few of his own dates. After leaving Peterson, Brown settled in Los Angeles, worked in the studios, continued recording jazz, and worked as a manager of several artists (including the Modern Jazz Quartet and Quincy Jones). He played with the L.A. Four starting in 1974, did a great deal to revive the careers of Ernestine Anderson and Gene Harris, and recorded extensively for Pablo and Concord. The Ray Brown Trio featured pianists Gene Harris, Benny Green, and Geoff Keezer, along with drummers Jeff Hamilton and Greg Hutchison, and recorded for Concord and Telarc. He continued touring up until his death, dying in his sleep while napping before a show in Indianapolis on July 2, 2002. His last batch of sessions, working as a trio with pianist Monty Alexander and guitarist Russell Malone, were released that fall. ~ Scott Yanow