By the age of 15 he had begun collecting songs in the Appalachian Mountains, a habit he would continue while serving as a ferry pilot in the U.S. Air Corps during World War I. Niles remained in France after the war, studying music at the Universite de Lyon and the Schola Cantorum in Paris. He would continue his studies for two more years at the Cincinnati Conservatory of Music upon returning to the United States. In 1921, he came to New York where he met the singer Marion Kerby. Kerby shared his love of folk music, so the two decided to work as a team, traveling throughout Europe and the United States.

Niles collected folk songs in the Southwest while working as a guide and chauffeur for photographer Doris Ulmann. During the '20s and '30s, he began publishing collections of folk songs, including Singing Soldiers (1927), Songs My Mother Never Taught Me (1929), and Songs of the Hill-folk (1934). In the '30s he began to perform solo, traveling widely and singing at high schools, churches, and colleges. He dressed in bright-colored shirts, wore corduroys, and sang in a striking, high falsetto. Barry Alfonso, recalling the first time he heard Niles on record, wrote, "Out of my stereo came his startling, other-worldly voice, the sound of someone enraptured -- or maybe possessed. He seemed to embody his dire ballad, rather than to merely perform it."

Niles wrote a number of classic folk songs that are often mistaken for traditional material, including, "Black Is the Color of My True Love's Hair," "Go 'Way From My Window," and "I Wonder as I Wander." He recorded numerous albums, including Early American Ballads (1939) and American Folk Lore (1941). He also composed more formal music, writing the oratorio "Lamentation," which would receive its first performance at the Indiana State Teachers College in 1951. Between 1967 and 1970 he would compose a work based on the poetry of Thomas Merton titled "The Niles-Merton Songs." The Songs of John Jacob Niles was published in 1975 and Niles would continue to perform publicly until two years before his death in 1980. Part Renaissance man, part traveling minstrel, Niles left an invaluable body of recordings, folk song collections, and compositions behind. His work has greatly aided the preservation and continued vitality of American folk culture. ~ Ronnie D. Lankford, Jr.