It could be argued that there are two reasons to improvise:
a) to satisfy the urge to explore the inner self and to build self-identity through creative expression
b) to support the rituals of one's society and to build a community identity
It is interesting how many composers, when asked why they do what they do, reply, "Because I have to!" Their creativity is needs driven. Most would link their needs to reason a).
The advent of music publishing, which brought so many benefits to the development of music in Western civilisation, had a serious downside in its potential to strangle invention. Nevertheless, for over two hundred years, improvisation managed to flourish alongside notated works, probably because composers left opportunities within their published scores for performers to add their own ideas.
From the Romantic period, however, composers began to fill in all the detail. The music was their creation, their expression and there was little scope for the performer to contribute. A common repertoire and standardised performance practice began to emerge. These became the norm, wherever the notation was used. Music recording brought about even narrower standardisation. One thing is certain with musical skills - if you don't use them, you lose them! With few creative demands placed upon performers, improvisation began to disappear from classical music.
Today, many classical musicians believe themselves incapable of improvisation, although they have never tried it! Players are even afraid of any element of a personal sound or style emerging in their performance. Young instrumentalists look, enviously, to the rich variety of world music and jazz, where exploration and experimentation is not only legitimate but expected. It is time to restore improvisation to its central role in classical music!
Improvisation with The Full Pitcher Music Resources