Improvising can be scary, but...
It doesn't have to be. What is improvising? Making stuff up. So guess what, you can't be wrong! It might be uninteresting or safe, like sitting in the stands....or crazy excited like the new tennis player who hits the ball out of the court.
So, we build our skills for listening, context, rhythm, feel. When we improvise we're making up a melody or rhythmic pattern to sing along with the groove and chords for the song. With practice, you'll find that you repeat certain patterns when you sing that you can use again and again in different combinations. I like to think of using these phrases like making a necklace or bracelet from beads. You can order the beads any way you want, or change them completely to make a new design -- but they're still the same beads.