If you are using manual clutch, try to lift your left foot slower.
Anyway, in addition to what Flotch said, you can wait until your RPMs are lower to reduce the impact of downshifting. Sometimes shifting 2 or even 3 gears in a row is better than maxing the RPMs in each downshift. It varies on car to car, driving style and even track surface and race line.