My experience in AA servers was that I'd commonly like to start at the end of the grid because of that problem. Less with an XFG only race, but with XRGs in the grid, you have the random XRG crasher just because he can accelerate out of there faster. I don't consider myself a fast driver, but I usually end up at least third unless some faggot rams me from behind. The problems usually are caused in Lap 1, T1, chicane, and T2. Thereafter, it gets a bit better. But on the occasion, we do get very good drivers, and dogfighting with them is really fun. Just keep it calm, and make slowly squeeze the brakes. And if someone crashes you, quickly clear the track, and get back on track once everyone has passed. If anything, and you know the person who is crashing is a crasher, just say "[name]: Please don't crash" in a very calm way, and people should acknowledge that it isn't your problem.
EDIT: Usually, you can pass at least 3 or 4 people if it is a full grid, and you are at the back because they end up f*cking each other from behind xD.