There are actually a lot of people drifting, and some servers are always filled with people. It doesnt matter what you do in LFS, if it's drifting, racing, or cruising. You'll always enjoy it
It all has it's own excitement and skill.
I've personally done it all, in the 5years of LFS. But the nice thing is, you can all combine it, to make it your own style.
Example: Can you race, but not with RWD, because you spin out every time? Go drift for a month, and get back to racing. You'll now be able to control your car better when the back end skids out.