You do not have pitch or roll attributes in InSim so all your calculations will have to assume the car is on all four wheels. An upside down car or on a 45 degree tilt will be the same as being flat.
However you still have car heading direction, X,Y,Z co-ords and also you have an approximate car length and width. You should use the car's heading, not the car's heading of travel which is easy to confuse the two.
Use this diagram:
Use the car's X,Y,Z co-ordinate and direction of heading to figure out the four verticies of each corner of the vehicle. The car's X,Y,Z co-ordinate is the approximate centre of mass.
Giving points
A,B,C & D
Shift
A & B using the direction of heading axis by an equal distance to give points
A & B
(use sine sin/cos/tan trig AND car heading to figure out the new
A & B points)
Now you have a really long pink rectangle
A,B,C & D. Use an intersecting rectangle formula to figure out if the pink rectangle intersects each of the other red rectangles individually. (You must pre-calculate each other car's vertices first)
Here is a thread to working out intersecting rectangles.
Once you find which red rectangles are intersecting the pink one, use pythag to calculate which is the closest red one to the blue car to see which one is in-line of the blue car "first".
In the cases where hairpins are going to give incorrect results, use the cars nodes to figure out which are closest in relation to the track terrain. Keep in mind that a spinning car would give results of cars behind it's position as in-line. So you must keep an eye on that event.
I can't help more now, I need to go. But the theory should be apparent.
edit: ah well someone else posted a solution. I thought of that first, but wrote up this method because I thought that incremental method might take too long to scan 50 meters or 100 meters, it would take 100 calculations cycles for 100 meters for example. Probably wouldn't make much difference with todays CPU's. But another reason why I didn't opt for that is because the 1 meter points might skip over the 'car in front's corner, but if you're scanning fast enough, then you would catch it in another cycle.