The online racing simulator
Objective Criteria for Rating Sim Physics
(15 posts, started )
Objective Criteria for Rating Sim Physics
People are always making vague statements like "the physics are better" but they seldom explain specifically what about the physics is more or less realistic. Even when I read reviews you just get very subjective satements like, "the physics are not quite up to snuff" but no objective information.

What OBJECTIVE tests do you use when rating the quality of a racing games car handling physics?
I usually try to compare car phsycs to real life ones. Nothing more.

Many games have weird physics as, u turn a bit and the car is instantly in the wall. Some games are too smooth. Not sure, but in some cases lfs feels too smooth and slippery for me. I dont remember RWD cars going sideways that easy Fix me if needed.
A good question. Sims are difficult to validate for a number of reasons:
a) Vehicle simulation is complicated and detailed, so there are lots of places and ways to get things wrong, or where there is no right, do things differently
b) Most people haven't driven most of the cars available in most sims, in real life, to make for the best comparison
c) Getting all the required data, at sufficient accuracy, to model a vehicle in the simulation, is arguably hardest of all (without having the real car in front of you), so even with a perfect simulation, the numbers fed in can still throw the handling off (GIGO). Working on the assumption that you will never get all the numbers you need, it all depends on how well the rest of the numbers were estimated.
d) Vehicles interact with the terrain, and modelling the terrain accurately for a completely fair comparison is another large headache in itself, one that has been tackled far less.

When you consider that any objective tests you can do (skidpan for outright grip, acceleration and top speed testing, making sure locked wheels don't steer the vehicle, suspension moves in a correct looking manner, throttle has the correct response, and so on), will only ever cover any one basic aspect of physics (at a time), it means you can pass all such tests, and not really have a good idea of whether the sim is accurate or not.
Lap time and stats are really the only objective tests. No objective tests (for simulators) will ever come out of the mouth of a human - every human sees the world differently and thus has a different take on reality.

As long as we are not able to simulate, in real time, all of the intricacies of suspension, tire movement, drivetrain, engine and all the rest, we will not achieve an accurate simulation. Even then, things may be tweaked and adapted to fit the vision of some designer and will most likely not be accurate in every respect. As it is, simulators are mere approximations. Most people, including myself, won't realize exactly how inaccurate they are until the next generation of simulators arrives. Much like graphics. Doom 3 used to look real to me a few years ago, now it looks so very old. Grand Turismo 1 was a very realistic simulator for it's time, but now we have iRacing and LFS.

In 10 year's time, we will look back and realize how unrealistic LFS and iRacing are/were.
While matching laptimes are ultimately the final result of accurate physics, they are completely useless in judging the accuracy. Comparing them might be objective, but you may as well not bother at all.
Even when laptimes match, there are near infinite wrong ways to achieve this! You also have the complexity that you can't judge a physics engine separate from how well optimzed the numbers that go into it are.

Ultimatly you can't really look at the total car. Sure laptimes could be right, even G forces could be right, but you could still achieve this by having the wrong tire loadsensitivity and wrong aero to compensate.

I think you'd have to break it down more and draw tire charts from each sim, from which you can derive a few key things such as loadsensitivity, stiffness etc.

Chassis wise, you can calculate the ride frequency and TLLTD and see if it makes sense, and the suspensions can be analyzed for roll center position, camber change, bump steer, anti dive etc..

Then of course, which is better? Despite all the hard work above It'll probably still be a lot of forum ramblings..
Quote from AndroidXP :While matching laptimes are ultimately the final result of accurate physics, they are completely useless in judging the accuracy. Comparing them might be objective, but you may as well not bother at all.

Well, if a car is doing a lap 10 seconds faster than it can do in real life, I'd say there's something pretty wrong with the simulation! Either with the length of the track or the car's grip levels or power. It's not the be-all-end-all of an accurate simulation, just another yardstick with which to measure.
I bet there are quite a few rfactor mods where laptimes are sort of realistic. But tires loosing 20% grip purely from going 300km/h, having peak slip angles 3x greater than realistic and grip curves that are from a different planet... THe actual driving experience is not realistic at all.


If I may brag a bit about my Z06. I have real tire data which is detailed enough for not only peak slip angle and stiffness, but also load sensitivity. We also know the weight, weight distribution and have a good guess on how height the center of gravity is. Then we have suspension geometry from a c5 corvette; which is roughly the same, giving us usable roll center locations and migrations, anti dive and squat etc. We have a detailed torque curve and know drivetrain losses because the Yanks love to dyno their cars. Coupled to gear ratios and a known diff lock, we know quite a few things. Aero lift is known as well, as is drag.

We don't know exact springrates (or rather wheelrates) though again we have some C5 numbers which might be somewhat close. We also don't know the anti roll rates, but again with plenty of videos available you can visually decide on something that is 'close' in body roll.
The chassis balance, TLLTD (total lateral load transfer distribution) is basically the front vs rear chassis roll stiffness, so each ends combination of roll center height, wheelrate and anti roll. Typical figures are in the 60% range for sporty behaviour, but in case of the corvette, the extra grippy rear tires could shave a few % off without making the ride all too hairy or dangerous.

Less known is tire traction; but you could say that modern ABS systems get pretty much all of the available grip out of the tire. With a reasonable brake bias of ~75% (thats only a few % in front of 'critical!!) you can decide on the longitudinal tire grip. Of course, this should also be enough for the car to do its 0..60 times.

Now, if the physics engine is good, first it should allow me to put in all this real data. Secondly the results of this data should match published real car data.

In case of the Corvette, just creating a car like I mentioned above, will pull about 1.04G on the skidpan. No tweaking of grip, just using the tire data, suspension data and a believable 58% TLLTD. Thats just impressive because the real car measured at 1.04 by Chevy themselves, and a few reviews got 1.02 / 1.03. Brake performance is within a yard, and 0..150mph is within a few tenths.

I think such an approach not only proves that the physics engine is pretty good, it also shows that this realistic aproach is the only way to go. Sadly the tire data I have for the Z06 isn't available for many tires at all, so sooner or later you're going to have to rely on common sense and clever interpretation of 'other data' to get a car going.. And of course the physics engine isn't perfect and things like combined tire forces are mainly down to the physics engine. So even in this case, while the numbers are truly realistic and the engine is good, the actual driving experience might still be somewhat off due to imperfections in force combining and perhaps other areas...
I understand the OP a little different(ly?) than the rest. While you guys are comparing simulation with reality, I think that the OP is about comparison of different simulators.

If that is the case, then the sims can be "rated" by these criteria (that I just made up):
  • Number of features being simulated - Tyre flex, changing weather, mechanical damage etc.
  • Accuracy (edit: "precision" might be a better word) of these features - Physics engine frequency, number of segments of a tyre, aero-model complexity etc.
  • Correspondance of these features with reality (if there is any) - does the car oversteer as much as it would IRL in the same conditions? etc.
Grade each of these on a scale of 1-10 and then compare with other sims and there you go.
Quote from breadfan :Grade each of these on a scale of 1-10 and then compare with other sims and there you go.

Very objective, isn't it?
You can have all the tire flex in the world running at 100000000000000ghz but that won't compensate for entering the wrong values for peak slip angle etc..

Often MORE numbers just means more things to get wrong!
Quote from Niels Heusinkveld :You can have all the tire flex in the world running at 100000000000000ghz but that won't compensate for entering the wrong values for peak slip angle etc..

Often MORE numbers just means more things to get wrong!

That's why I also mentioned "Correspondance of these features with reality". If the values are wrong, the "correspondance" is reduced. I just can't express myself properly.
I still think the only thing you're going to achieve, regardless of your best attempts, is an internet brawl on what IS realistic and what isn't. That is almost inevitable as there is very limited real data available, and at least half of this 'real data' about tires has turned out to be rather wrong.

We have all the ingredients for a brawl here..
Quote from Niels Heusinkveld :I still think the only thing you're going to achieve, regardless of your best attempts, is an internet brawl on what IS realistic and what isn't. That is almost inevitable as there is very limited real data available, and at least half of this 'real data' about tires has turned out to be rather wrong.

We have all the ingredients for a brawl here..

This is why I rate a sim on how much fun it feels to drive rather than sit and look at spreadsheets of numbers.
Quote from Niels Heusinkveld :I still think the only thing you're going to achieve, regardless of your best attempts, is an internet brawl on what IS realistic and what isn't. That is almost inevitable as there is very limited real data available, and at least half of this 'real data' about tires has turned out to be rather wrong.

We have all the ingredients for a brawl here..

I am not attempting to achieve anything. I just stated my opinion, plus the fact that this is my third post in a row in this thread makes it clear that I am bored out of my mind

Objective Criteria for Rating Sim Physics
(15 posts, started )
FGED GREDG RDFGDR GSFDG