PHPInSimMod - PRISM 0.3.3 Discussion
1234
6
(159 posts, started )
#126 - DarkTimes
Quote from Dygear :To be quite honest I don't know if I can fix that. PHP just might be too slow. It just might not be possible with PHP. But I'll keep on trying.

I'm looking through the code and there's something I'm slightly unsure of. It seems you're executing the timers between select returns, which is fair enough, but it looks like your select timeout is almost always 1 second. Now I might be wrong on this (it's very likely, as I don't know the codebase or PHP), but wouldn't that mean your timers can only be fired at minimum once a second? From looking at T3's output it's likely that the timer is always elapsing after one second, which would bare my theory out. If I'm not wrong then you could try putting the select timeout to a very low value, such as a 0.01 seconds or something, basically just enough to prevent it from flooding the CPU. pyinsim uses select as well and in that I use a timeout of 0.05 seconds, but this is a pretty arbitrary value that I came to after a discussion on StackOverflow. I could of course be completely wrong about all of this, in which case just slap me with a big fish.

Quote from T3charmy :Will you be adding the new packets? (Collision ones coming in new test patch?)

No point adding them until the new patch is out, as it'll have to be tested, also hard to know the ins-and-outs without an updated InSim.txt. I've put the new packets into InSim.NET, but personally not going to release it until I have a chance to test it.
#127 - morpha
Quote from DarkTimes :I'm looking through the code and there's something I'm slightly unsure of. It seems you're executing the timers between select returns, which is fair enough. But it looks to me that your select timeout is almost always 1 second. Now I might be wrong on this (it's very likely, as I don't know the codebase or PHP), but wouldn't that mean your timers can only be fired at minimum once a second? From looking at T3's output it seems that the timer is always elapsing after one second, which would bare my theory out.

Obviously incoming packets will also resume execution, which could potentially double (almost) the delay.
  • Timer is set to 1 second
  • select() suspends execution with a timeout of 1 second
  • a packet arrives after 900ms and is handled within 50ms, the timer has not yet expired and thus does not trigger
  • select() suspends again with 1s timeout
  • a second goes by without any packets
  • timeout triggers the timer, which is now 950ms late
Even with a very low timeout the timer resolution will be limited and vary greatly depending on the complexity of the script, as it's all running in a single thread of execution. Still, dynamically calculating the timeout will greatly improve the timer reliability, i.e.

<?php 
php
$timeout = min($regular_timeout, $time_till_next_timer_elapses);
?>

Quote from DarkTimes :If I'm not wrong then you could try putting the select timeout to a very low value, such as a 0.01 seconds or something, basically just enough to prevent it from flooding the CPU. pyinsim uses select as well and in that I use a timeout of 0.05 seconds, but this is a pretty arbitrary value that I came to after a discussion on StackOverflow. I could of course be completely wrong about all of this, in which case just slap me with a big fish.

I don't see any reason for pyinsim to use a timeout at all, since it can use (virtual) threads for timers and keep the socket blocked until there actually is something to do, including handling socket exceptions (disconnects mostly). More to the point, it would just use actual threading.Timers or something similar, I'm fairly certain those provide sufficient resolution.
#128 - DarkTimes
pyinsim 2.0 doesn't use threads at all, it uses the Python asyncore module, which is basically just a wrapper around select. Yes, if you just want to create a timer you can use the threading.Timer class, but all of the send/receive stuff in pyinsim is done on a single-thread with non-blocking sockets.

Quote :Even with a very low timeout the timer resolution will be limited and vary greatly depending on the complexity of the script, as it's all running in a single thread of execution.

Yeah, I realise this, but still changing the select timeout would improve things a lot, as looking at the current code for PRISM a 1.0 second timeout looks like a bug. A better solution would of course be to use a threaded timer for the timer events, but I don't remember enough about PHP to advise on how easy/hard that would be to implement.

Frankly I was just impressed I managed to figure out that there might be a timeout issue within 30 seconds of downloading the codebase.
#129 - morpha
Quote from DarkTimes :pyinsim 2.0 doesn't use threads at all, it uses the Python asyncore module, which is basically just a wrapper around select. Yes, if you just want to create a timer you can use the threading.Timer class, but all of the send/receive stuff in pyinsim is done with non-blocking sockets.

Should probably check that out sometime, I'm still running 1.5.3


Quote from DarkTimes :Yeah, I realise this, but still changing the select timeout would still improves things a lot. A better solution would of course be to use a threaded timer for the timer events, but I don't remember enough about PHP to advise on how easy/hard that would be to implement. Frankly I was just impressed I managed to figure out that there might be a timeout issue within 30 seconds of downloading the codebase.

With just PHP and its most common extensions alone, it's impossible. Threading is obviously a very low priority item for what is still primarily a hypertext preprocessor.
#130 - DarkTimes
Quote from morpha :Should probably check that out sometime, I'm still running 1.5.3

Yeah, in the time between pyinsim 1.0 and pyinsim 2.0 I learnt to hate threads with a passion (not Python's fault really), so I rewrote the whole thing to use non-blocking sockets. You can still set the main loop to run on a background thread by setting pyinsim.run(background=True), but that's only if you need to use it from a GUI app, otherwise it's all blissfully single-threaded.

Quote :With just PHP and its most common extensions alone, it's impossible. Threading is obviously a very low priority item for what is still primarily a hypertext preprocessor.

I'm actually surprised that PHP doesn't have a proper timer library. Oh well, I'll just add it to my long list of PHP oddities.
#131 - Dygear
I made the timeout to be a second. I must of forgot that I left it to default to a second as the lowest timeout. Your both right on the subjects, PHP is odd that it does not offer a proper Timer Module, and I had to do the same thing you did DarkTimes to ensure that a timers would be handled as a part of the select call. But again, it runs in just one thread, that's something that I really wish to fix in a C++ version of PRISM if I ever get there. It's hard to test tho, as I don't have an LFS server with clients on it to test on.
#132 - DarkTimes
If you are using select to synchronize the whole system (which there is nothing wrong with), then you should set the timeout to as low a value as you can without saturating the CPU. This is something I learned when developing pyinsim. Frankly the way you have implemented the timer is the way I would have done it too, but the whole one sec timeout thing is hurting you for this kind of operation (as I say it took a stack overflow question for me to figure this out myself).
-
(Victor) DELETED by Victor : come to think of it, my post is N/A regarding this timing issue.
#133 - Dygear
How the select calls works with timers and packets within PRISM.
  1. Sort the list of timers, with the timer to expire next at the top.
  2. Get the time to expire, and subtract that from now.
  3. Make the delta of this time the timeout for the select_timeout function.
  4. Call socket_select with the read, write, and exception socket arrays, and the timeout delta.
    • A packet is received before the timeout expires.
    • The timeout is reached.
  5. Handle any packets as needed.
  6. Execute any expired timers.
  7. Goto Item 1.
Best part? If there are no timers, PRISM will wait until there is a packet making it very efficient in this state.
#134 - cargame.nl
Well... It indeed always is a second (under normal circumstances).

Quote :$this->createTimer('tmrCallbackFIN', 0.05, Timer::REPEAT);

0.05 does not speed up the timing by 20.

I'm using my local I7 processor btw. That is my concern, I can live with the current drifting but I'm afraid what happens when I run it in an environment with more CPU load / slower CPU.

And I turned off MCI packets (don't need that at this moment), I think it worsens the problem if I do.
-
(DarkTimes) DELETED by DarkTimes
#135 - DarkTimes
Quote from Dygear :How the select calls works with timers and packets within PRISM.
  1. Sort the list of timers, with the timer to expire next at the top.
  2. Get the time to expire, and subtract that from now.
  3. Make the delta of this time the timeout for the select_timeout function.
  4. Call socket_select with the read, write, and exception socket arrays, and the timeout delta.
    • A packet is received before the timeout expires.
    • The timeout is reached.
  5. Handle any packets as needed.
  6. Execute any expired timers.
  7. Goto Item 1.
Best part? If there are no timers, PRISM will wait until there is a packet making it very efficient in this state.

There seems to be a bug in the code that figures out the sleep value, as it appears to always be set to 1, no matter what the timers are doing.
#136 - DarkTimes
The way you are doing it seems over-complicated to me, what I would do is..

1. select set to timeout every 0.01 seconds (or other suitably low value)
2. when select returns check if any timers have elapsed and call their callbacks
3. handle all read, write, error sockets
4. Goto 1

In badly written weird made-up code:

function main() {
while (any_sockets_are_connected) {
rlist, wlist, elist = get_socket_lists()

select(rlist, wlist, elist, 0.01)

handle_timers()
handle_sockets(rlist, wlist, elist)
}
}

function handle_timers() {
foreach (timer in timer_list) {
if (time_now > timer.due_time) {
timer.execute_callback()

if (timer.is_only_to_elapse_once) {
timer_list.remove(timer)
} else {
timer.set_next_due_time()
}
}
}
}

#137 - morpha
I disagree, select() should be left blocking indefinitely whenever possible. The whole point of select() is to have the application sleep when there's nothing to do, making select() perfect for passive/reactive applications, which most InSim applications are. That's probably especially true for those made with PRISM due to its limited support for interactivity (as in, GUI or console input).

The current approach is good, but sorting the timer list on every iteration isn't terribly efficient, should be done when timers enter the queue.
#138 - cargame.nl
This is with MCI turned on;

Quote :welcome::tmrCallbackFIN61 time: 39.489797
IS_MSO Packet from gtr.
V^1i^7ggen^8 finished 2nd
IS_MCI Packet from gtr.
welcome::tmrCallbackFIN62 time: 40.323637
IS_FLG Packet from gtr.
welcome::tmrCallbackFIN63 time: 40.958620
IS_MCI Packet from gtr.
welcome::tmrCallbackFIN64 time: 41.322348
IS_CCH Packet from gtr.
welcome::tmrCallbackFIN65 time: 41.754246
IS_FLG Packet from gtr.
welcome::tmrCallbackFIN66 time: 41.888076
IS_SPX Packet from gtr.
welcome::tmrCallbackFIN67 time: 42.212253
IS_MCI Packet from gtr.
welcome::tmrCallbackFIN68 time: 42.320785
IS_SPX Packet from gtr.
welcome::tmrCallbackFIN69 time: 43.197046
IS_MCI Packet from gtr.
welcome::tmrCallbackFIN70 time: 43.316710
IS_FLG Packet from gtr.
welcome::tmrCallbackFIN71 time: 44.205758
IS_MCI Packet from gtr.
welcome::tmrCallbackFIN72 time: 44.324441
IS_FLG Packet from gtr.
IS_FLG Packet from gtr.
welcome::tmrCallbackFIN73 time: 44.699609
IS_SPX Packet from gtr.
IS_MCI Packet from gtr.
welcome::tmrCallbackFIN74 time: 45.315669
welcome::tmrCallbackFIN75 time: 46.315640
IS_MCI Packet from gtr.
IS_MCI Packet from gtr.
welcome::tmrCallbackFIN76 time: 47.322623
IS_FLG Packet from gtr.
welcome::tmrCallbackFIN77 time: 47.563074
IS_MCI Packet from gtr.
welcome::tmrCallbackFIN78 time: 48.318871
IS_FLG Packet from gtr.
welcome::tmrCallbackFIN79 time: 48.995023
IS_SPX Packet from gtr.
welcome::tmrCallbackFIN80 time: 49.173521
IS_MCI Packet from gtr.
welcome::tmrCallbackFIN81 time: 49.323718
IS_FLG Packet from gtr.
welcome::tmrCallbackFIN82 time: 49.593627

You see how much it speeds up the timing? It's not really usable this way.

Interval 1.0 btw. Not sure if it has anything to do with MCI packets, I think more the amount of packets in general and the early abortion of this timer due to that (somehow).


.
#139 - Dygear
Quote from morpha :The current approach is good, but sorting the timer list on every iteration isn't terribly efficient, should be done when timers enter the queue.

Good point. I'll have a think about that, unless you want to have a stab at a patch.

Ok, seeing as there is going to be a new patch coming out soon I am going to make PRISM a priority, but it's going to have to wait until after I have finished my skills exam. That's in 16 hours from now for Part 1, and Part 2 is Wednesday. I'll be going to NJ this weekend for a Tactical Combat Casualty Care Course. I am also at work right now, typing you this message on a 20 hour shift, so if anyone wishes to make the changes I'll be more then willing to review and allow the patch into the main branch.
#140 - T3charmy
dygear, have you found out to get the short name(like AS5. BL1) of tracks yet? :P
#141 - cargame.nl
I scrapped this whole timer thing.. Not reliable enough (sorry).

Instead I'm now checking if a certain time condition is true or not during every packet which is being handled. Not ideal or efficient but well, got to be creative.
#142 - Dygear
Quote from T3charmy :dygear, have you found out to get the short name(like AS5. BL1) of tracks yet? :P

Yeah? I am pretty sure I have that information already.

Quote from cargame.nl :I scrapped this whole timer thing.. Not reliable enough (sorry).

Not your fault, it's mine. I'll have a fix for timers before 0.4.0.

Quote from cargame.nl :Instead I'm now checking if a certain time condition is true or not during every packet which is being handled. Not ideal or efficient but well, got to be creative.

That's not the most reliable way to do it, but I guess it's better then what you have.
#143 - DarkTimes
Hey there, I seem to have found a bug, it crashed when I was fast-forwarding through a replay.
Attached images
prism_error.png
#144 - cargame.nl
What replay? Who renamed?
#145 - DarkTimes
I think I figured it out, basically PRISM is not requesting the connection list when a replay starts. I think it just requests the conns list when first connecting to LFS. As soon as any packet that needs the UCID is received, it crashes as it cannot find the connection in the list (the fact it was on a CPR packet is just a coincidence).

I solved this in InSimSniffer by requesting the conns/players list whenever a ISM packet is received, which seems to work pretty well.
#146 - Dygear
Quote from DarkTimes :I solved this in InSimSniffer by requesting the conns/players list whenever a ISM packet is received, which seems to work pretty well.

Yeah, I never checked that use case. (Kinda busy with, ya know, everything else.)
#147 - T3charmy
Ok, so i am thinking of an option for my cruise to use ini files(or if the user selects database), so i was wondering, is there any way, for a plugin to use PRISM's already existing ini creator/reader/editor?
#148 - Dygear
Quote from T3charmy :Ok, so i am thinking of an option for my cruise to use ini files(or if the user selects database), so i was wondering, is there any way, for a plugin to use PRISM's already existing ini creator/reader/editor?

I'm not sure, but I think IniLoader is within scope for the plugins as well. You should be able to directly call those functions.
#149 - T3charmy
Quote from Dygear :I'm not sure, but I think IniLoader is within scope for the plugins as well. You should be able to directly call those functions.

how would they be used?
#150 - Dygear
If you look through the source code you'll see how Victor used those functions.
1234
6

PHPInSimMod - PRISM 0.3.3 Discussion
(159 posts, started )
FGED GREDG RDFGDR GSFDG