The online racing simulator
0.6A1 InSim.txt and LYT.txt
2
(49 posts, started )
#26 - Krayy
Is there a list of freeride track codes, or are they just the normal track IDs with freeride enabled?

If thats the case, I cannot see any InSim packet that identifies if we're in freeride mode which may affect split times by people taking shortcuts etc. Am I missing something quite obvious?
#27 - GeForz
You mean the open configs?
Those have the normal track code appended by either X or Y.

e.g. for AS1

AS1 - normal track
AS1R - reverse track
AS1X - open config with normal AS1 start position
AS1Y - open config with reverse AS1 start position
#28 - Krayy
That would be the obvious thing that I've missed then.

Thanks
#29 - EQ Worry
Quote from Mischa NED :I think Scawen wrote something about invalid packet sizes before. Shouldn't they be ignored if not dividable by 4? In my code (which is slightly different) I added this check: ...

Well, yes, I do that packet size check later on when the read packet is actually being processed, this is just TCP (whole) packet reading code.

Also, I added code to notify me about split packets and indeed it happens sometimes. It seems the code is now correctly handling both split and merged packets though, so it was a great note/tip Scawen made.
#30 - Mischa NED
Quote from EQ Worry :Well, yes, I do that packet size check later on when the read packet is actually being processed, this is just TCP (whole) packet reading code.

Also, I added code to notify me about split packets and indeed it happens sometimes. It seems the code is now correctly handling both split and merged packets though, so it was a great note/tip Scawen made.

Good to hear I check the package size in TCP handling already and when the packet size is wrong I terminate the connection. This never happened though (according to my log). My handling is a little different and I think it always handled split and merge packets right. I read the packets like if it's one big stream. All packets go into a buffer and I read only (or until) the size I expect for the specific packet. I will add some logging to see it actually happens. Do you know if there is a specific situation that this splitting/combining occurs or is it completely random?
#31 - EQ Worry
Hmm, I'm really not sure when it happens. Airio had troubles with remote connection to servers (running on another PC, data transferred over the Internet), especially when the connection was not perfect. I guess split packets were more frequent then, leading to erroneous disconnects for bad packets. But it may as well be just my imagination.
#32 - Mischa NED
Quote from EQ Worry :Hmm, I'm really not sure when it happens. Airio had troubles with remote connection to servers (running on another PC, data transferred over the Internet), especially when the connection was not perfect. I guess split packets were more frequent then, leading to erroneous disconnects for bad packets. But it may as well be just my imagination.

Hmmm I'll try that... and the erroneous disconnects for bad packets? How should I imagine that? Insim got disconnected with some error message or?
#33 - Victor
on the Relay i normally see split packets whenever there has been some lag. Because then after the lag is gone, I'll receive a big bunch of data all at once. And because that data may be bigger than the network buffer, it's just split at the buffer's boundary.

You can say it like this too : LFS always sends small packets. These are so small that they'll always fit into one network packet. Even if LFS needs to send several packets (like MCI) at once, 4 or 5 of them may still fit into one network packet. So these will not be split, but they will be concatenated -> a stream.
But now imagine there can be situations (lag / packet loss) where the amount of data exceeds the network packet's window size - then TCP will split the data after the network packet is full and continues with the data in the next TCP packet.

---

Ok now that I'm writing this anyway, I'd like to make a recommendation : (hm two it seems)

In your piece of network code you're constantly allocating memory for buff. This means the garbage collector will be doing a lot of work. Better create a byte[] buffer once (when you construct the networking class) and use that for all your network buffer reading needs.
In fact, I think reading byte for byte / packet for packet from the socket isn't good practise either. You're better off reading all the data in the network buffer asap, into a separate buffer. Then call a method that processes that data buffer to find the next valid packet.
That way your socket buffer will be more available, because you empty it completely as soon as you detect that it holds any data. Under stress, this allows the kernel to quickly fill the buffer again, keeping the data flow between two sockets going quicker.

So :

-create one buffer property in the class, where you read network data into. I would make it 4KB in size.
-always read the entire data in socket's buffer into our local buffer.
-after reading all data from socket, process the buffer to find the next packet(s)
#34 - EQ Worry
Victor, great explanation and awesome suggestions, thank you! In short, make one big static buffer, always fill it with everything available and process the data. Nice.

As for the erroneous disconnects, I meant that my earlier code was not processing split packets correctly, so it lead to bad packets being detected, especially when run remotely, which meant disconnects, probably just because of bad code and not bad packets.
#35 - Victor
ah, yeah.
Well if you write that bigger 'static' buffer, then every time you have read data from the socket, you just check the first byte in the buffer, which must always be the size byte. Is it % 4? Is there enough data in the buffer? Then you have a packet. Take the packet out of the byte buffer and pass it to whatever method will be processing individual insim packets.
#36 - Mischa NED
Quote from EQ Worry :Victor, great explanation and awesome suggestions, thank you! In short, make one big static buffer, always fill it with everything available and process the data. Nice.

As for the erroneous disconnects, I meant that my earlier code was not processing split packets correctly, so it lead to bad packets being detected, especially when run remotely, which meant disconnects, probably just because of bad code and not bad packets.

Thanks indeed Victor! I already used a "receive buffer", but now I notice I always copy the packet out of the buffer to use in my packet handler. So.. there is the overhead too. I should change that. I think i have the same problem as EQ with the "bad code" ;-) That tcp stuff is one of the first things you make when building a InSim library and hmm... I don't remember how long ago I did it, but when I watch the code I don't understand much of it anymore I'll try to optimize it and will let you know
#37 - EQ Worry
LOL, Mischa, you're absolutely right, same here.
#38 - Mischa NED
I analyzed my code tonight and even found a bug in it. I already worked with a read buffer and continued reading from the tcp if packets weren't complete. Combined packets were handled correctly, but if there would be a split packet (I never saw them yet ) the rest of the packet was written on the wrong position in the buffer. I still have to test it and hope it's fine now I'd like to share it with you guys. I removed the error handling to make the code more readable:

// buffer
const int _BUFF_SIZE = 8096;
byte[] _anbBuff = new byte[_BUFF_SIZE];
int _nBuffUsed = 0, int _nBytesRead = 0, int _nOffset = 0; // start reading from the first position in buffer

public void Listen (IAsyncResult iaResult) {
// get tcp data
_nBytesRead = _nsNetworkStream.EndRead (iaResult);

if (_nBytesRead > 0) {
_nBuffUsed += _nBytesRead; // _nBuffUsed will contain total number of bytes in buffer

// as long as the buffer contains (more) data and the size of the data is at least equal to the size of the processed packet
while (_nBuffUsed > _nOffset && _nBuffUsed >= _nOffset + _anbBuff [_nOffset]) {

// if packetsize % 4 (valid packet size)
if (_anbBuff [_nOffset] % 4 == 0) {
// copy and handle the full packet
byte[] anbPack = new byte [_anbBuff [_nOffset]];
Array.Copy(_anbBuff, _nOffset, anbPack, 0, anbPack.Length);
_handlePacket (anbPack);
}
else {
// invalid packet found, throw exception and terminate connection
}

_nOffset += _anbBuff [_nOffset]; // put offset on next packet
}

// if all packets in buffer could be processed, reset buffer pointers
if (_nBuffUsed == _nOffset) {
_nBuffUsed = _nOffset = 0;
}

// wait for more tcp data
_nsNetworkStream.BeginRead (_anbBuff, _nBuffUsed, _BUFF_SIZE - _nBuffUsed, new AsyncCallback (Listen), _tcpClient);
}
else {
// no data received, throw exception and terminate connection
}
}

#39 - Mischa NED
Quote from EQ Worry :I've noticed this also earlier, though there are no doubt exceptions: Posting any code makes the thread die!

You were right EQ :-)
#40 - EQ Worry
Well, yes. Just one thing I noticed during experiments: Split packets are rare but possible, merged (joined) packets quite very common. But even the simple code in one of my previous posts works OK. I believe Victor is right that creating new byte arrays constantly is not optimal performance-wise, but on the other hand it is simple and Airio never uses more than 1-3% of CPU power for normal processing, so... Uhm...
#41 - Mischa NED
Quote from EQ Worry :Well, yes. Just one thing I noticed during experiments: Split packets are rare but possible, merged (joined) packets quite very common. But even the simple code in one of my previous posts works OK. I believe Victor is right that creating new byte arrays constantly is not optimal performance-wise, but on the other hand it is simple and Airio never uses more than 1-3% of CPU power for normal processing, so... Uhm...

I still haven't seen split packets, but will test it with dedi on another server, maybe I see them then. The only way I got "split" packets was when I made my receive buffer to small so not all (combined) packets fit into it, so the packet was split in my own buffer. The handling is the same though, so I'm sure it will handle split packets correctly. Have a nice day!
#42 - DarkTimes
You can look at the packet receive code that InSim.NET uses, which is in the TcpSocket class. Undoubtably not perfect, but it seems to work pretty well. Of course it deals with multiple packets and split packets etc.. If you have any improvement suggestions I'm happy to hear them.

This is the simplest receive loop I can think of. Obviously not the most efficient code, but it makes it easy to see all the parts working.


<?php 
// Store buffer in generic list.
List<byte> buffer = new List<byte>();

// Temp array to store received data.
byte[] temp = new byte[2048]; // Receive up to 2kb of data at a time.

while (true) {
    // Receive data from socket.
    int received = socket.Receive(temp);
    
    // Nothing received means connection dropped.
    if (received == 0){
        throw new Exception("lost connection");
    }

    // Append received data to buffer.
    buffer.AddRange(temp.Take(received));
    
    // Loop through each completed packet in the buffer. The first byte in 
    // the buffer is the size of the first packet.
    while (buffer.Any() && buffer.Count >= buffer.First()) {
        int size = buffer.First();
        
        // Check packet is multiple of 4.
        if (size % 4 > 0){
            throw new Exception("corrupt packet");
        }
        
        // Read first packet from buffer and remove it.
        byte[] packet = buffer.Take(size).ToArray();
        buffer.RemoveRange(0, size); 
        
        // Do something with the packet.
        // HandlePacket(packet);
    }
}
?>

Edit: It should be noted as well, as many InSim devs seem to forget this, that when sending packets you also need to pay attention to how many bytes are sent. The send function returns the number of bytes transferred, and if this is less than the size of the packet you need to resend any data that was omitted. This is simple however.


<?php 
byte[] buffer = GetPacketToSend();

// Keep looping while there is unsent data.
int sent = 0;
do {
    sent += socket.Send(buffer, sent, buffer.Length - sent);
} while (sent < buffer.Length);
?>

#43 - cargame.nl
Quote from EQ Worry :and Airio never uses more than 1-3% of CPU power for normal processing, so... Uhm...

Ever seen the power which is being used when 400 Mb internal memory is being sorted by a !top or !pi command without indexes?

#44 - t1ger
First of all, a big thank you for implementing the ISP_HLV packet - I have used it to good effect on our qualifying server so we no longer need to build layouts to stop corner cutting. Instead, all laps just have to be HLVC compliant and it is so easy because people can use hotlapping to perfect their lap before joining our quali server.

Anyway, I also just used it to watch for corner cutting during races (by analysing the replays with ISP_HLV turned on) and I found a strange thing. I am tracking all cars with the MCI packets, and I wanted to know which node the HLV violation was fired so I could trend it to see if they cut the same corner every lap.

The corner in question is node 445 on KY3 (which is the corner coming off the oval back on to the road track where there is a nice chicane (perfect for cutting)). On lap 5 and 6 of the replay for a particular car, the HLV packet fires before the car has actually reached the off track bit. When I get the HLV packet, the car is at node 443, which is actually the middle of the track on the line they have taken. It does then cut the corner at 445, where I would expect the HLV packet to be fired.

Is this something to do with lag? Is the HLV coming from the client and being saved in the replay (maybe?) instead of being generated again by the replay when it is running? If it is, I doubt there is an easy fix, but just wanted to understand the reason if nothing else.
#45 - Dygear
As far as I know, the replay data is engine events from the servers point of view after dealing with lag. InSim packets, are then made based off those events when you run your replay by the engine on the fly. So that might be why there is some inconsistency.
#46 - Scawen
Dygear is correct. The HLV violation was originally detected on the guest's computer in the correct place. In the MPR your computer is processing that information packet before the car actually gets there. It seems that the car position packets run a little later than the game packets when you watch an MPR. You can see the same effect in an MPR at the start of a race - the lights go green and the cars seem to just sit there for half a second or so before moving away.
#47 - t1ger
Thank you both for the clarification, and it confirms my suspicions that it is just the way it is and not a fault in my code (which I always like to prove ).

Thanks
Tim
#48 - paparazzi
Hey, there. Why not damage packages? I see this fabulous idea from bluejudas in old thread of this and it would be nice to see into insim too at least more useful utilities can be made with this on.

http://www.lfsforum.net/showthread.php?p=1223721

hope your attention,
greats papa
#49 - DarkKostas
Recently i started playing with InSim again and i remembered 2 things i wanted. The 1st one is this one which was posted long time ago.
Quote from DarkKostas :As the previous topic about insim changes is locked ill post it here.
I would like a small change to this

struct IS_AXI // AutoX Info
{
byte Size; // 40
byte Type; // ISP_AXI
byte ReqI; // 0 unless this is a reply to an TINY_AXI request
byte Zero;

byte AXStart; // autocross start position
byte NumCP; // number of checkpoints
word NumO; // number of objects

[B]char LName[32];[/B] // the name of the layout last loaded [B](if loaded locally)[/B]
};

Is it possible to make it so when a remote player loads a layout, we can get the name of it? Because i want to store layout name and use it.

The 2nd is to make PLC packet work as /cars command. What i mean is that when someone for example is driving with XFG(and its the only available car on server), when you type /cars UF1, he will get spectated AND when he tries to join in the race, he goes to the car selection menu to change car.

These are not happening with PLC packet. If you send PLC packet and remove all cars available he 1)wont get spectated, 2)when he goes to pit, he is still able to join the race with same car(he should go to the car selection menu).

A good and simple use of this "fix" use to make a !spec_all command which will send a PLC packet with value 0 and one more with value 0xffffffff.

Even if it there wont be any new LFS version soon, at least please implement it for the future version.

kthxbye
Dark_Kostas
2

0.6A1 InSim.txt and LYT.txt
(49 posts, started )
FGED GREDG RDFGDR GSFDG