Hello programmers,

Since i have been started with updating LFSLapper insimlibrary, i want to solve old bugs.
One of them is a character bug.

This bug does not display the japanese characters when sending an string to LFS.
After some research on google, i found something that these Japanse characters works with Double Bytes.

After looking into the codepage sourcecode, i do not quite understand what the previous developer (Gai-Luron) has done with the code. And i have no idea how to edit the code to support Double Byte characters.

So my idea is to start from scratch and i want to ask you guys how to start with encoding strings in a proper way.

Just a small note: My C# experience is bad.

Thanks in advance

Hey, I'm working on PHP insim tracker, here is my implementation of it:
https://bitbucket.org/K0Z3L43V3R/kingtracker-v2/src/10fd383a511ee47b35e532f5007bec612131d4b9/src/AppBundle/Component/LFSTranslator.php?at=master&fileviewer=file-view-default#LFSTranslator.php-97

Even if it's in PHP it should be similar in C#. Lets say you have some Japanese string in your application encoded in UTF-8. LFS is for Japanese expecting CP932 codepage. All you have to do is transliterate string from one encoding to another encoding and prefix it with "^J"

Oke thanks for the feedback, will look into it.

For someone who's is interested in the codepage source.
Sending a MTC packet looks like this.

The codepage source can be found in the attachment below.

Well , after some code editing, and i even have used the encoding code of InsimDotNet ( JUST FOR TESTING!!!). I still cannot manage to get the japanese characters working.

Ill explain what i already did.

First i have set a Japanese Char as my Nickname.



Then i type a command to display my nickname.
In Lapper you use the following function to send a private message to your self or someone else.
In this case i sent a message to myself.


In the lapperconsole i get the following output , within the yellow square:
The first line is the sended message right next after 'Message:'
I dont know, if the output '^J T' is correct.



As last i show you the code after i used InsimDotNet's Encoding code.



So if someone got an idea about to fix this. Feel free to post.

Thanks in advanced.

Why don't you dump the content of the InSim packet from a working situation (InsimDotNet?) and dump the content of your project with a tcpdump program and compare the difference to see what you are doing wrong?

Quote from Bass-Driver :

In the lapperconsole i get the following output , within the yellow square:
The first line is the sended message right next after 'Message:'
I dont know, if the output '^J T' is correct.

I don't think it is. After the first two bytes 94, 74 (^J for japanese) should come bytes 129 and 153 for the encoded star – however, your message seems to have a terminator right after the first two.

LFS encodings are a messy business.. a product of its time, unfortunately. For example (on the decoding side), consider the following bytes (ASCII representations below where applicable):

The first two bytes on each, once again, mark that what follows is in the Japanese code page. However, only on the latter case do the bytes ^L mean shifting back to Windows-1252; in the first case the "control character" is actually a trailing byte of a multibyte character, and L is just a regular character after that (as would any other following characters be until next change of code page). Even though the last bytes are equal on both, they stand for different characters altogether in each case: you definitely need incremental decoding here to keep track of used code pages – now contrast this code page mess to the simple world of Unicode only

Edit: speaking of difficulties; checking through my logs, I think even AIRIO managed to mangle someone's nick once.

sorry for the long respond.

I have tried to encode it first to UTF8 and Unicode and Windows-1251 and none of them works.
I also digged deeper into the sourcecode of lapper to encode the NickNames. Same result.

Have searched on the web for the Heximal and Decimal value of the "White Star". It came up with this.


But i couldnt find out how to convert this into bytes. I might to think to difficult about this :S

I think '81' is one byte and '99' is the another one. But none of them are the same in as i can see in my LapperConsole.

Atm i have no clue what todo.
See the last Packetcode below.



I know i'm new with this and you guys might think this i'm stupid. But this is a big learning process for me and i enjoy it.

Thanks in advanced.

OK.. going to take some risk. This is part of Airio code. Does it help?

Quote from Bass-Driver :

Have searched on the web for the Heximal and Decimal value of the "White Star". It came up with this.

//0x8199(HEX) 0x2606(DEC) #WHITE STAR

But i couldnt find out how to convert this into bytes. I might to think to difficult about this :S

I think '81' is one byte and '99' is the another one. But none of them are the same in as i can see in my LapperConsole.

Actually you already have the latter there (but you're not seeing it as you print the byte values in decimal, whereas your search results have them in hexadecimal. (0x99 = 153) Your message is mangled for some reason, though.

LFS encoded message for the white star would be as follows (4 bytes):

The first two bytes (in ASCII, "^J") once again tell that following sequence – in this case the remaining 2 bytes – is in Japanese code page. (I had a typo in my earlier post regarding the byte value of 'J')

oke thanks,
But the weird thing is that i get those "Weird Bytes" before its encoded.

So the problem may be deeper, if i'm right.
So much fun

So what i think it goes wrong with getting data from the lapper scripts.
or saving the Nicknames of a new connection
or must go wrong with creating the package.

What i want todo now ism a quick test with the sourcecode. But i need to know is, how to add a 'White Star' as text. Because i cannot work with bytes than. it must look like normal text.

Thanks

Can't tell from screenshots, are you calculating packet length correctly and adding last zero bytes?

From docs:
byte Size = 8 + TEXT_SIZE (TEXT_SIZE = 4, 8, 12... 128)
char Text[TEXT_SIZE]; // up to 128 characters of text - last byte must be zero

so minimal packet data (16 bytes) will looks like this (^J☆) in hex:
10 0e 00 00 ff 00 00 00 5e 4a 81 99 00 00 00 00

Let computers do the hard work with bytes

Well after some test it must be a problem between the scripts and the sourcecode of lapper.

I'm still using the Packetcode from post #7 of this topic.

=====================================================
TEST 01:
=====================================================
I created a new function and sended a MTC packet with "^J☆" as text, and that works how it should be.
see the code below.

In the Lapperscript i use 'testfunction();' as command.



=====================================================
TEST 02:
=====================================================

So the next test because the problem must be somewhere else.

When i copy/paste a 'white star' into the script. I get a weird "HOP" text. And when sending the message, i receive a Square and a Questionmark. The Script is converted to UTF-8.

And i use in the sourcecode the next line as a string.
string text = args[0].ToString();



=====================================================
TEST 03:
=====================================================

In the script i used a parameter to be send to the testfunction.

ingame i type: !tf ☆
testfunction($argv);

$argv = ☆

but now it doesnt work at all , like in the beginning.

in the console it looks like this:



So, could it be a convertproblem between script UTF-8 to the sourcecode (where can i find this)??
Or should i convert the script to a another encodingpage.