Updated version here.

There are a few reasons why I'm putting this here instead of the Unofficial Addons section, for now, which will become clear shortly. Granted this is technically breaking the "rules", so mods feel free to move it.

However, I present luaLFS. luaLFS was the by-product of another project I was working on a number of months ago. Basically it's an InSim client, which does the "heavy" network lifting and then fires off events to a set of LUA scripts.

I've cleaned it up a little bit and decided that a few other coders might be interested. Despite this, it's still very hacky and was put together in an evening - hence lots of hardcoded values, and so on. Over the next few months I fully intend to clean it up properly, put together a nice set high level API functions for use in the Lua scripts, make a nice Lua based configuration and then release it for the masses.

However, I thought that perhaps a few other members might be interested and might even want to run with the project, so I've licenced it under the Apache2 licence, zipped up the source with a binary version.

Currently the process is as follows:

• luaLFS creates a Lua environment
  
• luaLFS loads the core scripts (./scripts/core/core.lua), and then loads all other files with the substring "lua" in them, from the ./scripts directory. These files are parsed
  
• luaLFS creates an InSim connection
  
• luaLFS monitors the connection, keeping it alive, and watches for timeouts. For all packets, other than VER packets, they are fired off into the core Lua scripts.
  
• The core scripts implement a very, very, very simple event system and call functions associated with a set of events. Currently there is a couple of custom events, such as start up, shutdown, etc. The rest of the packets are passed, as they come from LFS directly to the bound functions for that packet type.
  
• When LFS closes luaLFS attempts to stay around for 2 minutes before timing out.
  
• On timeout disconnection event is fired, then shutdown and the lua environment is shutdown
  
• Program exits

Current faults:

• Very hacky, lots of stuff that needs cleaning up
  
• LFS needs to be running and listening for an InSim connection first. It will not attempt to reconnect.
  
• LFS needs to have no admin password set
  
• Lua garbage collection needs to be called explicitly once every few minutes on a very busy connection. This doesn't get taken care of automatically at the moment. Simple to add though
  
• No real configuration, requires port 29999 to be used on LFS and uses 29998 as the listening port. Will be fixed when I refactor a Lua based configuration snippet from another project
  
• No API. As the packets are passed as binary strings, unless they are unpacked, and then functions bound, its reasonably complex to get your head around. I do plan on creating a simple to use API

Pros

• It should be cross platform capable because I've chosen Lua and all the network code is using the berkely functions, and the WIN32 stuff is conditionally invoked
  
• It's cool and easily extensible, once the API is written
  

I'm sure there are others I've forgotten, but you get the hint.
This implementation is in C, which I know some people won't care for. It's what I love though.

Bundled in this inital toy version are a few examples of how luaLFS will work in the future.

I will stress again that I am releasing this as a proof of concept and the current code is very hacky. I make no guarantees that it won't crash, nor of any release timescales. If anyone wants to submit patches, feel free.

Updated version here.

Very cool proof of concept. While I don't have a need for it myself, prefering to stay in compiled land as much as possible, the possibilities of this are fantastic and could open all sorts of new Insim based mods to different set of users.

That's freaking awesome!

Quote from sdether :

Very cool proof of concept. While I don't have a need for it myself, prefering to stay in compiled land as much as possible, the possibilities of this are fantastic and could open all sorts of new Insim based mods to different set of users.

So do I usually, but the original project this came out of was literally taking 15 minutes to compile. When you only wanted to change an arbitrary function or something else daft, it took bloodly ages.

Incidentally if anyone has any suggestions on what might make it better, I'm all ears

Quote from Dygear :

That's freaking awesome!

Thanks... I think

Hello angry_angel,
it's really nice idea. I played with your sources under Linux (Ubuntu dapper). Here are my notes:
- it doesn't work with LUA below 5.1, it's ok, but it could be mentioned somewhere, e.g. in the README,
- the main binary has to be started from the same directory, where the "scripts" directory is.
After some minor modifications it works :-).

In my Linux system, the declaration of a variable named "type" (in the file scripts/ISM.lua) overrides the lua language "type" function, used e.g. in event.lua like this: type(func) == "string". This leads to "attempt to call global 'type' (a number value)'" errors. It's ok after renaming the variable, e.g. to "type_ism" in ISM.lua (Disclaimer: I have no experience with lua ).

I modified the .c sources at several places, my modifications probably won't always compile under windows. I added src/Makefile and scripts/MSO.lua files, feel free to use them (e.g. put them into a trash, if you don't like them :-)).

Just one more note to insim_recv function. I don't like "busy waiting" code with "sleeps". I would recommend using either blocking recv or using "select" ("pselect") functions, what dou you think about it?

I attached the patch file and modified sources. I really like the idea of easy to use lua scripts, thank you for that .

Cheers for the comments Kada, I've not yet downloaded your ammended source / patches, so I apologise if I've said anything that you've already taken care of

Quote from Kada_CZ :

it doesn't work with LUA below 5.1, it's ok, but it could be mentioned somewhere, e.g. in the README,

That totally slipped my mind. I'll add it shortly

Quote from Kada_CZ :

the main binary has to be started from the same directory, where the "scripts" directory is.
After some minor modifications it works :-).

That was originally by design / lack of fore thought, well pointed out though :up:

Quote from Kada_CZ :

In my Linux system, the declaration of a variable named "type" (in the file scripts/ISM.lua) overrides the lua language "type" function, used e.g. in event.lua like this: type(func) == "string". This leads to "attempt to call global 'type' (a number value)'" errors. It's ok after renaming the variable, e.g. to "type_ism" in ISM.lua (Disclaimer: I have no experience with lua ).

My brain is a bit fuzzy at the moment (too much beer). I'll sort out a nice solution to this at some point.

Edit: I've just taken a look at your patch, despite telling myself that I really shouldn't in this state. I can see what you're on about now and I totally understand what you meant now :up:

Quote from Kada_CZ :

I modified the .c sources at several places, my modifications probably won't always compile under windows. I added src/Makefile and scripts/MSO.lua files, feel free to use them (e.g. put them into a trash, if you don't like them :-))

I'll see if I can get them compiling tomorrow afternoon / evening and integrate any changes, and add them as conditional blocks if I have to Cheers for attempting it under ubuntu. I hadn't actually gotten around to compiling this revision under my debian install yet :up: I really, really appreciate that

Quote from Kada_CZ :

Just one more note to insim_recv function. I don't like "busy waiting" code with "sleeps". I would recommend using either blocking recv or using "select" ("pselect") functions, what dou you think about it?

I don't like blocking sockets, because they don't allow you to timeout gracefully in a nice way. The reason I've not used select is because last time I used it, the Windows implementation was somewhat... well broken. This has probably been fixed, since it was quite a few years ago that I did anything along those lines, so I'll patch it tomorrow as well.

Quote from Kada_CZ :

I attached the patch file and modified sources. I really like the idea of easy to use lua scripts, thank you for that .

I absolutely love the fact you've put some time towards this Kada, thank you for putting stuff towards this :up:

I've love some thoughts on the following ideas as well;

I had an idea of possibly implementing a quick config implementation tomorrow based on tables i.e.

Not necessarily set out like that, but who knows. I'd love some comments on the idea of using tables as the main configuration concept though. Perhaps its not the best way? Interating through tables is a bit clunky as it requires pushing functions on and off the Lua stack and then getting the results, sadly.

This would allow the system to be easily extensible and for third party scipts to add their own configuration with hooks in the API to read / parse the conf also... The only alternative I can think of right now is to pass back a pack'ed structure to C, which isn't really as flexible as I'd like

That led onto something I went mad on this afternoon and basically I sketched out a quick league system (á la the STCC league), where player results are stored in a table, and then pickled / serialized for storage later. Theoretically it would work pretty well, and be very easily configurable for just about anyone. Speed concerns me in this instance though Would that be going too far? Or what this is ideal for?

Quote from the_angry_angel :

I don't like blocking sockets, because they don't allow you to timeout gracefully in a nice way. The reason I've not used select is because last time I used it, the Windows implementation was somewhat... well broken. This has probably been fixed, since it was quite a few years ago that I did
I've love some thoughts on the following ideas as well;

Ok, your way is reasonable then. I'm sorry, I don't have much experience with programming under windows .

Quote from the_angry_angel :

I had an idea of possibly implementing a quick config implementation tomorrow based on tables i.e.
...
Not necessarily set out like that, but who knows. I'd love some comments on the idea of using tables as the main configuration concept though. Perhaps its not the best way? Interating through tables is a bit clunky as it requires pushing functions on and off the Lua stack and then getting the results, sadly.

I agree with the concept "by tables" to allow user scripts access configuration. I don't see a better solution, but maybe someone else does :-).

Quote from the_angry_angel :

That led onto something I went mad on this afternoon and basically I sketched out a quick league system (á la the STCC league), where player results are stored in a table, and then pickled / serialized for storage later. Theoretically it would work pretty well, and be very easily configurable for just about anyone. Speed concerns me in this instance though Would that be going too far? Or what this is ideal for?

I wouldn't be afraid of speed of a Lua code. On the other hand, I don't know how fast the Lua garbage collector works. But if the garbage collector would be the bottle neck, then we could use manual "finalizing" of values.

Quote from Kada_CZ :

Ok, your way is reasonable then. I'm sorry, I don't have much experience with programming under windows .

The bug I hit years ago has been fixed. The next version will be using select most likely

Quote from the_angry_angel :

I had an idea of possibly implementing a quick config implementation tomorrow based on tables i.e.

cfg = {
insim = { "address" = "127.0.0.1, "port" = 29999, "password" = "pies", flags = etc. blah blah },
lua = { "forcegarbagecollection" = yes },
blah blah
}

Not necessarily set out like that, but who knows. I'd love some comments on the idea of using tables as the main configuration concept though. Perhaps its not the best way? Interating through tables is a bit clunky as it requires pushing functions on and off the Lua stack and then getting the results, sadly.

Use glib? The following would be very useful:
http://developer.gnome.org/doc ... ey-value-file-parser.html

Quote from wabz :

Use glib? The following would be very useful:
http://developer.gnome.org/doc ... ey-value-file-parser.html

I'd rather keep the config a lua script, so it allows you to do some funky stuff at runtime Thanks for the suggestion though :up: I might invest some time in looking at glib again for future projects

I like glib, but I would suggest to manage luaLFS without external libs, if possible (except libs for LUA). What I really love in this project, is the fact that it's cross platform capable. Each external library could cause problems.

I've been pretty sick the last 2 days, but this afternoon I felt like doing something relaxing at a slow pace and turned to luaLFS for an hour or so to prevent me from catching up with work I'd missed.

For version's sake, this is 0.2 and comes with a few funky features;
* Configuration via a config.lua script
* Now uses a select and non-blocking socket hybrid monster to monitor time outs and send keep alive's to LFS, whilst not hammering the system as much - this is configurable, but will be changed properly later
* I've just started implementing the API within the lua scripts. Right now it's pretty basic still, but you can get the sort of idea of where I'm heading with this. I obviously need to do a few other things, like make racer tracking automatic, and make the API easier to use. I gave up at that point deciding I felt sick again (who knows if that was the Lua or the illness... )

I've integrated the code from Kada, and added a few other bits here and there, but it's not the full clean up I was planning on releasing.

Incidentally I realise that the tabbing, etc. is severely cocked up now - thats my fault for changing editor and PC half way through this afternoon.

Anyway, here it is.

Update this afternoon - whilst working on the trunk (development) version I realised that with the code written last night it's possible to timeout due to how the internals worked. I've quickly hacked something onto the side of this to prevent it from happening and added a few "advanced" configuration options that govern this.

I've also bundled a few other samples and added a fix to the API so that the value of an MSO is correctly returned, and added an InSimPack function. I've also correctly readded Kada's Makefile for linux (this tag is untested on anything other than Windows. Trunk runs fine however, so it should be ok). I've also changed slightly how verbose mode works.

I'd appreciate anyone who can give me any comments on the API as I'm not entirely happy with it at the moment, but it's all I can think of to make InSim packets easier to understand, without going far away from the whole InSim concept (which would be a bad thing).

Current plans:
* Make it binary relocatable for non-windows OS'. Due to the fact that there's no real fixed way of doing this under non-windows across all platforms and various distros, I might simply add a command line argument
* Clean up the code
* Finish the API, add racer tracking facilities (Primary goal right now)
* Integrate 2 power patches into the luaLFS client, so that Lua itself doesnt have to be patched for bitwise operators and !=, if possible.
* Possible GUI tools, perhaps

Keep going .
I'll try to spend a couple of hours on it in the next few days. Your work is greatly appreciated.

Vain

I made some changes to make it work under my Ubuntu (gcc (GCC) 4.0.3 (Ubuntu 4.0.3-1ubuntu5)), LFS patch W. In the attachements are the patch file and all sources.

My comments (for the 0.3 version):
- the LICENCE file is missing,
- I added some useless returns, because of "control reaches end of non-void function" warnings,
- the function "lua_error" returns "int" in my LUA 5.1-0 instalation, so I changed a couple of lines like this:

The original code has "%s", maybe in your LUA the lua_error function returns string, am I right?
- I have to run LFS first, then to type "/insim 29999" and then to run luaLFS. If I run luaLFS before LFS, it doesn't work, because the ISI and VER request packets gets lost. Maybe you could keep sending version request packets until the VER is received.

Quote from the_angry_angel :

Incidentally I realise that the tabbing, etc. is severely cocked up now - thats my fault for changing editor and PC half way through this afternoon.

You may try linux utility "indent" to clean it up. I'm not sure, if there is a version for Windows.

I didn't play with the new API yet :-(.

Great work, keep it up .

Quote from Kada_CZ :

- the function "lua_error" returns "int" in my LUA 5.1-0 instalation

Quite right, that should be %s. With respect to the warnings, I'm ignoring them for the moment, as I plan on sorting the mess that is the master InSim struct soon.

Quote from Kada_CZ :

- I have to run LFS first, then to type "/insim 29999" and then to run luaLFS. If I run luaLFS before LFS, it doesn't work, because the ISI and VER request packets gets lost.

By design for the moment. It's not something I care about right now, as until I get the api and what-not cleaned up I don't plan on running it as a pseudo-daemon for some time.

Quote from Kada_CZ :

You may try linux utility "indent" to clean it up. I'm not sure, if there is a version for Windows.

Eventually when I get my main desktop & development machine with a new debian install (my motherboard requires a 2.6.18+ kernel, and I'd need to install dmraid before installation) etc. Plus my soundcard currently has no support, so my only options are to export all the code to my production box (urgh), or do run a VM - which I was until I screwed the image

Cheers for the thoughts guys :up:

Woah, now even LFS gets Lua'ed
Just like GMod did some time ago.... Many games I played got Lua support at a certain time moment, now it's LFS.

Sorry fot O/T

Will this recieve an update for the new InSim?
I'd still like to use it to make an /echo based pitboard.

Vain

It's mostly updated, I just want to add a few new bits - such as log file location, and finish off the API. But in terms of making it InSim v4 compatible, it's all there.

luaLFS 0.4

Changelog:
* Added support for InSim v4 - now uses TCP. Currently no support for previous versions of InSim, or UDP.
* Can now attempt to connect multiple times
* Now has an auto-reconnect (currently a bit dangerous - a compromise for running against servers for the moment)
* More API implemented (still not complete right now though)
* Updated basic server-api
* Restructured internal code
* Should now build without any warnings
* Can now resolve addresses of computers running LFS instances
* Added rehashing support, so that an instance of luaLFS can re-read all the scripts and run them from scratch.

Todo:
* Add support for path argument
* Add support to redirect output to a log file (natively)
* Finish API
* Create a few more examples and finish the server-api example
* Add HTTP query support (this is sort of already implemented, with the age old libLFSW - it's currently working in my test version, but needs work)
* Add LFSW Api
* Bundle or link to a SQL package (possibly rolling my own with libdbi orientated, possibly pre-existing luaSQL package)
* Make luaLFS respond to "keepalive" packets, as per the InSim spec (currently I don't do this, I just send my own keepalive packets randomly)
* Add support for Outgauge packets

Thanks to:
Kada and Vain for the support, suggestions and stuff, and the Fragmaster team for letting me sap their bandwidth as an idle user, testing luaLFS against a client, rather than a server.

Edit: Forgotten the makefile :doh:

Quote from Quint999 :

Angel is this intended to be an AMX / Metamod style engine for LFS & the scripts become plugins for that.

Sort of - it was originally just a test of an InSim library I was working on, and bits ripped from another project. But in short it's ended up going towards the Metamod/AMX way basically

Quote from Quint999 :

I'd be interested in learning a little more programming, would this be a good place to start ? , my current knowledge is fairly straightforward stuff and limited to Oracle PLSQL and bits of Perl / shell scripting etc.

If you're used to BASH or similar, you'd probably get on quite well with Lua. I've half implemented an API to talk to LFS with (which you can see in use with the bundled scripts in the scripts directory) - it very loosely echos the actual InSim packets.

For that reason, I'd say it's a good way to learn what all the various InSim packets do, and a good intro to coding.

Should compile under GCC. I've not tested it under anything else

Hi, I made it work under my Ubuntu. The patch and all_in_one_zip are in the attachements. The most important change is in insim.c line 96, the select call. I changed

to

The select call modifies the timeout argument, so I made a local copy. The first argument has to be "the highest-numbered file descriptor in any of the three sets, plus 1". I'm not sure, if the windows implementation uses these arguments differently. Without this change the select call returns always zero for me.

The other changes are minor, some typos and warnings. I slightly change the rest of the insim_recv function just for more comfortable debugging.

I spent most of the time discovering, that lfsw_pb.lua example is written for an old version [evt_bind("MSO"...) instead of evt_bind(ISP_MSO...)].

The API looks very good. I hope, that I'll build something bigger soon.

Tested configuration: (GCC) 4.1.2 (Ubuntu 4.1.2-0ubuntu4), liblua5.1-0-dev 5.1.1-2build1, LFS W17, wine-0.9.36.

Quote from Kada_CZ :

The select call modifies the timeout argument

It does? Not that I don't believe you but I can't find any reference to that anywhere...

Quote from Kada_CZ :

The first argument has to be "the highest-numbered file descriptor in any of the three sets, plus 1". I'm not sure, if the windows implementation uses these arguments differently. Without this change the select call returns always zero for me.

Indeed Windows ignores this. I think I originally had it set as the fd and changed it for some reason.. I clearly didn't make a note of why

Quote from Kada_CZ :

The other changes are minor, some typos and warnings. I slightly change the rest of the insim_recv function just for more comfortable debugging.

It's probably because I've written it, but I much prefer the monster of the recv function as it was I can see the point of checking rc == 0 earlier, rather than leaving it to default. That said, we've implemented Lua into an engine - cycles probably isn't something we should care about too much. Being an arse I'm gonna leave it as it is - for now. Plus I've got an interesting idea I'm trying out in my developmental version which makes it rather tricky to implement that part of the patch right now

Quote from Kada_CZ :

I spent most of the time discovering, that lfsw_pb.lua example is written for an old version [evt_bind("MSO"...) instead of evt_bind(ISP_MSO...)].

Hehehe.. I got so fed up with that script after an hour of testing and not disabling it - which is why I'd left it

With regards to perror having crept into insim_connect - not sure of the relevance there?

Quote from Kada_CZ :

The API looks very good. I hope, that I'll build something bigger soon.

Awesome At least someone other than myself has an opinion on it. Clearly needs finishing, and I want to produce some nice docs detailing the gotcha's on rehash/disconnection handling and stuff. I'd quite like to pop an Outgauge receiver in there some how as well, which is sort of leading me down the path of pthreads - but it's not something I'm overly enthrawled doing with a lua state given my experiments eating crap loads of memory and multiple W17 servers.

Quote from the_angry_angel :

It does? Not that I don't believe you but I can't find any reference to that anywhere...

Indeed Windows ignores this. I think I originally had it set as the fd and changed it for some reason.. I clearly didn't make a note of why

It's written in 'man select' in my Ubuntu box (package manpages-dev_2.39-1_all.deb), I can't find exact online version [EDIT, man 2 select], but I found this, see "Select Law" at the bottom (2. -- nfds, 10. -- timeout).

Quote from the_angry_angel :

It's probably because I've written it, but I much prefer the monster of the recv function as it was I can see the point of checking rc == 0 earlier, rather than leaving it to default. That said, we've implemented Lua into an engine - cycles probably isn't something we should care about too much. Being an arse I'm gonna leave it as it is - for now. Plus I've got an interesting idea I'm trying out in my developmental version which makes it rather tricky to implement that part of the patch right now

Ok, the main reason, why I added the rc==0 test was debugging the select call.

Quote from the_angry_angel :

With regards to perror having crept into insim_connect - not sure of the relevance there?

I'm sorry, I forgot to remove it. The perror just prints the reason, why "Connection to <host> failed" (e.g. "Connection refused" when the insim port in LFS is not set to the right value).

0.6 (yes, 0.5 was missed publically)

This is a new version I've hacked together from my development version in the last few hours;
* Hooks backported - this allowed me to separate the socket code more from the Lua engine / keepalive packets, etc. and potentially produce a threaded version. This also keeps the Lua VM per connection, rather than per executable - this means lots of multiple server loveliness. Will lead me to spawn lib_insim shortly (more for my own use, but it's there if anyone wants it).
* Modified the keepalive, so that it now responds to LFS instead of blindly sending as per InSim v3
* Added a new sleep function for use in Lua

I've not touched the rest of lua API in this release whatsoever, or done much else. I've a working version of LFSW querying in my dev version, but it's damn slow (not the HTTP fetch, the actual parsing) so I'm working on that atm.

I've not tested this under anything but Win32 - I'm too lazy to boot up my VM for this piddly little release

Have fun, abuse on a postcard to the usual address, etc.. Now time to sort my washing

Edit: Forgot to mention that ISI inits with a request for NLP and MCI packets - probably not a good idea, but I left it in for testing purposes.

luaLFS 0.7

Yes I made a huge cock up in 0.6 - as in the lua bit wasn't working - which was entirely my fault (Beware, semi-technical ramble follows; if anyone cares, I'd not been thinking much about the actual lua bits, and more of implementing the hooks, and in the process forgotten about pointer indirection for the hooks ctx pointer on create (i.e. it was assigning memory, but not assigning it to the ctx pointer, thus although the lua state was created, it wasn't being passed to the other functions for the lua hook)).

Attached is the fixed version - if you try to compile it yourself, I'd fully expecting it to whine about stuff being incorrect.

Aside from the fix, nothing new.