Host log files - access and cleanup
(18 posts, started )
Host log files - access and cleanup
Hello Hosters,

As some of you may know, I've been looking around in the hosting system a lot recently and have done some fixes and updates that were necessary as some things were going wrong. It's a big system that Victor created but as he has a separate full time job, he prefers to be less involved with Live for Speed these days. He's still there to help but a lot of the website and hosting maintenance must now be done by me.

One thing I noticed is that the host log files just keep on growing endlessly but I don't think this is a good idea. For example, the biggest one is 15 GB in size which I think is just too big for a file that no-one ever looks at. Six other hosts also have a log file over 1 GB.

This is not an urgent problem but I think it should be fixed before too long. I'm wondering how it should really be done.

A few questions I am wondering about.

Do you even have access to these log files? I know you can see a few pages in the 'Virtual console' window - is that it?
What should you really be able to see, or what would you like to see?
I'm wondering if there could be some practical way of preventing the endless growth but provide what is needed. For example, maybe you only really need a few days' worth of log to be visible?

Just brainstorming now, but how about if each day a new log would be started, and only keep the logs from a few days? Maybe there's a normal way to do this kind of thing, I don't know so that's why I'm asking. Just try to keep it simple and practical please. Smile
Quote from Scawen :Do you even have access to these log files?

No,it's really just what is accessible in console,nothing more. I would prefer at least like 3 days worth of log - just in case something needs to be checked (like last Crashmas where a guy joined with 2 accounts).
Quote from Scawen :
Do you even have access to these log files? I know you can see a few pages in the 'Virtual console' window - is that it?
What should you really be able to see, or what would you like to see?

Hello.

The virtual console window has many positive aspects. But unfortunately there are also negative aspects.

It's great to see who uses admin commands. However, since there is no page in the virtual console, we can display a specific message. I think that's the worst part.

If it is possible to view page by page, I am in favor of seeing 1 week of console history.

Also, in the virtual console window, when a person connects to the server, they send too many messages. (IP addresses become spam) I think there is no need for the server owner to view them.

We cannot view the entire console history anyway. If this is hidden, it's for you. But since you're asking us about our access to console history, that's a different story.

Error notification: "Download bans.ban file" link on the bans list page does not work. It was working before, but recently it's not working. If there is a system such as adding a layout when we download this file, the banned players on the list can be uploaded to the ban page of the servers we own and banned from the server. Maybe I couldn't explain it, but I hope you understand.
#4 - Racon
I mostly use my insim logs for checking things out, but I've used the virtual console a few times. I can't remember what for exactly (sorry!).

3 days worth of log is good enough for me, chunking it into days is no problem.

I was wondering about replay storage also, just because we have many months' worth but I only really need the same few days, maybe a week to be safe. At the moment it's just on/off, but if there were a shorter option I'd pick it. Maybe it's not so big a deal on servers without a 24/7 lurker Wink
In theory if someone does feel like or has a system where it depends on long-storage of logs, he should connect an InSim to it and scrap&save all the logs to his storage instead, shifting the matter of archiving these things onto whos using them such way.
We should be able to have access to logs back tracking at least 7 days. After that, it may start overwriting the oldest logs untill the 7 day buffer gets full again.

The log should contain everything from the chat window, plus admin typed inSim commands. Not sure if anything else is really needed.
By default, daily or weekly options are offered. Server owners choose the one that suits them. It can be deleted after a maximum of 7 days or 5 days. In addition, I do not think that everyone always checks the records. Now the INSIM system has improved a lot.

Finally, it would be nice for users to have the opportunity to download this log file.
Forgive me if I'm teaching you things you already know.

Quote from Scawen :Maybe there's a normal way to do this kind of thing, I don't know so that's why I'm asking. Just try to keep it simple and practical please. Smile

Generally speaking in the unix world generally most systems leverage a tool called `logrotate`, or implement their own variation in-process. You configure it to keep X copies, how often they're rotated, if they're compressed, and then what to do before and after rotation - for example restarting processes.

For a game server, I cannot imagine any more than 1-2 weeks of logs are realistically needed, imho?
I only use the virtual console for diagnostics for InSim (Shutdown/(Re)starts) or hosting in general.
3 days worth of logs is enough for me.
Quote :Jul 21 00:00:00 0.7E15 Sunday

This is now how every day starts - sadly everything older is gone. I hope it's just some temporary setting as now it's impossible to check stuff on the next day after an event for example.
You use airio system, which logs does Airio miss that you have to view the logs from the webpanel? (aside from airio disconnect error logs)
Btw. don't forget to update `REPLAYS AND LOGS` section on bottom right in the lfs.net/hosting site. Now it says logs preservation of 3 months, but I assume it just changes now.
I am in the middle of working on it.

It now saves a week of logs, plus the current day, but that has only started yesterday (Saturday 20 July).

I haven't made the interface for it yet but you can already access your host's logs like this:

Visit your host's "activity" page, so you see a URL like this: https://www.lfs.net/hosting/admin/12345/activity

Then you can add ?log=Saturday to the end of that line to see yesterday's log.

Or add ?log=Today to get today's log.
This is very nice. But there are many spelling errors in the text file. They appear in a way that I don't understand. Maybe this can help you.

There is a ^T sign before the letter ğ. This is something that is done in InSim texts. But the players send these texts. I detected it in the letter ş and the letter ı. I don't know if these are a mistake, but my eyes got tired a lot while examining the text file. Because there are meaningless sentences and trying to read and understand them is very tiring on the mind.

Jul 21 15:57:49 ProDrft^H¡¹iYoTaKee^L connected (emrekaans)

Jul 21 10:47:15 J^H¢Ò¢Û^T™[A] : ^Lhay^Tır baslama
Jul 21 10:47:18 J^H¢Ò¢Û^T™[A] : ^Lbu güzel de^Tğil
Jul 21 10:47:24 J^H¢Ò¢Û^T™[A] : ^Lljdsnkjdsc
Jul 21 10:47:28 Repaired Mesh
Jul 21 10:47:28 C^TİTORLEN-NEMO 1.4 : ^Lba^Tşlatın
Jul 21 10:47:30 THE_YUFUS[A] : ^Lgüzel de^Tğil diyon basıyon
Jul 21 10:47:34 J^H¢Ò¢Û^T™[A] : ^Lgü^Tşlückle


------------------------------------------------------------------

Additionally, when we view the servers we own and manage on the website, there is a "join" button instead of "edit". The old version was better. We could already join by clicking on the server name in the online server list on the website.
Quote from RealistAdam :This is very nice. But there are many spelling errors in the text file. They...

That is the LFS code page identifiers as seen in the plain text log file.

I've added an option to see an interpreted version of the file in a browser tab.

Add to the end of the line &html=1

E.g. https://www.lfs.net/hosting/admin/12345/activity?log=Sunday&html=1

I hope to add the links to the activity page today.


About the Join link, that change was intentional. Nearly every time I tried to edit one of my hosts, I clicked the name. Instead of editing the host, my browser tried to start LFS to join the host. This became irritating. It seemed to me the interface wasn't intuitive, so I changed it.
A first version of the generated log links system is available now on the activity page.
This is great, I quite like the new hosting option and logging changes, and it's very intuitive to use now, tnx Scawen.
Quote from the_angry_angel :Forgive me if I'm teaching you things you already know...

Thank you, any tips might help as my Linux experience is still low.

In this case I did my own implementation which I think was the easiest way forward with all the logs still being produced in the same folder. No code changed at that point. The new code moves them all into one of the 7 new day folders each time a new day comes around. It's naturally limited to one week of logs.

Host log files - access and cleanup
(18 posts, started )
FGED GREDG RDFGDR GSFDG