EDIT: The new mod approval system is now live: https://www.lfs.net/forum/thread/95940


Hello Mod Creators,

You may know we have been trying to sort out the mods approval system. We want it to be easy to find high quality mods, but we would like to allow people to experiment online with early stage WIP mods.

Our current system to try to ensure reasonable quality has involved talking a lot with mod creators, sometimes trying to get them to improve the mods quite a bit before we publish them. But we know that isn't helpful for early mods, and it's just too much work for the reviewers.

So we've come up with a new plan that involves mods going through two stages. Currently we have called these stages WIP and APPROVED.

The idea is to make it much easier to get early stage mods on the system but still provide a strong quality control system for approved mods.

How the system would work:

If a submitted mod is legal, moral and not going to cause problems, we would allow it onto the system, as a WIP mod. Then, after a while, and probably various updates (and usually requiring community support, by various ratings) good mods could eventually be checked by the reviewers and APPROVED.

By default, lists of mods (on website or in game) would show only approved mods. Servers can be started with "allow WIP" if they want to be testing grounds. I think that servers that define their own shortlist of mods don't really need to think about "allow WIP" because the list defines which mods can be used anyway.

When I was checking comments yesterday (after formulating this plan the day before) I came across a post by xspeedasx which seems to match our plan quite closely. Start from "So the takeaway..."
https://www.lfs.net/forum/post/1969572#post1969572

Only he says "unverified" and "verified" where we have called it "WIP" and "APPROVED"

Additional information:

Each of those early stage mods would have a comment thread accessible directly from the mod's own page (and probably visible right there on the mod's page) for feedback and help that community members may wish to provide.

About the voting system, our current thoughts are that we need a voting system to get the mod to that stage where the reviewers can be asked to approve it. The public evaluation is supposed to take a lot of work off the reviewers. But we suggest that the user of a WIP mod can have the option to zero the votes at any time (e.g. if votes were low but now they have uploaded a new version).

The thing is that we need something numerical so that the mod reaches a certain quality level before the user can request approval. But a single star rating is too crude for this purpose. For example a mod might be great fun but its model is far too underdeveloped for the mod to become APPROVED.

An initial list of voting points (that could be on the mod's own page or comments thread) are:

- Overall rating (the one you can select in game)
- Technical quality (shadow errors, drive side errors, light functions)
- Model graphical quality (detail level, completeness)
- Physics quality (handling, default setups)
- Engine sound quality

We look forward to hearing any comments you may have on the plan.

Thank you for all the testing and experimentation!

Is it possible to add the ability to rate the mod in-game, say after jumping to pits with the car and without the need to come back to LFS.net? Sort of how "Rate the connection quality" of video chat @ facebook works directly after ending the conversation.

EDIT: Figured out you can already rate the car in-game, but yea, one could put it on a plate for ppl to leave feedback.

Quote from Ziomek21 :

Is it possible to add the ability to rate the mod in-game, say after jumping to pits with the car and without the need to come back to LFS.net? Sort of how "Rate the connection quality" of video chat @ facebook works directly after ending the conversation

that's already possible by clicking on the stars at the mod catalog

We are still wondering about the ratings. Trouble is a single rating like that can be misleading on a WIP mod. And we can't implement the full works into LFS, so we're wondering if the rating in LFS for a WIP mod could (after a confirmation dialog) open your browser and send you straight to the web page for the mod, where you could do a detailed rating on several points, and/or leave a comment.

I didn't quite agree with your first idea of mod approving system. but this sounds like a really good plan. Mods can still be used online without being "APPROVED" and servers that don't want all these shitty mods that have been created on them can easily disable them.

Great job!

A problem I can see with popular votes like these, it can become a vote of popularity as well.

A) Popularity of the cars: Cars that are popular will most likely be voted more favourable then unpopular cars or unknown categories. I assume if there were 80s touring cars the BMWs and Jaguars of the era will get better ratings then the Ladas or Trabants of the time even if the letter were equally good or even better. If it becomes a matter of approval, this might become frustrating.
B) Popularity of the creators: Popular mod creators might get better reviews then people with less reputation even when they create a "worse" product. While this might not be too much of a problem when it does not affect the approval, it might be a problem, when the creator himself is quite unpopular and they get "vote-bombed" or similar, even if they created a good or even great mod.

For B) I could see a solution by having WIP mods being tested blind, without disclosure of the creator, at least on the in-game and file download sides. This would require some form of comment function on the mod files, so feedback can be given no matter what. If they choose to disclose their identity on the forum or on discord anyways, it still provides a partial blinding as people who just vote through the in-game function will not necessarily read and follow discord/forum/... discussions.

Thank you for the feedback. I've asked the reviewers about one more thing. It is the use of the terms WIP and APPROVED. Current thoughts are that these two concepts are not really exclusive, so should be kept separate.

Mods system as it exists now:

• Mod must be thoroughly checked before publishing
  
• Mod can be freely set as WIP by the author
  

Plan as described above:

• Mod can be published easily if legal
  
• Mod starts WIP and may become APPROVED
  

Probable alteration:

• Mod can be published easily if legal
  
• Mod can be freely set as WIP by the author
  
• Mod starts by being published and may become APPROVED
  

In this case, WIP would remain as a tag that the author can set. But getting APPROVED is the key point of the new plan. Some mentioned the terms "unverified" and "verified" but in a discussion with my family we are thinking that "verified" might sound as if it is more about legitimacy, as used on Twitter or YouTube.

Uh huh I don't like this at all

So with this system the only way for a good mod to be visible for a regular user is for it to be accepted first, then enabled in an online server, then for people -reviewers- to join that server, pick that mod, test it, and then rate it highly. Only then is it going to be visible on mods page. Actually even then it is doubtful, as it may just not get approved.

I don't see this happening at a large scale and I see this as quite restricting. Players really don't have that much of a problem when they see a bad mod. In fact seeing those shitmods makes players admire when they see a good mod. I just see here too many hurdles which may demotivate the modders.

Only thing I see wrong with the current system is star rating being too ambiguous in terms of what the thought was when user rated the mod 1 star. If you could add a lil comment within the game along with a star, that'd give the modder an idea of what the general complaint is. Not sure if you meant this when you said implementing full works.

I'd change "Staff Picked" into "Approved" so that it sets a good example for a kind of quality that is welcomed and maybe expected. If you're really disgusted by those low quality mods, I'd just add "Show WIP mods" button in mod browsing page and keep WIP's hidden otherwise. Or I'd reserve the most visible part of the page for approved mods. In either case, if a user wants to get his hands dirty he either scrolls down or clicks "Show WIP mods".

For the mods that are really fun but will not ever be at a "LFS standard" level simply because of what they are, if they are rated well, I'd give them trophies for each milestone, like in lfsworld. E.g. 100x5 stars = liked mod, 200x5 stars, loved mod or top mod or stuff like that. Ofc 4* should also count as a good vote, but some algorithm is needed. These could be used when sorting as well. They won't get the approved sign but they are still fun, and fun is why we play so they should be easy to access for the users imo.

No. Regular users will be able to see any mod if they switch off a filter:

APPROVED mods only [YES]

The idea is that filter is [YES] by default, so you only see the approved mods. If you want to see the longer list of all mods including really early WIP and so on, just switch off that filter.

If you are going online and want to see the not approved mods, you should choose a server that allows unapproved mods.

In the post I didn't see anything about a filter so I naturally assumed they just wouldn't be visible at all.

But what if reviewers try to approve of a mod that is very fun but just not what "approved" would stand for? Who makes the final decision on this, staff?

And should there be a need for the modder to send it again for approval, after reviewers approve it and staff is ready to approve it as well? Modder may just forget to send it again for all I know. So imo it could be reviewed by staff and approved without another urge from the modder, just like Staff's Pick, which I assume is going away.

Quote from nexttime :

In the post I didn't see anything about a filter so I naturally assumed they just wouldn't be visible at all.

There was a lot of meaning in my words "By default"

The final decision for approval is by the reviewers. We hope the ratings will provide an accurate guide to which mods have reached the required standard. The algorithm that takes ratings and rating counts, and converts that to "mods that should be approved today" I expect will need adjustment and we will have to see how that goes.

I think I agree that the mod creator should not have to ask for their mod to be approved.

I really like the new proposed system. It helps to get actual WIP models out to people so they can test it and provide valuable feedback and help the creator to improve it, before it reaches the bigger audience. Similar to "preview" or "canary" channels on Windows and many other software products.

Being able to drive those "beta" mods is much better than just looking at pictures on the forum.

Servers not interested in WIP models (or rather "unapproved" models) could simply disable those with a single option, if I understood that correctly. Many perfect models still carry the "WIP" tag, probably because they are too afraid to the commitment so "unapproved" is rather the word we're looking for.

big +1 from me

Can something be included for evaluating engine sounds?

It seems quite common for modded engine sounds to massively clip, which is "okay" for one car, but when there's loads on track, plus echoes, it can be really unpleasant...

Quote from PeterN :

Can something be included for evaluating engine sounds?

Good point - added "Engine sound quality" to the list of rating points.

We have done most of this new system today (filters, server options, separate ratings, etc) and hope to release it tomorrow.

One important part of the system is the comments section on each mod's page. We have already released that. You can now give helpful comments to the mod's creator there. Obviously stay nice and constructive.

If you already have a thread for your own mod in the Work in Progress section, you can add that URL in your mod's details, and that thread itself will appear as the comments thread below your mod. Otherwise, the first comment should create a new thread (in a currently hidden location).

For example, Flame's Stock Car mod now has his discussion thread displayed below the mod.
https://www.lfs.net/files/vehmods/98F5ED

WIP Rating Idea:
Let them expire after 6 or 12 months.
Let the most recent 10 persist beyond expire date.

Thank you for the feedback.

The new mod approval system is now live: https://www.lfs.net/forum/thread/95940