IMPORTANT:
Converting textures is now implemented in LFS !!!
Removed links as they are not needed anymore


Ever since I started LFS I use hi-res textures, and one thing that has bothered me a lot is slow track load. Using lower resolution textures was not an option. Recently I started doing some tests regarding LFS texture load. First I pregenerated mipmaps* for all files in my dds folder. I recorded approximately 30% faster load of track.
*Mipmap - original texture converted to a series of lower resolution ones used when viewing from a distance

Somehow I felt there is still more room for improvement. Scawen kindly advised me to use original dds format as LFS use. I did that and achieve quite a remarkable results. For most tracks load time has decreased three times or more.

Thing is most of user made textures are saved in "wrong" format which require LFS to convert them each time while loading track. Saving in those "wrong" formats doesn't benefit from neither side : more waiting, details which user perhaps tried to save are lost anyway after conversion by LFS and often these formats use more space (my dds folder dropped from 1,8GB to 1,4GB).

I made little program which can convert your textures too!

That's pretty much it, enjoy faster LFS.

Benchmark (stopwatch) and post your load times before and after.
Before my BL would load in 65 s, after conversion approximately 5s.

I do not use many of hi-res textures (except FE), but every option to speed up loading is good.

Time measured from pressing OK to complete track load (open config tracks loaded).


Add this to the description, for people who do not have the MSVCR120.dll file:
http://www.microsoft.com/en-us ... oad/details.aspx?id=40784

Draggo, thanks for your results
Program primarily speedup hi-res texture load, it will also speedup a little load of default textures (LFS wont have to calculate mipmaps, just load files), but most people probably wont notice it as load of default ones is already quite fast.

Quote from Draggo :

Add this to the description, for people who do not have the MSVCR120.dll file:
http://www.microsoft.com/en-us ... oad/details.aspx?id=40784

I've updated version now, so it doesn't need any MSVC ...dll Thanks for info

I've made a short video which show the possibilities when using RAM disk, also SSDs are very close.
LFS loading converted textures from RAM Disk

I haven't tried it yet, but judging from the posted figures, this (or directly the converted textures) should be part of next version vanilla LFS.

Is this just in a new patch or my track loading went very slow now.? Didnt used this programm yet. Just wanted to ask.

Quote from Litro :

Is this just in a new patch or my track loading went very slow now.? Didnt used this programm yet. Just wanted to ask.

Program is unrelated to new patch, it just optimizes texture files for faster load.
And don't forget to make a backup of both dds and pic folder.

i didnt calculate loading time in term of ms, but i tested on my pc and loading time are dramatically decreasing, now i have like 0.1 second to load any track even high res lynce FE with my trees textures! 0o

great work Daniel! that what lfs beta tester can do !

Quote from DANIEL-CRO :

[..]Scawen kindly advised me to use original dds format as LFS use[..]

What do you mean ?
I remember reading somewhere (also devs' recommendation) to save a DIY texture to raw format and then LFS will automatically convert it to dds. And this only the first time. So why reconvert the dds ? Or are you referring only to the textures made by "others" who didn't know to save as raw ?

Quote from lfsrm :

great work Daniel! that what lfs beta tester can do !

Thanks
I wish ... : )

Quote from luchian :

What do you mean ?
I remember reading somewhere (also devs' recommendation) to save a DIY texture to raw format and then LFS will automatically convert it to dds. And this only the first time. So why reconvert the dds ? Or are you referring only to the textures made by "others" who didn't know to save as raw ?

About raw files, this is probably the post you are referring to.
As far as I know raw files won't speedup load process at all, LFS just can convert them. Other than can there is no real advantage. This program doesn't convert textures to raw.

All textures are saved in dds format, default, user made, ... However there are still various formats in dds so called pixel formats, program convert textures between those pixel formats. LFS by default use DXT1 (for regular textures) and DXT3 (for some alpha textures). Difference primarily is that DXT3 allow explicit alpha e.g. you set 30% alpha... DXT1 has only one bit for alpha, which means pixel is transparent or not. As LFS request textures to be in specified formats DirectX has to convert them on load if the format is not right, for example if LFS request DXT1, but texture is saved in DXT5 DirectX will have to convert texture prior to load. And thing is that conversion happen each time you load track, which is absolutely not needed. Moreover DXT3 use 2x more space compared to DXT1, so by removing "wrong" textures you get some extra space and faster load (additional details are lost anyway when LFS convert textures).
Also each time LFS load textures it has to make mipmaps of textures, my program will generate all mipmaps to minimize need for any calculation by LFS. While creating mipmaps I use triangle sampling, that why some textures may look different (power lines) hopefully better

Also some may have noticed that ADS in pic folder are no longer openable with some simple picture viewer, the reason for that is they are now in DDS DXT1 format while still keeping .jpg extension. That is much more elegant solution than JPG when using hi-res ADS, for example my ADS in 4096x4096 JPG took 4s to load per file!, after conversion to DDS it is <0.01s. JPG is highly uneffiecient!

Thanks for the explanation Daniel .
I was not understanding why lfs' own converter (raw to dds) would not convert the textures to the best format for its use (one time process, not every time when loading the track).

Anyways, thanks for the program, it's always nice to speedup loading something .

Think I know where the source of confusion was, I used word "raw" while I actually meant just load and no need for conversion by LFS

Yes, would be cool if LFS could convert textures itself, maybe an option or so. For now this program will do the job

I did not measured with stopwatch,but loading times are really low now - before it took 3-7s (felt) seconds to load any track,now it was almost instant by any track.
Wondering why LFS hasn't those mipmaps added right away?

While converting, i noticed few lines saying "failed to open".
I looked for a log but there wasn't any.
Can I activate it in some way (command line switch)?

Also, does it "hurt" to re-run it (if I download new textures packs, for example)? Do the already converted textures get unnecessarely reconverted?

Quote from Eclipsed :

Wondering why LFS hasn't those mipmaps added right away?

Probably the reason is devs wanned to keep texture folder at lower disk space, mipmaps infact increase size a bit (114 MB compared to 101 MB – original textures). That little increase if often compensated by more significant decrease caused by wrong formats in hi-res textures.

Quote from Ripley :

While converting, i noticed few lines saying "failed to open".
I looked for a log but there wasn't any.
Can I activate it in some way (command line switch)?

Also, does it "hurt" to re-run it (if I download new textures packs, for example)? Do the already converted textures get unnecessarely reconverted?

Failed to open? Do you have these textures in your dds folder? Did LFS run at the time? If texture is opened by LFS probably program won't be able to convert it.
You can hit Pause/Break button if it doesn't go too fast or increase Command Prompt line buffer size so you can view log from start.
Re-runing shouldn't hurt, already converted textures will go much faster than first time (format check, ...)

Neat tool, there is definitely a speed improvement However, I'm not sure I like to lose the visual details in the pic folder. I wonder if there is a big performance difference if you only the convert the dds files.

I was in doubt too, but as I have some relatively big ADS it took quite some time to load. Only dds folder converted about 18s, after conversion of textures in pic folder it was like 4s. That is huge improvement if you are using hi-res ADS.

IrfanView (which I use) can open just fine everything in pic folder after conversion, no matter if jpg, dds, ... thing is Windows Photo Viewer cannot open dds, which I believe most people use

If you don't like having pic in dds, you can always restore a backup after conversion

Quote from DANIEL-CRO :

...Failed to open? Do you have these textures in your dds folder?...

I didn't have time to see the failed files' names.

After conversion I noticed some error messages while loading Blackwood.
After restoring the 2 backupped folder, error messages disappeared.
I'll try to grab some screens.

Quote from DANIEL-CRO :

...Did LFS run at the time?...

No, it didn't.

Quote from DANIEL-CRO :

...Re-runing shouldn't hurt, already converted textures will go much faster than first time (format check, ...)

This format check hopefully make it skip the reconvertion?

Great that Daniel's program is helping people load textures so much faster!

Quote from Eclipsed :

I did not measured with stopwatch,but loading times are really low now - before it took 3-7s (felt) seconds to load any track,now it was almost instant by any track.
Wondering why LFS hasn't those mipmaps added right away?

That depends on the track. Originally when LFS saved textures as dds, saving space was the main consideration. But later (I think when Blackwood was being updated) I noticed that some of the mipmaps (particularly some of the transparent ones) created by DirectX on loading were not very good, so I tried saving with mipmaps that had been created directly by LFS from the original raw files. That's when I found out about the much faster load times as well, so that's how LFS does it now. But we did not re-export all the old textures from all the other tracks, which would have made a big patch. I just thought we would release them when any track was updated. I guess it's BL and SO that have the mipmaps saved in their textures.

Scawen, good to see you in here too!
Here are my screenshots:


While converting




Original DDS folder (yes, I have those files)




Bug while loading Blackwood, converted textures

http://i.imgur.com/OUt4jUF.png


No bug using the backupped DDS folder

http://i.imgur.com/kunlWHx.png

Sorry for the dimensions!
I 've tried to add "width" parameter in IMG tag, but didn't work.

I see "failed to open" with some files during the conversion. Maybe you can post a few of those files here and Daniel may be able to find out why they failed to open.

There you go.

I also attach the converted BWOLDROAD file, which is one of those files LFS cant'open from the in-game screen.

Textures you posted converted just fine for me as well as load in LFS.

message "failed to load" happen when D3DXCreateTextureFromFileEx doesn't return an object.

Looking at converted texture you posted I see that part is missing.

I don't know what happened, but I will try to add more checks to code

Can you try to convert original textures again, not the one already converted?

I'll do.

Now other files are not converted...
As I'm sure you already noticed from my screens, I'm on XP. Maybe it has something to do with it?