Hi guys,

Those of you who frequent the programmers subforum may know I've been working every now and then on a project called xi4n. If you're not aware of it, in general terms its a client and server InSim program that does everything through plugins.

One of the main plugins I've been working on is the "TV Director". It's aim is to automatically pick the most interesting thing on track and present it to you. At the moment it is very dumb, but giving me good prototyping information.

What I'd like you to do, if you have a spare 10 minutes, is to watch the following video and give me feedback on the quality of the TV directing - rather than the actual race or the quality of the video.

I am aware of several issues with it, and as I say this is very much a first shot at something thats usable, but needs work - but before I go any further I want other spectator feedback so I can focus on not only what I find interesting, but other spectators find interesting.

Critique it just as you would a human being. I want to know when the program made a good decision, when it made a bad decision and any other general thoughts. During the whole filming I was completely hands off, it was all controlled through the program.

Main video link

Some direct time codes for you to skip over some chatter:
What I want from you, followed ... e talking over it (1m02s)
My thoughts (6m55s)
Overview of how it makes very dumb decisions (8m28s)
What I think is an good decision made by the program (9m41s)

I do have plans for this to go a lot further, so custom camera positions, rather than the default ones, making it a lot more intelligent. This is very much a massive prototype right now Hope thats uber clear

I apologise for the quality of the video and audio, I'm very much learning what are the best settings to use when uploading to YouTube, in combination with LFS, recording and compression, etc..

I appreciate your time.

IMHO at times, the camera stays too long on the car that was involved in a random crash. For example, at 5:50 in the video, it shows the car for nearly 20 seconds until it starts moving. Though, perhaps there wasn't anything "interesting" going on on the track in that time frame from the program's viewpoint, I dunno. It could be either one or the other.

In general, I think the camera's on time pretty well when a crash occurs, but it could be a little more... sensitive. That's a personal preference though.

Thanks for your input hp999. Thats one thing that annoyed me whilst filming it. You've correctly spotted that there is a "cooldown" to prevent rapid changes of camera angle - at the moment thats very very dumb. I've added a vehicle speed sensitivity factor to my todo list

Something I'll be playing with in more detail is trying to catch the start of a potential crash or interesting overtake much, much more quickly.

Again, thank you for your time hp999 - it is appreciated :up:

Ok here's some criticisms

At 1m45s the plugin stays on the crashed car a little too long imo (same at 5m55s).

Might be interesting for the program to pay attention to a car that is closing a gap very fast to a few cars infront (for upcoming battles). On the same approach the plugin could pay attention to one of the top dogs if he's losing too much time to the racers following behind. (wasted tyres?).
Maybe a few more % of the race time could be dedicated to the race leader.

As for custom camera position I would love to see something close to onboard DTM, camera with little FOV (around 45-50 degree) close to the dash, it gives a good sense of immersion.


Best of luck with your project.. looks amazing so far.

Quote from Skytrill :

At 1m45s the plugin stays on the crashed car a little too long imo (same at 5m55s).

:up:

Quote from Skytrill :

Might be interesting for the program to pay attention to a car that is closing a gap very fast to a few cars infront (for upcoming battles).

On my list, glad to see someone else has the same thought

Quote from Skytrill :

On the same approach the plugin could pay attention to one of the top dogs if he's losing too much time to the racers following behind. (wasted tyres?).

Hadn't considered that though, totally going on the list :up:

Quote from Skytrill :

Maybe a few more % of the race time could be dedicated to the race leader.

Shall see how it looks

Quote from Skytrill :

As for custom camera position I would love to see something close to onboard DTM, camera with little FOV (around 45-50 degree) close to the dash, it gives a good sense of immersion.

Had to go looking, since I don't get the opportunity to watch DTM, and you're totally on the money there - cheers for the pointer

Thanks for your time Skytrill

For being on auto it isn't that bad. It needs to not focus on a stopped car for as long (as previously stated). Otherwise it does seem to pick up on battles and incidents quite a bit. I noticed you said something about using it (or someone else using it) for events/broadcasts, in this case it is normally best to not use auto. In this case, do you by chance have a manual option already in the works?

Plus I just noticed your livemap plugin, that looks fantastic!

I was actually looking for something that done exactly this! Excellent!

It'd be nice if it somehow distinguished between possible classes? I think if that is taken into account it may react better.

Also spend less time on cashed out cars.

Also, it might be a good idea to totally ignore cars that are finished as with LFS many pull over to the side, which is going to cause a crash detection.

Custom views may also be nice, rather than just the default tv view for LFS.



I did wonder why you wasn't replying to me during that race :P - and I'm in the video rather a lot, thankfully I didn't crash that race :P

That's a really outstanding job, the result looks pretty good for an automated system.

Sometimes I had the feeling that the camera was changing a bit too hectically, especially at the start of the race. I realize that stuff like this is a pain to get right, but perhaps you could introduce a "race start" mode that would focus more on the entire pack and battles in the front instead of numerous little bumps and fights in the middle of the field?

Oh god why did you pick the one race where I took two people out and crashed twice, all picked up by your stupid programme - grrrrrr!!!!!!! (j/k, obviously )

I think the fact it picked them all up, and pretty quickly, shows the detection algorithm is pretty good. As you alluded to in the video, I think the multi-class-ness probably didn't help due to the large speed differentials and blue flags. I think maybe it needs to focus on one battle for a bit longer before switching to a different one, that UFB battle looked quite good but it did keep switching off it a bit to look at me plough into yet another backmarker

And having watched that, I thought me (Blue FXR with the green wing mirrors) hitting that UFB was his fault, having looked at it there, maybe not. Although he did turn in much more than I expected

I have 2 suggestions:

1) 3:09-3:15 When you are focusing a car, and there is an accident viewable (in this case, the car just behind goes off-track), camera should jump inmediatly to that car. (If/when you use custom cameras it should have a smooth transition)

2) IMO the leader should take the focus during his last 2 corners. Unless there is something more important to show.

The other issues are already commented

I have 2 suggestions:

1) 3:09-3:15 When you are focusing a car, and there is an accident viewable (in this case, the car just behind goes off-track), camera should jump inmediatly to that car. (If/when you use custom cameras it should have a smooth transition)

2) IMO the leader should take the focus during his last 2 corners. Unless there is something more important to show.

The other issues are already commented


Probably you will hate me for bringing this idea xD, but I believe if instead of working live, you use a replay, you could make replays of the accidentes...just saying

Maybe better to say it here rather then on Facebook but;

Can you repeat this on saturday evening when there is only one class racing. Or... When not possible, sunday (FBM day) . I think it gives a better view for league organizers. Our normal races are way to messy, too many different car classes. Not easy for a spectator to follow that.

Oh wow, this has gotten more views than I expected Thank you for your time one and all, I do appreciate it

Quote from PMD9409 :

do you by chance have a manual option already in the works?

I intend on having a little GUI that allows you to stop the automatic hunting, etc. picking a vehicle and a view manually, but it's not written yet.

Quote from Krammeh :

It'd be nice if it somehow distinguished between possible classes? I think if that is taken into account it may react better.

Definitely something it needs

Quote from Krammeh :

Also, it might be a good idea to totally ignore cars that are finished as with LFS many pull over to the side, which is going to cause a crash detection.

I'm considering a counter that determines how many times a vehicle goes off track, if they hit a certain threshold per minute then start ignoring them. Thoughts on that?

Quote from Krammeh :

Custom views may also be nice, rather than just the default tv view for LFS.

Totally in the works

Quote from Krammeh :

I did wonder why you wasn't replying to me during that race :P - and I'm in the video rather a lot, thankfully I didn't crash that race :P

Ah sorry dude, I wasn't really looking at player names or chat Had I noticed I would've totally said hi

Quote from MadCatX :

perhaps you could introduce a "race start" mode that would focus more on the entire pack and battles in the front instead of numerous little bumps and fights in the middle of the field?

Hadn't considered something like that either - I'll see what works out on that front

Quote from J@tko :

Oh god why did you pick the one race where I took two people out and crashed twice, all picked up by your stupid programme - grrrrrr!!!!!!! (j/k, obviously )

Well I thought it was funny

Quote from J@tko :

that UFB battle looked quite good but it did keep switching off it a bit to look at me plough into yet another backmarker

Yeah that sort of thing it where it really falls down. At the moment it treats all events equally. I'm going to start looking at some sort of prioritisation and starting and ending event detection to help mitigate that.

Quote from J@tko :

And having watched that, I thought me (Blue FXR with the green wing mirrors) hitting that UFB was his fault, having looked at it there, maybe not. Although he did turn in much more than I expected

It did look like you had room, but tbh the driver could've gone either way

Quote from Whiskey :

camera should jump inmediatly to that car. (If/when you use custom cameras it should have a smooth transition)

Not sure if I'll be able to do transitions, but it'll be interesting to try

Quote from Whiskey :

2) IMO the leader should take the focus during his last 2 corners. Unless there is something more important to show.

I did have that in an earlier version, however I often found that I was missing good overtakes back in the field. I'll reintroduce it with the newer event detection system and see how it goes :up:

Quote from Whiskey :

Probably you will hate me for bringing this idea xD, but I believe if instead of working live, you use a replay, you could make replays of the accidentes...just saying

Ooooh, now thats a cool idea Would be interesting to try that out

Quote from cargame.nl :

Can you repeat this on saturday evening when there is only one class racing. Or... When not possible, sunday (FBM day) . I think it gives a better view for league organizers. Our normal races are way to messy, too many different car classes. Not easy for a spectator to follow that.

Totally, what sort of time is the single class on Saturdays?

Is this intended as a real time thing for watching a race live, or for post processing for some sort of broadcast?

If it's not live, then you could use a 'look ahead' approach, that allows your software to make decisions by 'looking into the future'. So for example, it can get the camera set and ready a second or two before a big crash, or a series of overtakes... or similarly, it can spot when an event isn't really interesting after all based on nothing happening for x seconds after the initial 'incident'.

Obviously, this doesn't work for real-time live viewing.

cheers

Col

Quote from the_angry_angel :

Totally, what sort of time is the single class on Saturdays?

17h BST - 23h BST

Depends a bit because it's initiated and ended after a full track cycle (~ one hour)

Few comments, some of which have already been captured;

- At start of race (00:21), camera should probably be bit further forward to see pole sitter in frame and starting off.

- Camera staying too long on cars that go off or rejoin after an off (01:42 / 02:10 / 02:20 / 03:32 / 03:46 / 05:48, etc).

- If nothing happening, should maybe show leader more - seems to show person in 2nd place more often.

- Sometimes focus on cars going round on their lonesome with no obvious reason why this should be so (04:18 + 06:23).

- Focus for long time on cars doing nothing then suddenly focus on battle that must have been going for while (circa 06:23).

- On close battle (e.g. 05:16), maybe some on-board shots from closely following car.

Possible additions:

- Maybe focus on a car going over start/finish line if it gets close to or beats WR time?

- Focus on leader crossing finish line at end of race, unless something more epic going on in background?

Having said all that, it still looks as good if not better than some F1 races I've seen in the past, like when ITV where showing races, and the director was focussing on either his favourite driver, a local driver, or was in loo on comfort break, and missed all the action!

Quote from col :

Is this intended as a real time thing for watching a race live, or for post processing for some sort of broadcast?

Live, I'm afraid

Quote from sinanju :

- At start of race (00:21), camera should probably be bit further forward to see pole sitter in frame and starting off.

I'll add that into the "start mode"

Quote from sinanju :

- If nothing happening, should maybe show leader more - seems to show person in 2nd place more often.

In this instance that was just more luck than anything else But I totally understand where you're coming from on that :up:

Quote from sinanju :

- Maybe focus on a car going over start/finish line if it gets close to or beats WR time?

I'll add it as an interesting event :up:

Thanks for your input gents :up:

Quote :

struct IS_CPP // Cam Pos Pack - Full camera packet (in car OR SHIFT+U mode)
{
byte Size; // 32
byte Type; // ISP_CPP
byte ReqI; // instruction : 0 / or reply : ReqI as received in the TINY_SCP
byte Zero;

Vec Pos; // Position vector

word H; // heading - 0 points along Y axis
word P; // pitch - 0 means looking at horizon
word R; // roll - 0 means no roll

byte ViewPLID; // Unique ID of viewed player (0 = none)
byte InGameCam; // InGameCam (as reported in StatePack)

float FOV; // 4-byte float : FOV in degrees

word Time; // Time in ms to get there (0 means instant)
word Flags; // ISS state flags (see below)
};

This is what I meant with smooth transition. Though I never tried it, I guess it does the job.
Though I'm not saying it's easy to do hehe

Really interesting and very well executed.

It's not perfect, but neither is a real life broadcast - so I'd actually prefer a few imperfections, and for it not always immediately cut over to every off or accident.
I liked that you included in-car crosses.
There are some obvious things to fix that have already been mentioned, like lingering on the crashed TBO. Sort those things, and it's a winner.

Would love it for replays too.

Congrats.

Quote from the_angry_angel :

Oh wow, this has gotten more views than I expected Thank you for your time one and all, I do appreciate it

No problem, I personally wouldn't mind a better TV Director program for when I'm doing broadcasts, so I want to help as much as possible.

Quote :

I intend on having a little GUI that allows you to stop the automatic hunting, etc. picking a vehicle and a view manually, but it's not written yet.

Alright thank you, exactly what I wanted to hear. That's the part I'm most looking forward to.


Also someone mentioned "race starts" being different. Is there a way to set a different set of camera angles just for lap 1? Like a TV1 category of cameras for lap 1, and a TV2 category for other laps where its more about focussing on one car. With lap 1 cameras you could make cameras at end of straights etc to focus more on the field and not just a couple cars when it comes to the starts.

I hope that made sense.

Quote from Whiskey :

This is what I meant with smooth transition

Hadn't even noticed that existed. I shall experiment after work this evening :up:

Quote from farcar :

Would love it for replays too.

Not sure if LFS puts out InSim packets whilst watching a replay, but I'll have a play one evening this week and figure it out

Quote from PMD9409 :

Also someone mentioned "race starts" being different. Is there a way to set a different set of camera angles just for lap 1? Like a TV1 category of cameras for lap 1, and a TV2 category for other laps where its more about focussing on one car. With lap 1 cameras you could make cameras at end of straights etc to focus more on the field and not just a couple cars when it comes to the starts.

I hope that made sense.

Totally makes sense, and again something I'd not considered. It'd need to go in after I've started messing about with custom cameras, but I don't see why not..

Quote from the_angry_angel :

Not sure if LFS puts out InSim packets whilst watching a replay, but I'll have a play one evening this week and figure it out

It even outputs when you click the end of the time-bar, very useful when using LFS Stats and such programs

I liked a lot watching a race like this:
http://www.lfsforum.net/showthread.php?p=1119906#post1119906

A TV-director split screen feature working with different angles and cameras for sure would be interesting and look nice IMO.

I must confess to not staying interested for long, as there were two critical flaws right from the start:

1) The choice of camera at the start, a special start of race directive needs to be there to show the action at the front, and passes nearest the front - perhaps with more use of cameras looking back at the field to get a clearer image of the race start.

2) The camera focus' on 1 car, it's a classic sim racing mistake in every sim out there... But real TV does not do that, the camera typically tracks two cars and it is possible to do this in Insim as I did it for X-Cam. ( http://www.youtube.com/watch?v ... 5myGUc&feature=relmfu )

I've been silently wishing for something like this for a long time.

Since not every online race is a championship race, not every position is relevant for making interesting material. It seems to favor off track outings and collisions. IMO the top position racers are more interesting than crashes/collisions because they are racing for the win. Most of the camera views were irrelevant toward the progress of the race. Watching lap leaders with a gap of a second is more interesting that a back-marker's outing. Prioritize focus during "dull" racing would be a big plus.

On a side note, since that was a multi-class server, distinguishing between class leaders would make for more live material to switch between, especially in dull times, in this case.

Seems like most of my input has already been suggested though.