Technical challenges connected to simulating light refraction on armored windshields

[CSW_Tommy544]

As the title says, I was wondering what are the current technical limitations preventing flight sims from implementing some realistic light refraction. In my opinion, armored windshields are the main candidate (other than let's say bodies of water) that would benefit from this effect the most, as they take most of our monitor/VR headset screen space during flight. I have seen light refraction effect being utilized in other non-flight sim games, though never in such a scale as looking at the whole simulated world through a material that has this effect added. However, it can work well with puddles, lakes or small glass objects.

 

I am a software engineer, but I have a very limited experience with 3D rendering. I can see that some of the popular 3rd party game engines support light refraction at least to a certain degree, but I don't know whether you could simulate an armored windshield using these default tools. For example: https://docs.unrealengine.com/4.27/en-US/RenderingAndGraphics/Materials/HowTo/Refraction/

 

I think it would be really interesting to have the view of the world in front of you slightly shifted while looking through the glass as is evident from this video:

 

Since as far as I know no flight sim simulates this effect, it makes me wonder, would it be too expensive to simulate it with the current technology or is there perhaps any other reason for not including it?

[Eisenfaustus]

46 minutes ago, CSW_Tommy544 said:

would it be too expensive to simulate it with the current technology or is there perhaps any other reason for not including it?

Good question- in tankcrew there are corner mirrors- which should technically be applicable as solution here. 

[JimTM]

This thread (started in 2014) may interest you.

Edited April 21, 2022 by JimTM

[CSW_Tommy544]

6 hours ago, JimTM said:

This thread (started in 2014) may interest you.

Thanks, that thread has a lot of good information.

 

I wonder if there really hasn't been any progress made in technology or techniques since then, which could lower the performance impact of implementing refraction. 

[Eisenfaustus]

8 hours ago, CSW_Tommy544 said:

Thanks, that thread has a lot of good information.

 

I wonder if there really hasn't been any progress made in technology or techniques since then, which could lower the performance impact of implementing refraction. 

In tanks we have multiple corner mirrors per crew member - why shouldn’t a single one for a pilot be possible?

[AEthelraedUnraed]

20 hours ago, CSW_Tommy544 said:

Since as far as I know no flight sim simulates this effect, it makes me wonder, would it be too expensive to simulate it with the current technology or is there perhaps any other reason for not including it?

To quote one of the Devs in the thread JimTM linked:

  

On 2/1/2015 at 8:02 PM, Han said:

Flight Model (plus tanks, AAA, AI planes and so on) calculations may slow down the simulation thread - you will see "slow-motion" in this case, in other words in this case the PC CPU will not be awailable to maintain x1 time scale of physical world modelling.

 

Graphic features are calculating in graphics thread and lack of graphic performance can be seen as loss of Frame Rate (FPS).

 

BoS is a multithread game.

 

We have reflections now, you can see them on plane glossy surfaces. But there are many "cheats and hacks" performed to make them not too hurt for FPS. And MIRROR in your cockpit can't be done by same technology - detalization, resolution, view range of image in mirror will be not enough for usability.

This why other games have a special option for resolution of mirrors. Top resolution is cool in them, but performance is falling down dramatically.

 

Same situation with refraction. Refraction is possible, but both ways are very hurt for FPS if you want to have a good resolution in your "thick window" baceuse render algorythms becomes several times more complex in this case.

 

Nothing unknown here, all approaches of such features realization are known a long time ago. But flight simulator is loading GPU close to limit allready due to long-ranges of view, huge scene areas, high speeds and impossibility of "slicing" of the scene area.

 

One day, may be.

 

To summarize: technically, it's possible. However, it's relatively expensive GPU-wise since there's a less clear relationship between pixel location and visibility/Z-buffer/etc. checks. Implementing refraction would cause a large FPS hit.

[1PL-Husar-1Esk]

1 hour ago, AEthelraedUnraed said:

To summarize: technically, it's possible. However, it's relatively expensive GPU-wise since there's a less clear relationship between pixel location and visibility/Z-buffer/etc. checks. Implementing refraction would cause a large FPS hit.

as were mirrors at that time but they finally made them possible and without big fps impact,  btw they changed render since that Han post so it might be different now 

[AEthelraedUnraed]

1 hour ago, 1PL-Husar-1Esk said:

btw they changed render since that Han post so it might be different now 

They changed from forward to deferred rendering. AFAIK that doesn't make much difference in the FPS hit for refraction.

 

Basically, "traditional" graphics rendering as used in IL2 determines which object is visible in each pixel (this is the so-called Z-buffer). It then draws this topmost visible object for each of the pixels. The problem with refractions (and also mirrors) is that this z-buffer is not valid anymore since all objects show up in a different place than before. The game engine therefore has to calculate/write the Z-buffer twice, including all the necessary calculations. Of course, there are some optimizations you could employ so it likely doesn't take literally twice as long. Also, for small refraction displacements (e.g. raindrops on the canopy), you might likely be able to get away with using post-processing to simply select slightly displaced pixels from a second frame buffer that you've rendered beforehand.

 

Regardless, refractions are hardly free and will definitely have some kind of FPS impact. The fact that they implemented mirrors and switched to deferred rendering changes little about that.

 

(Note that things such as refractions are one of the strong points of Raytracing, where it doesn't really have much of an impact at all for very impressive results. But Raytracing has other big disadvantages, mainly its very low speed which makes it generally unsuitable for real-time rendering such as in games.)

[Eisenfaustus]

19 hours ago, AEthelraedUnraed said:

Regardless, refractions are hardly free and will definitely have some kind of FPS impact. The fact that they implemented mirrors and switched to deferred rendering changes little about that.

Ok - the corner Mirrors in the tanks basically do exactly the same thing as an armored windscreen in an Airplane would do (move the corresponding pixels lower than they originally would be...)

As Panther commander you are surrounded by several of these. The game doesn't become a slideshow - neither in 2D nor in VR.

Honest question: Can you explain to me, why the performance impact of an amrored windshield should be so much worde?

[AEthelraedUnraed]

53 minutes ago, Eisenfaustus said:

Ok - the corner Mirrors in the tanks basically do exactly the same thing as an armored windscreen in an Airplane would do (move the corresponding pixels lower than they originally would be...)

As Panther commander you are surrounded by several of these. The game doesn't become a slideshow - neither in 2D nor in VR.

Honest question: Can you explain to me, why the performance impact of an amrored windshield should be so much worde?

Well, first of all, who said anything about a slideshow? Without wanting to immerse myself all too deep into shader theory (it's been couple of months or so since I last wrote one), I don't think much else besides the z-buffer would need to be rewritten, so for the pixel shading itself (which is likely the most expensive operation) I think it would hardly matter. The game won't instantly become a slideshow if you'd have refraction.

 

That said, refraction would probably be a rather fixed amount of calculations (depending of course on how many windscreen pixels are visible), which would hence disproportionately affect those systems that are already struggling. I guess that for those happy owners of a 3080 it would hardly matter, but for those people who are already struggling on low GFX settings, it could be the death blow. Unless of course if you'd make refractions an optional graphics setting, but then gameplay comes into play as those people with refractions off would have an advantage over those with refractions on in things such as aiming.

 

Regarding your question itself, the Panther commander doesn't have corner mirrors, only the cupola windows. I guess you mean the driver with one corner periscope, and another one ahead of you if you close the window? Those do not acually move any pixels. They're basically just a holdout shader - for each of the pixels of the periscopes, the game basically just stops drawing the tank geometry and only draws the terrain (and other objects of course) outside of the tank. If you take a close look, you can see that the periscopes exactly match your eye height, rather than the height of the periscope "exit" a bit higher up the tank as you would see with a real periscope.

 

A better comparison would be the mirrors you find in some of the aircraft (although a different technique). On the lower mirror settings they use quite a bit of simplifications (no geometry of your own aircraft, for example), yet there's a noticeable performance impact. Now imagine a mirror that almost fills the whole screen (looking upward in an Fw-190) .

[Eisenfaustus]

Thanks - wasn’t aware that it weren’t actual corner mirrors in the tanks. Guess I‘ll have to recheck