Aircraft Survivability Comparator - BoBp Plane Set

[Stoopy]

This is for those of you who have lots of time on your hands (about an hour and a half) and are interested in potential ways to compare damage models among aircraft.  I've been playing around with it a while and since we may have updated damage models coming soon, this seems as good a time as any to inflict my misery and incompetence with the Mission Editor on the rest of you all. 

 

This mission establishes an "obstacle course" of light AA ground fire (Mg's only) of various types - MG34, M2, etc. plus some parked aircraft with trigger-happy tailgunners on the ground.  Then, 8 different AI aircraft are triggered to fly at very low level through the converged fields of fire along this obstacle course,  one at a time, and at the exit point of the obstacle course they continue to fly an additional ~22km to evaluate their survivability while being damaged.  Any aircraft that reaches the finish line is deemed to have survived, otherwise the number of PK's, crashed and/or critically damaged aircraft are counted.   The results are displayed on a scorecard that I put on the GUI Map. Each aircraft goes through the course 8 times, for a total of 96 total passes for all aircraft, making one data measurement set.  More than one full set should be run in order to get as much data as possible.  The idea being that you kick off the mission, sit back and watch the results accumulate (or go to dinner), and at the end you have a fairly representative evaluation of the relative survivability of each aircraft.   It takes about an hour and a half to go through the whole set.

 

The current aircraft set in the mission is based on the BoBp plane set:  P-47, P-51, P-38, Fw190A-8, Spitfire Mk.IX, Tempest, Bf109K-4 and Me262 (just for kicks).  I'm also modifying this for attack aircraft and BoS, BoM plane sets, but am focusing on completing my Stalingrad Interactive Playground right now (nearing completion at last!) so it will be a while. 

 

It's not in quite the finished state I'd like; there are two aircraft (Tempest to a degree, but more so the Me262) that I haven't been able to make fly through the obstacle course at the exact same speed as all other aircraft.  The 262 just cruises through unscathed most of the time since it spawns at a much higher airspeed.  So it's not a fair assessment of survivability in a strict regard, although one could argue that the 262's higher airspeed is a basis for survivability.  I would just expect folks 'round here to argue the point both ways (knowing how you people get  ). 

 

There are custom sounds that I added just to make it easier to track what might be happening by ear if you don't want to stay glued to your monitor (or HMD) for  all that time. Plus they're fun.  But in order work you need to unzip this into your IL2-Sturmovik/data/missions folder, such that the resulting path is:

 

{installpath}\IL-2 Sturmovik Battle of Stalingrad\data\Missions\Aircraft Survivability Comparator

 

If you don't , the sounds won't work (not may fault, it's an IL2 thing) and both myself as well as the collective staff at Foozle Aerospace will be supremely disappointed.

 

It would be VERY COOL of any of you who actually do have the fortitude to run this mission for the full 96+ minutes, to post a screenshot of your results as shown on the GUI map scorecard, as shown below.  Then we can build a collection of objective data to argue subjectively about.  ?

 

Here is a download link from Google Drive and I'll attach it to the very bottom of this post (the D/L link shown under the scoreboard screenshot is f'ed up) for download as well:

https://drive.google.com/file/d/1aEsUZNFoSNSnK8FuFnksAqz3WIqYE888/view?usp=sharing

 

Final comment:  It takes a minute or so for the action and flow to become apparent after unpausing at the start.  Be patient and all will be revealed... the FW190 on the ground is just there for atmosphere. 

 

Enjoy, comments and suggestions for improvement welcome, although I have a backlog of ME stuff to work on before I can do anything serious with this for a while. 

 

Aircraft Survivability Comparator v1.0.zip

 

Aircraft Survivability Comparator v1.0.zip

Edited February 15, 2020 by =[TIA]=Stoopy

[Luger1969]

Thanks Stoopy. I think it is a very good test with an expected or unexpected outcome. Just recently finished reading osprey gm...mig alley. Dont have the book with me now but this was discussed in it also https://www.google.com/amp/s/sobchak.wordpress.com/2012/04/24/article-why-the-u-s-air-force-did-not-use-the-f-47-thunderbolt-in-the-korean-war/amp/

 

 

Edited February 17, 2020 by Luger1969

[Stoopy]

19 hours ago, Luger1969 said:

...Just recently finished reading osprey gm...mig alley. Dont have the book with me now but this was discussed in it also https://www.google.com/amp/s/sobchak.wordpress.com/2012/04/24/article-why-the-u-s-air-force-did-not-use-the-f-47-thunderbolt-in-the-korean-war/amp/

 

 

Thanks very much for that link - great reading!  Appreciate it.

 

To be honest I'm hoping to hear back form anyone else that may have found time to let this mission run to completion (or even halfway) and if the results are similar.  They DO change from run to run, to an extent.   But I know it's a big ask since it takes a while. 

 

[1PL-Husar-1Esk]

Good tool , I will use it to test the beta build.

 

BTW how in the world P-51 aka Spam Can can be less vulnerable that the P-47 Jug. This is not what we can read about Jug exceptional reliability. 

I hope this to change

 

Edited February 19, 2020 by 1PL-Husar-1Esk

[ROTER_BART]

Poor P-47. The 12th one made it with critical damage. Just.

 

It looks like being fast and/or having multiple engines is a big advantage in this scenario.

 

It'll be interesting to see how this compares when the new DM is introduced.

 

[Stoopy]

On 3/14/2020 at 12:02 PM, ROTER_BART said:

It looks like being fast and/or having multiple engines is a big advantage in this scenario.

 

It'll be interesting to see how this compares when the new DM is introduced.

 

 

Very much suspect you're right.  I haven't found a way to better equalize the speed of each aircraft, I suspect adding a few turns at the start of the course might do it. I need to record a TacView track and see what's really what with each plane's speed.  But then again. they're also faster or slower in the game so it all counts.   And hey at least one of your 262's got taken out.    

 

Thanks for running it to completion and posting results!

 

Edited March 16, 2020 by =[TIA]=Stoopy

[Stoopy]

Here's something kind of interesting:  I finally got around to looking at a recorded track in Tacview and found that all aircraft speeds are pretty closely matched in the section where they fly through the firing area, with exception of the Me-262.  Speeds shown below are approximate TAS (rounded) just as they enter the firing area, as reported by Tacview :

 

P-47: 212 kts / 393 kph
P-51: 212 kts / 393 kph
P-38: 217 kts / 401 kph
FW190A-8: 211 kts / 390 kph
Spit IX: 213 kts / 394 kph
Tempest: 215 kts / 398 kph
Bf109K-4: 215 kts / 398 kph
Me262A: 270 kts / 500 kph

 

Nowhere near as much variance as I expected.  Will toy around with getting the 262 speed reduced just for the heck of it, but I consider that plane an outlier for more at least a couple reasons anyway.  Different engine type, fuel, etc..

 

[ROTER_BART]

That is interesting. I thought the P-47 would have shown up as being much slower. It's usually blown out of the sky before it even crosses the centre of the airfield.

 

The results pretty much align with what the anecdotal "gut feeling" of the community is in regards to the current DM, particularly in regards to a too weak P-47 and an OP P-51. The theory that the problem with the DM was related to all radial engines being made of glass doesn't seem to hold true in relation to the Fw though.

 

And speaking of the Fw, is there any reason for not including the Dora in the test?

 

[Stoopy]

9 hours ago, ROTER_BART said:

That is interesting. I thought the P-47 would have shown up as being much slower. It's usually blown out of the sky before it even crosses the centre of the airfield.

 

The results pretty much align with what the anecdotal "gut feeling" of the community is in regards to the current DM, particularly in regards to a too weak P-47 and an OP P-51. The theory that the problem with the DM was related to all radial engines being made of glass doesn't seem to hold true in relation to the Fw though.

 

And speaking of the Fw, is there any reason for not including the Dora in the test?

 

 

Re: the P-47 and radial engine damage, it's quite possible that it has its own numbers for the engine damage model apart from the FW-190's engine. And considering that the updated damage models that we will see soon are for the airframe damage model only and not the engine (since Jason specifically said he had engine damage turned off in the video he shared in the last dev diary), we just might continue to see a glass engine in the P-47, albeit wrapped inside a stronger airframe.  Since most of the P-47 failures in this test seem to come from critical damage that leads to failure - which I'm assuming means damage to the engine or engine-related components - we could continue to see the P-47 suffer in this test.  It's just harder to have the wings blown off by cannons, which isn't measured here.   That's my theory at least... we'll see soon.

 

I didn't include the D-9 because the design of the mission and the display was set for 8 planes (seemed a good number at the time) and since the airframes between the A-8 and D-9 are so similar it seemed more important to include the Me262.  Seeing what an outlier it is, it's no problem to swap the Dora in for the 262, I can make that change tonight and upload an updated mission.  Try as I might I could only get the 262 to slow down to 240 kts / 448 kph and not for very long either.

 

[Stoopy]

@ROTER_BART, OK swapped out the 262 for the FW190 D-9.  It cruises through the firing course at a steady 212 knots.

 

P-47: 212 kts / 393 kph
P-51: 212 kts / 393 kph
P-38: 217 kts / 401 kph
FW190A-8: 211 kts / 390 kph
Spit IX: 213 kts / 394 kph
Tempest: 215 kts / 398 kph
Bf109K-4: 215 kts / 398 kph
FW190D-9: 212 kts / 393 kph

 

Here's the new version to download, I haven't had time to let it run a full cycle so it will be interesting to see what color smoke we get:

Aircraft Survivability Comparator v1.1.zip

 

Edited March 23, 2020 by =[TIA]=Stoopy

[ROTER_BART]

Cool. I'll give it another run and post the results when I get the chance.

[Stoopy]

Hmmmmmm.... I started it, took a nap and came back just in time to watch the last & final D-9 suddenly explode in a fireball over the middle of the firing range, showering the airfield with various pieces and broken wings.  I've never seen that before in this test. From MG fire.

 

This may explain why every time I debate about taking an A-8 or a D-9 in multiplayer, I get that sudden premonition not to take the D-9, like a voice is telling me "Don't take that plane, you'll never make it back"....?

 

 

(I never make it back anyway) ?

 

Edited March 24, 2020 by =[TIA]=Stoopy

[ROTER_BART]

Test 2 with the D9 which did very well.

3 hours ago, =[TIA]=Stoopy said:

D-9 suddenly explode in a fireball over the middle of the firing range

Same thing happened to one of the Tempests in this test. I think the 20mm ammo in the right wing exploded. The weirdest one was where a Spitfire pilot lost conciousness, flew a 180 in a long arcing turn, woke up and then flew to the finish.

16 hours ago, =[TIA]=Stoopy said:

Re: the P-47 and radial engine damage, it's quite possible that it has its own numbers for the engine damage model apart from the FW-190's engine. And considering that the updated damage models that we will see soon are for the airframe damage model only and not the engine (since Jason specifically said he had engine damage turned off in the video he shared in the last dev diary), we just might continue to see a glass engine in the P-47, albeit wrapped inside a stronger airframe.  Since most of the P-47 failures in this test seem to come from critical damage that leads to failure - which I'm assuming means damage to the engine or engine-related components - we could continue to see the P-47 suffer in this test.  It's just harder to have the wings blown off by cannons, which isn't measured here. 

Agree, if the engine DM is not being remodelled in the next patch. It seems that any hit to the P-47 engine kills it very quickly. The last D9 in this test stopped streaming coolant a fair way before the finish but still made it without the engine seizing. A liquid cooled engine without coolant should die a lot quicker than an air cooled one.

 

[Stoopy]

Sounds like suspicions about a potential glass engine wrapped inside a stronger airframe DM may have been true - glad to hear that shouldn't be the case now!

 

 

[JaMz]

On 3/24/2020 at 4:29 AM, ROTER_BART said:

Test 2 with the D9 which did very well.

Same thing happened to one of the Tempests in this test. I think the 20mm ammo in the right wing exploded. 

 

 

Is that even modeled? I highly doubt it.

 

through their own admission and certainly to most combat sim players, the DM here is pretty lame. hence why its been the subject of many threads and is now being updated which comes with a lot of appreciation from me, that's for sure..

Edited March 31, 2020 by JaMz_SoLo

[ROTER_BART]

11 hours ago, JaMz_SoLo said:

Is that even modeled? I highly doubt it.

I'm not sure, but I thought I'd read somewhere on this forum that it is.

 

The latest news that the engine DMs are now being worked on to be included in the next patch is good news too. Hopefully P-47 survivability will increase accordingly.

[Stoopy]

Here are initial results with the 4.005 update.  One test does not lead to a conclusion, and this is also a very specific, limited-scope of criteria to evaluate, but this indicates improvement at least!

 

[ROTER_BART]

Massive improvement in survivability of the P-47.

 

Overall survivability seems higher as well.

 

Big +1 to this latest DM.

 

[hairogIII]

I've put this question to the brains at Quora and never got an answer. I find it hard to believe that somebody down in their basement has not done these calculations. You've done something similar here, but it involves a simulation and a rather small number of planes and ways to die. 

 

OK, here goes...

 

What was the survivability rate of the major fighter planes in WWII? In other words, if there were 100,000 sorties flown by the P-38 during the war, how many pilots survived and returned to base safely out of those 100,000 sorties

 

 

Edited October 3, 2020 by hairogIII

[Avimimus]

On 10/2/2020 at 9:32 PM, hairogIII said:

I've put this question to the brains at Quora and never got an answer. I find it hard to believe that somebody down in their basement has not done these calculations. You've done something similar here, but it involves a simulation and a rather small number of planes and ways to die. 

 

OK, here goes...

 

What was the survivability rate of the major fighter planes in WWII? In other words, if there were 100,000 sorties flown by the P-38 during the war, how many pilots survived and returned to base safely out of those 100,000 sorties

 

There were a couple of studies (I think the USAAF did one)... but a lot depended on how the aircraft were deployed. For instance, once the Typhoons were moved into the Hurribomber ground attack role the would start encountering a lot of anti-aircraft fire that the Tempests were not... and this has to do with the role they are being used for more than it does the airframes themselves.

 

What might be interesting is looking at the attrition rates faced by specific units (taking into account the aircraft used and the role/theatre/context of the combat)...! One could do a number of case-studies using specific groups which are well documented.