DM Test: P-47 vs 37mm and 20mm Flak

[unreasonable]

The objective is to compare the results of firing tests in the US Report ADA 800394 "Airplane Vulnerability and Overall Armament Effectiveness" (the USR) with the results of game testing.

 

The P-47 is used in the USR and is in the game so this was chosen as the benchmark.  The German Flak 36 and 38 were chosen to represent the 37mm and 20mm HE shells used in the USR.

 

Summary of Method

A test using an AI aircraft flown over AA guns was chosen, to save time and automate the process, eliminating human judgement and bias as far as possible.  A number of runs is required to get a representative distribution of results. I ran sufficient runs to get a total of 50 aircraft being hit at the required angle.  

 

Since we do not yet have US AAA in the game, their German equivalents were used. These are a very close match for the 20mm and 37mm weapons used in the USR. (See table later).

This post will deal with the first test series using the 37mm Flak 36. 

Details of Test 

 
The test mission is attached. From the tank Ctrl-F2 to the plane.  A single AI P-47 flies towards a waypoint, being fired at by a series of AA guns.

The guns are placed so that the direction of fire (first burst) approximates to the static angle used in the USR: frontal, 20 degrees offset to the side and below.

 

Only hits during the first burst are counted. If the plane is hit by subsequent bursts the run is stopped and the results of the final hit ignored. 
(If anyone knows how to program the AI using the ME so that they only fire one burst or turn off as the target gets close please let me know!)

 

The game is paused after a hit and effects recorded, assessing:

1) Status of plane: 
 
- Immediate kill: crashes before next hit or end of run, due to loss of control, fire, PK, or engine failure.
- Delayed kill: trailing black smoke, green fuel, difficulty of maintaining control

 

2) Type of hit: 

- Engine: black smoke, irregular running, oiled canopy, engine fire 
- Fuel: green fumes, lower fuselage fire, explosion (never seen)
- Pilot: slumped in cockpit
- Structure: everything else

 

Results of Test

 

The results are analysed as far as possible in the same format as the USR.  In particular, while the possibility of surviving more than one hit was assessed, the probability of kills was calculated for the first hit.

 

The test results show a much higher lethality than the USR results. The test probability of a hit producing a kill is over double that in the USR, when treating "Immediate Kills" as equivalent to "A Kills", and "Delayed Kills" as equivalent to "B Kills minus A kills". 

 

 

In particular, the probability of a hit producing a kill due to engine/fuel damage is much higher that in in the USR.  Distinguishing between the two can be tricky for the P-47, except in the case where either engine or tank catches fire, but this is not too important: combining the two categories delivers the same conclusion.

 

Pilot kills are very rare in each case.

 

 

The probability of a hit producing a kill due to structural damage is also higher in the tests than in the USR.

 

Overall, a game P-47 has only a 12% chance of surviving a first hit from this angle, compared to 66% in the USR.

Calculating the probability of surviving more than one hit becomes very uncertain, since the number that survived the first hit was low. 

 

Potential Problems

There are various possible objections to the method that might affect the comparison.

 

1) Not the same gun/shell. True, but the AC M9/ HE M54 used in the USR test firings is very similar in MV and shell weight and composition to the 3.7cm Flak 36 used in the test. 

 

2) Not the same angle, or closing speed. The angle is an approximation, but given that only first burst hits were counted only shots from in front and below were included.  The plane is traveling at 350kph =~ 100 m/s. This would yield an additional ~25% KE compared to a static firing, given the slowdown in shell speed compared to MV this would increase the KE a little more in the test runs. 

 

CE is unaffected by impact speed, assuming that there is no mechanism in the DM to allow this. So the total energy at impact in my test might be in the range of 10-20% higher than in static tests. I do not consider it likely that this is enough to account for the difference in results.

 

 

3) Not the same damage assessment. True, but in terms of Immediate and A kills being equivalent there is little room for error. Assessment of Delayed and B kills is harder: it is possible that some planes I assessed as Delayed kills would have been allowed to RTB in the USR, but I suspect this error is minor. Since nearly 90% of kills from this weapon are delayed Immediate or A type, we can focus on those.

 

4) If the engine was less vulnerable, the structure kills would be greater in number. True: although the probability of an I kill due to structure in the test was still higher than in the USR.

 

5) Bias in the results: the test hits may not have been randomly distributed over the visible aspect. This is quite true, and I did not record the location of hits to rebalance for this. TBH, given that almost every hit produced at least some damage decals on the engine area, it seemed pointless.

 

5) Uncertainty due to small sample sizes.  The rarer events in the test - such as p of a PK - will have wide confidence margins.  50 hits, however, is a good size sample for the overall kill rate. Note that the 90% confidence limits of the USR and game tests do not even overlap. (The USR "sample" number is a guestimate since their estimates are built up from a variety of component firing tests). 

 

 

 

6) This is not interesting as it is not about the MK108. No comment.

Implications

This is one gun vs one plane compared to one report, so early conclusions must be tentative. The difference in outcomes, however, is large. 

 

So either one, or both, of the game test and the USR are "wrong", or this could suggest either or both of the following:

 

1) The game P-47 is too weak, especially in engine (and possibly fuel) components

 

2) The game 37mm Flak 36 is too stronk

 

When I say "wrong" I simply mean unrealistic to a degree. What is considered "right" in terms of acceptable game-play is more subjective and I make no comment on that.

 

Next Steps

 

Run the same tests with the 20 mm Flak 38. 

 

Kill Probability P-47 Flak 36.zip

 

 

 

 

 

Edited December 29, 2018 by unreasonable
added charts, correct error

[ZachariasX]

You have a very particular way of playing your game.

[Talon_]

Really excellent research. Regardless of how you look at the data the R-2800 shows a clear lack of ability to withstand fire compared to its real life counterpart.

 

Indeed it's with good reason that the designers of radial engines considered the block as adequate pilot armour.

[unreasonable]

1 hour ago, ZachariasX said:

You have a very particular way of playing your game.

 

There is only so much flying around in circles I wish to do: sometimes I just want to know how things work. 

 

42 minutes ago, Talon_ said:

Really excellent research. Regardless of how you look at the data the R-2800 shows a clear lack of ability to withstand fire compared to its real life counterpart.

 

Indeed it's with good reason that the designers of radial engines considered the block as adequate pilot armour.

 

Thanks! The other factor I noticed was as I mentioned that even a shot hitting well out on the right wing, coming from the left, almost always creates some engine damage decals, and sometimes the whole smoke, fumes, kill result. Shrapnel direction is not completely predictable from flight direction, as shells can tumble before detonation, but I wonder whether the shrapnel pattern is too spherical. Have not figured out a way to test that yet. 

[JtD]

Thanks for your testing. Interesting that you arrive at a result that in game shows higher vulnerability than the reference - though I think as you have mentioned a large part of the difference might come from a not so random hit distribution.

 

About structural kills: Do you still know what exactly (or roughly) they were? Wing shot off or fuselage cut in half, or rather severed control rods or something?

 

I'm doing a bit of my own testing, trying to pinpoint the parts I feel bad about. I keep noticing that wings are remarkably tough, both structurally and in resistance to incineration (wing tanks were present), whereas the fuselage displays fairly reasonable resistance to weapon fire overall, except for structural strength. So distribution matters a lot.

Edited December 29, 2018 by JtD

[Bremspropeller]

On 12/28/2018 at 11:36 AM, unreasonable said:

I ran sufficient runs to get a total of 50 aircraft being hit at the required angle.

 

How did you do that?

 

I'd have to weigh in with JtD:

The report states an area-distribution kill chance.

If the AI manages to hit the critical spots all the time (instead of having a more accurate, random hit-distribution), then even though not matching the real world probability-data, it's still accurate in terms of damage-modelling.

[unreasonable]

49 minutes ago, JtD said:

Thanks for your testing. Interesting that you arrive at a result that in game shows higher vulnerability than the reference - though I think as you have mentioned a large part of the difference might come from a not so random hit distribution.

 

About structural kills: Do you still know what exactly (or roughly) they were? Wing shot off or fuselage cut in half, or rather severed control rods or something?

 

I'm doing a bit of my own testing, trying to pinpoint the parts I feel bad about. I keep noticing that wings are remarkably tough, both structurally and in resistance to incineration (wing tanks were present), whereas the fuselage displays fairly reasonable resistance to weapon fire overall, except for structural strength. So distribution matters a lot.

 

Actually I do not think the distribution of impacts accounts for the results at all: I did not locate each individual hit, because it became apparent while I was doing my initial trials of the test set up that the majority of hits anywhere on the plane were damaging the engine (and possibly fuel) areas leading to at least "Delay" kills.  The kills due to engine in my report were not always, or even mostly, shells that detonated on the engine area. They were mostly due to splinter damage.   This may be different from the USR treatment where a composite average was constructed out of three different sets of firing trials on engines, fuel tanks and structure.  

 

With a Normal AI gunner setting the spread of shots is fairly wide, I certainly did not notice any particular bias in the location of hits except what you would expect: the left wing gets hit more than the right, and the nose more than the tail.  Impact distribution would be comparatively easy to check separately, also to set the AI Gunners to Low which will help them spray their rounds around even more, so I may do this before moving on to the 20mm. 

 

On the structural hits: from my spreadsheet there were 6 first hit structural Immediate kills, all of which were either the outer section of the wing detaching or the whole wing coming off at the root.  On 2nd/3rd hits there was one  crash from a plane losing it's vertical stab and rudder. There was also one plane that lost control and dived into the ground, with no obvious pilot or control surface damage, so I am inclined to put that one down to loss of elevator control rods, but impossible to be sure so I did not include it in the "Noted from all hits".  

 

For structural kills the overall kill p is very close to the USR's. So if the test hits were biased towards the engine, the lethality of hits on the structure is too high compared to the USR.  

 

 

46 minutes ago, Bremspropeller said:

 

How did you do that?

 

 

By only counting hits from the first burst from each gun. If you load the mission and play it you will see that the guns open up and fire a burst from in front and below. I am not claiming the angle is exact: it cannot be for a moving target, and I used the Lapino map, for a variety of reasons, which is not completely flat,  but I am happy with the approximation. No hits from behind or directly to the side were counted. When that happened I just stopped the run and restarted.

[Operatsiya_Ivy]

Do we have any knowledge on how flak is affecting the DM and how the ballistic test was conducted in the ADA report? Did they use the aircraft armament to do the test? The shells might be close in values but that doesn't necessarily mean that they are close in performance in the game too.

[unreasonable]

9 minutes ago, Operation_Ivy said:

Do we have any knowledge on how flak is affecting the DM and how the ballistic test was conducted in the ADA report? Did they use the aircraft armament to do the test? The shells might be close in values but that doesn't necessarily mean that they are close in performance in the game too.

 

The USR gives the weapons and shells for it's tests in Appendix A. This does not include the P-47 structure firings, but it a fair bet that they used the same equipment.  All of the single shot test for 37mm were shown as using the 37mm AC M9. "AC" here standing for auto-cannon: it is a variant of the M1 AA gun used on PT boats.  

 

Table 7 in the report lists the M9 as having a muzzle velocity of 2600ft/s, (792 m/s) which is pretty similar to the Flak 36.  (Typo in my table, I will change it).Firings were done at 500 yards against a static target. The shell is the HE M54 - so a known quantity again similar to the Flak 36 HE shell.  So if the game is faithfully replicating the basic facts about the shell and MV of the Flak 36 they are a very good match. That I cannot check as I have not tried to unpack any files.

 

 

 

[unreasonable]

New Test

 

I have updated my test to narrow down the range and angle of the hits, using the Kuban map which has a completely flat area and check zones to delete AA guns at 300m cylindrical range (~350 direct los).  The test was rerun until I had 50 first hits, counting hit locations and range at impact.

 

The results in terms of p of being shot down are almost the same as the first test.  

 

Bias could be from non - random locations of hits, or it could be from the angle of the shot differing from the USR. Shots from a wider angle have a lower % chance of hitting the engine cowling or prop disc.

 

 I looked at the relative area of the engine cowling and prop disc, modifying for shots able to pass through the disc, to come up with an effective relative area for the angle of the test.  I measured that at ~22%. The USR gives the relative area of the engine in the text as 19% of the total aspect. When measuring their photo, I came up with 16%. The photo does not show prop blades clearly, so it is possible that this accounts for the difference, but we do not know how the prop area was accounted for. The USR does count prop hits as hits to the engine: one is listed in the detailed appendices.  So in comparing USR and Test it is not possible to be completely sure that we are comparing areas exactly like with like.

 

When assessing where a shot hit in the test, I counted 23% of all hits on the cowling or prop disc, 26% for first hits only.  So if there is any bias in areas hit I believe it to be very small.

 

 

Conclusions 

 

The lethality of 37mm HE shells vs the P-47 in the game is about double what the USR states for a similar target, range and angle.  This is far more than can be put down to any residual bias in the Test. 

 

I believe that part of the explanation for the difference is implicit in the text of the USR. It states that the probability of a random hit causing a kill due to a hit to the engine, is equal to the probability that a random hit would hit the engine, times the probability that a hit on the engine would cause a kill.  From that and the USR, you can calculate the implied p that a hit on the P-47 engine would cause a kill. For a B kill this is 1.00 (actually 1.02 which I put down to rounding error in the 19% engine aspect).

 

The game models this last probability: hits on the engine or prop disc almost universally cause a kill, nearly always Immediate.

 

The big difference is that hits on areas 5m or over from the cowling area can damage the engine in the game, even if the shell has passed the engine by some distance before detonation. While I can accept that this is possible, I do not see it as likely due to the fragmentation pattern. See the picture.

 

 

In the USR such occasions are ruled out. I can see that some damage to the structure apart from direct hits to the engine could cause engine failure; but in the USR these hits are registered as Structure hits.  In my test, just counting  wing and similar component loss, my results were similar to but slightly higher than the USR's structure numbers. 

 

Not knowing exactly how the DM works - ie if the fragmentation zone is spherical or another shape, where the centre of the sphere or other shape is relative to the point of detonation etc - all I can really say is that if the fragmentation pattern was more forwards of the impact the results in the test would, I believe, more closely resemble that of the USR.

 

Now on to 20mm.

 

  

Edited January 3, 2019 by unreasonable

[JtD]

Nice to see the test to be reproducible.

 

The engine sensitivity appears to compensate a lot for structural toughness/lack of effect of aerodynamic damage in my tests, though the 37mm in your test does not seem to have this much of a problem taking a wing right off.

[unreasonable]

Here we go.  P47 Flak36 Test Kuban.zip

 

21 wing hits yielded 4 wing-offs - in all cases just the outer section during Test 2, although there was one wing off at the root in Test 1 IIRC. These hits also yielded 4 kills due to prop stopped. 17 kills in all: the rest (9) put down as Delay kills due to streaming liquids.  To get away without a Delay kill the hit had to be near the wingtips.

 

If you run the test you will see that the number of rounds each gun varies: by gun and by run, anywhere from 0 to about 10. All the guns are programmed the same: there is an RNG in the AI response time to an aircraft entering their zone, I think, in addition to the normal dispersion of point of aim. 

 

Based on the USR figures I would say that they thought the p of a wing loss was lower than my test result: using the same reversal to estimate the probability that a structure hit causes a B kill you get 0.10/(1-0.19) = 0.12   for all structure hits.  (edit: being a bit lazy and ignoring fuel and pilot hits).

 

 

Edited January 3, 2019 by unreasonable
correct error

[=EXPEND=13SchwarzeHand]

On 12/28/2018 at 11:36 AM, unreasonable said:

Only hits during the first burst are counted. If the plane is hit by subsequent bursts the run is stopped and the results of the final hit ignored. 
(If anyone knows how to program the AI using the ME so that they only fire one burst or turn off as the target gets close please let me know!)

 

Just out of curiosity, how do you make sure that only one round really hit the target? Considering the Flak36 had a fire rate of 150-250 rpm and the plane is flying at rather slow speed of 350kph and sound and visual animations do not always correspond to the actual amount of hits, how do you make sure you pause after the 1st bullet strikes the plane?

[unreasonable]

1 hour ago, =EXPEND=SchwarzeDreizehn said:

 

Just out of curiosity, how do you make sure that only one round really hit the target? Considering the Flak36 had a fire rate of 150-250 rpm and the plane is flying at rather slow speed of 350kph and sound and visual animations do not always correspond to the actual amount of hits, how do you make sure you pause after the 1st bullet strikes the plane?

 

It is impossible to always pause after one and only one hit; so what I did with the 37mm case is simply to discard any hits where there appeared to be more than one impact before I was able to press pause, in which case the run was stopped  and those hits not counted.  Similarly if the second hit came after pause but before I could assess damage. It just means that you have to run more trials to get n first hits. 

 

In the 37mm case that happened relatively rarely: perhaps 1 time in 4? Same procedure as discarding hits from too oblique an angle, now solved in Test 2 by using check zones which not only restricts the angle of fire but limits the average number of shots fired.

 

 

I considered using "[" key to slow down the game once I saw tracers, but since I am not 100% sure that is all it does I did not, but trying it out does make it easier to distinguish hits and hit locations.

 

If you watch a Flak 36 firing from the ground you will see that every firing sound is matched by a recoil animation and a tracer. In SP I think that the sounds, animations and hits correspond well, (another reason why all DM testing should be done offline) so I am confident that my count was close enough.

 

This procedure would work with the 20mm  but would be enormously time consuming: the guns are getting more than one hit in about 50% of hit groups.  So I am recording [edit: I mean on paper not on a track] all of the results (still time consuming: running another ten aircraft and entering results has just taken well over an hour!)   I will still be able to look at the results from the initial hit, although my sample size will be smaller, but by recording them all I can also see the number of shots needed to get a kill. 

 

 

Edited January 5, 2019 by unreasonable

[unreasonable]

20mm HE test results now out.

 

Summary of Results.

 

The results for 20mm HE are very similar to those for 37mm: the probability of a single hit downing a P-47 is about double that estimated in the USR. The reason is also similar: a very high incidence of engine failure and fuel tank leaks in the tests.

 

The test design was similar to the 37mm test, with a few modificatons outlined below. The analysis also had to take into account the difficulty of measuring the results of single hits when multiple hits were common. See discussion for details.

 

Outline of test

 

The test flew a P-47 with 60% fuel and default munitions loadout past a line of 20mm Flak 38 guns, using attack area and check zone commands to restrict the flak firing arc to achieve as close as possible a 500 yard range and 20 degree front and below angle as described in the USR.

 

As soon as the plane was hit the game was paused and hit location(s), range to the gun and damage results noted. The game was then unpaused and the process repeated until either the plane crashed (I Kill = immediate), had a fuel leak (D Kill = delayed) or had arrived at the last waypoint. At any of these points the mission was ended.

The test was run until I had 100 planes that had been hit.

 

 

Analysis

 

The results are laid out to conform as far as possible with the USR tabulation.

 

Hits came in "groups"; these represented hits close enough together that they all happened before I could hit pause after the first hit. In order to reduce the problem of multiple hits causing compound damage, the probabilities for kills were taken from the first hit group only. Many of these contained more than one hit, but you can see from the table that the number of kills attributed to those groups was about the same per hit as for the first groups with only one hit, suggesting a high level of independence between hits. You need quite a few hits to get significant evidence of compound effects. 

 

 

 

Possible Issues and Solutions

 

1) Miscounting hits.  In an initial trial I had some difficulty in counting accurately when the plane was hit by more than two shells close together. The mission was changed to use a player plane on autopilot so that the eventlogs.txt could be used to check the hit numbers.

 

2) Angle and range variation. The mean range (and hence angle) came very close to the required range. There is some variation due to the impossibility (AFAIK) of restricting the gunners completely: they appear to have a random element in how quickly they react to a plane entering their zone. This is good for gameplay but makes testing a little harder.

 

 

3) Bias in the hit locations. The overall spread of hits was very similar to that of the 37mm test and not far off the estimated aspect, except that:

 

- In all cases the right wing took more hits than the left despite presenting a smaller aspect. This may be due to some factor in the gunners' aiming calculations
 
- In the case of first hits, the prop disc/cowling area took a higher than expected share of hits even though the mean range of the first hits was the same as the mean range of all hits. This was probably due to chance and may also reflect the difficulty of assigning hit locations for hits that arrived in the area of cowling/fuselage/wingroot. It would only be possible to be 100% sure of hit location in every case using track recording, which would have slowed the testing down to an impracticable degree.

 

Consequently the kill probabilities have been presented both with and without reweighting for this issue.

 

 

 

4) PKs - there were none in this run; that could be entirely due to chance as the hit area is very small, but there have been other threads suggesting possible issues in this area. The reweighted probabilities include the USR's estimates to correct for the test results being due to chance.

 

5) AI pilot behaviour and engine management. I set the waypoint speed low so that even if the plane was damaged the AI pilot would not be forced to strain or damage the engine. Looking in the cockpit with damaged aircraft i could see no evidence that  the AI was in any way making things worse, but I make no claim to expertise in this.

 

6) Mixing up I and D kills. The USR states that an unsealed fuel leak in a P-47, given the mission criteria, is always at least a 100% B kill, so I have followed their rules. Some D kills assigned as Fuel kills also had some engine damage and might have crashed within the USR's 5 minute deadline.

 

Discussion of the Results

 

The reason for the high lethality of the shell is their ability to damage engines and fuel tanks.  

 

Structural damage is in line with the USR. No wings or tails were shot off. Although there were some flap and aileron losses, the two crashes of planes with undamaged engines had no visible control surface damage but appeared to have lost elevator control. Structural damage is very unlikely to cause a one shot kill, but precise quantification of small probabilities is impossible with these sample sizes. 

 

If you accept the test results are real, and also that the USR's estimate give a good base case for reality, you have to decide whether the discrepancy represents a problem for the game. I think it does: leaving aside the whole issue of air-air combat, much of the player base uses the game for ground attack. It is bad enough that the gunner AI settings range up to the superhuman, but when you add in a shell lethality of double "reality", you are going to make surviving even a single ground attack mission against a defended target very difficult; surviving a ground attack career impossible. 
 
Now that we know that this is not a 37mm problem specifically, question is whether there is a general problem with engine and fuel tank or a P-47 problem.

 

Unfortunately the USR did not test the P-51 in the same way (at least we do not have a report on it) so any other test will have to be relative to the P-47 case and interpretation somewhat subjective . I was thinking of testing another radial, one of the Fw190s perhaps,  any suggestions?

 

 

Edited January 15, 2019 by unreasonable
add charts

[Venturi]

For an engine like the R-2800 to be instantly stopped, the shell would need to directly impact the engine block and cause severe structural damage to major reciprocating parts: Not something I would necessarily expect from a HE shell impacting the skin or outer engine components first. Fragments would be unlikely to penetrate the engine block. 

 

There is an ancedotal story of a P-47 from Gabreski's squadron, if I recall, of a plane returning from a combat mission over Europe. The ground crew noted afterwards that it had sustained a hit from an armor piercing 20mm shell in the engine, which had penetrated one of the cylinders. The R2800 had continued running, although the shell was bounced round inside the cylinder by the piston at ~2500rpm, the remainder of the mission: The pilot had not noticed too much wrong. 

 

Nice tests. ?

[=EXPEND=13SchwarzeHand]

9 hours ago, unreasonable said:

you accept the test results are real, and also that the USR's estimate give a good base case for reality, you have to decide whether the discrepancy represents a problem for the game.

 

If the DM is fine tuned to single shot kill Ps it must have a) a hitbox System that is detailed in a way in which it is not nearly now and b) you must find a sensible way to extrapolate your single shot kill P to more than

one shot.

Since a) is probably not achievable financially and b) not mathematically any tuning of the DM to the data you presented would yield results that would be way more off reality than what we have now.

Edited January 15, 2019 by =EXPEND=SchwarzeDreizehn

[unreasonable]

5 hours ago, =EXPEND=SchwarzeDreizehn said:

 

If the DM is fine tuned to single shot kill Ps it must have a) a hitbox System that is detailed in a way in which it is not nearly now and b) you must find a sensible way to extrapolate your single shot kill P to more than

one shot.

Since a) is probably not achievable financially and b) not mathematically any tuning of the DM to the data you presented would yield results that would be way more off reality than what we have now.

 

I disagree completely: 

 

a) The hitboxes can be unchanged.  All that needs to happen, based on these test results, is that the engine and fuel tank components (in the P-47 at least) need more "hit points" relative to the range of damage done by the shell.  If you know the distribution of the range of damage done by the shell (in HPs), setting the HPs of the components to so that the ratio of hits/kills is at a required level is easy. 

 

b) The compound effects of multiple hits happen entirely naturally. They are partly a result of the distribution of hits over the surface - being shot down by losing both ailerons, for example - or due to the cumulative damage of two or more hits damaging the same hit box.  They already happen with the current DM and would also happen after any change. It would simply take more - or fewer - hits on average to damage a particular component.

 

This is easily achieveable using the current DM hit boxes and splinter mechanism.  The developers appear to have just done something very like this with respect to the HPs of wing components, producing structure damage results that are well within the range of plausibility - at least in these tests!

[=EXPEND=13SchwarzeHand]

I disagree completley

 

simple example.

Let´s say P for a kill after the hit on a wing tip is really small in your data as it should be.

According to your calculations the second, third maybe even the tenth hit will not change the Kill P dramatically in your calculations.

In reality the second hit might be the fatal one ripping the entire wing tip off and you jump from 1% to 90% within one hit. Your assertions presume a  relatively low kill P even after the second hit.

Your tested single hit kill P is irrelevant to a simplified DM engine, and it´s not the figure I expect ti to get right. In fact I do not expect the game to get the real cumulative effect right, since calculations are complex.

I presume that the game incorporates cumulative effects by increasing damage done by each single hit, so as to arrive at a valid number wrt to the average number of hits needed. The results are exactly what you get in your testing i.e. higher one hit kill Ps than data suggests (which is fine, because shk Ps are not the target value).

Your approaching the entire DM from the wrong end so to say. With all due respect, calibrating to single hit kill P´s -especially for a model for which you do not know how it treats cumulative effects- is questionable.

Edited January 16, 2019 by =EXPEND=SchwarzeDreizehn

[unreasonable]

The test is about seeing to what extent the game results correspond to the USR's.  The USR data is given as single shot probability, so that is what I tested.  If you want to test the results of multiple hits in one location, that requires a different kind of test, which other people have been doing more or less systematically.

 

The results are clear: for these two shell/plane combinations, the game's results are nowhere near the USR's conclusions for damage to fuel tanks and engines.

 

Your assumption - unsupported by any evidence, since none of us knows in detail how the DM works - would mean that the number of hits required to reach any given level of damage for a component could only ever be right for one and only one number of hits.  Cumulative damage created by the addition of successive hits is handled naturally by the hit box - hit points mechanism.  One way to model a compound damage - where other things being equal a second hit would do more damage than the first -  would be for the record to apply a multiplier to damage as it comes in in series. This would not be particularly complex. For a fuel tank, it could just be a logic gate representing if the tank is holed or not and assessing damage accordingly, for instance to decide if a fire is started.  Perhaps the game already does this - you know no more than I do.

 

Since you have started your usual snide ad hominem comments I am putting you back on my "ignore" list, this time permanently. 

 

 

[=EXPEND=13SchwarzeHand]

32 minutes ago, unreasonable said:

Your assumption - unsupported by any evidence, since none of us knows in detail how the DM works - would mean that the number of hits required to reach any given level of damage for a component could only ever be right for one and only one number of hits.

 

No I am not saying this at all. It can be right for both.

There are is a major problem with the implication though.

 

1. We are lacking data for more than one hit (which as explained is hard to extrapolate from shk Ps)

2. As you correctly point out we do not know how exactly the DM works.

 

With these two points in mind, I would rather have the game arrive at a plausible average number and have the single hit kill p deviate than the other way around.

I am not questioning your results, I am just saying that the figure on which your test focuses does not have the relevance to be the main focal point of a DM calibration.

 

Not trying to offend you.

Edited January 16, 2019 by =EXPEND=SchwarzeDreizehn