FC Flight Model Benchmarking

[No.23_Starling]

All, most of you will be aware that Anders' AKA @Holtzauge recently published a book looking at WW1 aircraft performance, creating models tuned by extant archival test datasets and utilizing modern aerospace techniques learned from his many years in the industry. In order to better understand how this could be used to improve the older Rise of Flight models being ported into the BoX engine, a number of us in the community including @J5_Wolff and @=IRFC=Artun spent a number of weeks testing some of the Volume 1 plane set. The results and comparison to Anders' models can be found below. In terms of the tests, we used the following conditions:

 

Test Parameters

1. All undertaken in the same map without wind

2. All tests performed multiple times (3-5) then averaged, with outlier results removed and additional tests conducted to replace erroneous readings

3. Timings made by stopwatch - some human error will be present in line with observation human response times

4. FC planes to be flown by veterans of both Central and Entente planes to ensure near-max performance of types

5. Planes should have full fuel and standard ammo weight as given in FC with no additional modifications

6. Tests taken:

 i) Sustained 360 degree turn at 1km maintaining altitude

 ii) Instantaneous turn starting at 200kph IAS, usually entering from a dive from 5km to 4km, maintaining altitude in the turn

 iii) Top IAS at sea level and 5km, auto-leveling for at least 1 minute allowing speed to settle

 iv) Climb times from 0 - 5km, timed at 1km intervals

 

Test Results and Comparison

Data can be found here if image is unreadable.

 

Key Findings

There were a couple of stand-out findings which fit with many of the other discussions and datapoints/anecdotes shared both here and in the Rise of Flight forum previously:

 

1. N28: turn performance at all altitudes is far too poor. The N28 should be a competitive dogfighter vs most in-line engine scouts

2. DVa & Diiia: turning too well, too slow, and not climbing fast enough. The Diiia in particular is 21kph too slow at sea level. Again, there have been many discussions previously about these planes not behaving like energy fighters, and forcing the SE5a/SPADs to fly exclusively in BnZ

3. Dr1: speed is too slow at sea level by 15kph. That extra speed would make a big difference to its performance on the deck, and has been suggested in data shared elsewhere

4. DXII(F) & SE5a: climbing too poorly for their engine echoing comparisons with German test data for the DXII(F)

5. Sopwith Tripe & SPAD XIII (220hp): speed also far too low on the deck. The SPAD issue is likely due to the lower compression 200hp engine being present in the game (see bug report)

 

Overall

1. Turns too fast: true at 1km and more so at 5km for most types. High alt turning is very optimistic in most cases.

2. Speeds too low: true for most types at sea level

3. Climbs too slow: true for almost all types to 5km

 

Ranking Changes

The results of these tests show that the relative performance of many planes in the set today would change, some dramatically. The N28 would go from being the worst sustained turner at 1km to mid-table. Likewise, the DXII(F) would see a substantial improvement in turn and climb:

 

 

Summary

What the data does show is that 777 Studios got a lot more right than they did wrong. The majority of types are acceptable in performance in most aspects, however there are key outliers as noted above, and some notable issues with all types in terms of high altitude maneuverability. The older FMs also make it harder to add new, better FMs, if the relative performance is awry.

Should the devs choose to come back to the FMs, a good place to start would be the 5 points listed above, plus the overall issue on high altitude turning.

 

I have also provided a link to the raw dataset; please do analyse and check my numbers, and/or post your own tests which might differ from ours, but make sure you follow our test parameters exactly (particularly keeping altitude in the turns).

 

Rummy.

[JGr2/J5_Baeumer]

Thanks for the consolidation summary @US103_Rummell.  Very helpful.

 

The camel is conspicuously absent from the list.  Is this an oversight or is it preforming within historical and data predictive modeling parameters in terms of turning ability etc?

[Holtzauge]

Outstanding initiative @US103_Rummell! The meticulous testing you and the others have done together with the simulation results gives a very solid base to stand on I think. In addition, your summary and the tables comparing the results give a good picture of where we are today. Now let’s just hope this gets the attention it deserves from the forum and the developers. 

 

 

Edited October 3, 2023 by Holtzauge
Reason for edit: To delete reply to another post because this inane forum engine STILL INSISTS on auto-merging posts that have no connection whatsoever. This needs to be FIXED!!!!!!

[=IRFC=Tunes]

27 minutes ago, J5_Baeumer said:

Thanks for the consolidation summary @US103_Rummell.  Very helpful.

 

The camel is conspicuously absent from the list.  Is this an oversight or is it preforming within historical and data predictive modeling parameters in terms of turning ability etc?

3rd from bottom!

 

[Holtzauge]

29 minutes ago, J5_Baeumer said:

Thanks for the consolidation summary @US103_Rummell.  Very helpful.

 

The camel is conspicuously absent from the list.  Is this an oversight or is it preforming within historical and data predictive modeling parameters in terms of turning ability etc?

 

No, there is no oversight and the Camel is actually quite good but it's hampered by its big fuel load compared to the Fokker Dr.I. And with a fuel load for similar endurance it outperforms the Dr.I even more. In addition, while the Dr.I is good at low altitudes, the low power of the engine at high altitude in combination with the short stubby wings leads to poor turn performance at higher altitudes. This is not easy to explain why in just a few sentences, and for those who find it hard to believe I say read the book! It's all explained there.

[No.23_Starling]

42 minutes ago, J5_Baeumer said:

Thanks for the consolidation summary @US103_Rummell.  Very helpful.

 

The camel is conspicuously absent from the list.  Is this an oversight or is it preforming within historical and data predictive modeling parameters in terms of turning ability etc?

You’ll see in the table that the FC model for the Camel isn’t bad. What would change however would be that the Dr1 would have the boost of the extra speed to chase them down, and the DVa and DIIIa would have more of a chance in an energy fight with the additional speed and climb performance. They can’t turn with it anyway so a drop in two circle performance doesn’t make much difference.

 

As Anders says, it does vary by altitude and fuel load. Up high the Camel beats the Dr1 for the reasons Anders mentions.

[ST_Catchov]

OMG the Se5a is screwed!  It seems the FM was modelled on top speed only disregarding everything else. It is a disgrace to the old girl. Mind you, I have had my suspicions for some time. 

 

Devs .... 

 

Great work Rummy. 

 

 

[No.23_Starling]

9 hours ago, ST_Catchov said:

OMG the Se5a is screwed!  It seems the FM was modelled on top speed only disregarding everything else. It is a disgrace to the old girl. Mind you, I have had my suspicions for some time. 

 

Devs .... 

 

Great work Rummy. 

 

 

Remember that this data is for the 200hp Viper (old Spad VII engine with modifications) and that the early 1918 SE5as with the Hispano 8B engines got the 220hp high compression pistons. McCudden, who fitted his own, noted that it gave her an improved climb and ceiling. Engine reliability aside, that would have been the best-performing SE of the conflict, when the gearing wasn’t buggered. We can see this in the British test data too.

[Russkly]

I have to confess that all this excellent work and that of @Holtzauge being met with a firm "Nyet" from the devs (via @LukeFF ) saddens me.

 

Whilst I take Luke's point that it was always made clear that FC would be a 'simple' port of RoF into the Il2 engine, now that people have put so much time and research into demonstrating empirically that the FMs are ahistorical in many cases, it seems such a shame that this game will remain unoptimised.

 

For me as an offline player only there are three critical elements to a combat flight sim:

 

1.0  AI (friendly & enemy)

2.0  FM

3.0  DM

 

The deficiencies of 1.0 are well documented, and apparently some work is still being done on the FC AI - I await this work, or at least information about it, with bated breath.

 

The DM has seen a significant overhaul since RoF, and the lack of complaints about it these days indicate that people are relatively content with it in its current form.

 

From the above posts and other empirical research posted on this forum and elsewhere, however, it is clear that some of the FMs (the very basis of any flight sim of course) still require further work to make them more like the actual planes themselves.

 

At the end of the day, we will have paid USD320 for the four (non-early-access) FC modules, and yet we will know that the FMs do not really represent the aircraft in the way that they could/should.

 

It's a shame, because now every time I fly one of the main culprits in terms of FM inaccuracy, I will feel that I'm playing something like a vague approximation of how that aircraft might have flown (á la War Thunder) rather than a historically accurate flight simulator.

 

For USD320 I would expect FM accuracy, and now that 1C has the empirical data available, there should be no excuse for any shortfall. I understand that this would require significant dev team input, but for USD320, I feel that FM accuracy is a must-have not a nice-to-have.

 

Unfortunately I have pre-ordered FC3, but I'm not sure I'll bother with FC4 on this basis - it's as if 1C are now treating FC as a cash cow to help fund the new project, whatever that is.

[ST_Catchov]

7 hours ago, US103_Rummell said:

Remember that this data is for the 200hp Viper

 

Compelling data it is too Rummy. And it just illustrates what all serious Se5a Viper drivers have known for years since the RoF glass engine "fix" was implemented. Good intentions from AnP for sure but, well .... you know. That said, the pre-"fix" Viper pretty much overperformed except for the glass engine issue. God, how I loved her for my own evil and selfish pleasures. But enough of that.

 

I have to say I admire your comprehensive analysis of all the FM issues. It puts everything into perspective. Lot of work there by you and the other "test" pilots along with compiling the report. Well done to all participants. And of course, the invaluable data from @Holtzauge.

 

But alas, what will become of it?

[No.23_Starling]

2 hours ago, ST_Catchov said:

 

Compelling data it is too Rummy. And it just illustrates what all serious Se5a Viper drivers have known for years since the RoF glass engine "fix" was implemented. Good intentions from AnP for sure but, well .... you know. That said, the pre-"fix" Viper pretty much overperformed except for the glass engine issue. God, how I loved her for my own evil and selfish pleasures. But enough of that.

 

I have to say I admire your comprehensive analysis of all the FM issues. It puts everything into perspective. Lot of work there by you and the other "test" pilots along with compiling the report. Well done to all participants. And of course, the invaluable data from @Holtzauge.

 

But alas, what will become of it?

 

 

 

 

This is only a sub-set of all the content. There’s some odd things going on with the Dolphin and the rest of the Albatross series, not to mention the DH2 etc etc. There is little reason for additional modelling however if there is no chance of it being used beyond general interest.

 

I would add that Anders can also model the performance changes for engine variants such as the high/over compressed versions which opens up the possibility of additional content including some not seen in RoF.

[BMA_Hellbender]

Thank you so much for compiling all this data, it confirms many of the suspicions I have held for a long time.

 

I would like to point out a few things that need to be looked at in the data:

 

- The delta for the DVII (Mercedes DIIIa) at 5km sustained turn is wrong

 

- Top speed of the Sopwith Camel ASL in FC is 190km/h, not 185km/h. This brings it closer to the correct 188.7km/h, but let’s be honest, this is only true for a Camel with a later British license-built 140hp Clerget 9B(f), not one you could find in 1917 with an early French 130hp Clerget 9B or Le Rhone 110hp.

 

If the devs are not interested in your data and/or making corrections, we should go ahead and make our own sim with it.

[RNAS10_Mitchell]

47 minutes ago, =IRFC=Hellbender said:

If the devs are not interested in your data and/or making corrections, we should go ahead and make our own sim with it.

Do it. Is kick start still an option?  I think myself and many others would support some competition for the WW1 market.

[JGr2/J5_Baeumer]

20 hours ago, US103_Rummell said:

5. Planes should have full fuel and sta

This part of the methodology seems worthy of more scrutiny as it potentially hides certain performance characteristics as well as produces data that may not be valid.  Data can show a variety of different outcomes.  All we need to do is look at the test methodology and make changes (in this case fuel load) to do so. For our purposes, it isnt at all reflective of how planes are flown in game and probably in teal life as well.

 

It would be more interesting to see a standard fuel load for say 30 minutes or even an hour.  When was the last time a camel took off with full fuel load or a DFW or Gotha as well? 

[RNAS10_Mitchell]

53 minutes ago, J5_Baeumer said:

When was the last time a camel took off with full fuel load or a DFW or Gotha as well? 

Bloody April... ? kidding,  but you have a point..

Edited October 4, 2023 by RNAS10_Mitchell

[No.23_Starling]

58 minutes ago, J5_Baeumer said:

This part of the methodology seems worthy of more scrutiny as it potentially hides certain performance characteristics as well as produces data that may not be valid.  Data can show a variety of different outcomes.  All we need to do is look at the test methodology and make changes (in this case fuel load) to do so. For our purposes, it isnt at all reflective of how planes are flown in game and probably in teal life as well.

 

It would be more interesting to see a standard fuel load for say 30 minutes or even an hour.  When was the last time a camel took off with full fuel load or a DFW or Gotha as well? 

Fair point on in-game fuel loads vs real

life. @Holtzauge writes about this in the book. It depends on whether he is willing to rerun all the simulations with the weights adjusted. He can also give a view on how much of a difference it would make to each test at say 50% fuel

load.

[Holtzauge]

4 hours ago, US103_Rummell said:

This is only a sub-set of all the content. There’s some odd things going on with the Dolphin and the rest of the Albatross series, not to mention the DH2 etc etc. There is little reason for additional modelling however if there is no chance of it being used beyond general interest.

 

I would add that Anders can also model the performance changes for engine variants such as the high/over compressed versions which opens up the possibility of additional content including some not seen in RoF.

 

Absolutely. And the only reason I'm not doing the Dolphin and the DH2 in C++ is that it seems like a waste of time right now. But if the developers change their minds and want to tweak the Dolphin and DH2 FM's I will of course model them in my simulation and support with the data they need.

 

2 hours ago, =IRFC=Hellbender said:

If the devs are not interested in your data and/or making corrections, we should go ahead and make our own sim with it.

 

I wrote a book and can produce simulation data as needed. I would be happy and supportive to any developers of WW1 sims be it Il-2 FC, DCS, WoFF or this new amazing sim you speak off. Where do I sign up for it? 

 

55 minutes ago, J5_Baeumer said:

This part of the methodology seems worthy of more scrutiny as it potentially hides certain performance characteristics as well as produces data that may not be valid.  Data can show a variety of different outcomes.  All we need to do is look at the test methodology and make changes (in this case fuel load) to do so. For our purposes, it isnt at all reflective of how planes are flown in game and probably in teal life as well.

 

It would be more interesting to see a standard fuel load for say 30 minutes or even an hour.  When was the last time a camel took off with full fuel load or a DFW or Gotha as well? 

 

Not sure I understand or agree with your statement that we have produced data that hides performance characteristics and that may not even be valid since it's unclear what you mean by that and what part of the methodology you think needs scrutiny.

 

When it comes to fuel load, the convention is to use T/O weight for all aircraft when doing performance analysis. This is not something I invented but "industry standard" so to speak. But this is only needed to get the relative performance right and to be able to tune with historical data. The models will then of course display much better performance in-game if you start with say 25% fuel only.

 

@US103_Rummell: Hope the above answers you question. Once we have the models tuned, they will behave depending on fuel load just as IRL.

[JGr2/J5_Baeumer]

You cannot use full fuel tanks for these tests for the following reasons if you want most people (and perhaps the developers) to take your data seriously. 

 

Foregoing the reality that there is every incentive for someone who has spent time and money writing, printing, and publishing a much needed book (similar to a researcher publishing their research in a peer reviewed chronicle) to defend a glaring hole in their research methodology, those reasons are thus:

 

1.  These are not commercial modern planes where we are interested in range much less modern test processes for commercial modern airplanes. So lets get real and dispense of the full fuel load argument.  

 

We are interested in relative combat performance of planes 100 years ago, in their normal operating condition (even commercial and recreational pilots here know that they dont fill their fuel tanks full on many flights.  Its based on what they need to get where they are going, given their weight which inludes fuel weight, which they try to minimize.

 

2. Your methodology doesnt allow you to even test several planes that couldnt even take off with full fuel loads (DFW and Gotha) or perform even close to how they actually perform in use (Camel).

 

3. The fact that your data claims the Camel is in the ballpark is as obvious a reason as any to show there is some critical flaw in this methodology.  To claim that a plane everyone knows is ubered and operating outside its actual envelope in terms of lift and turning ability shouts warning to anyone thinking critically about the data.

 

Finally, as a simple solution to obtain the data we actually need and that will provide the insight we need, simply recalculate using fuel loads of 30 and 60 minutes and publish both sets. They will compliment the full fuel tank methodology.  It would be  incorrect to look at alternative % fuel loads for testing unless all the planes have the same size fuel tanks.  They dont.  Fuel tank sizes are arbitrary.

 

So rerun the tests (I know you will, privately because you are intelligent enough to see how the  methodology to date using full fuel tanks has no real world application, not even in this game....and has skewed the data ....again possibly why your data is ignored by the devs).

 

Publish the data as an addendum to your book using fuel loads based on standard operating times.  I have proposed the very relevant and applicable 30 and 60 minute fuel loads.

 

Unfortunately, until this is done, the data, much needed by everyone, is accurate but invalid.  Useless to provide meaningful insight to solve the problems we need solved. 

[=IRFC=Tunes]

4 minutes ago, J5_Baeumer said:

We are interested in relative combat performance of planes 100 years ago, in their normal operating condition (even commercial and recreational pilots here know that they dont fill their fuel tanks full on many flights.  Its based on what they need to get where they are going, given their weight which inludes fuel weight, which they try to minimize.

Have to add that the British aircraft would routinely take off and fly with nearly full fuel-loads on 2-3 hour patrols. This was pretty much standard operating procedure. Keep in mind, also, that they were generally flying 10-15 miles+ behind enemy lines. If a fuel pump failed or a fuel tank began leaking, they needed reserves to get home. Meanwhile, German aircraft had much smaller fuel tanks on average, as they only needed to climb up and intercept enemy planes over their own lines. Thus, from a historical context, a dogfight between a nearly full fuel Camel and a full-fuel, but much lower relative fuel Dr1 most certainly occurred commonly. 

Low-fuel flights are generally a multiplayer fiction that we exploit, simply because who don't have to (or typically want to) patrol with realistic flight times. From a historical research point of view, I'm not sure that targeting the book figures to that specifically makes quite as much sense.

With that said, the four of us compared turn figures using equal fuel that Anders was kind enough to supply us during the collection of these figures. Whereas the Dr1 out-turns the Camel by 0.3 seconds using full fuel (in the C++ figures), the Camel handily out turns the Fokker Triplane when the fuel loads are equalized, albeit at a higher speed.

In most cases, altering the fuel loads to be equal, rather than with full tanks, doesn't really change the pecking order much. Remember that a full-fuel Camel is still about 400 kilos lighter than a dry-weight Albatros. 

[1PL-Husar-1Esk]

28 minutes ago, J5_Baeumer said:

The fact that your data claims the Camel is in the ballpark is as obvious a reason as any to show there is some critical flaw in this methodology.  To claim that a plane everyone knows is ubered and operating outside its actual envelope in terms of lift and turning ability shouts warning to anyone thinking critically about the data.

Looking thru historical data, Camel was a  uber plane in hand of skillful pilot. How otherwise it  accounted for the most kills of any WWI aircraft ?

[US103_Baer]

4 hours ago, J5_Baeumer said:

You cannot use full fuel tanks for these tests for the following reasons if you want most people (and perhaps the developers) to take your data seriously. 

 

Foregoing the reality that there is every incentive for someone who has spent time and money writing, printing, and publishing a much needed book (similar to a researcher publishing their research in a peer reviewed chronicle) to defend a glaring hole in their research methodology, those reasons are thus:

 

1.  These are not commercial modern planes where we are interested in range much less modern test processes for commercial modern airplanes. So lets get real and dispense of the full fuel load argument.  

 

We are interested in relative combat performance of planes 100 years ago, in their normal operating condition (even commercial and recreational pilots here know that they dont fill their fuel tanks full on many flights.  Its based on what they need to get where they are going, given their weight which inludes fuel weight, which they try to minimize.

 

2. Your methodology doesnt allow you to even test several planes that couldnt even take off with full fuel loads (DFW and Gotha) or perform even close to how they actually perform in use (Camel).

 

3. The fact that your data claims the Camel is in the ballpark is as obvious a reason as any to show there is some critical flaw in this methodology.  To claim that a plane everyone knows is ubered and operating outside its actual envelope in terms of lift and turning ability shouts warning to anyone thinking critically about the data.

 

Finally, as a simple solution to obtain the data we actually need and that will provide the insight we need, simply recalculate using fuel loads of 30 and 60 minutes and publish both sets. They will compliment the full fuel tank methodology.  It would be  incorrect to look at alternative % fuel loads for testing unless all the planes have the same size fuel tanks.  They dont.  Fuel tank sizes are arbitrary.

 

So rerun the tests (I know you will, privately because you are intelligent enough to see how the  methodology to date using full fuel tanks has no real world application, not even in this game....and has skewed the data ....again possibly why your data is ignored by the devs).

 

Publish the data as an addendum to your book using fuel loads based on standard operating times.  I have proposed the very relevant and applicable 30 and 60 minute fuel loads.

 

Unfortunately, until this is done, the data, much needed by everyone, is accurate but invalid.  Useless to provide meaningful insight to solve the problems we need solved. 

 

Surely the most critical element here is to ensure an apples-to-apples comparison.  Thus the testers HAVE to use exactly what the model used.  Which they did.

In his book  @Holtzauge checks and compares his model results carefully against what real life test data is available. I didn't notice any variations of concern when reading thru at the time. 

 

Lighter fuel loads will change some performance metrics sure, but the point is to establish accurate baselines at low and high altitudes, letting the data land where it does, regardless of perceptions. 

 

Also what actual aerodynamic flaw would be illuminated by 50% fuel load testing that wouldn't be highlighted by full fuel tests? I'm not an aeronautical engineer so I tend to trust their established industry approaches. 

 

I do have questions regarding the DVII with Merc DIIIa model results,  but will go back to the book and reread those sections.  Am sure Anders must've triple checked and noted why they're so poor up high.

[Holtzauge]

@J5_Baeumer: I think you have misunderstood the intentions of the tests and how to interpret them. But @US103_Baerhas already covered that very well in his post above and I really don’t have much to add on top of what he wrote since I think he did a very good job of explaining the rationale why we have done the tests the way we did and how they should be interpreted.

 

But anyway, here are some additional points and from me:

 

First of all, the analysis that @US103_Rummellposted concerns how the aircraft compare as modeled in-game compared to how they behave in my C++ analysis. So the object here is to compare the FM models under similar conditions. This in no way means that the list should be taken to represent gospel on how they compare under combat conditions since this will depend on the loadout. However, as long as you have one reliable data point to tune the FM models, you can from this with good accuracy predict what an increase or decrease in power or weight will result in. So if the in-game FM models are tuned correctly, you will following this update be able to fly in-game with whatever fuel and ammo loadout you find appropriate and have the aircraft behave much more realistically.

 

And this is the key takeaway I think one should have from the OP: There are discrepancies in the basic FM’s concerning their aerodynamic performance which need to be ironed out. Once/if this is done, then the aircraft can be flown in-game with whatever fuel load you find appropriate for the mission or you think represents “combat” conditions. This means for example that if you fill up a Camel with the same fuel load as a Dr.I, say 70 l, that the Camel will handily outperform the Dr.I since the Camel is pretty close to the Dr.I even when carrying its full fuel load of 168 l. Which is exactly what would have happened IRL.

 

Another thing: I can understand if some here are disappointed about the Albatros’ performance. In fact I was so myself when I first analyzed the results of the simulations. And in fact I write about that in the book. I always thought that the Albatros was a very good plane, mostly based on its sleek looks. However, it has physics working against it: It’s simply too heavy and underpowered. And this is not only my simulations: I have two very comprehensive tests reports by the French Stae on a couple of captured examples and they tell the same story.

[US41_Winslow]

One more thing that hasn't really been mentioned yet, in defense of the Camel.  It was always fairly competitive down low even at the end of 1918, but it was much worse higher up, especially compared to the DVII.  So, right now, when German airplanes come and try to dogfight Camels at low altitude, what do you expect to happen?  Until most of the fighting that occurs favors the strengths of the DVII, rather than the Camel, the Camel will continue to do ahistorically well.

[No.23_TaxDollarsAtWork]

On an unrelated note.

@Holtzauge

In another thread here and on the aerodrome I saw your neat charts on the Pfalz 12.

 

It got me to thinking, did you ever model fighters like the SPAD 17 with the 300 HP engine or Pfalz 14 with less rigging?

 

I know these models take lots of effort but some data on these more obscure types would be great. Just out of morbid curiosity of how they'd compare to their normal SPAD 13 and Pfalz 12 ancestors

[No.23_TaxDollarsAtWork]

On 10/4/2023 at 11:26 PM, 1PL-Husar-1Esk said:

Looking thru historical data, Camel was a  uber plane in hand of skillful pilot. How otherwise it  accounted for the most kills of any WWI aircraft ?

If this game is truly a facsimile of WW1 in its current state, it must've been that those RFC boys got on late at night to farm bots on the airstart. As that is the case for a certain other popular airframe.

 

Let's hope the real war wasn't like flugpark...

Edited October 6, 2023 by No.23_TaxDollarsAtWork

[Holtzauge]

2 hours ago, No.23_TaxDollarsAtWork said:

On an unrelated note.

@Holtzauge

In another thread here and on the aerodrome I saw your neat charts on the Pfalz 12.

 

It got me to thinking, did you ever model fighters like the SPAD 17 with the 300 HP engine or Pfalz 14 with less rigging?

 

I know these models take lots of effort but some data on these more obscure types would be great. Just out of morbid curiosity of how they'd compare to their normal SPAD 13 and Pfalz 12 ancestors

 

No, I have not done the SPAD 17 or Pfalz 14 but maybe in the future. Before the sad news that the plan is to only port over FM's from RoF to FC, I was thinking of doing the Sopwith Dolphin and DH2 as next projects but as things stand, working on WW2 FM seems like time more well spent.

[NO.20_Krispy_Duck]

On 10/5/2023 at 3:56 AM, Holtzauge said:

@J5_Baeumer: I think you have misunderstood the intentions of the tests and how to interpret them. But @US103_Baerhas already covered that very well in his post above and I really don’t have much to add on top of what he wrote since I think he did a very good job of explaining the rationale why we have done the tests the way we did and how they should be interpreted.

 

But anyway, here are some additional points and from me:

 

First of all, the analysis that @US103_Rummellposted concerns how the aircraft compare as modeled in-game compared to how they behave in my C++ analysis. So the object here is to compare the FM models under similar conditions. This in no way means that the list should be taken to represent gospel on how they compare under combat conditions since this will depend on the loadout. However, as long as you have one reliable data point to tune the FM models, you can from this with good accuracy predict what an increase or decrease in power or weight will result in. So if the in-game FM models are tuned correctly, you will following this update be able to fly in-game with whatever fuel and ammo loadout you find appropriate and have the aircraft behave much more realistically.

 

And this is the key takeaway I think one should have from the OP: There are discrepancies in the basic FM’s concerning their aerodynamic performance which need to be ironed out. Once/if this is done, then the aircraft can be flown in-game with whatever fuel load you find appropriate for the mission or you think represents “combat” conditions. This means for example that if you fill up a Camel with the same fuel load as a Dr.I, say 70 l, that the Camel will handily outperform the Dr.I since the Camel is pretty close to the Dr.I even when carrying its full fuel load of 168 l. Which is exactly what would have happened IRL.

 

Another thing: I can understand if some here are disappointed about the Albatros’ performance. In fact I was so myself when I first analyzed the results of the simulations. And in fact I write about that in the book. I always thought that the Albatros was a very good plane, mostly based on its sleek looks. However, it has physics working against it: It’s simply too heavy and underpowered. And this is not only my simulations: I have two very comprehensive tests reports by the French Stae on a couple of captured examples and they tell the same story.

 

I think you've hit on good points. What you found also matches with the German accounts finding the Alb. D.V and D.Va series disappointing and not offering much beyond the D.III they already had. Until very recently, we were seeing only the D.Va end of the series, which was lackluster by 1918 standards. The real period when the Alb series shined was late 1916 through about mid-1917 with the earlier models, and we're only just now starting to get aircraft in the game in that range. The Alb series and the v-strut Nieuports up to the 27 shared a parallel history in which they initially were dominant but then evolution hit a ceiling and became rather lackluster by the end.

[Holtzauge]

On 10/6/2023 at 8:17 PM, NO.20_Krispy_Duck said:

 

I think you've hit on good points. What you found also matches with the German accounts finding the Alb. D.V and D.Va series disappointing and not offering much beyond the D.III they already had. Until very recently, we were seeing only the D.Va end of the series, which was lackluster by 1918 standards. The real period when the Alb series shined was late 1916 through about mid-1917 with the earlier models, and we're only just now starting to get aircraft in the game in that range. The Alb series and the v-strut Nieuports up to the 27 shared a parallel history in which they initially were dominant but then evolution hit a ceiling and became rather lackluster by the end.

 

Yes, the Albatros is simply too heavy and underpowered. But so is the Fokker D.VII unless it has the BMW engine in it. Both these aircraft weight close to a ton and a Mercedes engine delivering 160-180 hp is just not good enough.  Maybe the Albatros would have regained some standing with the BMW engine in 1918, but there were never enough of them to go around even for the Fokker D.VII’s so why put one in an obsolescent Albatros when everyone wanted Fokker D.VII’s?

 

Returning to Flying Circus, my pet theory of why the Albatros turns so well in-game is that at the time the original FM's were done way back in RoF, there was only data for speed and climb to tune to. And since the in-game FM’s seem very elastic and pliable, because to me it looks like you can dial in whatever you want with levers and knobs (If I have misunderstood this and someone knows better then please let me know!).

 

So this probably gave a lot of leeway when the developers tweaked the turn performance for the original Albatros we now live with. And since the SPAD S.XIII handily beats the Albatros D.Va in speed and climb and there was no turn data to go on except that it had a very cambered wing profile, it was tempting to give the Albatros an advantage in turn so that one plane did not dominate across the board. Total supposition of course. But I would not be surprised if this was the case.

 

 

Edited October 8, 2023 by Holtzauge

[JGr2/J5_Hotlead]

Just stumbled upon this thread now. This is fantastic work! Thank you @US103_Rummell for creating it, to everyone who tested to provide the data, and (of course) to @Holtzauge for his excellent book. Really hope that the devs listen and revise the FMs. I think it could potentially happen down the road, given time. I remember the days where the DM was in really bad shape for years, and that the devs fixed it when they had time. (Still super-grateful for this, thank you devs! ?) At any rate, I think this thread is a great starting point to see where things could be improved. 

[No.23_Starling]

I see @Holtzauge and his book was mentioned in the recent YouTube video diary. I think the devs are looking to better understand the original datasets use to tune the models in the book.

 

Is any aware of exactly what they need to explain the discrepancies found by @J5_Wolff?

[ZachariasX]

57 minutes ago, US103_Rummell said:

I think the devs are looking to better understand the original datasets use to tune the models in the book.

Fingers crossed.

[ST_Catchov]

It seems pretty clear to me that 1C will not be considering any FC FM updates based on a book with "incomplete sources/references."  So mote it be. 

[No.23_Starling]

8 hours ago, ST_Catchov said:

It seems pretty clear to me that 1C will not be considering any FC FM updates based on a book with "incomplete sources/references."  So mote it be. 

@Holtzauge does give source references but doesn’t print the original data, probably because of permissions of use. He also not going to put the C++ code into the book either. This would have to be explained verbally - Han and co must realise that this.

[LukeFF]

Guys, let's please watch the comments about the team's intentions with the WWI FMs. For those that haven't watched the video, I encourage you to watch it so you can see what was actually said.

[No.23_Starling]

9 hours ago, ST_Catchov said:

I'm just relaying what the dudes on the Q&A video said. Or should I say, the translators. I want the dodgy FM's to be revised of course, but 1C clearly have no interest in doing so even when presented with compelling data. 

 

 

Now we know. 

Just to echo Luke’s comments, the video indicates a need to see the primary data sources used by @Holtzauge to train his models, and more info about the C++ builds. That wasn’t included in the book so hopefully we can link up Anders with Han and co to fill in the gaps.

[Holtzauge]

2 hours ago, US103_Rummell said:

Just to echo Luke’s comments, the video indicates a need to see the primary data sources used by @Holtzauge to train his models, and more info about the C++ builds. That wasn’t included in the book so hopefully we can link up Anders with Han and co to fill in the gaps.

 

Anyone who reads scientific papers should have grasp of how the scientific method works: A new paper contains what is new and then REFERENCES supporting material. The complete contents of the references are for understandable (to most) reasons not included. If you don't have the references and feel a need to see them, you either download them yourself or ask the author for them if you can't get hold of them yourself. If you have done neither, then you have no leg to stand on and no business criticizing the paper or in this case the book. It's as simple as that.

 

 

Edited January 20, 2024 by Holtzauge

[Gavrick]

Hello, everyone. I'll try to provide some clarity from my side.
Without reference to Anders' book - I heard that the FM of the Albatros 5 and Newport 28 should be checked long time ago. These airplanes are very old, in ROF they were made before my time, so how things were done with them back then - I simply don't know. But when the Newport-28 came out at Circus, I noted the talk of it turning.
Again - back in the day, the previous producer decided to port Circus "as is", without researching/fixing the FM (only obvious bugs were fixed). And that's why there was - and still isn't - no time for research tasks on Circus.
We have few engineers, and lots of airplanes, and lots of tasks. "Fixing the FM" of a notional Albatross or Newport is to check the sources and what's wrong with the current model (let's say there's something wrong with the aerodynamics). Then you have to find data on the new aerodynamics, put it all into the mod, check it, take the characteristics, reconfigure the ASS and AI. This time, in the most wonderful and extremely optimistic case - a week (but this is fantastic). More realistically, a couple weeks.
For example, we have P-47. I myself was aware that there was "something wrong" with the P-47 after the debate at the time of its release, but data was scarce. About a couple years ago they finally collected data on it, I've been communicating with people in private messages and through constructive and polite communication I was persuaded to check and try to reconfigure the whole thing. I've also since then started looking for time to do it. Found it literally now, we are trying to do something with P-47 (and that without guaranteeing the result - again, there is less time than I would like).
So I can say that I have taken note of the Albatross-5 and Newport-28 and put them on my list of fixes. Until we can find that considerable, as I said above, time to check and correct them, I can only say that I have taken note that there is some material and calculations that may (or may not) be useful. When/if we make the corrections - then we can discuss everything else in detail.
And in addition, I have a list of problems that I would like to fix. But I personally don't like to make empty promises, so I usually start collecting materials and communicating with people only when I am sure that I will do something with particular problem.

[No.23_Starling]

35 minutes ago, Gavrick said:

Hello, everyone. I'll try to provide some clarity from my side.
Without reference to Anders' book - I heard that the FM of the Albatros 5 and Newport 28 should be checked long time ago. These airplanes are very old, in ROF they were made before my time, so how things were done with them back then - I simply don't know. But when the Newport-28 came out at Circus, I noted the talk of it turning.
Again - back in the day, the previous producer decided to port Circus "as is", without researching/fixing the FM (only obvious bugs were fixed). And that's why there was - and still isn't - no time for research tasks on Circus.
We have few engineers, and lots of airplanes, and lots of tasks. "Fixing the FM" of a notional Albatross or Newport is to check the sources and what's wrong with the current model (let's say there's something wrong with the aerodynamics). Then you have to find data on the new aerodynamics, put it all into the mod, check it, take the characteristics, reconfigure the ASS and AI. This time, in the most wonderful and extremely optimistic case - a week (but this is fantastic). More realistically, a couple weeks.
For example, we have P-47. I myself was aware that there was "something wrong" with the P-47 after the debate at the time of its release, but data was scarce. About a couple years ago they finally collected data on it, I've been communicating with people in private messages and through constructive and polite communication I was persuaded to check and try to reconfigure the whole thing. I've also since then started looking for time to do it. Found it literally now, we are trying to do something with P-47 (and that without guaranteeing the result - again, there is less time than I would like).
So I can say that I have taken note of the Albatross-5 and Newport-28 and put them on my list of fixes. Until we can find that considerable, as I said above, time to check and correct them, I can only say that I have taken note that there is some material and calculations that may (or may not) be useful. When/if we make the corrections - then we can discuss everything else in detail.
And in addition, I have a list of problems that I would like to fix. But I personally don't like to make empty promises, so I usually start collecting materials and communicating with people only when I am sure that I will do something with particular problem.

Would any of the work ever be something you’d open up to the community? WW1 simming is a niche within a niche so it is hard to justify the effort on your side, but just look what Team Fusion have done with Cliffs of Dover or the two (yes two) developers at OBD have done and continue to do with Wings Over Flanders Fields.

 

Youve got a former military aerospace engineer who has collected primary sources and created new datasets to cover those which will never be found, e.g. any turn performance for any of these birds, who is willing to give time to tweak models with direct understanding of the various scientific elements involved. Getting models close to the truth is an art to some extent and no model is perfect, just some better than others ( @Holtzauge says as much in his book if you read the last chapter) but I doubt we will find anything better or anyone with his level of experience willing to help pro bono.

 

Youve also got a small army of passionate players who would happily spend their evenings testing and testing again - this is what this thread is all about; all-in the three of us put a few weeks of nights into collecting the FC data.

 

My fear is that nothing will happen if the decision is purely commercial. It’s totally understandable if that is the situation and would have been a fairer response to Anders in the Dev Diary video.

Edited January 22, 2024 by US103_Rummell

[Gavrick]

14 minutes ago, US103_Rummell said:

Youve also got a small army of passionate players who... ...put a few weeks of nights into collecting the FC data.

I can only regret that there is no opportunity to spend time not for formal communication behind the monitor screen, but with live communication in good company, with a bottle of good wine, old books and graphs and schemes on napkins. Alas, alas...

[Holtzauge]

1 hour ago, Gavrick said:

Hello, everyone. I'll try to provide some clarity from my side.
Without reference to Anders' book - I heard that the FM of the Albatros 5 and Newport 28 should be checked long time ago. These airplanes are very old, in ROF they were made before my time, so how things were done with them back then - I simply don't know. But when the Newport-28 came out at Circus, I noted the talk of it turning.
Again - back in the day, the previous producer decided to port Circus "as is", without researching/fixing the FM (only obvious bugs were fixed). And that's why there was - and still isn't - no time for research tasks on Circus.
We have few engineers, and lots of airplanes, and lots of tasks. "Fixing the FM" of a notional Albatross or Newport is to check the sources and what's wrong with the current model (let's say there's something wrong with the aerodynamics). Then you have to find data on the new aerodynamics, put it all into the mod, check it, take the characteristics, reconfigure the ASS and AI. This time, in the most wonderful and extremely optimistic case - a week (but this is fantastic). More realistically, a couple weeks.
For example, we have P-47. I myself was aware that there was "something wrong" with the P-47 after the debate at the time of its release, but data was scarce. About a couple years ago they finally collected data on it, I've been communicating with people in private messages and through constructive and polite communication I was persuaded to check and try to reconfigure the whole thing. I've also since then started looking for time to do it. Found it literally now, we are trying to do something with P-47 (and that without guaranteeing the result - again, there is less time than I would like).
So I can say that I have taken note of the Albatross-5 and Newport-28 and put them on my list of fixes. Until we can find that considerable, as I said above, time to check and correct them, I can only say that I have taken note that there is some material and calculations that may (or may not) be useful. When/if we make the corrections - then we can discuss everything else in detail.
And in addition, I have a list of problems that I would like to fix. But I personally don't like to make empty promises, so I usually start collecting materials and communicating with people only when I am sure that I will do something with particular problem.

 

Well in addition to the Albatros D.Va and Nieuport 28 C1, there are other WW1 FM's that come to mind as well: For example the S.E.5a seems to be underperforming in turns. Another thing that could be good to discuss is the turn performance at higher altitudes, which I think in general is a bit on the optimistic side in flying Circus for all planes. Anyway, it's getting late here so I will call it a day for now, but good to hear from you and if you have any questions about the modeling, references or other things in my book then please feel free to post these here or in the PM thread we have with Daniil and Victor where we could also establish contact via e-mail.