What do YOU want tested in new plane?

[J2_Trupobaw]

First, this thread is a hell of experiment, please keep it on the rails. It is not thread to discuss what actually happens in Beta testing, or design decisions - can't and won't do that, if the discussion goes that way I'll ask Haash to end it. What happens in testing stays in testing. It's about your feedback and expectations as customers and helping us testers make you satisfied with the product :). Please keep it strictly at that. 

 

With this out of  the way, I'd like to ask for community (yours!) input on what plane aspects you believe should be tested in new machine, and what methodology to use. I have my own routine, but there is always room to expand. I can not promise that all your suggestions can or will be tested, or that anyone but me looks at them. I do promise to treat every suggestion as something that needs to be looked into. Discussions of perfecting the methodology is  most welcome, especially if you can link to IRL sources. Other discussions are likely to sink the experiment. 

Edited September 30, 2021 by J2_Trupobaw

[J99_Sizzlorr]

When I was a tester in Rise of Flight I did the following checklist for every new plane that arrived.

 

1. Check 3D model and weapon system on the ground.(Check if MGs work correctly if the levers are animated if the misfire rate is in line with the other planes and if the guns overheat correctly. Check the 3D model for errors also in external view at diffrent LoDs)

2. Check cockpit in the air. (is everything animated mixture and radiator, stick, throttle, RPM indicator, Fuel indicator, bank indicator. temperature, speed gauge, altitude gauge)

3. Check Flight model in the air with additional instruments on in the HUD. (Climbrate, top speed on the deck, turning abilities, dive speed, roll behaviour)

4. Check damage model (mostly in multiplayer with other testers and against other planes also to get more information about the relative performance and about weak spots)

5. Check weapon mods and loadouts (do they work, are they correct, what is allowed and so on)

6. Check plane description (look for typos etc.)

7. Running wild with the new plane.

[US41_Low]

This games biggest problems don’t seem to me to be a tester problem, but rather a management problem. Seems to me you testers do as good a job as you can.

 

Only suggestions I have are:

 

1) Check personal bias

 

2) Fly plane in its strong and weak regimes and see how it behaves. Is the diving plane doing well in a dive? Is the turny plane doing well in a turn? Etc 

 

Though you probably already do those things. Bugs always slip through regardless, can’t be helped.

[No.23_Starling]

41 minutes ago, J99_Sizzlorr said:

When I was a tester in Rise of Flight I did the following checklist for every new plane that arrived.

 

1. Check 3D model and weapon system on the ground.(Check if MGs work correctly if the levers are animated if the misfire rate is in line with the other planes and if the guns overheat correctly. Check the 3D model for errors also in external view at diffrent LoDs)

2. Check cockpit in the air. (is everything animated mixture and radiator, stick, throttle, RPM indicator, Fuel indicator, bank indicator. temperature, speed gauge, altitude gauge)

3. Check Flight model in the air with additional instruments on in the HUD. (Climbrate, top speed on the deck, turning abilities, dive speed, roll behaviour)

4. Check damage model (mostly in multiplayer with other testers and against other planes also to get more information about the relative performance and about weak spots)

5. Check weapon mods and loadouts (do they work, are they correct, what is allowed and so on)

6. Check plane description (look for typos etc.)

7. Running wild with the new plane.

Great stuff. Would add to that:

 

3. test energy retention in a zoom and RPM recovery vs rated. Does the type zoom back to X height after dive despite drag profile? Sustained turn with full deflection to stall could be investigated too. Prop type and pitch would likely be related and worth matching vs best available data. The pitch and RPM recovery might also impact the zoom ability.
 

8. Check vs any reliable performance data from extant sources (official armed force’s reports preferred) for airframe and stated engine variant - we might need to build a library for this purpose. Do we reach expected RPM and expected max speed at given alts?

7.a. As a final sanity check after all others completed (science and data always comes first), draw sample of testers of both new type and opponents for broad combat performance vs expectations drawn from anecdotal sources e.g. Dr1 would largely be expected to beat an SE5a in a sustained turn. Holtze’s chart from the N28 thread is helpful (let’s leave the N28 for now) on broad positions.

 

With any of the above where we NEED data either from owners of survivors e.g. Shuttleworth for the SE5a or replicas (indicative), or extant contemporary sources, then a call should be put out to the community, and data added to the library.

[No.23_Triggers]

Here's a thought...might be interesting to come up with a basic "data sheet" template for FC planes - each tester could fill it in and post it, compare results...

[[ER]Nordseefischer]

Vor 1 Stunde sagte US93_Larner:

Hier ist ein Gedanke... könnte interessant sein, eine grundlegende "Datenblatt" -Vorlage für FC-Flugzeuge zu erstellen - jeder Tester könnte sie ausfüllen und posten, Ergebnisse vergleichen ...

??

[J99_Sizzlorr]

2 hours ago, US93_Rummell said:

Great stuff. Would add to that:

 

3. test energy retention in a zoom and RPM recovery vs rated. Does the type zoom back to X height after dive despite drag profile? Sustained turn with full deflection to stall could be investigated too. Prop type and pitch would likely be related and worth matching vs best available data. The pitch and RPM recovery might also impact the zoom ability.
 

8. Check vs any reliable performance data from extant sources (official armed force’s reports preferred) for airframe and stated engine variant - we might need to build a library for this purpose. Do we reach expected RPM and expected max speed at given alts?

7.a. As a final sanity check after all others completed (science and data always comes first), draw sample of testers of both new type and opponents for broad combat performance vs expectations drawn from anecdotal sources e.g. Dr1 would largely be expected to beat an SE5a in a sustained turn. Holtze’s chart from the N28 thread is helpful (let’s leave the N28 for now) on broad positions.

 

With any of the above where we NEED data either from owners of survivors e.g. Shuttleworth for the SE5a or replicas (indicative), or extant contemporary sources, then a call should be put out to the community, and data added to the library.

The problem with these tests were that the developers were interested in testing what they have done and not in what they haven't done or should do. Reports about wrong engine variants and other sources for performance figures got mostly ignored. We were told that we are testers not researchers or game developers. 

Edited September 30, 2021 by J99_Sizzlorr

[No.23_Gaylion]

I have nothing to add to what's been said. Good advice here.

 

Good luck!

[No.23_Starling]

16 minutes ago, J99_Sizzlorr said:

The problem with these tests were that the developers were interested in testing what they have done and not in what they haven't done or should do. Reports about wrong engine variants and other sources for performance figures got mostly ignored. We were told that we are testers not researchers or game developers. 

So would a Dr1 that got out turned in a sustained circle by all Entente scouts not be counted as a bug? What are the tests looking for? Is it more things like the SE5a compass being backwards?

[NO.20_Krispy_Duck]

I think J99's methodology is pretty good. The one thing I will say is that the "final proof" should be in several MP battles that involve human players. I realize lag and server are involved there, but beyond just "how does the plane look and perform", one also has to consider how the planes perform relative to one another when human pilots are at the controls of each. A small issue in one plane flying offline in quick battle may become a huge issue in a MP game where live players are flying other planes that might have similarly small errors, but which then compound together to make one plane a death trap.  A couple of small-ish errors in performance of each plane may amplify one another, and this can really come out when playing other live pilots as opposed to offline AI. I like the idea of testing the planes in a "live environment". The other thing is that FC should have its own group of WW1-oriented testers. WW1 is a niche theater with some unique aspects, and thankfully we have some members of the community who have been flying WW1 simulators for many years, over many different game platforms. We have some of the same names since the RB3d multiplayer days.

Edited September 30, 2021 by NO.20_Krispy_Duck

[J2_Trupobaw]

Guys, please let's not speculate on qhat will be useless and why, or second guess the developers, or each other:). If a suggestion cannot be tested because it does not match testing process, it won't (like I said, can't promise that). I don't thinks devs will be happy with public discussion of actual testing process. Let's keep pouring the ideas and how they will be used apart.

 

(Great stuff so far, thank you everyone!)

Edited September 30, 2021 by J2_Trupobaw

[US41_Low]

1 hour ago, J99_Sizzlorr said:

The problem with these tests were that the developers were interested in testing what they have done and not in what they haven't done or should do. Reports about wrong engine variants and other sources for performance figures got mostly ignored. We were told that we are testers not researchers or game developers. 

Like I said, management issues. Not testing issues. Thanks for taking the time to report the inconsistencies even though they got turned down.

[Cynic_Al]

8 hours ago, J2_Trupobaw said:

what plane aspects you believe should be tested in new machine

 

I'm not sure how to interpret that statement.  Do you mean performing tests on new planes as they are released, and if so is the same methodology to be applied to existing planes?

[No.23_Triggers]

2 hours ago, NO.20_Krispy_Duck said:

The one thing I will say is that the "final proof" should be in several MP battles that involve human players. I realize lag and server are involved there, but beyond just "how does the plane look and perform", one also has to consider how the planes perform relative to one another when human pilots are at the controls of each. A small issue in one plane flying offline in quick battle may become a huge issue in a MP game where live players are flying other planes that might have similarly small errors, but which then compound together to make one plane a death trap. 

Totally in agreement with this - it might not be as 'quantifiable' as engine data, climb times, obvious and reproducible FM bugs, etc, but I think it's an integral part of the 'quality control' to see how the plane performs in PvP. If the devs then turn around and say to the testers "Fine, but we're not changing it" then C'est la Guerre - but it should be tested. Thoroughly. 

[ST_Catchov]

10 hours ago, [ER]Nordseefischer said:

"Datenblatt"

 

I do like the sound of that. Splendid idea.

[Cynic_Al]

10 hours ago, J2_Trupobaw said:

If a suggestion cannot be tested because it does not match testing process, it won't (like I said, can't promise that). I don't thinks devs will be happy with public discussion of actual testing process.

 

It seems you welcome suggestions for methodologies, but don't want to discuss those that you may have or may have been instructed to use. No problem.

 

What is not clear is the standard of testing under discussion, namely are you required to test to stipulated parameters, performance relative to a reference aircraft, or do you just have to be able to say something like: "Yeah I gave it a good workout and it seems to handle pretty much OK" ?

 

 

Edited October 1, 2021 by Cynic_Al

[Drum]

I would like each cockpit checked so the pilot's default head position is exactly centered over the seat and is the proper distance below a closed canopy, just as one would experience if siting in one with your chute on and seat properly adjusted for maximum ride height, etc.

[US103_Baer]

FC is in a unique situation. 

We've been told that essentially planes are being brought over 'as is', from RoF to FC. 

 

Does this mean we should be testing FC for it's alignment with RoF, or against specs? 

Obviously the latter, but the likelihood of any changes from RoF behaviour is pretty low. So there's going to be issues. It's not 2010 anymore and how long should we put up with an N28 that can't turn and a D12 that literally cannot dive.

So when beta-testing these planes is this acceptable behaviour?

- No, it must be reported

- However immediate fix expectation: low

 

But what about new issues that RoF didn't have? The Spad 7s never shook violently about the roll axis in RoF and the modeling is just bizarre. Its definitely a bug.

- Must be reported

- Immediate fix expectation (before release): High.

 

Despite all this the biggest issue in FC has been the DM, and testing DM requires PvP. 

So I would urge the testers to work together and ensure the devs are left in no doubt when  DM changes are going to have a major impact, the nature of that impact, and what a more acceptable expectation would be.

Tell them, whether they want to hear it or not. You have to be the community's voice in that moment. 

 

Edited October 1, 2021 by US28_Baer

[US41_Low]

9 hours ago, Drum said:

I would like each cockpit checked so the pilot's default head position is exactly centered over the seat and is the proper distance below a closed canopy, just as one would experience if siting in one with your chute on and seat properly adjusted for maximum ride height, etc.

Ok I’m glad I’m not the only OCD person that wants this detail lol

[LukeFF]

10 hours ago, Drum said:

I would like each cockpit checked so the pilot's default head position is exactly centered over the seat and is the proper distance below a closed canopy, just as one would experience if siting in one with your chute on and seat properly adjusted for maximum ride height, etc.

 

1 hour ago, US93_Low said:

Ok I’m glad I’m not the only OCD person that wants this detail lol

 

Well, good, because that's already being done. In the case of planes with off-center gunfights, the default view is also off-center, because not everyone uses some sort of head tracking device. 

 

And, besides that, it's dead simple to change the camera view if the default one isn't to your liking. 

[US41_Low]

Wait, are you being snarky? There’s no reason for that lol. I’m aware of how easy it is to adjust the default and snap views. I make good use of it. It’d be a cool detail though, and an example of being passionate about the accuracy of the sim, to make it so the default head position mimicks real life. Most of us won’t get the opportunity to get into one of these cockpits. So if the company , being the ww1 enthusiasts they are, would do the research and put in the effort it’d make for an overall improved product.

 

Or not.

[LukeFF]

46 minutes ago, US93_Low said:

Wait, are you being snarky? There’s no reason for that lol. I’m aware of how easy it is to adjust the default and snap views. I make good use of it. It’d be a cool detail though, and an example of being passionate about the accuracy of the sim, to make it so the default head position mimicks real life. Most of us won’t get the opportunity to get into one of these cockpits. So if the company , being the ww1 enthusiasts they are, would do the research and put in the effort it’d make for an overall improved product.

 

Or not.

 

I'm not being snarky at all. The default head positions already mimic where they should be in reality - with the noted exception of the off-center gunsights. I've checked and rechecked practically every single plane - both First and Second World War - and I've found no issues with the default camera views.

 

And, again, if you don't like the default head positioning, it's easy enough to change. For instance, I didn't like that the Ju 88 pilot's view is aligned with the gunsight, so I changed it to what I like and bam, it's done.

 

So, I really don't see what the issue is here.

[BraveSirRobin]

1 hour ago, US93_Low said:

Wait, are you being snarky? There’s no reason for that lol. I’m aware of how easy it is to adjust the default and snap views. I make good use of it. It’d be a cool detail though, and an example of being passionate about the accuracy of the sim, to make it so the default head position mimicks real life. Most of us won’t get the opportunity to get into one of these cockpits. So if the company , being the ww1 enthusiasts they are, would do the research and put in the effort it’d make for an overall improved product.

 

Or not.

Are you using a mod gunsight?  If so, the default head position is aligned with the default gunsight.

[Drum]

It's not that I can't center my head like I've done oodles of times already, or that it isn't dead simple to do so either.  It's simply why should I or anyone else have to?

Why not make a new programable button that switches to an offset gunsight for those non-head tracking guys.  This is just my feedback and what my expectations are as a customer after eight years of having to move my head around to center it proper.  I'm just helping you testers to help me become even more satisfied with the product, which I am 99.9%.  ?  You testers have done a fantastic job making this sim number one, IMO.  Other then that little, itty bitty .1% exception, that is. 

Edited October 1, 2021 by Drum

[NO.20_W_M_Thomson]

Maybe I'm wrong but don't they have like a snap view that you can program to a button so your head goes to the gun sights? I know I used to use that but can't remember what it was call. Pretty sure it was snap view. 

What did they do in real life, not every pilot was the same size so did they adjust the seat? add cushions? 

[US41_Low]

4 hours ago, LukeFF said:

 

I'm not being snarky at all. The default head positions already mimic where they should be in reality - with the noted exception of the off-center gunsights. I've checked and rechecked practically every single plane - both First and Second World War - and I've found no issues with the default camera views.

 

And, again, if you don't like the default head positioning, it's easy enough to change. For instance, I didn't like that the Ju 88 pilot's view is aligned with the gunsight, so I changed it to what I like and bam, it's done.

 

So, I really don't see what the issue is here.

 

There's no issue if the default position mimicks real-to-life head position. 

[LukeFF]

39 minutes ago, US93_Low said:

 

There's no issue if the default position mimicks real-to-life head position. 

 

And it already does. ?

[NO.20_Krispy_Duck]

Something like this event WW1 Aviation Heritage Trust event in Britain might have some information to add. They're not flying in a true combat environment, but some of the basic handling observations might have something to add. And maybe it would be a venue to ask questions of the pilots.

 

https://ww1aviationheritagetrust.co.uk/index.php/2021/10/15/the-boys-are-back-ww1-fighters-compared/