1PL-Husar-1Esk Posted March 19, 2024 Posted March 19, 2024 (edited) 2 hours ago, Aapje said: Yes, although here the reward function is even harder to judge, since the goal is not to perform the best, but in a realistic or fun way, depending on the goal of the human player who faces the AI. Yes it must be the way to introducing errors or just choosing less well trained AI model as preset for difficulty level. Maybe in the future foundation model without supervised learning would do the trick. Edited March 19, 2024 by 1PL-Husar-1Esk
Aapje Posted March 19, 2024 Posted March 19, 2024 2 hours ago, Araosam said: Training on event servers could do the trick, no? Not at all perfect, but with more people engaged in realistic behaviour, following orders, formations and such. Yes, assuming that those generate enough data. You could also try to have the AI weigh behavior on the event servers more heavily, so it would still learn from other servers, but would behave more like the event servers. 1
HazMatt Posted March 21, 2024 Author Posted March 21, 2024 Have any of you guys tried the offline missions with AH3? It's free to download and try if you're interested. While still limited I've noticed the drones there seem to be much more capable. They immel if they are fast, the spit-s if they are slow. If they are on the deck and out of e they flat turn inches about the trees (it appears that they are trying to get a maneuver kill) As far as I know there's only one guy left at AH3, the founder. From what I understand he coaded this stuff. Not some AI specialist or team. I'm not sure that it's as complex as it's made out to be. I think if you coded the drone to stay as close as possible to corner airspeed climbing when above, diving when below and keeping the lift vector on the enemy plane that it would make for much more exciting fights. This wouldn't require any AI, you could just code it straight up. For sure you might want to add some sort of random number generator that threw in something unexpected or maybe a function to be called when the drone needed to switch to defensive maneuvers. In my distant memory back in the day of SWTOL, BoB and other of those early flight sims, I seem to recall the drones being more challenging.
AEthelraedUnraed Posted March 22, 2024 Posted March 22, 2024 (edited) On 3/15/2024 at 8:12 PM, Aapje said: Yes, but your choice is to not output the actual human behavior (the control inputs), but a abstract simplification (the maneuvers). So then you introduce the problem of translating those maneuvers into actual control inputs. The issue is that your decision to not output the control inputs means that you inherently limit yourself to either rather unrealistic behavior, or a very complicated mapping of maneuvers to control inputs, where you effectively have to redo part of the work that you did for generating the maneuvers in the first place. And because you do not in fact output the same thing that a real human outputs, you also introduce the problem of how to score your model against human behavior. You have to then either measure the performance of the combination of the machine learning AI that outputs the maneuvers and the code that translates this to control inputs, or you have to grade the maneuvers itself somehow. The issue with the first solution is that you effectively have to build out the entire system fully first and cannot test the two parts separately, which is a very bad idea from a programming point of view. What part of your system have failed if the final control outputs are wrong? Did it output the wrong maneuvers? Is the translation wrong? A bit of both? You can't really know how to fix things if you can't judge which part is not working correctly. A system can't you can't debug and thus not fix is trash. And if you grade the maneuvers on their own, then how do you know that your grading is accurate? You are almost certainly going to have to depend on your preprocessed data that you used for training, but if you have bugs in the preprocessing that cause you to determine the maneuvers wrongly or too simplistically, then you will use the exact buggy data to grade the system. So the system will pass inspection if it outputs the same garbage that you put in. But then you actually try to fly with the model and find out that it is garbage. So then you have to fix the preprocessing, run the preprocessing again over all the data, then start over completely with the training and then fly again. Rinse and repeat. In my opinion, there is no way that outputting maneuvers can achieve the same quality as outputting control inputs. If you grade your entire system based on whether the final control inputs have the smallest loss function and thus are most similar on average to the real pilots, you are going to find structural problems where your translation layer from maneuver to actual control inputs will fail to consider the very same data that you need to determine the proper maneuver in the first place. You cannot fix these issues without reintroducing a lot of complexity that you removed, and then you still have all the complexity that you added to remove the other complexity, that you put back in! I do not believe that this is how the brain works. Brain research suggests that some decision making is in fact a post-action rationalization of subconscious behavior that gets perceived as a conscious choice. Also, I believe that there is constant mutual feedback between the mostly subconscious operation of the controls and conscious decision making. This is part of the reason why force feedback and motion platforms improve not just the sensation of flying, but also allow for better decision making. So if you implement your system with a one-directional flow, where the decision making precedes operation of the controls, you are not in fact mimicking nature. So your simile seems rather flawed. My solution can partially handle the feedback loop problem, because the data that gets fed into the system is going to include things that a human pilot would perceive a bit later. This will then both impact the decision making and the actual flying, while your model cannot do this. Yes and no. My model would indeed have to deal with more parameters and more outputs, but your choice comes at the expense of having to do way more and very complicated preprocessing, having to do more and very complicated post-processing and making it much harder to check how well your system works. And I believe that your solution means that you will forever suffer from significant quality deficiencies. I think that it is far from obvious that your choice would result in an overall system that is cheaper to make, quicker to get working and would offer superior quality with the same resources (for example, a similar load on the CPU). This debate between us seems very similar to the debate between those who attempt to create strong AI by trying to write a lot of clever code that they think is able to interpret or generate human behavior, versus the Jeremy Clarkson method (POWERRRRRRRRRRRRRRRR). So far, the Jeremy's of the world seem to be winning. And this is not just relevant to what the best approach is, but also to what hardware and software will increasingly become available to increasingly efficiently train and execute large models with lots of parameters. And in the world of large AI models, even with all the parameters I suggest, we would still be very, very far from truly large AI models. So even a relatively modest NPU could do wonders. Only if you handwave away the fact that you have simply excised a lot of that complexity from your proposal and have never actually explained how you plan to make that work. Of course, if you pretend that you can do some actually very complicated parts perfectly and very easily, you can argue that your solution would be superior in theory. But your theory is flawed, as it often is. Einstein also noticed how often this is the case when he said: "In theory, theory and practice are the same. In practice, they are not." How do you plan to make your translation of the maneuvres to the actual control inputs take the environment into account? The status and FM of your own plane? Of the other planes? How will you model the impact of stress on how well the maneuvres are executed? How will you model the impact of being blinded by the sun? How will you model losses of spatial aweness, loss of sight, misjudgements of the behavior of the opponent, etc that cause the maneuvers to be executed differently? All of this stuff is already in IL-2 and is simulated for AI planes. So why would this suddenly be overly complicated if we make the AI do it? And yet they put a trillion parameters into ChatGPT 4 and it works out. A combat flight AI would be way, way simpler and you could still feed it with quite a lot of data. I still don't agree at all with much of what you've said, and almost all of the concerns I brought up still stand as you tend to respond to things I never said, but I cannot afford to keep wasting my time on writing whole essays like I've done before. So this is the last thing I'm going to say on the topic of a "NN implementation of IL2's AI." I find it ironic that it's you who quotes Einstein, since that quote perfectly aligns with my own argument. Theoretically, your method is very good. If we had infinitely much training data, an infinite amount of time, if SGD would always find the global optimum, if there were no hardware limitations on the complexity of our inference network, your method would objectively be the best possible method there is. I'm not disputing that. But as Einstein said, "In theory, theory and practice are the same. In practice, they are not." In practice, we have a limited amount of training data. We don't have a server park at our disposal. SGD (or AdaGrad or whatever you use) isn't very good. Gradients get smaller and smaller. The end result needs to be run on a home PC while simultaneously running a game. In cases like this, adding complexity is only rarely the answer. Granted, a certain amount of complexity is always needed, but there equally always comes a point beyond where having your net do more, or adding additional parameters, gives less good results. So while added complexity may or may not improve the "human-ness" of the behaviour, "human-like behaviour" is just one of the requirements of the total system. And each of those other requirements get ever harder to achieve if you add complexity. Now you may agree with me or not - I don't care. All I can say is: go work with Neural Nets and complex tasks for a couple of years, and see if you've still got the same opinions after that. (As an aside, I just finished a project which had a state-of-the-art network by Meta (the guys with the money) as one of the essential constituent parts - it was the absolute best network I could find for that task - and I spent about half of my time devising workarounds to deal with the horrible results this network kept outputting. Even though it was a task the average six-year old child would've been able to do just fine. Neural Nets are great, but for many tasks they are nowhere even near the level of a human.) 2 hours ago, HazMatt said: Have any of you guys tried the offline missions with AH3? It's free to download and try if you're interested. While still limited I've noticed the drones there seem to be much more capable. They immel if they are fast, the spit-s if they are slow. If they are on the deck and out of e they flat turn inches about the trees (it appears that they are trying to get a maneuver kill) As far as I know there's only one guy left at AH3, the founder. From what I understand he coaded this stuff. Not some AI specialist or team. I'm not sure that it's as complex as it's made out to be. I think if you coded the drone to stay as close as possible to corner airspeed climbing when above, diving when below and keeping the lift vector on the enemy plane that it would make for much more exciting fights. This wouldn't require any AI, you could just code it straight up. For sure you might want to add some sort of random number generator that threw in something unexpected or maybe a function to be called when the drone needed to switch to defensive maneuvers. In my distant memory back in the day of SWTOL, BoB and other of those early flight sims, I seem to recall the drones being more challenging. What you describe is AI as well; an AI doesn't necessarily have to be a Neural Network I do fully agree with the rest though. Although I've never played any other combat flight sim than IL2 1946 and BoX, I've heard positive things about the AI in several sims. It is definitely possible to write a good and realistic AI without needing to train a neural network - in fact, I'm arguing that it is much easier that way. One final remark about the "more challenging" part; many sims (and games in general) make the AI harder by letting them cheat. Edited March 22, 2024 by AEthelraedUnraed 1
Aapje Posted March 22, 2024 Posted March 22, 2024 @HazMatt I never played that game, but it is indeed quite plausible that 1GCS could do a way better job than they do now, with a procedural approach. @AEthelraedUnraed Well, I never argued that it definitely would generate good enough results in a realistic implementation. I would like for a game company to try, although it is questionable whether 1GCS can afford to, or even find the right talent, or be able to self-study to a high enough level. Ultimately, you learn the most by actually trying something, just like 1GCS had to try to make clickable cockpits before they decided that it wasn't worth the effort. Quote One final remark about the "more challenging" part; many sims (and games in general) make the AI harder by letting them cheat. Yes, but this is exactly what a procedural approach pushes you towards, since it is very hard to generate a spread of realistic quality differences using that approach. But even more so, with such an AI, the AI will typically have significant weaknesses like being predictable, so it never becomes truly good. Then cheating is a way to reach higher difficulty levels.
HazMatt Posted March 22, 2024 Author Posted March 22, 2024 I agree about the cheating. I saw some of that in the old sims back in the day. Drone/AI planes would pull high g turns and never lose any speed for example. What I was saying is to have the drones/AI fly within the flight model of the given plane but just keep at the corner speed. I don't think it would be too complicated to use just basic tables to determine what a plane would do as a drone vs the plane it was fighting. For example you could have a table that had the climb rates, speed and turn rates of all planes in could be used a reference for the done/AI to determine how to fight. Based on this chart the drone/ai would determine that it was a better turning plane for example and would fly lift vector on maneuvers vs the plane that wasn't as good a turner or if it determined it was a better climbing plane it would fight more in the the vertical. I think that the stuff you guys are talking about would be nice to implement, however I don't think that it would be required to make the drones/ai more competitive. I fly mostly online on the small servers with the air starts as I enjoy the ACM. The problem for me is sometimes there's nobody online so I go offline and do the quick start missions. 1vs1 even on expert there is no challenge unless I've flying a plane that doesn't climb well against one that does and then I just extend and drag them down and still win. Due to this I usually put up multiple enemy planes but if I put them on experts it's like they are a bunch of snipers. It's interesting that from what I've read in real life that the most experienced pilots didn't open fire at long range, however when you set the drones/ai to expert that seems to be exactly what they do. I sometimes wonder if it's that the programmers don't understand ACM well more then them lacking the ability to translate the moves into code.
1CGS LukeFF Posted March 22, 2024 1CGS Posted March 22, 2024 Guys, if you've not seen it yet, Albert and Han go into the details of why the AI is the way it is and what their future plans are for it, in today's dev blog entry. 2
HazMatt Posted March 22, 2024 Author Posted March 22, 2024 1 hour ago, LukeFF said: Guys, if you've not seen it yet, Albert and Han go into the details of why the AI is the way it is and what their future plans are for it, in today's dev blog entry. Where do I find this blog?
Aapje Posted March 22, 2024 Posted March 22, 2024 1 hour ago, HazMatt said: Where do I find this blog?
atomsic Posted May 25 Posted May 25 AI Aircraft Unstable Behavior – Rapid Wing and Control Surface Oscillations Hello, I've encountered a recurring issue with AI-controlled aircraft in IL-2 Sturmovik: Great Battles. While flying missions with aircraft such as the Focke-Wulf 190 or P-51, I’ve noticed that AI planes often exhibit strange behavior during maneuvering. Specifically, the AI frequently starts shaking the wings rapidly left and right, and the same happens with the elevator and ailerons – they begin oscillating quickly either side to side or up and down. It looks as if the aircraft is constantly trying to stabilize but overcorrects repeatedly, resulting in a very unnatural and erratic movement pattern. This behavior is immersion-breaking and can occur even when the aircraft is not significantly damaged. I've seen it in both scripted missions and in quick missions. Has anyone else observed this? Is this a known issue? I’d appreciate if the devs could take a look into this, as it affects the realism of the AI flight model quite a bit. Thank you!
Recommended Posts