How could the devs apply machine learning to improving AI?

[JaffaCake]

from the way you put your statements it seems you are   thinking from a research point of view alone. In  real   world scenarios, specially in small companies, if you can do it in another way that is cheaper it IS the correct way!

 

Less effort with good enough result always wins when you are talking about  companies with small budgets.

 

While you seems to know a lot about IA,  I would dare to ask, since you  toned your answers with a  subtle  anger towards other people answers,.. how many  products have you worked on  since their theoretical inception  up to market  delivery?  You  sound  a lot like academic person.   Safety  always  wins on the small company scenarios. If you have a way that works and is cheap  and you can do a  small increment design   or you can go  full in a  cutting edge  way with revolutionary techniques... you never ever chose the second option (unless your company  can afford "throwaway money projects". Another sim already failed hard because it aimed too high in recent past...

 

 

I profoundly apologise if I came across as aggressive. Maybe frustrated towards the end would be a better word to use though.

 

I have worked both in industry an academia and realise the points you raise very clearly, which is why my earlier estimation basically said that there is no chance that IL2 could benefit from full-blown AI project due to cost and complexity. It is also why I said that IL2 could open up API to research groups to use the sim for ML development - it would certainly bring some interest with time and could result in workable agents that could later be integrated into the game. Hell, even I'd give it a shot in my spare time if I had the access.

 

From the commercial point of view the only area of ML that is viable is supervised learning. 

 

This is, however, not to say that ML can be dismissed as non-commercial applications only. AlphaGo (reinforcement learning) was a research project made by a commercial company. There are now many other commercial implementations of AlphaGo made by other companies. Beyond large entities I am also aware of several ML-related developments made by smaller companies, they are usually results of successful PhD projects in the area.

Edited July 18, 2017 by JaffaCake

[Gambit21]

Likely. On the upside though, you and I will likely be long gone before that happens.

It's happening a lot faster than you think.

[=gRiJ=Roman-]

I wonder if the AI could be taught to make the defensive circles. It could be cool to simulate that real life strategy.

[AustinP]

@JaffaCake: very interesting post.

 

 

More edit : The only serious hope I see for a flight sim to gain any traction with ML is to release API for the researches in the field to be able to experiment and develop algorithms. For example, GTA V is already used to train agents for self-driving cars : https://github.com/ai-tor/DeepGTAV There are few air sims available that have the FM fidelity of IL2 in the research field though.

 

As a developper with a bit of professional experience in ML and passionate about the domain, I also think that he only way to move forward would be to get access to APIs to IL-2.

You mention the example of GTA5. There is also the truck (!) sim "Euro truck simulator 2" and the https://github.com/marshq/europilot project for training an agent to drive the vehicles. I've downloaded it and plan to play a bit with it in the following weeks...

Edited November 6, 2017 by AustinP

[PatrickAWlson]

I would see the machine learning as more of a lab exercise than something that is distributed with the software.  Otherwise each distribution would have only a single source of input - that player.  Set up a server and let people play against the AI, then translate the learned actions into a data set that is distributed with the software.

 

It would require a total and complete rewrite of the AI.  The whole thing would have to be rewritten to set up weighted responses to circumstances.  That means you would have to define all of the circumstances that the AI should consider.  Then do the weighting.  Test the AI responses.  Rinse, wash, repeat.  Not an easy task as there are a huge number of scenarios.  Vastly more than,say, chess.

 

If you did that (maybe or maybe not possible in this decade) then you would have the next problem.  I made my AI really, really good.  Now how do I make it really, really good sometimes but really, really bad at others.  That would involve pruning the decision tree based on AI capability.

 

It would be a most awesome development and one that I would love to work on for what is left of my career.  Too bad nasty things like bills prevent it.

[AustinP]

I would see the machine learning as more of a lab exercise than something that is distributed with the software.  Otherwise each distribution would have only a single source of input - that player.  Set up a server and let people play against the AI, then translate the learned actions into a data set that is distributed with the software.

On the contrary, I think the machine learning could be the way the game developper "develops" the AI of fighters. The dev would write and tune the various learning algorithms, and then let the sofware trains itself. Maybe the dev could provide some initial bootstraping info, for example recordings of player-to-player dogfights. That's is the way that Alphago learned to play Go after all. Then, after this learning phase, the resulting decision tree would be distributed with the software.

 

That means you would have to define all of the circumstances that the AI should consider.  Then do the weighting. 

I disagree. With Reinforcement Learning, the 'circumstances' and related choices of actions, the 'weighting' between multiple opportunities would be learnt by the machine. You can check the video of 6 virtual agents playing Doom at 

 

If you did that (maybe or maybe not possible in this decade) then you would have the next problem.  I made my AI really, really good.  Now how do I make it really, really good sometimes but really, really bad at others.  That would involve pruning the decision tree based on AI capability.

I wouldn't worry too much about this aspect. Indeed you could prune and/or alter the decision points of the decision tree to dumb down the AI once you have a good one. I think this would be quite easy compared to the task of succeeding in developping a great AI in the first place 

 

But you're right: it would be a huge effort to rewrite an existing code base with this approach. What I have in mind is new sims/games.

Edited November 10, 2017 by AustinP