Dynamic campaign generator - anyone up for a design discussion?

[Hylo]

There have been a number of discussions about the creation of a dynamic campaign generator for this game (e.g. here)

 

Personally, my interest in the game hinges on it - I just cannot get excited about BoS and BoM without the prospect of this kind of gameplay.

 

If anybody is interested I'd like to have a serious discussion about designing such a tool.

 

Reading what others have already posted and given my general experience of watching these kinds of projects and to some degree from working on them, a number of things are becoming very apparent:

1. Based on what we've heard from the game's dev team, this tool is something that will never appear unless a community effort produces it. Also, hanging that responsibility onto one person at a time is unrealistic. Nobody has an inexhaustible well of free time, interest and motivation.
2. Development has to happen out in the open. The Il-2 series has a history of long-running projects where the source stays under wraps. This establishes a situation where one person becomes very tired of supporting a project which no longer holds their interest - see Lowengrin's DCG and FBDj.
3. Development really should be modular. There are a lot of different components involved in solving this problem. There are also a number of community members who could really be considered experts in areas such as the game's log file and mission file structure and who might just be willing to work on such a project. This helps with item 1 as it's usually more motivating to work on problems where you understand the domain well.

To set out my position:

• I think a JVM language would be most appropriate as the community already has a number of successful Java projects.
• I find it very difficult to work in imperative languages now. My preference would be Scala. I've seen the benefits of using this language in production for myself.
• I have no problem at all with helping others to become acquainted with that language and paradigm and neither do I have any problem with helping out those with a relatively low level of experience of programming in general.

Is anybody interested in working on such a project (or even just contributing to the discussion)?

Edited August 2, 2015 by TheGrunch

[Ace_Pilto]

I'm interested, only know a bit of C# though, I don't know that it would be much use but if there is a resource like w3schools or something for java based languages then it wouldn't be too difficult to pick up enough syntax to get something going. Someone who is fluent in the mission editor would be a major asset, especially if they have the time/inclination to teach others. 

 

Anyway, my study load is quite light for the next 2 months so I have time and it has been on my mind, I just need somewhere to get started since I'm a bit lost on my own with this kind oif thing.

[Sim]

Here is my input on this,

 

I think from a design point of view - we are talking about two separate modules here. One is a mission generator and another one is a campaign manager (not a career manager however).

 

Mission generator

 

The mission generator is responsible for creating historically-plausible missions and does not need to have any UI at all. It needs to be configurable (over command-line for example) and should be totally separate project from campaign manager. It's also the place where new theaters and battles are implemented (future BoM for example).

 

It does not need to use the same programming language as a campaign manager. All interaction between the two parts can be done over simple command-line or other inter-process communication methods (IPC).

 

I am actually working on this part. I have made enormous progress on this, including some notable breakthroughs (for example, by being able to generate .msnbin files directly by reverse engineering the format). I plan to release the first version when summer/autumn maps are available.

 

Campaign manager

 

Campaign manager on the other hand is a UI tool for story telling. This is where you choose a battle and create your pilot. You can advance the time and fly missions that are assigned to you, get awards/promotions and interact with other squadron pilots/mechanics/etc.

 

This is also the place to tell the story not only related to your squadron, but also of the battle. While the mission generator generates missions that include day-by-day front lines - it does not really tell you what is happening on the ground. This is the place to tell that story.

 

Regarding what language this should be coded - I personally think that native GUI framework would be overkill. If it was my choice - I would go with Electron (where you can use HTML/CSS/JavaScript to code the UI as a single-page web application).

 

---

 

You have also mentioned "dynamic" campaign generator. What do you mean by dynamic exactly? I don't think you should be able to influence situation on the ground, by influencing the changes to front lines or even the entire outcome of the battle for example. However what I would love to see is persistence. For example, if your wingman was shot at X/Y location - you could still fly later a mission and find a wreck on the same spot. This is all possible with BoS.

[Sim]

Two more points:

 

1. If you have historical squadrons - your campaign may be too short. For example, the P-40 units of the VVS were operational for around two weeks before they were withdrawn due to losses. Obviously, you could handle this in the campaign manager with a side story. For example, giving the player a choice to join another unit still operational in the area due to his "connections" or something.

 

2. A relatively simple thing the developer should do is allow launching BoS with a mission file set over command line. This is necessary so you could click a button on the campaign manager and get in the mission directly, without giving the player instructions to go to Mission screen, look for mission named "XYZ" etc. This feature however is in the 1C game studios hands..

[JG4_dingsda]

 

 

2. A relatively simple thing the developer should do is allow launching BoS with a mission file set over command line. This is necessary so you could click a button on the campaign manager and get in the mission directly, without giving the player instructions to go to Mission screen, look for mission named "XYZ" etc. This feature however is in the 1C game studios hands..

 

Yes, I suggested that one to Jason already and I consider it quite important.

[Hylo]

Thanks for the input guys. I'm glad to see we're pretty much on the same page. Sim, I'm not familiar with the RoF mission format in much more than a shallow way. It looks simple and easy to parse, a bit like YAML or HOCON.

Question: can the game not load a mission without msnbin? Is the only usual way to generate the binary format saving it in the editor? Presumably this is why you were forced to reverse engineer it instead of just using the markup?

 

What I envisaged was:

 

- A campaign state engine that's essentially a log-watcher and parser - this would manage the persistence that you describe

- A mission generator that generates missions based upon the campaign state when the campaign engine requests one

 

These two parts could easily run as part of the same process, probably as nothing more than a tray icon or something like that. If anybody has any strong feelings on the topic it could easily be a command line tool instead.

 

Out of these two components there are a couple of pieces that everybody working on a tool has implemented individually so far:

 

- Log file parser

- Mission file parser

- Mission file generator

 

I think it would make sense for these to be written as libraries so they could be reused. I'm particularly interested in the idea of a decent, composable API for mission construction (take a look at Scalatags).

[Sim]

Question: can the game not load a mission without msnbin? Is the only usual way to generate the binary format saving it in the editor? Presumably this is why you were forced to reverse engineer it instead of just using the markup?

 

The game can load missions from text .Mission files if .msnbin is not present - but it's very slow compared to binary format (especially if you include all static objects in the world, not just around the player or action zone). I believe you also need .msnbin files if you intend to generate multiplayer (coop) missions in the future - those are 10x smaller and get transferred over the network. AFAIK the only way to generate .msnbin files right now is with ME editor.

 

 

- A campaign state engine that's essentially a log-watcher and parser - this would manage the persistence that you describe

- A mission generator that generates missions based upon the campaign state when the campaign engine requests one

 

From a technical point of view - yes, I agree. But don't forget all the content - detailed airfields with taxi routes, front lines, historical unit research/placement. I think it's easy to look at just the code parts and forget the content. The players will expect something more than the current 1CG campaign generator can provide.

 

 

I think it would make sense for these to be written as libraries so they could be reused. I'm particularly interested in the idea of a decent, composable API for mission construction (take a look at Scalatags).

 

I believe coconut is working on similar API concept with his SturmovikMission project. Although you can see the problem here. You are pro Scala, coconut is probably pro F# and I personally prefer modern JavaScript.

[Ace_Pilto]

I think it would be best to have a large variety of pre-made dynamic missions that meet the various criteria rather than expecting a campaign generator to generate a unique mission each time. It would mean making a lot of dynamic missions but that would come in time as they are made. 

 

So missions that fall into categories like 

 

Recon, busy front activity, USSR defensive posture, good weather.

Recon, busy front activity, USSR defensive posture, fair weather.

Recon, busy front activity, USSR defensive posture, cloudy weather.

Recon, busy front activity, USSR defensive posture, rough weather.

Recon, busy front activity, USSR defensive posture, bad weather.

 

And so on with the variables being altered to accommodate other situations.

 

eg: Escort, busy front activity, German offensive posture, good weather. etc etc etc

 

It would be easier to integrate such a campaign within the GUI of BoS itself if that ever became an option. The main advantages that I can see are that:

 

1) Each mission would be dynamic in itself, so the player has a wider number of choices to make depending on what happens.

 

2) It would be possible to create a large variety of missions yet the mission builder would be able to work from a template and only have to make slight or moderate changes to the mission type to create fill out the various mission presets.

 

3) Presets would open up the possibility of non co-op and adversarial online play .

 

Of course it would be best if the campaign handler used the logs from the outcome to influence what came next by means of altering a master database for the campaign so that there was some continuity. (eg, player flight loses all aircraft so that squadron must be under strength until resupplied and the probability of a mission escorting resupply assets is higher than normal.

 

If we're talking HTML&CSS + JS for the UI then this could also be hosted on a server and client updates handled without the risk of user error, as well as leaderboards and organising co-op or adversarial online play. (And I would be able to make a more useful contribution since I can actually do stuff with these languages )

[Hylo]

From a technical point of view - yes, I agree. But don't forget all the content - detailed airfields with taxi routes, front lines, historical unit research/placement. I think it's easy to look at just the code parts and forget the content. The players will expect something more than the current 1CG campaign generator can provide.

I usually prefer to consider a minimum working set of functionality first.  One other point, I think the principle here is as Jason stated in the discussion threads about this; these aspects are essentially configuration. Our community is usually very happy to create content, especially if such configuration is easily accessible and well documented with some kind of example data set. For me this discussion is not so much about whether it's possible to do these things (because it is), but more a way to do them that is accessible and that would get the maximum number of people involved.

 

I believe coconut is working on similar API concept with his SturmovikMission project. Although you can see the problem here. You are pro Scala, coconut is probably pro F# and I personally prefer modern JavaScript.

Yes, SturmovikMission is an amazing piece of work. In any case, language is absolutely part of the discussion. I'm happy to work in other languages, I just prefer functional, statically-typed languages. I use JavaScript pretty frequently and I could do with getting better acquainted with it. F# is very interesting to me; it's basically Haskell.NET so the principles are very similar to my preferences but the syntax and the tooling are very different to anything I've worked with before. I don't think it's a very natural companion to a JS UI, but I could be wrong.

 

I think it would be best to have a large variety of pre-made dynamic missions that meet the various criteria rather than expecting a campaign generator to generate a unique mission each time. It would mean making a lot of dynamic missions but that would come in time as they are made. 

 

...

 

Of course it would be best if the campaign handler used the logs from the outcome to influence what came next by means of altering a master database for the campaign so that there was some continuity. (eg, player flight loses all aircraft so that squadron must be under strength until resupplied and the probability of a mission escorting resupply assets is higher than normal.

On a very high level this is essentially what all of the existing dynamic campaign generators for other sims do; they have a set of template mission types and a master mission and modify that mission programmatically dependent upon the campaign state. Then they feed the mission results back into the campaign state before they generate the next mission.

 

If we're talking HTML&CSS + JS for the UI then this could also be hosted on a server and client updates handled without the risk of user error, as well as leaderboards and organising co-op or adversarial online play. (And I would be able to make a more useful contribution since I can actually do stuff with these languages )

I would be very surprised if anybody suggested any other option for a UI.

Edited August 3, 2015 by TheGrunch

[coconut]

Development has to happen out in the open.

I agree 100% with that.

 

I never understood why people distributed their work for free and at the same time kept the source for themselves. Maybe it's a cultural thing. Nowadays, with github and the likes, sharing code and collaborating is just a few clicks away.

 

 

 

Yes, SturmovikMission is an amazing piece of work. In any case, language is absolutely part of the discussion

 

Thanks! SturmovikMission has three layers (mission sample analyser, parser, type provider), and I don't think it would be hard to extend the first layer to generate a parser in any language, e.g. Java, Scala, C, C++... The first layer parses a sample mission file and generates the types for all the MCUs (timers, commands, units...). From the types, it's pretty easy to generate the source code of a parser for those types.

 

Alternatively, one can build a command-line utility in F# around SturmovikMission, and call that from the software that keeps track of the campaign state.

 

I can take care of doing those things if we can agree on an API.

Edited August 4, 2015 by coconut

[Hylo]

Thanks! SturmovikMission has three layers (mission sample analyser, parser, type provider), and I don't think it would be hard to extend the first layer to generate a parser in any language, e.g. Java, Scala, C, C++... The first layer parses a sample mission file and generates the types for all the MCUs (timers, commands, units...). From the types, it's pretty easy to generate the source code of a parser for those types.

 

Alternatively, one can build a command-line utility in F# around SturmovikMission, and call that from the software that keeps track of the campaign state.

 

I can take care of doing those things if we can agree on an API.

That sounds good; can I ask you a bit more about SturmovikMission (and probably by extension F#)?

If I was starting from scratch to parse the .Mission files I would probably just define a set of types for my AST based upon the primitives of the mission editor and use some parser combinator library (e.g. parboiled2) to write the parser by hand, as a really cut down example it could probably look something like:

 

where:

~ = sequences rules and pushes their resulting values onto a stack

~> = pops values off the stack, into a constructor and then back onto the stack

 

def mission = rule { 

  versionString ~ options ~ zeroOrMore(missionObject) ~ endOfFileString ~ EOI

}

 

def missionObject = rule { 

  group |

  block |

  plane |

  mcu

}

 

def group = rule {

  whitespace ~ 

  "Group" ~ '{' ~

    name ~ index ~ description ~ zeroOrMore(block) ~

  '}' ~> Group ~ 

  whitespace

}

 

... with each of those lowercase identifiers replaced with further subrules until you get to the text level like

 

def name = rule { capture(oneOrMore(CharPredicate.alphaNum)) ~> Name }

 

Then I could run my parser over the input and receive some Mission object like (using e.g. RatasAndStukas as an example):

 

MissionParser.run(missionFile)

=>

Mission(

 1.0,

  Options(

    ...

  ).

  List(

    Group(

      "lay_vil_lin_01",

      2,

      "",

      List(

        Block(

          "Block",

          3,

          0,

          Position(5789.080, 85.551, 43725.631),

          Orientation(0.00, 10.49, 0.00),

          ...

        ),

        Block(

          "Block",

          5,

          0,

          ...

        )

     )

 

etc. etc.

 

If I wanted to produce a mission from scratch or from bits of missions I could instantiate my AST types myself or manipulate the members of parsed mission files.

 

I can see this is essentially what SturmovikMission does but I can also see that the approach is very different; I'm not at all fluent enough with F# or F# idioms like type providers to the extent that I can understand the code in DataProvider so it's difficult for me to understand the architecture at the moment, however the immediate advantage that I can see is that this dynamic member generation provides access to mission properties by named identifiers, i.e. T.m2.``Approach beacon`` from your example, while keeping some degree of type safety.

What is the function of the sample file that's given to the analyser? Do you make a pass over this file in order to enumerate all of the possible types? Does this provide some degree of robustness if the mission format changes?

 

Sorry for the raft of questions!

Edited August 6, 2015 by TheGrunch

[Ace_Pilto]

Grunch, It might be a good idea to comment some of this code as well as explaining it in a subsequent paragraph. 

[Hylo]

Hi Jimmy,

Of course, I'm happy to explain the details:

 

`def` - denotes a function definition; these particular functions return parsers which match some section of a mission file

`rule` - this part is specific to how the library is implemented, don't worry too much about it

 

The most simple kind of parser would be

 

def matchesA = rule { 'A' }

 

That parser would match the input "A" only.

 

~ is an infix function that says that first one parser should match, then another in order for the current rule to match

 

def matchesAthenB = rule { 'A' ~ 'B' }

 

This rule matches only the input "AB".

 

There are combinators such as zeroOrMore and oneOrMore to express repetition of a rule.

 

| expresses alternation, in other words "or".

 

def asOrBs = rule { oneOrMore (oneOrMore('A') | oneOrMore('B')) }

 

The above would match any sequence of uppercase A and B.

 

`capture` pushes the input matched by the rules it contains into parboiled's internal value stack

 

~> pops values off the stack and into an object constructor, then pushes the resulting object onto the stack. If the constructor takes 3 arguments it pops the last 3 arguments off the stack.

 

If we have an object Name which takes one String argument in its constructor, we can do this:

 

def name = rule { capture(asOrBs) ~> Name }

 

This rule would match a sequence of uppercase A or B and then push it into a Name object, so running the parser on "AAABBBAA" would get you a Name("AAABBBAA").

 

Hope that helps!

[coconut]

 

 

If I was starting from scratch to parse the .Mission files I would probably just define a set of types for my AST based upon the primitives of the mission editor and use some parser combinator library (e.g. parboiled2) to write the parser by hand

 

That's how I started doing it, but it was taking quite a bit of time going over all the constructs.

 

 

 

What is the function of the sample file that's given to the analyser? Do you make a pass over this file in order to enumerate all of the possible types? Does this provide some degree of robustness if the mission format changes?

 

Yes, that's what it does. This pass generates the AST types and the parser. The types will be "dynamic", so in order to retain type safety I use the type provider mechanism.

 

On the subject of parser frameworks, I'm not using any. Instead I use recursive descent using F# active patterns.

[Skoop]

Have you guys thought of collaborating with the guy who made the IL2 commander app ?  It looks like he made the front end interface for a career mode, it just needs the mission generation and persistent battlefield stuff you guys are talking about here.  Could be too many cooks in the kitchen though.   

 

Coconut, your one talented dude, I have been wondering when someone was going to make use of your SturmovikMission work.

[Hylo]

Coconut, I sent you a PR for SturmovikMission - just writing some FsCheck tests to get used to F#, I hope that's okay?

 

https://bitbucket.org/johdex/sturmovikmission/pull-requests/1/start-adding-some-fscheck-tests/diff

Edited August 8, 2015 by TheGrunch

[FlatSpinMan]

Keep going, fellas! Keep going! Even just discussing things openly helps drive things forward.

[LLv24_Zami]

Keep going, fellas! Keep going! Even just discussing things openly helps drive things forward.

 

Second that!

 

Would be awesome to get something like this for BoS.

[coconut]

Have you guys thought of collaborating with the guy who made the IL2 commander app ?  It looks like he made the front end interface for a career mode, it just needs the mission generation and persistent battlefield stuff you guys are talking about here.  Could be too many cooks in the kitchen though.

That's the way to go, I think. I have personally no interest in redoing what he has done.

Keep going, fellas! Keep going! Even just discussing things openly helps drive things forward.

At the moment I'm working on handcrafted missions using the parser with various blocks for missions. It has exposed a number of issues that should be improved in the parser if people other than me should be expected to use that.