Jump to content
Lupson

[Release] Bosparser - BoS after-action report application

Recommended Posts

Hi all,

 

being a single-player kind of dude I really like the Chir Front campaign and some other single missions. However, I do miss having an after-action report / debriefing to examine after each (non-campaign) mission.

 

Therefore I decided to brew something up myself based on parsing of the missionReport.....[n].txt files generated by each mission into the BoS installation /data directory, if one has enabled "mission_text_log = 1" in the startup.cfg file.

 

post-35556-0-27923500-1417988855_thumb.jpg

post-35556-0-97048000-1417988920_thumb.jpg

 

It's a pretty rather crude thing, but at least it records kills, individual hits on enemies and self and a few other things.

 

Technically, its a Java application that boots an embedded web application (accessible on http://localhost) which can scan, store and finally display mission reports using a simple web-based GUI.

 

Bosparser is free and open-source software licensed under the MIT License. The full source code for bosparser can be found on github: https://github.com/eriklupander/bosparser

Full license details can be read here: https://github.com/eriklupander/bosparser/blob/master/LICENSE

 

Of course, I have no affiliation to 1C or 777 whatsoever. I based interpretation of log files on the Rise of Flight server log description found here: http://en.wiki.riseofflight.com/index.php?title=Server_log

 

== Bugs etc. ==

There's probably lots of bugs. This is something I've thrown together during spare hours the past week mostly for my own enjoyment, but I thought I'd share it with the community if there's someone else wanting to try it out. Why some hits cause no damage while others do, I do not know. Neither what a "Damage 0.014" means except it seems as objects typically are destroyed when the damage value reaches 1.0. Also, it seems HE ammunition causes no damage itself, rather it creates an "explosion" entry that can cause damage to both plane, turret and crew if you're lucky enough...

 

The object structure is somewhat hierarchial with for example a Pe-2 being made up by plane having turrets having gunner for example. This is not very well reflected in the GUI.

 

Future plans? I have very limited time (work, family etc.) to work on projects like these so I simply do not know. I guess it's good enough for my private needs now but of course I'll listen if someone reports a bug or asks for a feature. Since the source code is open sourced anyone can fork it and continue to work with it.

 

Oh, I hope I'm not breaking any forum rules or EULA by posting this...

 

== Prerequisites ==

* Make sure you have a properly installed Java Runtime Environment on your computer, 1.6 or later required. The java executable needs to be on your PATH.

* Modify your BoS startup.cfg file, set property mission_text_log = 1

 

== Install instructions ==

* Download: https://www.dropbox.com/s/opdnpe61p89pctg/bosparser-alpha1.zip?dl=0

* Unzip it somewhere on your hard drive

* Modify 'config/application.properties' and change the reports.directory property to your IL-2 BoS install/data directory, such as c:\\Program Files\\SteamApps\\common\\IL-2 Sturmovik Battle of Stalingrad\\data

(Note the double-slashes)

* Run bosparser.bat from the folder you unzipped bosparser-alpha1.zip to

* Open your web browser and type localhost (or http://localhost) to open the web gui

* Press the "Scan for reports" button in the upper left corner. Browse reports as you see fit.

* To exit the application, press CTRL+C in the command shell or close the process using the Windows Task Manager.

 

  • Upvote 27

Share this post


Link to post
Share on other sites

Brilliant stuff. I wasn't sure there'd be a way of extracting this sort of information in stand-alone single missions. Shows how much I know!  :wacko:

 

Edit: Prangster beat me to itl!  :P

Edited by No601_Swallow

Share this post


Link to post
Share on other sites

Thx a lot !

This is needed if we want online campaign in the future !

 

The minimum requirement is a parser and a mission generator (missions files are only text files and can be generated but that need some works)

 

You did the parser so that's gonna help for sure ! Thx again mate

 

PS : I didn't checked yet but is it an offline SP parser ? Can you do a MP version of this parser ?

Edited by PB0_Foxy
  • Upvote 1

Share this post


Link to post
Share on other sites

Thanks all!

 

PB0_Foxy: I've only tested with single missions (both bundled and Chir Front), Quick Missions and Campaign missions. These missions seems to generate log files with the same structure. Works in both Online and Offline mode.

 

I can try an online mission or two to see what ends up in my client log file. I do suspect that the dserver.exe will yield log files having more info for individual players etc. compared to what the standard BoS.exe outputs.

 

One can also get some kind of binary .mlg log files. I havn't spent much time determining what format they actually are, it would be neat to have an SDK or some kind of API to work with them but since I wanted fast progress I chose the easiest path with the .txt files.

 

When I tried to look at the mission files (for example the Chir Front ones) they looked binary to me. I suspected they would be in a compressed format but at least 7-zip (which swallows most compressed formats) couldn't open them as archives. Anyone knows more on how to read these files?

 

In the long run, it would be interesting to construct some kind of war game engine that starts with a "starting war template", takes mission logs as input, updates war state and then generates the next mission based on the updated state, randomness etc. with a subset of the total war game state injected into the mission as objects, flights etc. I guess that's somewhat similar to how the old IL-2 campaign generators worked (DCG) and I've also read a bit about Pat Wilsons dynamic campaign for Rise of Flight which I also think is using log files. To do a standalone dynamic campaign mission generator one would probably need proper documentation on the mission format and/or access to the FMB to see what ends up in the mission file(s) given some unit/action/trigger made in the FMB. 

 

Regarding the AAR application: If you run into problems, please post any output from the command window into this thread, preferrably with a zip of your last mission's .txt logs, that would simplify troubleshooting by a great deal.

Share this post


Link to post
Share on other sites

Thanks all!

 

PB0_Foxy: I've only tested with single missions (both bundled and Chir Front), Quick Missions and Campaign missions. These missions seems to generate log files with the same structure. Works in both Online and Offline mode.

 

I can try an online mission or two to see what ends up in my client log file. I do suspect that the dserver.exe will yield log files having more info for individual players etc. compared to what the standard BoS.exe outputs.

 

One can also get some kind of binary .mlg log files. I havn't spent much time determining what format they actually are, it would be neat to have an SDK or some kind of API to work with them but since I wanted fast progress I chose the easiest path with the .txt files.

 

When I tried to look at the mission files (for example the Chir Front ones) they looked binary to me. I suspected they would be in a compressed format but at least 7-zip (which swallows most compressed formats) couldn't open them as archives. Anyone knows more on how to read these files?

 

In the long run, it would be interesting to construct some kind of war game engine that starts with a "starting war template", takes mission logs as input, updates war state and then generates the next mission based on the updated state, randomness etc. with a subset of the total war game state injected into the mission as objects, flights etc. I guess that's somewhat similar to how the old IL-2 campaign generators worked (DCG) and I've also read a bit about Pat Wilsons dynamic campaign for Rise of Flight which I also think is using log files. To do a standalone dynamic campaign mission generator one would probably need proper documentation on the mission format and/or access to the FMB to see what ends up in the mission file(s) given some unit/action/trigger made in the FMB. 

 

Regarding the AAR application: If you run into problems, please post any output from the command window into this thread, preferrably with a zip of your last mission's .txt logs, that would simplify troubleshooting by a great deal.

Mission files are in the same format as Rise of flight.

Veteran just gave you the binaries files because it's enough but there is also a mission file. More "readable".

 

I can help you with that if you need  ;)

Share this post


Link to post
Share on other sites

Mission files are in the same format as Rise of flight.

Veteran just gave you the binaries files because it's enough but there is also a mission file. More "readable".

 

I can help you with that if you need  ;)

 

Thanks, that would be interesting to take a look at. Sending you a PM.

Share this post


Link to post
Share on other sites

Yours is one of the best ideas recently arised in this forum  :salute:  . Great. Thanks.

 

AA_Engadin

Share this post


Link to post
Share on other sites

wauu Lupson

nice work!!!!!

 

will look for the binaries files :)

PN to me who need it

Edited by Veteran66

Share this post


Link to post
Share on other sites

Well done Lupson, although I hope it is only a temporary solution and the developers will officially add such a function that can be accessed from inside the game.

Share this post


Link to post
Share on other sites

Great and very important work . Many thanks.

 

Hopefully this can be applied to online results also. I am a sad individual who likes to read mission logs from multiplayer sessions. It worked nicely in RoF , would like to see something similar in BoS , including accuracy and maybe damaged parts if possible.

 

Certainly more important than just reading logs is creating a consistent online war. This will bring loads of new squadrons in and will keep existing player in game.

 

I admire your effort , this community needs more talented people like you , so it can grow. You should be getting a lot of support from the devs. Or at least a thank you ;).

Share this post


Link to post
Share on other sites

Thanks again, everyone. Nice to know it's appreciated.

 

Agree on that this functionality should be part of the core product. How is it in Rise of Flight, does it have after action reports?

 

Tried a few MP sorties but regrettably it seems no mission logs were produced on my computer. Probably its only the dserver.exe host computer that gets logs for online missions. So for now 'bosparser' is only applicable to single-player missions.

 

I would love having stats for individual hits on subcomponents such as engine or other vital parts, but the bosparser is limited to what's written to the log files. This far I've only seen turrets and pilot/gunners as subcomponents of an aircraft in the logs.

 

Have received a text-based mission file from the Chir Front series (thx to PB0_Foxy!) and a link to the RoF mission builder manual. Interesting stuff - lots of possibilities but also very very complex. Creating an acceptably useful standalone mission generator / dynamic campaign would be a huge undertaking to say the least.

Edited by Lupson

Share this post


Link to post
Share on other sites

REALLY BIG THANK YOU for this, I am sure this will come handy when I'm going to make my own  parser and hopefully simple generator too (PHP), but devs have to release dserver and mission editor first... and i got to get a decent pc to be able to play the game properly :D

Share this post


Link to post
Share on other sites

Hi all,

 

slightly updated version uploaded to Dropbox. This one fixes an issue where a damaged aircraft crashes but the "kill" is not associated with any attacker. Seems to happen when you damage an aircraft which keeps flying for a while before crashing or burning up/whatever. I now query the log files for hits on this aircraft by the player and the player with the highest _number_ of hits on the crashed aircraft is awarded the kill. This may not be 100% accurate every time as the mission logs doesn't record individual hits by AI on AI, but it should be good enough.

 

It might be an idea to rescan your reports to rebuild them using the updated parser logic after upgrading. To do this, first make sure you have all your "missionReport...txt" files in the /data (or otherwise specified directory), launch the bosparser application, click "Delete all reports" and wait until the dialog is shown. Then just click "Scan reports" (once) and wait while all files are parsed. On a slow computer, I guess it may take a little while if you have lots of reports. My logs dir currently holds logs for 38 missions and it takes about 10 seconds to perform a full parse. (i5 3770K)

 

New download link:

https://www.dropbox.com/s/wnbug08t4srqkq0/bosparser-alpha-1.01.zip?dl=0

 

-- Install --

If you havn't installed bosparser before, just follow the little guide in the first post of this thread.

 

-- Upgrade instructions --

If you have the old version installed, it's probably easiest to just unzip the new one somewhere other than the existing install location, then copy the bosparser-1.0-SNAPSHOT.war file from the new archive onto the old one, simply overwriting the existing one.

 

Alternatively, take a backup copy of your /config/application.properties, unzip the new archive on top of your existing install overwriting everything, then just copy back your application.properties into the /config directory.

  • Upvote 1

Share this post


Link to post
Share on other sites

Very Cool!  Thank you for this!  This is awesome!  I was a huge fan of DCG when I flew IL-2 1946 and the one big thing that kept me out of CLOD (even though I bought it at launch) was the lack of any sort of Dynamic Campaign Generator.

 

This will give me a great way to track my personal pilot kills etc.

 

I wonder if it would be possible to have it tell you what kind of targets were killed (2 JU-87D3), etc.  I'm thinking of things along the line of the old IL-2 MLR.

Share this post


Link to post
Share on other sites

Thanks!

 

Lupus, I assume you are referring to some kind of career stats with stuff like:

* Total number of missions

* N/o missions completed alive / destroyed

* Total n/o kills

* Total in-flight time

* N/o sorties per aircraft type

* N/o kills achieved flying each aircraft type

* Kills of each aircraft type

* N/o hits per ammunition type

* etc...

 

The above should be straightforward enough to implement. Just aggregation of data already present in the database on a per-mission basis.

 

I'll add this as soon as I have some spare time, hopefully a few days into 2015 or so.

  • Upvote 1

Share this post


Link to post
Share on other sites

Hi again,

 

time for a new update of bosparser. This iteration brings you career statistics:

 

post-35556-0-60039900-1419979092_thumb.jpg

 

As one might note, the scan/delete buttons have been moved below the header. Instead, one switches between "Missions" and "Career stats" in the top left corner. I've also added a loading spinner when scanning as its starting to take a little while with my current 57 missions in my sample log folder.

 

Some notes:

* "Missions destroyed" refers to the player aircraft state after the mission, not whether the pilot bought the farm or not.

* "Kills" and "Kills by object type" only contains "root" objects destroyed by player. Turrets on bombers, crew members etc. are not included in these figures.

* "Hits" and "Hits by ammunition type" contains hits on all game objects, including turrets, pilots and crew as well as planes, vehicles and static objects.

 

Download here: https://www.dropbox.com/s/opdnpe61p89pctg/bosparser-alpha1.zip?dl=0

 

Happy flying!

  • Upvote 1

Share this post


Link to post
Share on other sites

Hi again,

 

time for a new update of bosparser. This iteration brings you career statistics:

 

attachicon.gifcareer-page.jpg

 

As one might note, the scan/delete buttons have been moved below the header. Instead, one switches between "Missions" and "Career stats" in the top left corner. I've also added a loading spinner when scanning as its starting to take a little while with my current 57 missions in my sample log folder.

 

Some notes:

* "Missions destroyed" refers to the player aircraft state after the mission, not whether the pilot bought the farm or not.

* "Kills" and "Kills by object type" only contains "root" objects destroyed by player. Turrets on bombers, crew members etc. are not included in these figures.

* "Hits" and "Hits by ammunition type" contains hits on all game objects, including turrets, pilots and crew as well as planes, vehicles and static objects.

 

Download here: https://www.dropbox.com/s/opdnpe61p89pctg/bosparser-alpha1.zip?dl=0

 

Happy flying!

 

Wow!  That's Awesome!  Thank you!

Share this post


Link to post
Share on other sites

2 questions, how do we shut it down once opened?  And is there a way to have it automatically start up the browser and launch the page?  I tried adding the lines to the .bat file but had no luck.  Thanks for this, this is really awesome!

Share this post


Link to post
Share on other sites

I had somehow entirely missed this thread until right now. I have yet to try it, but it sounds excellent.

Thanks in advance for the thought, time and effort it has taken you to make this for the rest of us.

  • Upvote 1

Share this post


Link to post
Share on other sites

This little gem would be cool if it could be linked along with being able to name a pilot and to have a Russian and German pilot separate.

All going in the right direction though, keep it up.

 

Mick. :)

Share this post


Link to post
Share on other sites

Hi Lupson,

 

excellent work!

A suggestion: Auto starting the browser after web server is ready to serve requests would be probably a bit more convenient.

 

Do you have any information about another interfaces of BOS than text log files to get more data out of it?

I am looking for something that gives me runtime data of aircraft positions etc.

 

hdortim

Share this post


Link to post
Share on other sites

2 questions, how do we shut it down once opened?  And is there a way to have it automatically start up the browser and launch the page?  I tried adding the lines to the .bat file but had no luck.  Thanks for this, this is really awesome!

 

1. If you give the command-shell window focus you can press CTRL+C. I guess it also works to just close the command-shell window or use the Task Manager to kill the "java.exe" process.

2. This "sort of" works: Edit bosparser.bat so it looks like this:

 

start java -jar bosparser-1.0-SNAPSHOT.war

start "" chrome.exe http://localhost

 

I guess you can change chrome.exe to iexplore.exe or firefox.exe (or whatever) if you're using another browser. You may need to enter the absolute path to the browser executable if its not on the PATH environment variable.

 

The problem is that the browser will try to load the page before the web server has started so you'll have to manually refresh the browser window (F5 or CTRL+R) after the server has started. I guess it's possible to launch the browser from within the server application when it has booted, maybe I can add an option to application.properties for that in the future.

This little gem would be cool if it could be linked along with being able to name a pilot and to have a Russian and German pilot separate.

All going in the right direction though, keep it up.

 

Mick. :)

Thanks!

 

I think I already keep track of which country each object belongs to so a simple filter to show entries for missions flown as ALL|GERMANY|USSR can quite easily be added.

 

The log files contains the player name but since we can only have a single callsign per license of the software I don't think I can identify separate "pilots" except for one per country.

Share this post


Link to post
Share on other sites

Hi Lupson,

 

excellent work!

A suggestion: Auto starting the browser after web server is ready to serve requests would be probably a bit more convenient.

 

Do you have any information about another interfaces of BOS than text log files to get more data out of it?

I am looking for something that gives me runtime data of aircraft positions etc.

 

hdortim

 

Thanks!

 

See my post a few posts up in this thread about starting the web browser. To be more specific, one should be able to programatically start the browser using Java's Runtime.getRuntime().exec(), I'll see what I can do about that,

 

I don't know about any other interfaces into the simulator except for the log files. There are also those binary .mlg log files, maybe they contain more information than the textual ones.

 

It would be very neat to have the same kind of support that DCS offers for consuming simulation data using lua scripts and sockets. I prototyped a little Android tablet application a few years ago that rendered a few instruments and buttons for the Ka-50. If BoS offered real-time (or close to real time) data of aircraft positions, object state changes etc. over some interface (socket, memory-mapped file) one could do all kinds of neat stuff. For example making a web page showing the "air war" on a multiplayer server in real time. Once the dserver.exe is released its certainly possible to inspect the network traffic using Wireshark or similar, but I assume the data is encrypted due to anti-cheating reasons. It would of course be cheating to be able to see the position of your own aircraft on a map with expert settings or where the opponents are. But from a pure technological point of view it would be quite cool.

 

An interface to get info about the player plane flight data (speed, altitude etc.) might be more feasible, I guess its a prerequisite for sim pit builders.

Edited by Lupson

Share this post


Link to post
Share on other sites

 

 

It would be very neat to have the same kind of support that DCS offers for consuming simulation data using lua scripts and sockets. I prototyped a little Android tablet application a few years ago that rendered a few instruments and buttons for the Ka-50. If BoS offered real-time (or close to real time) data of aircraft positions, object state changes etc. over some interface (socket, memory-mapped file) one could do all kinds of neat stuff. For example making a web page showing the "air war" on a multiplayer server in real time. Once the dserver.exe is released its certainly possible to inspect the network traffic using Wireshark or similar, but I assume the data is encrypted due to anti-cheating reasons. It would of course be cheating to be able to see the position of your own aircraft on a map with expert settings or where the opponents are. But from a pure technological point of view it would be quite cool.

 

Hi Lupson,

 

that is exactly what I am intended to do. There was an interface in the old IL2 1946 which was called "DeviceLink". Something similar to that would be really cool and of course the major requirement before one could start to develop a second screen app with all available data. This would give a great overview especially for beginners.

I totally agree with you that this would be a kind of a cheat, but I guess the dedicated server will have a .ini file or so to set config parameters where this interface can be switched on and off, so the hard core guys will still be able to play in "full realism mode".

 

 

 

I don't know about any other interfaces into the simulator except for the log files. There are also those binary .mlg log files, maybe they contain more information than the textual ones.

Do you have more detailed information about the format of these .mlg files?

 

HDortim

Share this post


Link to post
Share on other sites

 

Do you have more detailed information about the format of these .mlg files?

 

HDortim

Not really. The .mlg files are generated just as the .txt ones with a few ones generated per in-game minute. Viewing them with a text editor shows some text strings similar to things present in the corresponding .txt file, so it may be the same data. It would have been neat if the log files were formatted as JSON (or BSON) or XML documents, that would have saved quite some time developing log file parsers.

 

I guess one can kindly send the devs a request to at least open up an interface for consuming player plane data, excluding position but including altitude, airspeed, fuel, temps etc. for future sim pit builders.

Share this post


Link to post
Share on other sites

Thanks to BoSparser I know that "I'm ace in a day" over Chir Front.  :)

 

Bos_Pacer_report.jpg

 

But due target fixation -  already out of ammo attack other P-2, got confused when weapons don't work, collided with target and bail out "behind enemy lines" - will be necessary
wait the end of the war to receive the Iron Cross... :P
 
Strange that statistic for "crew killed", I see some hit the silk.  :huh: 
 
 
 
 

Share this post


Link to post
Share on other sites

Hi,

 

Sokol1: Sorry for late answer - nice flying btw! Regarding "Crew killed" I'm not sure whether I got it right in bosparser. It may be that the log file reports bailed out crew with the same event as killed ones - in that case I may not be able to do anything about it. I'll have a look.

Share this post


Link to post
Share on other sites

New version of bosparser released!

 

This new version of bosparser introduces a "mission map" to the mission report. The map plots the flight path of the player and location of kills on an extracted in-game map (thanks to zeus-- for posting this map on the forums some time ago!). The map is fully zoomable/pannable. Route, kills etc. are rendered "vectorized" so they appear the same size regardless of zoom level.

 
One can click on kills on the map which will show a dialog with some details about the kill.
 
post-35556-0-32073400-1421361564_thumb.jpg
 
 
Install/upgrade instructions
 
 
Generally, the same instructions as before applies - for new installs, follow the guide at the top. For users upgrading, extract the latest zip somewhere and then copy the bosparser-1.0-SNAPSHOT.war file into your existing install dir, overwriting the old version of the file.
 
After updating, you will need to use the new "Rebuild reports" button to have your mission reports rebuilt with location data included.
 
Now you can open any report and use the "Show map" button to render the mission map. Depending on the performance of your computer and which web browser you are using, map rendering can take a few seconds.
 
The map rendering uses HTML5 canvas techniques so a relatively modern browser is required. I recommend using Google Chrome for bosparser. Firefox and Internet Explorer 11 also works but some map rendering gets weird with the latter two and the zooming the map using the mouse scroll wheel doesn't work.
  • Upvote 8

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×