Yet another simple parser - plog

[taleks]

I've made simple log parser for mission text logs last weekend.

 

to use it you can start it from console: plog some_path/missionReport(2015-08-11_20-31-25)[0].txt

It will parse this file, then missionReport(2015-08-11_20-31-25)[1].txt and so on, while sequence of files continues.

Both multiplayer and single mode mission logs can be parsed.

 

As result you will see cool colored text in console plus such files:

1) 2015-08-11_20-31-25-log.json

2) 2015-08-11_20-31-25-brief.json

3) 2015-08-11_20-31-25.json

 

1) is mission log, converted to json format. Useful if you want to do some research of log events, but game's file format is too scary even to try to start. JSON is much more human-friendly, reading to it would be simpler..

 

2) is some kind of  mission log analyze, brief part of it.

 

Interesting places in it are:

 

MissionStats - cummulative statistics all killed objects per coalition and country. Here you can find Frags subsection, list of which player killed which player, according to parsed data.

 

DeadStats - all killed game object, including those killed by NPCs or are dead for some other reasons (don't believe to DeathReason, parser guesses it correctly in about 80% for dead non-player objects).

 

PlayerStats - statistics per player, what unit types he destroyed, kill/death statistics, how his flights sessions ended (dead, landed, etc).

 

3) more detailed information, including

MissionFlow - key events of mission (take off, landing, death)

AllPlayers - all players with all related to them events per each flight during game.

 

You can read this data in text editor or, if you prefer, in one of many freely available JSON Viewers.

 

-----

Some notes.

 

1) I've done it in my personal time, it is unrelated to 1CGS and it is not built upon any code insights or game binary dependencies, only text logs parsing, thus there may be and there are errors, although I will try to eliminate most of them or produce workarounds in next few weeks. Notably, right now turrets kill/death are not counted properly. There may be some crashes as well.

2) information about how to count unit (which type: Fighter, Turret, AAA, etc) is taken from GameObjectsData.json. So if you get crash and the last message in console says something about unknown type of object like IL2, then this file should be updated with type for IL2.

3) plog is available as plog.exe (console tool), plog_gui.exe (simple GUI, for those who want to parse, but don't like console, even cool colored console. Anyway, it can parse specified log or set of logs in folder only, without visualization of results), ploggy.dll (parser library).

4) You're free to use code of ploggy library https://bitbucket.org/taleks/plog It is not well commented, though. It is managed .NET app, depends on NLog and NewtonSoft.Json only, and buildable with mono on Mac.

plog_test.zip

[Jason_Williams]

Thanks Taleks!

 

Jason

[SCG_Neun]

Thank you for your hard work on this.  I'll give it a try.  Salute!

[=LG/F=Kathon]

Thanks Now I can stop working on my own parser

[Juri_JS]

This shows very well what can be done with the log files, but it seems to be somewhat inconvenient to use.

To be honest, I wonder why one of the devs is doing this in his free time instead of officially adding such a feature inside the game. Could this be a first step to get a real mission stat function inside the game itself?

[SCG_Space_Ghost]

-snip-

 

 

-snip-

 

it is unrelated to 1CGS and it is not built upon any code insights or game binary dependencies

 

snip-

 

(facepalm emoticon)

[taleks]

This shows very well what can be done with the log files, but it seems to be somewhat inconvenient to use.

 

To be honest, I wonder why one of the devs is doing this in his free time instead of officially adding such a feature inside the game. Could this be a first step to get a real mission stat function inside the game itself?

That's true, it is not very convenient for player, instead it is good start for those who want to do, e.g. some statistics, but are stopped by need to develop log parsing module first.

 

As for why I'm doing it in personal time. 

Obviously, there is in-house parser, which works right now to analyze mission results, but it is tightly coupled with other code, which is company's IP and can't be shared with public. We have a lot of things to do right now, new multiplayer mode, BoM related stuff, some improvements in other areas and yet unannounced features, thus it is difficult to allocate time for official publicly available parser. At the same time, I have finished one of my other hobby projects, thus have some time to fill it in.

[Juri_JS]

I hope there will be time to add an in-game stat system for single player mission mode after the release of BoM. Together with the possibility for user-made campaigns this would be my biggest wish.
Because a stat system that runs outside the game is too complicated or inconvenient for the majority of players, I've created a simple debriefing screen for my missions in FMB as a temporary solution, but unfortunatly it can't track the player's individual victories, only results from the whole mission (player+AI).

[Gitano_Fraile]

gracias, gracias y mas gracias

si alguna vez nos encontramos yo invito las 

[ECV56_Cherar]

thanks taleks!!!

[PA-Sniv]

Thanks Taleks!

[[TWB]Pand]

This looks promising! Thanks!

 

Quick question regarding startup.cfg---- setting the "textlog_folder" value doesn't work to point the log output to a different location.

 

I pulled that variable from some old thread on the rise of flight forums---- should it be a different variable like "missionlog_folder" to make it register in BOS?

 

[KEY = system]

      chatlog = 1

      chatlog_folder = "logs\chat\"

      mission_text_log = 1

      textlog_folder = "logs\txt\"

[END]

[taleks]

As far as I remember, correct name for this option is text_log_folder (with underscore between text and log).

[[TWB]Pand]

You are my hero taleks!!! I've been trying to find that variable for months!  

[[TWB]Pand]

Taleks!  One more question...

 

Do you have a list of the Action Types that go into the raw mission text log (AType:XX) explanations?    We're looking at potentially implementing some custom parsing also, and if this is already documented somewhere, it will save a ton of time and testing on our side.

[taleks]

Taleks!  One more question...

 

Do you have a list of the Action Types that go into the raw mission text log (AType:XX) explanations?    We're looking at potentially implementing some custom parsing also, and if this is already documented somewhere, it will save a ton of time and testing on our side.

Well, no. I've no such list, I've looked to xedoc's sources and RoF's wiki, which is now offline but available in web archive (I've link to Russian version of it only). Plus a bit of guessing in some cases.  If you're familiar with C# or any other language with C-like syntax you can check my code of plog, especially file LogEntry.cs. At the top of it there is enumeration of all ATypes, rest of file contains different classes for each log entry, some examples, copied from logs, and regular expressions I use to parse log.

[HR_Pingu]

Nice work! Thank you for the effort!

[[TWB]Pand]

Taleks, thanks for sharing what you have!

[Dutch2]

Sorry did try this but it did not work, how to install this. 

[taleks]

Sorry did try this but it did not work, how to install this. 

Just unpack archive and start plog.exe in command line, specifying log to parse. It will present stats in console and save in .json files as well.

Attached in first post tool is a bit dated, more fresh can be compiled from provided sources. Probably I'll update it later and reattach to the first post.

[ECV56_Cherar]

a new horizon, we finally understand the log language, thanks again taleks 

[von_Tom]

I cannot locate any mission report to parse (whatever that means).

 

I'm looking for something to check what my hit % is  will this do this for me or do I need something else?

 

von Tom

[KpaxBos]

I cannot locate any mission report to parse (whatever that means).

 

I'm looking for something to check what my hit % is  will this do this for me or do I need something else?

 

von Tom

 

Hello,

 

Do you have enabled generation of log files in :

 

.....\1C Game Studios\IL-2 Sturmovik Battle of Stalingrad\data\startup.cfg

 

                            mission_text_log = 1

[von_Tom]

Hello,

 

Do you have enabled generation of log files in :

 

.....\1C Game Studios\IL-2 Sturmovik Battle of Stalingrad\data\startup.cfg

 

                            mission_text_log = 1

 

 

I will check later - thank you.

 

von Tom

[LoupVert]

 
That nice crash happen when I try to parse our last Wednesday's logs.
 
Any idea guys?
 
Here's my GameObjectsData.json file :

{
	"BotPilot_Bf109" : "Bot",
	"BotPilot_Pe2" : "Bot",
	"BotPilot_Il2" : "Bot",
	"BotPilot_FW190" : "Bot",

	"BotGunner_Ju87D3" : "Bot",
	"BotGunner" : "Bot",
	"BotGunner_Il2m42" : "Bot",

	"Wagon_PlatformAA-MG34" : "AAA",
	"MG 34 AA turret" : "Turret",
	"VehicleTurret" : "Turret",
	"Wagon_PlatformAA-Flak38" : "AAA",
	"Flak 38 turret" : "Turret",
	"BlocksArray" : "Static",
	"Bf 109 E-7" : "Fighter",
	"Bf 109 F-4" : "Fighter",
	"Bf 109 G-2" : "Fighter",
	"Bf 110 E-2" : "Fighter",
	"Fw 190 A-3" : "Fighter",
	"He 111 H-6" : "Bomber",
	"Il-2 mod.1942" : "Attacker",
	"Ju 87 D-3" : "Attacker",
	"MC.202 ser.VIII" : "Fighter",
	"P-40E-1" : "Fighter",	
	"I-16 type 24" : "Fighter",
	"La-5 ser.8" : "Fighter", 
	"LaGG-3 ser.29" : "Fighter",
	"Pe-2 ser.87" : "Bomber",  
	"Yak-1 ser.69" : "Fighter",       
	"52-K" : "AAA",
 	"61-K" : "AAA",
 	"DShK" : "FiringPoint",
 	"Flak 37" : "AAA",
 	"Flak 38" : "AAA",
 	"German Landlight" : "Static",
 	"German Searchlight" : "Static",
 	"leFH 18" : "Artillery",
 	"MG 34" : "FiringPoint",
 	"MG 34 AA" : "AAA",
 	"ML-20" : "Artillery", 
 	"Pak 38" : "Artillery",
 	"Pak 40" : "Artillery",
 	"Radio Beacon" : "Static",
 	"Soviet Landlight" : "Static",
 	"Soviet Searchlight" : "Static",
 	"ZIS-2" : "Artillery", 
 	"ZIS-3" : "Artillery",
 	"BT-7M" : "Tank",
 	"KV-1-42" : "Tank",
 	"PzKpfw III Ausf.L" : "Tank",
 	"PzKpfw IV Ausf.G" : "Tank",
 	"Sd Kfz 10 Flak 38" : "AAA",
 	"Sd Kfz 251 Wurfrahmen 40" : "Artillery",
 	"StuG III Ausf.C-D" : "Tank",
 	"StuG III Ausf.F" : "Tank",
 	"T-34-76 STZ" : "Tank",
 	"T-70" : "Tank",
 	"Locomotive_E" : "Vehicle",
 	"Locomotive_G8" : "Vehicle",
 	"AA Platform 61K" : "Vehicle",
 	"AA Platform Flak 38" : "Vehicle",
 	"AA Platform M4" : "Vehicle",
 	"AA Platform MG 34" : "Vehicle",
 	"Wagon_BoxB" : "Vehicle",
 	"Wagon_BoxNB" : "Vehicle",
 	"Wagon_ET" : "Vehicle",
 	"Wagon_G8T" : "Vehicle",
 	"Wagon_GondolaB" : "Vehicle",
 	"Wagon_GondolaNB" : "Vehicle",
 	"Wagon_Pass" : "Vehicle", 
 	"Wagon_PassC" : "Vehicle",
 	"Wagon_PlatformB" : "Vehicle",
 	"Wagon_PlatformEmptyB" : "Vehicle",
 	"Wagon_PlatformEmptyNB" : "Vehicle",
 	"Wagon_PlatformNB" : "Vehicle",
 	"Wagon_TankB" : "Vehicle",
 	"Wagon_TankNB" : "Vehicle",
 	"BA-10M" : "Vehicle",
 	"BA-64" : "Vehicle",
 	"Ford G917" : "Vehicle",
 	"GAZ-AA" : "Vehicle",
 	"GAZ-AA M4" : "AAA", 
 	"GAZ-M" : "Vehicle",
 	"Horch 830" : "Vehicle",
 	"Opel Blitz" : "Vehicle",
 	"Sd Kfz 251-1 Ausf.C" : "Vehicle",
 	"ZiS-5" : "Vehicle",
 	"ZiS-6 BM-13" : "Artillery",
 	"Turret_Bf110E2" : "Turret",
 	"Turret_He111H6_1" : "Turret",
 	"Turret_He111H6_1M" : "Turret",
 	"Turret_He111H6_2" : "Turret",
 	"Turret_He111H6_3" : "Turret",
 	"Turret_He111H6_3M" : "Turret",
 	"Turret_He111H6_4" : "Turret",
 	"Turret_Il2m42" : "Turret",
 	"Turret_Ju87D3" : "Turret",
	"Turret_Pe2s87_1M" : "Turret",
	"Turret_Pe2s87_1" : "Turret",
	"Turret_Pe2s87_2" : "Turret"
 }

[LoupVert]

Does anybody else experienced the issue I'm facing? Just so I know if it's on my side or not.

 

Here are the log files I'm trying to parse if anyone have time to gibe it a try : http://superdupontzeserveur.fr/ff/missionReport(2015-12-16_21-12-45).zip

Edited December 21, 2015 by -LaFrench-LoupVert

[taleks]

If you're using tool from first post (not compiled it from sources), then it is a bit outdated.

 

Newer version writes

---

Vehicle type BotGunner_Ju87D3 has wrong binding, check GameObjectsData.json. Results will be wrong.

---

and doesn't crash. However that behavior is strange. It may be related to message above: attempt to create turret object failed because bot with the same id is still exists. I'll check it later, today is a bit busy day.

[LoupVert]

Thank you Talek, looking foorward for the newer version then (I can't find it on Bitbucket).

[taleks]

Thank you Talek, looking foorward for the newer version then (I can't find it on Bitbucket).

Well, on bitbucket I host sources only, thus you can't find it there.

But here is version of binary, which is able to process log with errors, but without crash.

 

Please note, that if you're using .json files produced by this tool, there were slight changes in its format somewhere in September. And there were some changes in kill accounting: to avoid kill stealing, only maximum damager is awarded kill and only if damage was not too long time ago. It is not perfect solution, though, but a bit better than it was before.

 

As for issue with posted by you log, most probably I'll be able to look to it tomorrow.

plog.20151222.zip

[LoupVert]

No error with that newer version, thanks a lot Taleks.

[[Pb]Vapor]

Sorry to resurrect the dead (forum thread).  Any chance you have updated this Taleks?  I know this was a "free time" project and so I understand if you haven't had the time.  It is just such a useful tool I was hoping you had an updated copy.  :D

[Ala13_Zetas]

I came to late to this post ?.

It should be outdated but anyone have the parser and can re-upload it? 

 

Thx Taleks!