IL2 server stats (unofficial)

[LLv34_Temuri]

Temuri     I need to talk to you about the Finnish server 

Oh, that was news to me. I'll check that.

[DeadMeat0383]

compacted-2018-01-14_22-33-28.txt 

you yourself renamed so the file?

If not, this means that some third-party software does this (not statistics).

Then such problems will be in the future, because some kind of third-party program hinders the work of statistics.

The mission log was already named as the compacted name, the only thing besides the statistics server running on that computer is the dserver software itself. It may have been a setting in the il2 config that did that. Either way it may be something to note for future iterations of the stats server

[SAS_Storebror]

Hi guys,

 

I've got a couple of questions concerning IL-2 Stats installation.

The readme says

1) Download and install PostgreSQL 9.5

 

Is there any point in sticking to version 9.5? Or will version 9.6 or even 10.1 do as well?

I'm not a big fan of running outdated software on my servers, that's why I'm asking...

 

2) Download and install Python 3.5

 

Similar as before, any reason not to use Python 3.6.4?

 

If the server is already use any other web server -  there are several options:

(...)

c) Run website statistics with web server as a WSGI or FastCGI application. Used framework Django.

 

Sounds cool, so using IIS and running the Django Framework in FastCGI mode is supposed to work? Anyone got experience with this?

I'd rather like to avoid operating a second Webserver in parallel to the existing IIS for security reasons.

 

Thanks in advance for your answers.

Cheers!

Mike

Edited January 18, 2018 by SAS_Storebror

[=FB=Vaal]

PostgreSQL 9.6 did not check, but I think it will work

Python 3.6 can be used

About IIS. Django no longer has built-in support for FastCGI. Need to look for other solutions.

[SAS_Storebror]

Thanks for the reply.

There seems to be an issue though and I suspect it might be related to the newer Python Version:

$ ".venv/Scripts/python.exe" -m pip install --quiet --no-index --find-links=wheelhouse -r requirements.txt
  Could not find a version that satisfies the requirement argon2-cffi==16.3.0 (from -r requirements.txt (line 7)) (from versions: ) No matching distribution found for argon2-cffi==16.3.0 (from -r requirements.txt (line 7))

Cheers!

Mike

[=FB=Vaal]

Yes for another version of the python you need to download packages from the Internet.

run.zip

[SAS_Storebror]

Okay we're getting closer.

$ "../.venv/Scripts/python.exe" manage.py import_csv_data --verbosity 0
Traceback (most recent call last):
  File "manage.py", line 7, in <module>
    execute_from_command_line(sys.argv)
  File "C:\Python3\virtualenvs\il2stat\il2_stats\.venv\lib\site-packages\django\core\management\__init__.py", line 371, in execute_from_command_line
    utility.execute()
  File "C:\Python3\virtualenvs\il2stat\il2_stats\.venv\lib\site-packages\django\core\management\__init__.py", line 365, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "C:\Python3\virtualenvs\il2stat\il2_stats\.venv\lib\site-packages\django\core\management\base.py", line 288, in run_from_argv
    self.execute(*args, **cmd_options)
  File "C:\Python3\virtualenvs\il2stat\il2_stats\.venv\lib\site-packages\django\core\management\base.py", line 335, in execute
    output = self.handle(*args, **options)
  File "C:\Python3\virtualenvs\il2stat\il2_stats\src\stats\management\commands\import_csv_data.py", line 28, in handle
    'cls': row['cls']})
  File "C:\Python3\virtualenvs\il2stat\il2_stats\.venv\lib\site-packages\django\db\models\manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "C:\Python3\virtualenvs\il2stat\il2_stats\.venv\lib\site-packages\django\db\models\query.py", line 503, in update_or_create
    obj = self.select_for_update().get(**lookup)
  File "C:\Python3\virtualenvs\il2stat\il2_stats\.venv\lib\site-packages\django\db\models\query.py", line 914, in select_for_update
    obj = self._chain()
  File "C:\Python3\virtualenvs\il2stat\il2_stats\.venv\lib\site-packages\django\db\models\query.py", line 1156, in _chain
    obj = self._clone()
  File "C:\Python3\virtualenvs\il2stat\il2_stats\.venv\lib\site-packages\modeltranslation\manager.py", line 213, in _clone
    return super(MultilingualQuerySet, self)._clone(**kwargs)
TypeError: _clone() got an unexpected keyword argument '_rewrite'

As much as I understand, Query.clone was reworked in Django 2.0 and now all such lines need to be changed...

 

Cheers!

Mike

[=FB=Vaal]

The project does not use Django 2. It uses Django 1.11

The required versions of packages are specified in the requirements.txt

[SAS_Storebror]

I see. I'll have to check which Django Version pip installed.

Sorry for the confusion. I've been out of this Python Business for a couple of years...

 

Cheers!

Mike

[SAS_Storebror]

Okay, that was my mistake. In fact I've had Django 2.0.1 installed.

Why? Because in my previous attempt to get rid of the dependency error, before I figured that I'd have to skip the wheelhouse, I've replaced all == requirements by >=, which of cause made python think that it'd be okay to use latest versions for all modules.

 

install.cmd worked now and I'll proceed, will keep you updated.

 

Thanks for the help!

 

Cheers!

Mike

[SAS_Storebror]

Update: I've got the stats up and running with the builtin Django webserver ("waitress") but failed miserably on attempting to integrate it into IIS.

 

The latter part is a bit frustrating: FCGI or WSGI? Django support or not? There's a quadrozillion guides on the net about how to get this to work, but no two of them match and none of them even remotely work at all.

But that's not your fault and not il2_stat's issue.

 

Cheers!

Mike

[=FB=Vaal]

I can not help you. I do not have experience with the actual version of Django plus IIS.

[SAS_Storebror]

Yes I see and I don't blame it on you at all.

It's a mess created by MS and PSF.

If I ever get bored enough, I will give it another shot. Until then, I'll just leave it up to the builtin Django webserver, after all it seems to work well and isn't too resource hungry, so I'll leave it up to the sandbox to ensure my mental sanity 

 

Cheers!

Mike

[SAS_Storebror]

I have a potentially stupid question: How do I add a squad?

Currently the Squad list is empty and there seems to be no way to start a new squad or add players to a squad, so I don't know where to start.

 

This is what the main menu looks like:

 

If I click the lower "Squads" link or the "Change" link in that line, it takes me straight to the next image, if I click "SQUADS", I get this menu inbetween:

 

Whether I click "Squads" or "Change" here, takes me to this:

- No chance to add a squad - I can search for one, but as none exist, the search result of course remains empty.

 

In the player profiles I see this:

Can't edit any value here, so no chance to join a squad again.

 

Cheers!

Mike

Edited February 6, 2018 by SAS_Storebror

[KG200_Volker]

Through the site players sign up, one of them creates a squad and gives the special link to the other registered pilots to join the same squad.

The squad will appear in the list of squads after at least 2 (or 4 that I don’t remember that for sure) had flown at least 1 mission after they register to the squad. Passed mission flown (before a pilot joined the squad) won’t be added to squad stats.

[SAS_Storebror]

Hm okay, so it's solely up to the users to create, join, leave, edit squads and an admin can't do that.

That's a bummer.

Thanks for the answer nevertheless 

 

Cheers!

Mike

[SAS_Storebror]

Just ran update.cmd today to grab 1.2.19, but somehow the update doesn't seem to do much.

All output is as it should according to UPDATE.en.txt, but the website still states that it's version 1.2.18 and I don't have the files

src/stats/static/img/aircraft/a-20b.png    
src/stats/static/img/aircraft/bf 109 g-6.png    
src/stats/static/img/aircraft/la-5fn ser.2.png    
src/stats/static/img/aircraft/p-39l-1.png    
src/stats/static/img/aircraft/yak-7b ser.36.png  

which according to the update post should be there (only the A-20B.png is there, the others are missing).

I do have the "mangled" versions of these files in "static/img/aircraft" though (without "src/stats/"), e.g. yak-7b.0a46979096b6.png

Quite confusing, what's wrong with that update?

 

Cheers!

Mike

[=FB=Vaal]

Цитата

2) Update files to the new files from the archive

 

You did it? these files are in the archive

[SAS_Storebror]

Alright Vaal, thank you very much for your reply.

Turns out it was a permission issue on the folder structure of my server, I seem to be overcautious 

 

Thanks again and Cheers! 

Mike

[LLv34_Temuri]

Updated to latest version. Looks like the stats system doesn't recognize pilots taking off, as the Online Pilots isn't updated.

[SAS_Storebror]

Stats parser (stats.cmd) stopped working on first 3.001 mission to parse.

Log:

[2018.03.14 18:59:02] unexpected error
Traceback (most recent call last):
  File "C:\Python3\virtualenvs\il2stat\il2_stats\src\stats\management\commands\stats_whore.py", line 16, in handle
    stats_whore.main()
  File "C:\Python3\virtualenvs\il2stat\il2_stats\src\stats\stats_whore.py", line 69, in main
    online_timestamp = update_online(m_report_files=m_report_files, online_timestamp=online_timestamp)
  File "C:\Python3\virtualenvs\il2stat\il2_stats\src\stats\online.py", line 24, in update_online
    data = parse_mission_log_line.parse(line)
  File "C:\Python3\virtualenvs\il2stat\il2_stats\src\mission_report\parse_mission_log_line.py", line 231, in parse
    data = atype_handlers[atype_id].match(line.strip()).groupdict()
AttributeError: 'NoneType' object has no attribute 'groupdict'
[2018.03.14 18:59:02] Lock 3152032611184 released on C:\Python3\virtualenvs\il2stat\il2_stats\file.lock
Traceback (most recent call last):
  File "manage.py", line 7, in <module>
    execute_from_command_line(sys.argv)
  File "C:\Python3\virtualenvs\il2stat\il2_stats\.venv\lib\site-packages\django\core\management\__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "C:\Python3\virtualenvs\il2stat\il2_stats\.venv\lib\site-packages\django\core\management\__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "C:\Python3\virtualenvs\il2stat\il2_stats\.venv\lib\site-packages\django\core\management\base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "C:\Python3\virtualenvs\il2stat\il2_stats\.venv\lib\site-packages\django\core\management\base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "C:\Python3\virtualenvs\il2stat\il2_stats\src\stats\management\commands\stats_whore.py", line 16, in handle
    stats_whore.main()
  File "C:\Python3\virtualenvs\il2stat\il2_stats\src\stats\stats_whore.py", line 69, in main
    online_timestamp = update_online(m_report_files=m_report_files, online_timestamp=online_timestamp)
  File "C:\Python3\virtualenvs\il2stat\il2_stats\src\stats\online.py", line 24, in update_online
    data = parse_mission_log_line.parse(line)
  File "C:\Python3\virtualenvs\il2stat\il2_stats\src\mission_report\parse_mission_log_line.py", line 231, in parse
    data = atype_handlers[atype_id].match(line.strip()).groupdict()
AttributeError: 'NoneType' object has no attribute 'groupdict'

 

Cheers!

Mike

[=FB=Vaal]

Attach mission logs to messages.

[SAS_Storebror]

Done, please see attachment for the logs of the mission where stats.cmd bails out upon.

 

Cheers!

Mike

stats_issue_2018-03-14.zip

[LLv34_Temuri]

Here is ours. In missionReport(2018-03-15_09-50-12)[21].txt a pilot spawns in.

 

Looks like the log format has changed from:

"...INAIR:2 PARENT:-1 PAYLOAD:0 FUEL:1.000 SKIN: WM:1"

 

to:
"...INAIR:2 PARENT:-1 ISPL:1 ISTSTART:1"

missionlogs.zip

[=FB=Vaal]

The patch will be tonight.

[SAS_Storebror]

Thanks in advance 

[LLv34_Temuri]

Updated to the newest version, got error:

Traceback (most recent call last):
  File "manage.py", line 7, in <module>
    execute_from_command_line(sys.argv)
  File "C:\Ohjelmat\IL2-BoS-stats\.venv\lib\site-packages\django\core\management\__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "C:\Ohjelmat\IL2-BoS-stats\.venv\lib\site-packages\django\core\management\__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "C:\Ohjelmat\IL2-BoS-stats\.venv\lib\site-packages\django\core\management\base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "C:\Ohjelmat\IL2-BoS-stats\.venv\lib\site-packages\django\core\management\base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "C:\Ohjelmat\IL2-BoS-stats\src\stats\management\commands\stats_whore.py", line 16, in handle
    stats_whore.main()
  File "C:\Ohjelmat\IL2-BoS-stats\src\stats\stats_whore.py", line 62, in main
    stats_whore(m_report_file=m_report_file)
  File "C:\Ohjelmat\Python\Python35-32\lib\contextlib.py", line 30, in inner
    return func(*args, **kwds)
  File "C:\Ohjelmat\IL2-BoS-stats\src\stats\stats_whore.py", line 159, in stats_whore
    m_report.processing(files=m_report_files)
  File "C:\Ohjelmat\IL2-BoS-stats\src\mission_report\report.py", line 109, in processing
    self.events_handlers[atype_id](**data)
  File "C:\Ohjelmat\IL2-BoS-stats\src\mission_report\report.py", line 315, in event_game_object
    country_id=country_id, coal_id=coal_id, parent_id=parent_id)
  File "C:\Ohjelmat\IL2-BoS-stats\src\mission_report\report.py", line 421, in __init__
    obj = mission.objects[self.log_name]
KeyError: 'botgunner_pe2s87rear'

This time I don't know which log file might contain the reason, so here they are all.

missionReports.zip

[=FB=Vaal]

ohh hope this is the last

replace objects.csv and run the update

objects.zip

[LLv34_Temuri]

7 minutes ago, =FB=Vaal said:

ohh hope this is the last

replace objects.csv and run the update

objects.zip

It's working now. Thank you!

[=FB=Vaal]

once again

objects.zip

[WWSitttingDuck]

Vaal, do we have to run both of these updates, or just the last one?

[SAS_Storebror]

Thanks for the updates @=FB=Vaal, all is looking good now.

@WW_SittingDuck: You only need the last objects.zip or, alternatively, download the full installation file again and run the updater.

 

Cheers!

Mike

[WWSitttingDuck]

Ahh,,thanks for that info.  Had not realized a new version had come out so quick....back to up and running.

 

 

[KG200_Volker]

Vall do you think that there is a possibility for coop logs parsed by your program? I closed the readstats and put the logs in the logs folder and then I run it again but I get an error. I understand that the log is different but I was thinking maybe you can do it?

[=FB=Vaal]

What error? Attach the archive with the log files, I'll look.

[KG200_Volker]

1 hour ago, =FB=Vaal said:

What error? Attach the archive with the log files, I'll look.

 

OK I ll do it when I get home.

[KG200_Volker]

False alarm! Probably I did something wrong, I flew a Coop mission and then copied the files to my second pc where Dserver and site/stats are in the txt folder. It was parsed

 

http://kg200.net:8000/en/mission/1795/

 

But now I have a request IF possible.

It will help a lot for the coop missions which might get restarted due to an accident or a pilots disco etc etc.

 

How can we make the parser to ignore or not parse missions that lasted for example less than 10 minutes?

 

Edited March 24, 2018 by KG200_Volker

[KG200_Volker]

Hmmmm, in the end there is an error. I flew the aftermentioned mission alone and it was parsed, but when more people joined I can’t post it.

 

I hosted the mission from my computer through IL2 coop create server option, not from a dedicated server. (I noticed that while we where on the plane selection screen the server creates log files even before the mission start, I don’t know if it matters I am trying to give as much info as possible.)

 

When the mission ended (all pilots exited) I took the txt files in to the computer that runs the site and stats and tuned the collect stats. It gives an error. I uploaded the mission txt files that I used and the error that I get from the parser here:

 

https://www.dropbox.com/s/6q48onki8p4fyt0/Error coo

 

If you need more info ask me

 

 

[=FB=Vaal]

For some reason, you have more than one open tour in the database. All tours except one must be ended. This can be done through the admin panel.

[KG200_Volker]

Ahhhhhh thank you! Indeed I didn’t ended the previous one my bad!