Programmatic manipulation of .eng, .fra (etc) subtitle files

[Alonzo]

Hi everyone,

 

I want to write a weather randomizer for my server. The idea is to take a .Mission, pick a new time of day and cloud cover, then update the .Mission file and associated mission description. I'm good with updating the .Mission file, but so far I am stuck on the subtitle files.

 

If I look at MyMission.eng in Notepad++, it is apparently encoded as "UCS 2-LE BOM". This is a wide-character encoding due (I think) to the need to support languages other than English. I am loading the file using a Perl script, and it looks like it's reading okay and I can manipulate the strings in memory. But I can't figure out how to write a file that is in the correct format. The closest I have come ends up having (apparently) extra newlines in it. Or Perl just writes stuff that Notepad++ interprets as garbage characters.

 

So. Is anyone successfully manipulating or creating subtitle files, and if so, how? Can I just write the file using UTF-8? ANSI? Must it be UCS 2-LE BOM for it to work? Any hints appreciated.

[Alonzo]

Okay, I might have an approach for this. It's a bit of a hack, but here's how I'm doing the processing.

 

Use iconv to convert to UTF-8.

iconv -f UTF-16LE -t UTF-8 \"$subtitle_file\" > \"$subtitle_file.utf8\"

Load and process the file using Perl. Load and save it using an explicit UTF-8 encoding.

open(IN, "<:encoding(UTF-8)", "$subtitle_file.utf8")

# Do some processing

open(OUT, '>:encoding(UTF-8)', "$subtitle_file.utf8-out")

Finally, use iconv to convert back to UTF-16LE.

iconv -f UTF-8 -t UTF-16LE \"$subtitle_file.utf8-out\" > \"$subtitle_file\"

This seems to produce a file that the mission resaver is happy with, so I assume that dserver will also be happy with it.

 

Might be a processing limitation for Perl or something, I'm not sure why I need to jump through all these hoops. Oh well.

[Habu]

I can't help you on that point. it's out of my skill.

[[Pb]Vapor]

I have used python for most of my work playing with game files (lua's and what not).   I ran into some of the same problems you are describing.  It's not totally clear what encoding they used, or at least it doesn't seem to be a common encoding.  I remember trying latin-1, ascii, utf-8 and cp1251.  

 

I lucked out and for my use I could open with a generic encoding format in python (ignoring errors) and save as utf-8.  Only the comments in the code were messed up.  

 

It might be worth checking out cp1251 and others that are Cyrillic friendly (Russian).  It appears like many of the code comments/notation for IL2 are in Russian (no big surprise).  

[Alonzo]

It seems like the file format needs to be UTF-16 little-endian with byte order marker. At least that's how Notepad++ detects it. I used the command-line tool iconv to work with it in UTF-8, then convert back to UTF-16 once I'm done.

[Alonzo]

Old thread but in case anyone finds it, to write these files from C# use the following incantation to select Unicode:

 

using var writer = new StreamWriter($"{directory}/{baseName}.{extension}", false, Encoding.Unicode);

[Deacon352nd]

Wouldn’t it be easier to just go into the ME and just change the time of day and the weather conditions?  You wouldn’t need to change anything else in the mission and it would take you about a minute to do what you want. 

[Cynic_Al]

14 minutes ago, Deacon352nd said:

Wouldn’t it be easier to just go into the ME and just change the time of day and the weather conditions?  You wouldn’t need to change anything else in the mission and it would take you about a minute to do what you want. 

 

Unless I have completely misunderstood the OP, the operative phrase is: "a weather randomizer ".  If the weather conditions change, so must the description of the weather.

 

I recall someone attempting this in RoF; in that case I believe the idea was to have a set of prepared files for each mission, each configured with different weather and associated description.  Prior to each mission to be loaded in the rotation, a random set would be copied-over to overwrite the files of the following mission.  The only challenge then was for the app to detect the currently loaded mission, in order to know which mission to randomise next. 

[Alonzo]

3 hours ago, Cynic_Al said:

Unless I have completely misunderstood the OP, the operative phrase is: "a weather randomizer ".  If the weather conditions change, so must the description of the weather.

 

4 hours ago, Deacon352nd said:

Wouldn’t it be easier to just go into the ME and just change the time of day and the weather conditions?  You wouldn’t need to change anything else in the mission and it would take you about a minute to do what you want. 

 

Yep. This is for Combat Box, one of the multiplayer servers that runs missions 24x7. We need an automatic way to do things like changing the weather and time of day. Some of the servers have pretty advanced mission automation tech these days. TAW and Finnish create a dynamic campaign with moving front line and automated placement of targets. Combat Box has an Ai voice bot that understands the state of the mission and can create mission assignments, vector fighters to cover, and give you a heading home.

[SYN_Vander]

Sorry, must have missed this. In my python code I use 

encoding='utf_16'

to write the .eng file and that works. Took me a while as well to work that out. For the .mission file I use the default utf 8.

Edited April 19, 2022 by SYN_Vander