Control of Cockpit Buttons using leap motion (Your own hands)

[kelemenlajos]

Greetings All,

 

About a month ago i came across this vid:

 

I wanted to recreate this, because i'm very low on buttons on my hotas. I poured 2 afternoons into this and here is what i got:

- i got a working leap motion

- i got working drivers for SteamVR
- i got no control over other VR controllers right now (you will see why this is a problem)

- i somewhat control over hand gestures that are translated to vive controller actions

- i got a working in game working buttons

- i can interact with the buttons with my left hand (intended limit)

 

If you want to recreate this, you will need the following:

- buy a leap motion (if you don't have one). It's fairly cheap. It's got it's limitation, but in VR it can be very useful (https://www.ebay.com/sch/i.html?_from=R40&_trksid=m570.l1313&_nkw=Leap+Motion+LM-010&_sacat=0 )

- buy a decent USB cable for it, you will need the same lenght as your VR headset cable e2e (this way you will have +1m for the leap) (the best on is what is good for oculus rift sensor).

- Get a leap motion adapter for your headset. IMPORTANT: don't fix the leap motion DIRECTLY to your headset, it will overheat, and wont function right! I'm using this one for CV1: https://www.thingiverse.com/thing:1554303 and it's overheating about 2 hours gaming.

- download this: https://github.com/cbuchner1/driver_leap/releases (don't install yet)

- download both x86,x64 drivers from https://github.com/SDraw/driver_leap/releases (don't install yet)

- exit SteamVR (if it's running)
- exit Steam (if it's running)

- install leap motion Orion Beta: https://developer.leapmotion.com/orion

- RECALIBRATE leap motion with leap motion controller troubleshooter, until you get 90 or higher score

- TEST leap motion with leap motion controller visualizer. (It's should also work in Oculus VR, not sure about SteamVR). If it's not working, don't proceed, because you should debug this before the next step.
- you should see something like this:

 

- Install the drivers based on this: https://github.com/SDraw/driver_leap (install both 32/64 bit!)

- Open steamvr.vrsettings in Steam/config folder and add line in "steamvr" section:
"activateMultipleDrivers": true, of it's not present

- DON'T add the driver via vrpathreg (for now, it should be automatic!)

- Start Steam
- Start SteamVR
- You should see 2 green C icons blinking. If you put your hands before the leapmotion about 0.5m away, the blinking should stop and the C icons should be stable green. If this is working, then SteamVR is seeing your hands, but it's likely that you can't use them as controlller right now. IF this isnt the case, then check your steam logs in your steam dir.

- If everything is working then stop SteamVR

- search for the control_config.xml file on the SteamVR subdirectory (???\Steam\steamapps\common\SteamVR\drivers\leap\bin\Win64\) and if youre using your right hand for the stick, then modify lt like this:

<config>
    <param name="menu" value="false"/>
    <param name="appMenu" value="true"/>
    <param name="trigger" value="true"/>
    <param name="grip" value="true"/>
    
    <param name="touchpad" value="false"/>
    <param name="touchpadTouch" value="false"/>
    <param name="touchpadPress" value="false"/>
    <param name="touchpadAxes" value="false"/>
    
    <param name="gripOffsetX" value="0.0"/>
    <param name="gripOffsetY" value="0.0"/>
    <param name="gripOffsetZ" value="0.0"/>
    
    <param name="rightEnabled" value="false"/>
    <param name="leftEnabled" value="true"/>
</config>

This will turn off your right hand (not needed) and turn off unneeded features to have it more stable.

BTW turning off the touchpad will render the the Three-Way Switch and the Multi-Position Switch useless 

- Download, and unpack this: https://github.com/BManx2000/VirtualButtonBox/releases

- Read the wiki https://github.com/BManx2000/VirtualButtonBox/wiki

- create a proof of concept button setup with Profiler.exe

- Start up Virtual Button Box.exe

- Start up IL-2 BOS in VR (Important, if IL-2 is not the active window, then the control inputs wont work, not only the leap motion inputs, your hotas also!)

- Test it. You should get something similar:

 

If anyone can improve this any further, feel free to do so, and share it here.

I think right now these are the most important issues:

- remap the leap controlling in the driver somehow (i got a hint about that, if i can recreate and it's working i will share it here...)

- replace the controller with an actual hand (i havent looked into this, but i think there should be a way to overwrite the controller model in steamVR )

- Somehow share the settings with every IL-2 VR user. For now if there is no better method we should share the configs here in the forum. (The config is in ?:\Users\<USERNAME>\Documents\Virtual Button Box\)

- And the most important part: recreate the cockpits. Right now i think there is only one method to do this: create the buttons and place them to their places in VR and after that lock them in the profiler.

(also sorry for my limited english, i tried my best)

 

[BubiHUN]

Looks like it is very useful for VR users!

[kelemenlajos]

I mashed up a quick JU 87D-3 profile, you can find it in the attached zip file.

You need to unpack it to your ?:\Users\<username>\Documents\Virtual Button Box\ folder.

I think you should also modify the profile.json file, because it's pointing to my virtual button box directory. Just update this part to match your folder:

"directory":"E:\\Users\\kelem\\Documents\\Virtual Button Box\\JU 87D-3"

These buttons are mapped:

- Attack Siren

- Bomb bay doors toggle

- Canopy

- Cockpit lamps

- Landing lights

- Navigation lights

- Tail wheel

- Water Radiator

 

I also included the demo button set as default.

 

You can also use the buttons with vive or rift controllers, but who would do such a thing, if leap motion is more awesome?

 

Many thanks to -[HRAF]BubiHUN and -[HRAF]Amergin to help me out

 

All buttons are mapped to the stock key, based on this, thanks Arfsix for your work! 

 

 

Virtual_Button_Box_-_IL2 profile_0.1.zip

[kelemenlajos]

I found a possible solution for the overheating.

I will re design this holder to have a shield between the leap and the headset not touching any of them:

https://www.thingiverse.com/thing:2773859

[kelemenlajos]

this is the redesigned dock:

https://www.thingiverse.com/thing:3628059

 

It has more airflow than my previous dock (https://www.thingiverse.com/thing:1554303), not by much, but i tested it with a 2 hour session and the LM doesn't get hot as with the previous dock.

 

[=RS=Funkie]

Hi,

I was attempting to setup the Virtual ButtonBox with my leap motion controller in VR. but I've run into an issue.
I'm hoping someone can help me here.

I have the Leap Motion plugged in, and the VR viewer works and detects my hands well.

I have copied the driver over to the SteamVR>Drivers>Leap folder. (both win32 and win64 are there)

"activateMultipleDrivers" true, was already in the SteamVR Config file.

When I run SteamVR (beta 1.10.31) - I see greyed out icons for 2 controllers and a base station. But moving my hands in front of the Leap doesn't recognise them, or show them as active in the SteamVR window.

 

However, Virtual Button Box is set up and working well. I just find it a pain to use the rift touch controllers to press buttons.

 

 

[DD_Crash]

Could this work? https://forums.eagle.ru/showthread.php?t=218861

[dburne]

10 minutes ago, DD_Crash said:

Could this work? https://forums.eagle.ru/showthread.php?t=218861

 

That would require switches and button in the cockpit to be manipulated.

Unless I have missed something really big, IL-2 GB does not have that.

[THERION]

12 minutes ago, dburne said:

 

That would require switches and button in the cockpit to be manipulated.

Unless I have missed something really big, IL-2 GB does not have that.

 

This would mean - click-cockpits - thanks god they don't have it in IL-2 GB!

[VR-DriftaholiC]

Save your money on finger trackers, we're not far away from headsets like the Index and maybe WMR headsets being able to use their front facing cameras to track hands and be able to turn it into inputs in game. They already have hand tracking on the Quest in beta. Just need clicky cockpits..