Skip to content

Bleutooth controllers and leap motion . #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 52 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
182e652
remove most hydra specific folders and project files
cbuchner1 Apr 23, 2016
c2d4b6a
commit current state of the code
cbuchner1 Apr 23, 2016
6aadea4
Update README.md
cbuchner1 Apr 23, 2016
e24bdd2
Formatting: replace tabs with spaces
cbuchner1 Apr 23, 2016
2225ddc
Merge branch 'master' of https://github.com/cbuchner1/driver_hydra
cbuchner1 Apr 23, 2016
5cd94f0
update README
cbuchner1 Apr 23, 2016
c8fdba6
make the controllers disappear from screen when the Leap Motion loses…
cbuchner1 Apr 23, 2016
06d6981
Remove unneeded resource files. Change resting pose of hands to be pa…
cbuchner1 Apr 24, 2016
f1c8ba4
Add some experimental finger gesture recognition that maps to trigger…
cbuchner1 Apr 24, 2016
62c39a6
Update README
cbuchner1 Apr 24, 2016
b114fd7
Update README.md
cbuchner1 Apr 24, 2016
d73e11f
add an installer project (needs the Microsoft Visual Studio 2015 Inst…
cbuchner1 Apr 24, 2016
cb66ed8
Merge branch 'master' of https://github.com/cbuchner1/driver_hydra
cbuchner1 Apr 24, 2016
e9d2644
this isn't GNU GPL. So I use Valve's original LICENSE instead.
cbuchner1 Apr 24, 2016
6fed0a5
more debug information printed to log file, installer project fixes.
cbuchner1 Apr 24, 2016
46ab3ad
fix small syntax problem in streamvr.vrsettings file
cbuchner1 Apr 24, 2016
48462a6
add a bit of driver configurability (render models and rip angle offs…
cbuchner1 Apr 24, 2016
5d07f58
Update README.md
cbuchner1 Apr 24, 2016
65eb039
Update README.md
cbuchner1 Apr 24, 2016
2e5299b
Use the new GestureMatcher class to classify and quantify hand gestur…
cbuchner1 Apr 27, 2016
eebc076
Merge branch 'master' of https://github.com/cbuchner1/driver_hydra
cbuchner1 Apr 27, 2016
474bb14
add ILY, FlippingTheBird, Victory gestures.
cbuchner1 Apr 27, 2016
107d9bb
Look, Ma! New Gestures!
cbuchner1 Apr 28, 2016
46e7c4d
Add a gesture checker application that prints currently detected gest…
cbuchner1 Apr 28, 2016
5d68414
update README
cbuchner1 Apr 28, 2016
eb3fc54
update settings file
cbuchner1 Apr 28, 2016
9107d81
small updates for README and installer
cbuchner1 Apr 28, 2016
9832e25
Update README.md
cbuchner1 Apr 28, 2016
6296470
Update README.md
cbuchner1 Apr 28, 2016
76c221c
Update README.md
cbuchner1 Apr 28, 2016
f09e247
Update README.md
cbuchner1 Apr 28, 2016
619e8e0
Update README.md
cbuchner1 Apr 28, 2016
738faef
Update README.md
cbuchner1 Apr 28, 2016
6d47a03
Add touchpad X/Y axis emulation using the index finger of the other h…
cbuchner1 May 1, 2016
cbcb86f
small refactoring and update README.md
cbuchner1 May 1, 2016
fbd9c6f
Update README.md
cbuchner1 May 12, 2016
2e03fd5
Update README.md
cbuchner1 May 12, 2016
c9ad409
Update README.md
cbuchner1 May 12, 2016
066d7c9
Update README.md
cbuchner1 May 12, 2016
79bae9f
Update README.md
cbuchner1 May 13, 2016
18fc8ea
Add a config tool that registers the driver and cleanly merges/unmerg…
cbuchner1 May 21, 2016
7739d6b
Merge branch 'master' of https://github.com/cbuchner1/driver_hydra
cbuchner1 May 21, 2016
6367d2e
Change target platform of installer to 64 bit and use 64 bit config_t…
cbuchner1 May 21, 2016
fd60a10
Hotfix for config tool. It was picking up the wrong driver path from …
cbuchner1 May 21, 2016
484ee76
escape spaces in SteamVRPathReg() function, also look for OpenVR dire…
cbuchner1 Jun 4, 2016
6959ea5
add dependency to Visual C++ 2015 redistributables to installer project
cbuchner1 Jun 4, 2016
c52c935
finally, the bug fix that should make the revamped install system wor…
cbuchner1 Jun 5, 2016
7bf96c3
revert change to remove "leap" subdirectory in installation location.…
cbuchner1 Jun 6, 2016
e8da7d3
update to OpenVR SDK 1.0.0
cbuchner1 Jun 6, 2016
33ce811
Update README.md
cbuchner1 Jun 6, 2016
3de6a8a
config tool / installer fix. Proper escaping of command in exec() sub…
cbuchner1 Jun 10, 2016
56b3783
Merge branch 'master' of https://github.com/cbuchner1/driver_hydra
cbuchner1 Jun 10, 2016
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions Paths.props
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ImportGroup Label="PropertySheets" />
<PropertyGroup Label="UserMacros">
<SteamVRRuntimeDir>C:\Program Files (x86)\steam\steamapps\common\SteamVR</SteamVRRuntimeDir>
<OpenVRDir>d:\dev\openvr_hydra\sdk_20160123</OpenVRDir>
<SixenseSDKDir>C:\Program Files (x86)\steam\steamapps\common\Sixense SDK\SixenseSDK</SixenseSDKDir>
<InstallDir>D:\dev\hydra</InstallDir>
<SteamVRRuntimeDir>D:\Steam\steamapps\common\SteamVR</SteamVRRuntimeDir>
<OpenVRDir>C:\Users\Christian Buchner\Documents\Visual Studio 2015\Projects\Leap\openvr-1.0.0</OpenVRDir>
<LeapSDKDir>C:\Users\Christian Buchner\Documents\Visual Studio 2015\Projects\Leap\LeapSDK</LeapSDKDir>
<InstallDir>D:\Steam\steamapps\common\SteamVR\drivers\leap</InstallDir>
</PropertyGroup>
<PropertyGroup />
<ItemDefinitionGroup />
Expand All @@ -16,8 +16,8 @@
<BuildMacro Include="OpenVRDir">
<Value>$(OpenVRDir)</Value>
</BuildMacro>
<BuildMacro Include="SixenseSDKDir">
<Value>$(SixenseSDKDir)</Value>
<BuildMacro Include="LeapSDKDir">
<Value>$(LeapSDKDir)</Value>
</BuildMacro>
<BuildMacro Include="InstallDir">
<Value>$(InstallDir)</Value>
Expand Down
133 changes: 119 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,119 @@
# Razer Hydra Driver for SteamVR
# Leap Motion Driver for SteamVR

## Building
## Installation of this driver.

- Get the Leap Motion Orion Beta runtimes https://developer.leapmotion.com/get-started
- Install the Visual C++ 2015 Update 2 redistributables (32 and 64 bits) https://www.microsoft.com/en-us/download/details.aspx?id=51682
- Download the most recent zip file from the Releases section of this project and run the contained exe setup program https://github.com/cbuchner1/driver_leap/releases

Start SteamVR to see if two additional controllers show up (they should be blinking if your hands are
not in the field of view of the Leap Motion, solid otherwise).

### Troubleshooting

If you experience frequent crashes of SteamVR on exit and this bothers you, uninstall my driver. I will try to fix this ASAP, but at the moment I have no clue why SteamVR is crashing.


## Note about WORK IN PROGRESS

You're seeing an early version of this software. I've got positional tracking established now as well as hand pose tracking. Some experimental mappings of hand gestures to triggers and buttons were added:

Trigger:
- bending of the index finger maps to the trigger button, like you would fire a gun.

Grip:
- clenching the middle, ring, pinky finger to a fist maps to the grabbing buttons

Trackpad:
- the thumbpress gesture (just point the thumb in the direction of your palm) touches and clicks the trackpad, depending on the intensity of your gesture.
- pointing the index finger towards the are of the other hand's palm will emulate the touchpad. To press the touchpad in the desired position use the tumbpress gesture simultaneously while pointing.

Menu buttons:
- Flat hand held in front of you, palm towards face is used for application menu button
- the Timeout pose (as used in sports), registers as the system menu button


Swapped hands?

when SteamVR confuses the left and right controller hands (which will be indicated by little hand icons shown near the bottom of the displayed Wand controllers), simply cross your hands the first time you bring them into view after starting SteamVR. This reverses the hand assignment and can improve the gaming experience for example in Audioshield.




I am working on allowing to freely map gestures to buttons in the steamvr.vrsettings config file in your Steam\config folder. However note that the "leap_gestures" section is currently not parsed yet. It's merely a sign of things to come.

## Supported gestures

There are other gestures detected currently, but not mapped to buttons. If you want to try these out, click on the application "gesture_checker.exe" in the directory C:\Program Files (x86)\SteamVR Leap Motion driver\leap\bin\Win32

Then I also recommend that you simultaneously bring up your Leap Motion's settings and from there start the diagnostic visualizer (the windowed version, not the VR one). Press "v" once to switch it to headmount optimized mode.

Now pull both windows side by side and bring a hand into view. The command prompt running the gesture_checker program should output a series of numbers next to the names of the gestures. A "1.0" means confidential detection, a "0.0" means no detection.

You can practise some gestures this way and also cross-check your pose in the Leap Motion diagnostic visualizer against the detection confidence.

// Finger gestures (these would not throw your hand's orientation off much)
TriggerFinger, // bend your index finger as if pulling a trigger
LowerFist, // grab with your middle, ring, pinky fingers
Pinch, // pinch with your thumb and index fingers
Thumbpress, // point the thumb towards the direction of your pinky

// Hand gestures (these would significantly change the orientation of your hand)
FlippingTheBird, // flip someone off with your middle finger
ILY, // pinky and index finger extended, middle and ring bent
Victory, // V shape with your index, middle fingers, other fingers curled
FlatHandPalmUp, // flat hand, palm points upwards (relative to alignment of Leap!)
FlatHandPalmDown, // flat hand, palm points downwards (relative to alignment of Leap!)
FlatHandPalmAway, // flat hand, palm points away from self (relative to alignment of Leap!)
FlatHandPalmTowards, // flat hand, palm points towards self (relative to alignment of Leap!)
ThumbUp, // thumb points up, remaining fingers form a fist
ThumbInward, // thumb points towards the left for the right hand and vice versa

// Two handed gestures
Timeout, // both Hands form a T shape, signals a Timeout in sports
TouchpadAxisX, // Touchpad emulation: index finger of other hand points towards palm
TouchpadAxisY, // Touchpad emulation: index finger of other hand points towards palm


### Games/Experiences that work mostly

- the Blu (all three stages)
- Irrational Exuberance: Prologue
- the Rose and I
- The Lab (some experiences work, others are tricky)
- Final Approach
- Audioshield: somehow the controllers are swapped? Control is tricky and not very precise. Semi-playable though.

### Games/Experiences that are starting but not quite playable yet.

- Tilt Brush: starts and you can start doing things, but there is lack of complete trackpad support in my driver.
- Brookhaven Experiment: tracking only works while SteamVR window is in focus. Why? Gun in right hand needs a 60 degree uptilt angle (define this in steamvr.vrsettings config file in Steam config folder). Trigger gesture detection is way to imprecise, you won't even survive the first wave of Zombies.


### Demos that won't work at all
- n/a


## Known Issues

I am seeing SteamVR Server crash on shutdown a lot. This could be related to my driver, but I have not yet found the root cause for the crash.

The Brookhaven experiment seems to steal focus from StreamVR, so that Steam does not get any position tracking. Clicking on the SteamVR window restores tracking, but mutes the audio on Brookhaven. Meh.

Some games work better when no grip angle is added to the controller pose, other games actually require a steep angle to be playable (Brookhaven, Audioshield). We may have to add a feature to chose the preferred default pose at runtime.

Tracking is not quite reliable to always detect my trigger gestures. I think we will have to integrate small handheld controllers like the Wiimote or the Playstation Move Navigation controller in the future.

I do not think I will be able to get animated hands into the 3D view, as the render model you can assign to each controller is mostly a static object. There are some JSON files to map joystick axes and triggers to animated parts of the displayed controller. But the fingers do not directly map to joystick axes directly and hence cannot be shown. Also not all games make use of SteamVR's internal controller visualization.


## Building from Sourcecode (Developers only)

### Install Dependencies

1. Install SteamVR. It is under "Tools" in everyone's Steam Library. steam://install/250820
2. Install "Sixense SDK for the Razer Hydra". It is also under "Tools". steam://install/42300
3. Fetch the OpenVR SDK 0.9.19 from https://github.com/ValveSoftware/openvr . That version SHA is f1ffbf4e92f383bdb453d58f9583c51a5ec350d9. Newer version should also work with minor changes.
2. Install "Leap Motion Orion SDK V3.1.2". https://developer.leapmotion.com/get-started
3. Fetch the OpenVR SDK 1.0.0 from https://github.com/ValveSoftware/openvr .

The solution and project files are for Visual Studio 2015.

Expand All @@ -18,7 +125,7 @@ Under "Property Manager" in Visual Studio, expand any of the configurations and

You will probably want to build Release x86. You can also build x64. The post-build step will install the binaries and copy the resources to the configured InstallDir and register that path with SteamVR.

## Using The Hydra Driver
## Preapring The Leap Motion Driver for use (Developers only)

After building, the InstallDir should be a complete binary distribution. To use it:

Expand All @@ -32,25 +139,23 @@ After building, the InstallDir should be a complete binary distribution. To use
}```
3. If you are trying to use the Hydra driver without an HMD, you might want to enable driver_null (no HMD) or set "requireHmd": false.

After starting SteamVR, you should see controllers blinking in the status window. Inside the HMD you will see instructions to get the Hydra coordinates aligned with your HMD. Once you do, the controllers will turn solid green.
After starting SteamVR, you should see controllers blinking in the status window until you move your hands into the field of view.

You can use "vrcmd" (with no arguments) to see a list of devices to verify things are working.
use "vrcmd" to verify things are loading:

```...
Driver hydra : 2 displays
Hydra (Serial number hydra0_controller0)
Hydra (Serial number hydra0_controller1)
Driver leap : 2 displays
leap (Serial number leap0_lefthand)
leap (Serial number leap0_righthand)
...
```

You can also use "vrcmd --pollposes" (followed by an index number to limit the output) to see things are working.
You can also use "vrcmd --pollposes" (followed by an index number to limit the output) to see if things are working.

## Licenses

The code in this distribution is distributed under the terms of the LICENSE file in the root directory.

The render models are based on work by Zoltan Erdokovy <[email protected]> with permission.

The compiled driver and the install directory use the Sixense SDK. That development kit has a license which requires copyright acknowledgement for distribution. See the EULA that is displayed when installing the Sixense SDK.

The compiled driver and the install directory use the Leap Motion Orion SDK. Use subject to the terms of the Leap Motion SDK Agreement available at
https://developer.leapmotion.com/sdk_agreement.
38 changes: 0 additions & 38 deletions driver_hydra.sln

This file was deleted.

68 changes: 68 additions & 0 deletions driver_leap.sln
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@

Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio 14
VisualStudioVersion = 14.0.25123.0
MinimumVisualStudioVersion = 10.0.40219.1
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "driver_leap", "drivers\driver_leap\driver_leap.vcxproj", "{52D3F16D-A7A5-4D6F-8F17-5E4B459A1440}"
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "leap_monitor", "tools\leap_monitor\leap_monitor.vcxproj", "{BC06AF9C-36D6-455A-B421-00A9635684AD}"
EndProject
Project("{54435603-DBB4-11D2-8724-00A0C9A8B90C}") = "leap_installer", "tools\leap_installer\leap_installer.vdproj", "{60517323-2772-4341-9161-56C776DC1840}"
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "gesture_checker", "tools\gesture_checker\gesture_checker.vcxproj", "{9C28E205-C4CD-43D4-91BC-4852D7A588EC}"
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "config_tool", "tools\config_tool\config_tool.vcxproj", "{FD2D3DBF-C82D-4333-9710-7DD2C51FA8E1}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|x64 = Debug|x64
Debug|x86 = Debug|x86
Release|x64 = Release|x64
Release|x86 = Release|x86
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{52D3F16D-A7A5-4D6F-8F17-5E4B459A1440}.Debug|x64.ActiveCfg = Debug|x64
{52D3F16D-A7A5-4D6F-8F17-5E4B459A1440}.Debug|x64.Build.0 = Debug|x64
{52D3F16D-A7A5-4D6F-8F17-5E4B459A1440}.Debug|x86.ActiveCfg = Debug|Win32
{52D3F16D-A7A5-4D6F-8F17-5E4B459A1440}.Debug|x86.Build.0 = Debug|Win32
{52D3F16D-A7A5-4D6F-8F17-5E4B459A1440}.Release|x64.ActiveCfg = Release|x64
{52D3F16D-A7A5-4D6F-8F17-5E4B459A1440}.Release|x64.Build.0 = Release|x64
{52D3F16D-A7A5-4D6F-8F17-5E4B459A1440}.Release|x86.ActiveCfg = Release|Win32
{52D3F16D-A7A5-4D6F-8F17-5E4B459A1440}.Release|x86.Build.0 = Release|Win32
{BC06AF9C-36D6-455A-B421-00A9635684AD}.Debug|x64.ActiveCfg = Debug|x64
{BC06AF9C-36D6-455A-B421-00A9635684AD}.Debug|x64.Build.0 = Debug|x64
{BC06AF9C-36D6-455A-B421-00A9635684AD}.Debug|x86.ActiveCfg = Debug|Win32
{BC06AF9C-36D6-455A-B421-00A9635684AD}.Debug|x86.Build.0 = Debug|Win32
{BC06AF9C-36D6-455A-B421-00A9635684AD}.Release|x64.ActiveCfg = Release|x64
{BC06AF9C-36D6-455A-B421-00A9635684AD}.Release|x64.Build.0 = Release|x64
{BC06AF9C-36D6-455A-B421-00A9635684AD}.Release|x86.ActiveCfg = Release|Win32
{BC06AF9C-36D6-455A-B421-00A9635684AD}.Release|x86.Build.0 = Release|Win32
{60517323-2772-4341-9161-56C776DC1840}.Debug|x64.ActiveCfg = Debug
{60517323-2772-4341-9161-56C776DC1840}.Debug|x64.Build.0 = Debug
{60517323-2772-4341-9161-56C776DC1840}.Debug|x86.ActiveCfg = Debug
{60517323-2772-4341-9161-56C776DC1840}.Debug|x86.Build.0 = Debug
{60517323-2772-4341-9161-56C776DC1840}.Release|x64.ActiveCfg = Release
{60517323-2772-4341-9161-56C776DC1840}.Release|x64.Build.0 = Release
{60517323-2772-4341-9161-56C776DC1840}.Release|x86.ActiveCfg = Release
{60517323-2772-4341-9161-56C776DC1840}.Release|x86.Build.0 = Release
{9C28E205-C4CD-43D4-91BC-4852D7A588EC}.Debug|x64.ActiveCfg = Debug|x64
{9C28E205-C4CD-43D4-91BC-4852D7A588EC}.Debug|x64.Build.0 = Debug|x64
{9C28E205-C4CD-43D4-91BC-4852D7A588EC}.Debug|x86.ActiveCfg = Debug|Win32
{9C28E205-C4CD-43D4-91BC-4852D7A588EC}.Debug|x86.Build.0 = Debug|Win32
{9C28E205-C4CD-43D4-91BC-4852D7A588EC}.Release|x64.ActiveCfg = Release|x64
{9C28E205-C4CD-43D4-91BC-4852D7A588EC}.Release|x64.Build.0 = Release|x64
{9C28E205-C4CD-43D4-91BC-4852D7A588EC}.Release|x86.ActiveCfg = Release|Win32
{9C28E205-C4CD-43D4-91BC-4852D7A588EC}.Release|x86.Build.0 = Release|Win32
{FD2D3DBF-C82D-4333-9710-7DD2C51FA8E1}.Debug|x64.ActiveCfg = Debug|x64
{FD2D3DBF-C82D-4333-9710-7DD2C51FA8E1}.Debug|x64.Build.0 = Debug|x64
{FD2D3DBF-C82D-4333-9710-7DD2C51FA8E1}.Debug|x86.ActiveCfg = Debug|Win32
{FD2D3DBF-C82D-4333-9710-7DD2C51FA8E1}.Debug|x86.Build.0 = Debug|Win32
{FD2D3DBF-C82D-4333-9710-7DD2C51FA8E1}.Release|x64.ActiveCfg = Release|x64
{FD2D3DBF-C82D-4333-9710-7DD2C51FA8E1}.Release|x64.Build.0 = Release|x64
{FD2D3DBF-C82D-4333-9710-7DD2C51FA8E1}.Release|x86.ActiveCfg = Release|Win32
{FD2D3DBF-C82D-4333-9710-7DD2C51FA8E1}.Release|x86.Build.0 = Release|Win32
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
EndGlobalSection
EndGlobal
Loading