Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebXR AR hit test support #1926

Merged
merged 16 commits into from
Mar 22, 2020
Merged

WebXR AR hit test support #1926

merged 16 commits into from
Mar 22, 2020

Conversation

Maksims
Copy link
Contributor

@Maksims Maksims commented Mar 20, 2020

Implementation of WebXR Hit Test Module. Bear in mind, it is still in Draft. So is a subject to change.

New APIs:

app.xr.hitTest // instance of pc.XrHitTest

// pc.XrHitTest
app.xr.hitTest.supported // True if Hit Test Module is available
app.xr.hitTest.sources // list of pc.XrHitTestSource objects
app.xr.hitTest.start(startOptions); // starts hit test source with provided options. If profile string provided, will be created for input sources, otherwise uses spaceType (or default pc.XRSPACE_VIEWER).
var startOptions = { // all options are optional.
    spaceType: pc.XRSPACE_*, // reference space for hit test
    profile: string, // or profile for input source
    entitiTypes: [ pc.XRTRACKABLE_* .. ], // types of AR subsystem entities to make hit tests against
    offsetRay: pc.Ray, // offset ray
    callback: function (err, hitTestSource) { } // called when hit source been created or failed. 
};
app.xr.hitTest.on('add', function (hitTestSource) { }); // called when new hit test source is added
app.xr.hitTest.on('remove', function (hitTestSource) { }); // called when hit test source is removed
app.xr.hitTest.on('result', function (hitTestSource, position, rotation, inputSource) { }); // called when hit test source provides hit result, with position and rotation. Also can provide inputSource if created using profile string
app.xr.hitTest.on('error', function (err) { }); // called when hit test source failed to start or other related issues

// pc.XrHitTestSource
hitTestSource.remove(); // removes hit test source
hitTestSource.on('remove', function () { }); // called when hit test source is removed
hitTestSource.on('result', function (position, rotation, inputSource) { }); // called when hit test source provides hit result, with position and rotation. Also can provide inputSource if created using profile string

// pc.XrInputSource
inputSource.hitTestSources // list of pc.XrHitTestSource objects created against this input source
inputSource.hitTestStart(startOptions); // same as XrHitTest.start, but without spaceType and profile.
inputSource.on('hittest:add', function () { }); // called when related hit test source is added
inputSource.on('hittest:remove', function () { }); // called when related hit test source is removed
inputSource.on('hittest:result', function (hitTestSource, position, rotation) { }); // called when related hit test source provides hit result, with position and rotation.

// pc
pc.XRTRACKABLE_POINT // Point - indicates that the hit test results will be computed based on the feature points detected by the underlying Augmented Reality system.
pc.XRTRACKABLE_PLANE // Plane - indicates that the hit test results will be computed based on the planes detected by the underlying Augmented Reality system.
pc.XRTRACKABLE_MESH // Mesh - indicates that the hit test results will be computed based on the meshes detected by the underlying Augmented Reality system.

Basic example of painting grass on the floor by touching mobile screen in AR session:

var HitTest = pc.createScript('hitTest');

HitTest.attributes.add('grass', { type: 'entity' });

// initialize code called once per entity
HitTest.prototype.initialize = function() {
    var self = this;
    var lastGrassCreated = 0;
    
    // in AR touch on screen adds new input source every time
    this.app.xr.input.on('add', function(inputSource) {
        
        // start hit test for added input source
        inputSource.hitTestStart({
            // when hit test source created
            callback: function(err, hitTestSource) {
                if (err) return;
                
                // if result is reported
                hitTestSource.on('result', function(position, rotation) {
                    // clone grass not too often
                    var now = Date.now();
                    if ((now - lastGrassCreated) < 50) return;
                    lastGrassCreated = now;
                    
                    // clone grass
                    var ent = self.grass.clone();
                    
                    // position on picked location
                    ent.setPosition(position);
                    
                    // rotate randomly
                    ent.setEulerAngles(0, Math.random() * 360, 0);
                    ent.reparent(self.app.scene.root);
                });
            }
        });
    });
};

I confirm I have signed the Contributor License Agreement.

src/xr/xr-hit-test.js Outdated Show resolved Hide resolved
Copy link
Contributor

@willeastcott willeastcott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've made a few fixes to the JSDoc blocks (mainly fixing all the missing @link tags). Can you check out my comments on function signatures please. Thanks!

@willeastcott
Copy link
Contributor

This is great! Can you also include an engine example that exercises this new functionality please?

@Maksims
Copy link
Contributor Author

Maksims commented Mar 20, 2020

I've changed function, so now there is only one start. And it accepts options object, which can have various optional fields.
Simplified a bit API, now it is app.xr.hitTest.sources.
Added an simple example, where target is placed on hit test result position and rotated, based on viewing direction.

@willeastcott
Copy link
Contributor

OK @Maksims, I think this PR is now in pretty good shape. You might wanna edit the description to remove the comment about it being in draft now. 😄

For some reason, I'm still not that keen about the hitTestStart function signature. I mean, won't there always be a callback? So couldn't it be:

        inputSource.hitTestStart(function (err, hitTestSource) {

        }, options);

Instead of:

        inputSource.hitTestStart({
            // when hit test source created
            callback: function(err, hitTestSource) {
            }
        });

@Maksims
Copy link
Contributor Author

Maksims commented Mar 21, 2020

For some reason, I'm still not that keen about the hitTestStart function signature. I mean, won't there always be a callback? So couldn't it be:

        inputSource.hitTestStart(function (err, hitTestSource) {

        }, options);

Instead of:

        inputSource.hitTestStart({
            // when hit test source created
            callback: function(err, hitTestSource) {
            }
        });

"No callback" is very valid case, where you would subscribe to results event, instead of on hitTestSource specifically.
Easy example would be: create 4x4 hit test sources from Viewer reference space, with small offsets - it will provide hit tests for all of them, under results event, instead of subscribing to each individually.

And if you insist on adding callback as separate argument, then usually it is a last argument (most APIs do that way).
Let me know what is preferred:

  1. start(options);
  2. start(options, callback);

Additionally could be a Promise - but you know my personal stance to them ;)

@willeastcott
Copy link
Contributor

willeastcott commented Mar 22, 2020

Yeah, let's not go with Promises. 😆

Based on what you say, I think start(options); probably is best.

So I should go ahead and merge?

For some reason, CircleCI failed to run the tests.... I wonder why.

@Maksims
Copy link
Contributor Author

Maksims commented Mar 22, 2020

So I should go ahead and merge?

Yes please.

For some reason, CircleCI failed to run the tests.... I wonder why.

Indeed, not sure why.

@willeastcott willeastcott changed the title Ar hit test WebXR AR hit test support Mar 22, 2020
@willeastcott willeastcott merged commit 11700a6 into playcanvas:master Mar 22, 2020
TheJonRobinson added a commit to Mojiworks/playcanvas-engine that referenced this pull request Mar 30, 2020
* stable: (48 commits)
  [RELEASE] v1.26.0
  [FIX] null access when pushing SpectorJS marker without camera (playcanvas#1933)
  [FIX] pc.XrHitTest when WebXR not available
  Update package-lock.json
  Downgrade mocha to 6.2.2
  WebXR AR hit test support (playcanvas#1926)
  [ImgBot] Optimize images (playcanvas#1927)
  [DOCS] Make anim API private (playcanvas#1924)
  Update NPM dependencies.
  [FIX] Update some example paths
  Update to latest JSDoc template
  [FIX] Updating of particle systems local bounds
  Add pc.drawTexture to render texture to target using a shader (playcanvas#1920)
  Optimize Grab-Pass for WebGL 2 and fix when using anti-aliased RenderTarget (playcanvas#1918)
  [DOCS] Document drawQuadWithShader and copyRenderTarget (playcanvas#1919)
  [FIX] Mouse wheel events on elements (playcanvas#1916)
  glTF 2.0 support (playcanvas#1904)
  Update to latest JSDoc template.
  Add support for ES6 script classes (playcanvas#1908)
  Particle system randomize sprite animations (playcanvas#1911)
  ...
@Maksims Maksims deleted the ar-hit-test branch April 4, 2020 13:26
ellthompson pushed a commit to ellthompson/engine that referenced this pull request Apr 20, 2020
* webxr hit test

* lint fixes

* add missing error event

* fix build:all

* tipes -> types

* tipes -> types

* Fix links

* Fix links

* Fix links

* simplify hit-test start function

* small fixes

* XrHitTest.hitTestSources > XrHitTest.sources

* ar hit test example

* fix merge

Co-authored-by: Will Eastcott <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants