Tuesday, 14 June 2022

Marker Detection (AR.js) in A-Frame Project

I have some problems to extend an existing a-frame project which includes hit-testing with the ar.js library and marker detection.

Link to the project: https://glitch.com/~ar-starter-kit

Goal:

I would like to use marker detection along with the existing hit-testing approach of the project where I can place objects in the near surrounding based on the cursor position.

For this I need to add ar.js to the project.

My Approach:

I simply added the script tag in the head section after the aframe script tag like this

<!-- the AFrame library -->
<script src="https://aframe.io/releases/1.3.0/aframe.min.js"></script>
<script src="https://raw.githack.com/AR-js-org/AR.js/3.4.0/aframe/build/aframe-ar.js">

Then I added arjs to the a-scene tag.

<a-scene
    arjs
    webxr="overlayElement:#dom-overlay;"
    background="color:skyblue;"
    reflection="directionalLight:#dirlight;"
    ar-hit-test="target:#my-objects;type:footprint;footprintDepth:0.2;"
    shadow="type: pcfsoft"
    gltf-model="dracoDecoderPath: https://cdn.jsdelivr.net/npm/three@0.129.0/examples/js/libs/draco/gltf/;"
ar-cursor raycaster="objects: #my-objects *"
>

The effect of this is that ar.js injects a video-element which runs by default. At first this cant be seen on the screen but if we remove background="color:skyblue;" from the a-scene you can see the camera stream is already running (a <video> element is injected with the id="arjs-video"). Here the marker detection is working but hit-testing and placing the objects is not. When I now click on the AR-button I get a black screen.

I resolved this by adding this code snippet:

window.addEventListener("arjs-video-loaded", (e) => {
  var videoEl = document.getElementById('arjs-video');
  // now get the steam 
  stream = videoEl.srcObject;
  // now get all tracks
  tracks = stream.getTracks();
  // now close each track by having forEach loop
  tracks.forEach(function(track) {
    // stopping every track
    console.log(track);
    track.stop();
  });
  // assign null to srcObject of video
  videoEl.srcObject = null;
  videoEl.remove();
});

Now I can use the AR-button and all the features of hit-testing and placing objects but marker detection does not work.

Would be awesome if someone could hint me in to the right direction to use the existing projects together with marker detection. Thanks in advance.



from Marker Detection (AR.js) in A-Frame Project

No comments:

Post a Comment