Tuesday, 4 December 2018

Vuforia Dominoes: native UWP adaption

Working from Vuforia's native UWP sample for VuMarks I've been trying to implement functionality from the native Android Dominoes sample (both can be found here). Specifically, the option to click on the screen and get coordinates in the VuMark's target space.

Here is my code:

void VuMarkRenderer::OnScreenTapped(Point* pos, double logicalWidth, double logicalHeight) {
    // normalize coordinates
    float x = pos->X / logicalWidth;
    float y = pos->Y / logicalHeight;

    // normalized device coordinates
    Vuforia::Vec4F ndcNear(x, y, -1.0f, 1.0f);  // pn near plane
    Vuforia::Vec4F ndcFar(x, y, 1.0f, 1.0f);    // on far plane

    // to eye coordinates (i.e. multiplay with inverse projection matrix)
    auto nearPlanePoint = AppMathUtils::Vec4FTransform(ndcNear, m_currentInverseProjection);
    auto farPlanePoint = AppMathUtils::Vec4FTransform(ndcFar, m_currentInverseProjection);
    // normalize
    nearPlanePoint = AppMathUtils::Vec4FDiv(nearPlanePoint, nearPlanePoint.data[3]);
    farPlanePoint = AppMathUtils::Vec4FDiv(farPlanePoint, farPlanePoint.data[3]);

    // to object (world) coordinates
    Vuforia::Vec4F nearWorld = AppMathUtils::Vec4FTransform(nearPlanePoint, m_currentInverseModelView);
    Vuforia::Vec4F farWorld = AppMathUtils::Vec4FTransform(farPlanePoint, m_currentInverseModelView);

    // set up line (--> ray cast)
    Vuforia::Vec3F lineStart(
        nearWorld.data[0],
        nearWorld.data[1],
        nearWorld.data[2]);
    Vuforia::Vec3F lineEnd(
        farWorld.data[0],
        farWorld.data[1],
        farWorld.data[2]);

    // intersect with VuMark plane
    auto lineDir = AppMathUtils::Vec3FSub(lineEnd, lineStart);
    lineDir = AppMathUtils::Vec3FDiv(lineDir, lineDir.data[2]);

    Vuforia::Vec3F vuMarkPlaneCenter(0, 0, 0);
    auto planeDir = AppMathUtils::Vec3FSub(vuMarkPlaneCenter, lineStart);

    Vuforia::Vec3F vuMarkPlaneNormal(0, 0, 1);
    float n = AppMathUtils::Vec3FDot(vuMarkPlaneNormal, planeDir);
    float d = AppMathUtils::Vec3FDot(vuMarkPlaneNormal, lineDir);

    if (fabs(d) < 0.00001) {    // i.e. line is parallel to plane
         m_vuMarkView->UpdateDebugOutput("No intersection");
         return;
    }

    float dist = n / d;

    Vuforia::Vec3F offset = AppMathUtils::Vec3FScale(lineDir, dist);
    auto intersection = AppMathUtils::Vec3FAdd(lineStart, offset);
}

logicalWidth and logicalHeight are the UWP page's ActualWidth and ActualHeight respecively.

Here's where I get m_currentInverseProjection from:

[...]

// Calculate the DX Projection matrix using the current Vuforia state
auto projectionMatrix = Vuforia::Tool::convertPerspectiveProjection2GLMatrix(
    m_renderingPrimitives->getProjectionMatrix(Vuforia::VIEW_SINGULAR, state.getCameraCalibration()),
    m_near, 
    m_far);

XMFLOAT4X4 dxProjection;
memcpy(dxProjection.m, projectionMatrix.data, sizeof(float) * 16);
XMStoreFloat4x4(&dxProjection, XMMatrixTranspose(XMLoadFloat4x4(&dxProjection)));

XMMATRIX xmProjection = XMLoadFloat4x4(&dxProjection);

bool gotVuMark = false;

for (size_t v = 0; v < viewList.getNumViews(); v++)
{
    Vuforia::VIEW viewId = viewList.getView(static_cast<int>(v));

    // Apply the appropriate eye adjustment to the raw projection matrix, and assign to the global variable
    Vuforia::Matrix44F eyeAdjustment44F = Vuforia::Tool::convert2GLMatrix(
        m_renderingPrimitives->getEyeDisplayAdjustmentMatrix(viewId));

    // get current inverse projection matrix
    Vuforia::Matrix44F currentProjection;
    AppMathUtils::multiplyMatrix(projectionMatrix, eyeAdjustment44F, currentProjection);
    m_currentInverseProjection = AppMathUtils::Matrix44FInverse(currentProjection);

    [...]

And here is m_currentInverseModelView:

[...]

// Set up the modelview matrix
auto poseGL = Vuforia::Tool::convertPose2GLMatrix(result->getPose());

// save the current inverse model-view matrix
m_currentInverseModelView = AppMathUtils::Matrix44FInverse(poseGL);

[...]

The functions from the AppMathUtils are just copied and pasted from the two sample projects.

However, the results are completely off. I am pretty sure that I'm misunderstanding something about the different coordinate systems, but I can absolutely not figure out what it is...

Does anyone have an idea, what I'm doing wrong? I'd appreciate any help!

EDIT

I forgot to mention this article (section "Screen space to target space"), which originally pointed me to the Dominoes sample and which I tried to adapt.



from Vuforia Dominoes: native UWP adaption

No comments:

Post a Comment