LineLayer is not working on top of pointcloudlayer in firstpersonview

[iamravisingh]

Hi,

I am trying to calculate the distance between 2 points in pointcloudlayer for the .las file. I want to draw a line over 2 selected points. I tried creating the LineLayer based on this. However, the line is not rendering for my case ( I am missing something small ) but the layer is getting created with data props.

For an instance, this what I am trying to do.

Below images shows LineLayer created with data holding in it.

Code Snippet

import { useEffect, useState, useRef, memo } from "react";
import DeckGL from "@deck.gl/react/typed";
import { PointCloudLayer, LineLayer } from "@deck.gl/layers/typed";
import {
    COORDINATE_SYSTEM,
    FirstPersonView,
    FirstPersonController,
    PickingInfo,
    FirstPersonViewState,
    FlyToInterpolator,
    Layer,
} from "@deck.gl/core/typed";
import { LASLoader } from "@loaders.gl/las";
import { useAppDispatch, useAppSelector } from "../../store/hooks";
import {
    getCurrentPanoramaIndex,
    getPanoramaHeading,
    getPanoramaImageMap,
    getPanoramaNameList,
    getPanoramaViewerRotation,
    setCurrentPanoramaLocalCoordinates,
    setPanoramaViewerRotation,
} from "../../store/reducers/panoramaSlice";
import { getCurrentTopdownMeta } from "../../store/reducers/topdownSlice";
import { deg2rad, extractNumber, rad2deg } from "../../services/common";
import { LocalCoordinates } from "../../store/reducers/type";
import "./styles.scss";
import { setPointCoordinates, setSelectedPointPosition } from "../../store/reducers/pointcloudSlice";
import { usePointCloudQuery } from "../../query";
import { setPointcloudBusy } from "../../store/reducers/appSlice";
import store from "../../store/store";
import { deckGLStyles, INITIAL_VIEW_STATE } from "./constant";
import { Panel } from "./components";
import { HDSButton, HDSTooltip } from "@here/hds-react-components";
import { getSelectedPointRenderer, normalizeLng, getSelectedPointPosition, convertXYToLatLng } from "./helper";
import { CoordinateFactors, PointDetails } from "./type";
import { useBeforeUnload } from "../../hooks";

class MyFirstPersonController extends FirstPersonController {
    constructor(options = {}) {
        // TODO: remove this below ignore once the deck.gl updated version have type exposed for the class
        // @ts-ignore
        super(options);
    }
    handleEvent(event: any): boolean {
        if (event.type === "pan") {
            // Placeholder
        } else {
            super.handleEvent(event);
        }
        return true;
    }
}

const PointcloudComponent = memo((props: any): JSX.Element => {
    const { onLoad } = props;
    const [viewState, updateViewState] = useState<FirstPersonViewState>(
        INITIAL_VIEW_STATE.FIRST_PERSON as unknown as FirstPersonViewState
    );
    const currentPanoramaIndex = useAppSelector(getCurrentPanoramaIndex);
    const pointCloudQueryOptions = {
        currentPanoramaIndex,
    };
    // fetch .las file from catalog
    const { data: pointCloudQuery } = usePointCloudQuery(pointCloudQueryOptions);
    const deckRef = useRef(null);
    const [selectedPoints, setSelectedPoints] = useState<PointDetails[]>([]);
    const [lineData, setLineData] = useState<PointDetails[]>([]);
    const [showPanel, setShowPanel] = useState(false);
    const [layers, setLayers] = useState<Layer[]>([]);
    const [pointData, setPointData] = useState<PointDetails[]>([]);
    const [enableAnnotation, setEnableAnnotation] = useState(false);
    const [coordFactors, setCoordFactors] = useState<CoordinateFactors>();
    const dispatch = useAppDispatch();
    const panoramaNameList = useAppSelector(getPanoramaNameList);
    const panoramaPartId = extractNumber(panoramaNameList[currentPanoramaIndex]);
    const panoramaImageMap = useAppSelector(getPanoramaImageMap);
    const imageMap = panoramaImageMap[panoramaPartId];
    const currentPosition = [imageMap?.xPc, imageMap?.yPc, imageMap?.zPc];
    const currentHeading = useAppSelector(getPanoramaHeading);
    const viewerRotation = useAppSelector(getPanoramaViewerRotation);
    const currentTopdownMeta = useAppSelector(getCurrentTopdownMeta);
    const currentTopdownData = currentTopdownMeta?.data || undefined;
    // ** Assuming if any point is selected, we will attach a unload event handler
    useBeforeUnload({ shouldAddEventListener: pointData.length > 0 });

    const onDataLoad = (data: any) => {
        const { header } = data;
        if (header.boundingBox) {
            // File contains bounding box info
            if (currentTopdownMeta?.map) {
                if (currentTopdownMeta.map.source === "navvis") {
                    // Adjust position height as navvis panorama position is quite high
                    currentPosition[2] -= 0.6;
                }
            }
            updateViewState({
                ...viewState,
                position: currentPosition,
                bearing: rad2deg(deg2rad(currentHeading) + viewerRotation),
            } as unknown as FirstPersonViewState);
        }
        console.log("Pointcloud loaded, header", header);
        dispatch(setPointcloudBusy(false));

        if (onLoad) {
            onLoad({ count: header.vertexCount, progress: 1 });
        }
        if (currentTopdownData) {
            console.log("Topdowndata", currentTopdownData);
            const wgs84 = currentTopdownData!.cornerCoordsWGS84;
            const xDelta = Math.abs(currentTopdownData.xmin - currentTopdownData.xmax);
            const yDelta = Math.abs(currentTopdownData.ymin - currentTopdownData.ymax);
            const lngDeltaX = normalizeLng(wgs84.topRight.lng - wgs84.topLeft.lng);
            const lngDeltaY = normalizeLng(wgs84.bottomLeft.lng - wgs84.topLeft.lng);
            const xCoordScale = {
                lng: lngDeltaX / xDelta,
                lat: (wgs84.bottomLeft.lat - wgs84.bottomRight.lat) / xDelta,
            };
            const yCoordScale = { lng: lngDeltaY / yDelta, lat: (wgs84.topRight.lat - wgs84.bottomRight.lat) / yDelta };
            console.log("Coordscales", xCoordScale, yCoordScale);
            setCoordFactors({ x: xCoordScale, y: yCoordScale });
        }
    };

    const pointHandler = (info: PickingInfo) => {
        // if point is selected then open the panel
        if (!showPanel) {
            setShowPanel(true);
        }
        const updatedPointData = [...pointData];
        const index = info.index;
        const selectedPointIndex = updatedPointData.findIndex((point) => point.index === index);
        if (info.layer && info.layer.props) {
            const [x, y, z] = getSelectedPointPosition(info);
            const wgs84 = currentTopdownData!.cornerCoordsWGS84;
            const vector = [x, y, z] as ReturnType<typeof getSelectedPointPosition>;
            // converting x,y value to coordinates
            const [lat, lng, alt] = coordFactors ? convertXYToLatLng(vector, coordFactors, wgs84) : [0, 0, 0];
            if (selectedPointIndex !== -1) {
                // The point is already selected, so unselect it
                updatedPointData.splice(selectedPointIndex, 1);
            } else {
                // The point is not selected, so select it and change its color
                // assuming z as a altitude
                updatedPointData.push({
                    positionInLatLng: [lng, lat, alt],
                    position: [x, y, z],
                    coordinate: info.coordinate,
                    color: [255, 0, 0], // red color
                    index,
                });
            }
            setPointData(updatedPointData);
            dispatch(setSelectedPointPosition(updatedPointData));
            console.log("position:", x, y, z);
        }
    };

    const onPointRemove = (deleteAll: boolean, id?: number) => {
        const points = [...pointData];
        const filteredPoint = deleteAll ? [] : points.filter((point) => point.index !== id);
        setPointData(filteredPoint);
        dispatch(setSelectedPointPosition(filteredPoint));
    };

    const handleLineLayer = (object: PickingInfo) => {
        const index = object.index;
        const [x, y, z] = getSelectedPointPosition(object);
        const newSelectedPoint: PointDetails = {
            positionInLatLng: [x, y, z], // Add this line to store position in latlng format
            position: [x, y, z],
            coordinate: object.coordinate,
            color: [255, 0, 0], // red color
            index,
        };

        if (selectedPoints.length === 0) {
            // If no point is selected, add the clicked point to selected points
            setSelectedPoints([newSelectedPoint]);
        } else if (selectedPoints.length === 1) {
            // If one point is already selected, add the second point and draw the line
            const [firstPoint] = structuredClone(selectedPoints);
            setSelectedPoints([firstPoint, newSelectedPoint]);
            setLineData([firstPoint, newSelectedPoint]);
        } else {
            // If two points are already selected, clear the selection
            setSelectedPoints([]);
            setLineData([]);
        }
    };

    const clickHandler = (object: PickingInfo) => {
        console.log("Clicked", object, coordFactors);
        const index = object.index;
        if (object.layer && object.layer.props) {
            const oldCoordinates = store.getState().pointcloud.pointCoordinates;
            // below ts error is suppressed intensionally, as data props is not defined inside PickingInfo,
            // will be removed once deck.gl is upgrade with updated pickinginfo type
            // @ts-ignore
            const position = object.layer.props.data.attributes.POSITION;
            const x = Number(position.value[index * position.size]);
            const y = Number(position.value[index * position.size + 1]);
            const z = Number(position.value[index * position.size + 2]);
            console.log("position:", x, y, z);
            handleLineLayer(object);
            if (oldCoordinates && JSON.stringify(oldCoordinates) === JSON.stringify([x, y, z])) {
                dispatch(setPointCoordinates(null));
            } else {
                dispatch(setPointCoordinates([x, y, z]));
            }
        }
        if (enableAnnotation) {
            pointHandler(object);
        }
    };

    const getTooltipText = (info: PickingInfo) => {
        const coordinates = store.getState().pointcloud.pointCoordinates;
        const index = info.index;
        let tooltipText = "";

        let x = "N/A";
        let y = "N/A";
        let z = "N/A";
        let x1 = 0;
        let y1 = 0;
        let z1 = 0;
        if (info.layer && info.layer.props) {
            // below ts error is suppressed intensionally, as data props is not defined inside PickingInfo,
            // will be removed once deck.gl is upgrade with updated pickinginfo type
            // @ts-ignore
            const position = info.layer.props.data.attributes.POSITION;
            x = position.value[index * position.size];
            y = position.value[index * position.size + 1];
            z = position.value[index * position.size + 2];

            if (x && y && z) {
                x = Number(x).toFixed(2);
                y = Number(y).toFixed(2);
                z = Number(z).toFixed(2);
            }
            x1 = Number(x);
            y1 = Number(y);
            z1 = Number(z);
        }
        if (coordinates) {
            const x2 = coordinates[0];
            const y2 = coordinates[1];
            const z2 = coordinates[2];
            const distance = Math.sqrt((x2 - x1) ** 2 + (y2 - y1) ** 2 + (z2 - z1) ** 2);
            const height = Math.abs(z2 - z1);
            const width = Math.sqrt((x2 - x1) ** 2 + (y2 - y1) ** 2);
            tooltipText =
                "Distance: " +
                distance.toFixed(2) +
                " meters<br>" +
                "Width: " +
                width.toFixed(2) +
                " meters<br>" +
                "Height: " +
                height.toFixed(2) +
                " meters";
        } else {
            let lat = 0;
            let lng = 0;
            if (coordFactors && currentTopdownData) {
                const wgs84 = currentTopdownData!.cornerCoordsWGS84;
                lat = x1 * coordFactors.x.lat + y1 * coordFactors.y.lat + wgs84.bottomLeft.lat;
                lng = x1 * coordFactors.x.lng + y1 * coordFactors.y.lng + wgs84.bottomLeft.lng;
            }
            tooltipText =
                "X: " + x + "<br>Y: " + y + "<br>Z: " + z + "<br>lat: " + lat.toFixed(7) + "<br>lng: " + lng.toFixed(7);
        }
        return `<div className="tooltipContainer">
                ${tooltipText}
            </div>`;
    };

    const pointLayer = (enableAnnotation ? getSelectedPointRenderer(pointData.concat(selectedPoints)) : []) as Layer;
    const lineLayer = new LineLayer({
        id: "line-layer-selected-point",
        data: lineData,
        pickable: true,
        getNormal: [255, 255, 255],
        coordinateSystem: COORDINATE_SYSTEM.METER_OFFSETS,
        opacity: 1,
        getWidth: 5,
        // @ts-ignore
        getSourcePosition: (d) => d.position,
        getTargetPosition: (d) => d.position,
        getColor: (d) => d.color,
        updateTriggers: {
            getColor: lineData.length,
            getSourcePosition: [lineData],
        }
    });
    const allLayers = [pointLayer, lineLayer];
    const layerUpdateTrigger = {
        getColor: [pointData.length, lineData.length],
        getPosition: lineData.length,
    };

    useEffect(() => {
        console.log("Fetch pointcloud tile");
        dispatch(setPointcloudBusy(true));
    }, []);

    useEffect(() => {
        if (pointCloudQuery) {
            setLayers([
                new PointCloudLayer({
                    id: "point-cloud-layer",
                    data: pointCloudQuery,
                    onDataLoad,
                    coordinateSystem: COORDINATE_SYSTEM.METER_OFFSETS,
                    getNormal: [0, 1, 0],
                    sizeScale: 15,
                    autoHighlight: true,
                    getPosition: (d: any) => d.position,
                    opacity: 1,
                    pointSize: 0.2,
                    sizeUnits: "meters",
                    pickable: true,
                    loaders: [LASLoader],
                    highlightColor: [0, 255, 0],
                    loadOptions: {
                        las: {
                            colorDepth: 16,
                        },
                    },
                    onClick: clickHandler,
                    updateTriggers: layerUpdateTrigger,
                }),
                ...allLayers,
            ]);
        }
    }, [enableAnnotation, pointCloudQuery, JSON.stringify(pointData), coordFactors, selectedPoints, lineData]);

    const renderTooltip = (info: PickingInfo): any => {
        if (info && info.index !== -1) {
            return {
                html: getTooltipText(info),
                style: { ...deckGLStyles.pointToolTipStyles },
            };
        }
        // ** Hide the tooltip when object.index is -1
        return null;
    };

    // ** this will help to fly the view to selected point
    const onItemSelect = (item: any) => {
        const selectedItemPoint = pointData.find((p) => p.index === item.index) || {};
        const { positionInLatLng, position } = selectedItemPoint as unknown as PointDetails;
        if (positionInLatLng && "positionInLatLng" in selectedItemPoint) {
            updateViewState({
                ...viewState,
                position,
                transitionInterpolator: new FlyToInterpolator({ speed: 1.0 }),
                transitionDuration: "auto",
            } as unknown as FirstPersonViewState);
        }
    };

    console.log("layers in pointcloud >>>>>>>", layers, lineData, selectedPoints);
    return (
        <div className={`pointcloudComponent ${showPanel ? "panelOpen" : ""}`}>
            <HDSTooltip content={"Menu"} position="right" id="menu">
                <HDSButton
                    className="menuIcon"
                    icon="menu"
                    iconCategory="core-ui"
                    variant="floating"
                    size="large"
                    iconOnly
                    onClick={() => setShowPanel(!showPanel)}
                />
            </HDSTooltip>
            {showPanel && (
                <div className="panelContainer">
                    <Panel
                        onClose={setShowPanel}
                        pointData={pointData}
                        onRemove={onPointRemove}
                        onSelect={onItemSelect}
                        setEnableAnnotation={setEnableAnnotation}
                        enableAnnotation={enableAnnotation}
                    />
                </div>
            )}
            <DeckGL
                ref={deckRef}
                views={[new FirstPersonView({ focalDistance: 100, fovy: 80, controller: MyFirstPersonController })]}
                viewState={viewState}
                onViewStateChange={(v) => {
                    updateViewState(v.viewState);
                    if (JSON.stringify(v.oldViewState?.position) !== JSON.stringify(v.viewState.position)) {
                        dispatch(
                            setCurrentPanoramaLocalCoordinates({
                                xPc: v.viewState.position[0],
                                yPc: v.viewState.position[1],
                                zPc: v.viewState.position[2],
                            } as LocalCoordinates)
                        );
                    }
                    if (v.viewState.bearing !== v.oldViewState?.bearing) {
                        // Convert point cloud bearing to panorama viewer rotation
                        dispatch(setPanoramaViewerRotation(deg2rad(v.viewState.bearing) - deg2rad(currentHeading)));
                    }
                }}
                controller={{ keyboard: { moveSpeed: 0.5 }, doubleClickZoom: false, dragMode: "rotate" }}
                style={{ ...deckGLStyles.component }}
                layers={layers}
                parameters={{
                    clearColor: [0.93, 0.86, 0.81, 1],
                }}
                pickingRadius={10}
                getTooltip={renderTooltip}
                getCursor={(state): any => {
                    return state.isDragging ? "grabbing" : "pointer";
                }}
            />
        </div>
    );
});

export default PointcloudComponent;

Any help on this is appreciated. Thanks in advance.

[Pessimistress]

Each element in the LineLayer's data array represents one line, not one point. Your getSourcePosition and getTargetPosition are returning the same thing, so there is no visible line.

[iamravisingh]

@Pessimistress Ah! thanks for the quick reply. I changed the code based on LineLayer's data array represents one line.

Additionally, I would also love to get your thoughts on one of approach I am trying to achieve. I want to restrict user to only draw a line in horizontal/vertical, I mean I don't want user to draw a line diagonally. Is there any possibility in lineLayer, something in deckgl libraries itself or I'll have to do the custom maths?