Raycast to a Displacement Map

Description

In Threejs, when setting a materials displacementMap using a grayscale image, it will modify its vertices at the GPU level as it is rendered each frame. The displacement happens in the vertex shader and the altered information is not readable at the JavaScript layer.

The Threejs Raycaster won't ray-cast against the information modified in the vertex shader, but will read any information it has stored in the Buffer Geometry data in the JavaScript layer.

So if you are displacing a planeGeometry with a displacementMap, then at the JavaScript (CPU) side, the plane geometry will still be a flat geometry in memory since what happens in the vertex shader is not reflected in the BufferGeometry data JavaScript side.

So the Raycaster will ray-cast against a flat plane.

If you want to ray-cast against the displaced geometry, then one option you have is to displace the geometry at the JavaScript layer instead.

This example below, will load a grayscale image, and displace the vertices of the plane using JavaScript instead of using the materials displacementMap property.

This will then allow the Raycaster to ray-cast against the modified vertices that were modified JavaScript side instead.

Double-click the terrain to change the target of the OrbitControls.

<>

Note that the terrain displacement demonstrated in this example has been exaggerated for dramatic effect. It does not follow any particular measurement standard.

Also note that displacing a material using the inbuilt Threejs displacementMap functionality will give better visual results because of extra smoothing that happens in the vertex shader. Only displace the vertices JavaScript side, only if you really need to.

Example Script

./dist/client/index.html

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />
        <title>
            Three.js TypeScript Tutorials by Sean Bradley :
            https://sbcode.net/threejs
        </title>
        <style>
            body {
                overflow: hidden;
                margin: 0px;
            }

            #locationData {
                position: absolute;
                left: 50%;
                top: 50%;
                font-size: 12px;
                font-family: monospace;
                color: #00ff00;
                pointer-events: none;
                user-select: none;
            }
        </style>
    </head>

    <body>
        <div id="locationData"></div>
        <script type="module" src="bundle.js"></script>
    </body>
</html>

./src/client/client.ts

import * as THREE from 'three'
import { OrbitControls } from 'three/examples/jsm/controls/OrbitControls'
import { RGBELoader } from 'three/examples/jsm/loaders/RGBELoader'
import Stats from 'three/examples/jsm/libs/stats.module'
import TWEEN from '@tweenjs/tween.js'

const elem = document.getElementById('locationData') as HTMLDivElement
const coordinate = { lat: -41.5, lon: 146.5 }

const scene = new THREE.Scene()

new RGBELoader().load(
    'img/kloppenheim_06_puresky_1k.hdr',
    function (texture) {
        texture.mapping = THREE.EquirectangularReflectionMapping
        scene.background = texture
        scene.environment = texture
    }
)

const camera = new THREE.PerspectiveCamera(
    55,
    window.innerWidth / window.innerHeight,
    0.1,
    1000
)
camera.position.set(0, 60, 100)

const renderer = new THREE.WebGLRenderer({
    antialias: true,
    logarithmicDepthBuffer: true,
})
renderer.setSize(window.innerWidth, window.innerHeight)
document.body.appendChild(renderer.domElement)

const controls = new OrbitControls(camera, renderer.domElement)
controls.enableDamping = true
controls.maxPolarAngle = Math.PI / 2.1
controls.maxDistance = 200
controls.minDistance = 1
controls.target.set(8, 2, 28)

const geometry = new THREE.PlaneGeometry(180, 180, 720, 720).rotateX(
    -Math.PI * 0.5
)
const material = new THREE.MeshStandardMaterial({})
const ground = new THREE.Mesh(geometry, material)
scene.add(ground)

const textureLoader = new THREE.TextureLoader()

const texture = textureLoader.load('img/world.topo.bathy_tasmania.jpg')
material.map = texture
material.flatShading = true

textureLoader.load('img/gebco_tasmania.png', function (texture) {
    /* Displacing in the Vertex Shader */
    // material.displacementMap = texture
    // material.displacementScale = 100
    // material.displacementBias = -50
    /* End Displacing in the Vertex Shader */

    /* Displacing in the Javascript Layer */
    const canvas = document.createElement('canvas') as HTMLCanvasElement
    canvas.width = texture.image.width
    canvas.height = texture.image.height
    const context = canvas.getContext('2d') as CanvasRenderingContext2D
    context.drawImage(texture.image, 0, 0)

    const width = geometry.parameters.widthSegments + 1
    const height = geometry.parameters.heightSegments + 1
    const widthStep = texture.image.width / width
    const heightStep = texture.image.height / height

    const data = context.getImageData(
        0,
        0,
        texture.image.width,
        texture.image.height
    )

    const positions = geometry.attributes.position.array
    let w, h, x, y

    for (let i = 0; i < positions.length; i += 3) {
        w = (i / 3) % width
        h = i / 3 / width

        x = Math.round(w * widthStep)
        y = Math.round(h * heightStep)

        const displacement = data.data[x * 4 + y * 4 * texture.image.width]
        positions[i + 1] = displacement / 2.55 - 50
    }

    geometry.attributes.position.needsUpdate = true
    geometry.computeVertexNormals()
    /* End Displacing in the Javascript Layer  */
})

const seaPlane = new THREE.Mesh(
    new THREE.PlaneGeometry(180, 180, 1, 1),
    new THREE.MeshStandardMaterial({
        transparent: true,
        opacity: 0.25,
        metalness: 0.99,
        roughness: 0.01,
    })
)
seaPlane.rotateX(-Math.PI / 2)
scene.add(seaPlane)

const points = []
points.push(new THREE.Vector3(-90, 0, 0))
points.push(new THREE.Vector3(90, 0, 0))
const latLine = new THREE.Line(
    new THREE.BufferGeometry().setFromPoints(points),
    new THREE.LineBasicMaterial({ color: 0x00ff00 })
)
scene.add(latLine)

const lonLine = latLine.clone()
lonLine.rotateY(Math.PI / 2)
scene.add(lonLine)

const altLine = latLine.clone()
altLine.rotateZ(Math.PI / 2)
scene.add(altLine)

const mouse = new THREE.Vector2()
const raycaster = new THREE.Raycaster()

function onDoubleClick(event: MouseEvent) {
    mouse.set(
        (event.clientX / renderer.domElement.clientWidth) * 2 - 1,
        -(event.clientY / renderer.domElement.clientHeight) * 2 + 1
    )
    raycaster.setFromCamera(mouse, camera)
    const intersects = raycaster.intersectObject(ground, false)
    if (intersects.length > 0) {
        const { point, uv } = intersects[0]

        new TWEEN.Tween(controls.target)
            .to(
                {
                    x: point.x,
                    y: point.y,
                    z: point.z,
                },
                500
            )
            .easing(TWEEN.Easing.Cubic.Out)
            .start()

        new TWEEN.Tween(latLine.position)
            .to(
                {
                    y: point.y,
                    z: point.z,
                },
                500
            )
            .easing(TWEEN.Easing.Cubic.Out)
            .start()

        new TWEEN.Tween(lonLine.position)
            .to(
                {
                    x: point.x,
                    y: point.y,
                },
                500
            )
            .easing(TWEEN.Easing.Cubic.Out)
            .start()

        new TWEEN.Tween(altLine.position)
            .to(
                {
                    x: point.x,
                    y: point.y,
                    z: point.z,
                },
                500
            )
            .easing(TWEEN.Easing.Cubic.Out)
            .start()

        new TWEEN.Tween(coordinate)
            .to(
                {
                    lat: -38.0 - (1 - (uv as THREE.Vector2).y) * 7,
                    lon: 143.0 + (uv as THREE.Vector2).x * 7,
                },
                500
            )
            .start()
            .onUpdate(function () {
                elem.innerText =
                    coordinate.lat.toFixed(6) + ' ' + coordinate.lon.toFixed(6)
            })
    }
}
renderer.domElement.addEventListener('dblclick', onDoubleClick, false)

function onWindowResize() {
    camera.aspect = window.innerWidth / window.innerHeight
    camera.updateProjectionMatrix()
    renderer.setSize(window.innerWidth, window.innerHeight)
    render()
}
window.addEventListener('resize', onWindowResize, false)

const stats = new Stats()
document.body.appendChild(stats.dom)

function animate() {
    requestAnimationFrame(animate)

    controls.update()

    TWEEN.update()

    render()

    stats.update()
}

function render() {
    renderer.render(scene, camera)
}

animate()

Design Patterns in Python
		Kindle Edition Paperback
Design Patterns in TypeScript
		Kindle Edition Paperback

Design Patterns in Python
		Kindle Edition Paperback
Design Patterns in TypeScript
		Kindle Edition Paperback

Raycast to a Displacement Map

Description

Example Script

./dist/client/index.html

./src/client/client.ts

Comments