In the last post I used a method called Reoriented Normal Mapping(RNP) to combine two normal maps. But I didn’t know what I was doing, so I spend some time to figure it out.

Normal map

Fist I need to know what is a normal map, I found this Unity manual explains very well.

I baked a simple normal map to a two-meter by two-meter plane, the angle of the slope is 45 degrees. The normals are \((0, {\sqrt 2 \over 2}, {\sqrt 2 \over 2})\), \((0, - {\sqrt 2 \over 2}, {\sqrt 2 \over 2})\) and \((0, 0, 1)\). Values of the unit vector are in the range of [-1, 1], convert them to [0, 1] by add one then divide by two: \((0.5, {\sqrt 2 \over 4} + {1 \over 2} \approx 0.854, 0.854)\), \((0.5, - {\sqrt 2 \over 4} + {1 \over 2} \approx 0.146, 0.854)\), \((0.5, 0.5, 1)\)

normal map

Blender casts rays from the low-poly object inwards towards the high-poly object while baking therefore the Extrusion parameter should be at least \({2 \over 8} = 0.25m\) in order to bake the protruding part. And the normal map image node’s Color Space must be set to Non-Color, otherwise the Image Editor will display the normal map as an ordinary image.

Quaternion

Next I need to know what the heck is quaternion and why half angles are used in 3D rotation. The amazing website created by Grant Sanderson and Ben Eater is really helpful for me to understand the concept.

Reoriented Normal Mapping

I made a video to demonstrate how RNP combines two normal vectors by rotating vector :

Let \(\color{#83C167}{\boldsymbol{u}}\) be the first normal vector and \(\color{#58C4DD}{\boldsymbol{s}}\) is \(\color{#58C4DD}{(0, 0, 1)}\), \(\color{#FC6255}{\boldsymbol{t}}\) is the second normal vector, \(\color{#D147BD}{\boldsymbol{r}}\) is the reoriented normal and \(\theta\) is the angle between \(\color{#58C4DD}{\boldsymbol{s}}\) and \(\color{#FC6255}{\boldsymbol{t}}\). By finding the quaternion \(\boldsymbol{q}\) which rotates \(\color{#58C4DD}{\boldsymbol{s}}\) to \(\color{#FC6255}{\boldsymbol{t}}\), we can then use it to rotate \(\color{#83C167}{\boldsymbol{u}}\) to get \(\color{#D147BD}{\boldsymbol{r}}\). The length of the imaginary part of \(\boldsymbol{q}\) is \(\sin({\theta \over 2})\) and the length of the real part is \(\cos({\theta \over 2})\).

The orientation of the imaginary(vector) part of \(\boldsymbol{q}\) can be obtained by applying the right hand rule on \(\boldsymbol{s}\) and \(\boldsymbol{t}\), that’s the same orientation of \(\boldsymbol{s} \times \boldsymbol{t}\). Recall the length of cross product of two unit vectors is sine of the angles between them. Therefore the imaginary part of \(\boldsymbol{q}\) is:

\[\boldsymbol{q_v} = \boldsymbol{s} \times \boldsymbol{t} {\sin {\theta \over 2} \over \sin \theta}\]

Using the Pythagorean identity:

\[\sin^2\theta + \cos^2 \theta = 1\]

and the double-angle formulae:

\[\cos(2\theta) = 1 - 2\sin^2 \theta = 2\cos^2 \theta - 1\]

\[\boldsymbol{q_v} = \boldsymbol{s} \times \boldsymbol{t} \sqrt {1 - \cos \theta \over 2 (1 - \cos^2 \theta)} = {\boldsymbol{s} \times \boldsymbol{t} \over \sqrt {2 (1 + \cos \theta)}}\]

Recall the dot product of two unit vectors is the cosine of the angle between them:

\[\boldsymbol{q_v} = {\boldsymbol{s} \times \boldsymbol{t} \over \sqrt {2 (1 + \boldsymbol{s} \cdot \boldsymbol{t})}}\]

and the real part is:

\[q_w = \cos {\theta \over 2} = \sqrt {1 + \cos \theta \over 2} = \sqrt {1 + \boldsymbol{s} \cdot \boldsymbol{t} \over 2}\]

Use quaternions to transform \(\boldsymbol{u}\) to \(\boldsymbol{r}\):

\[\boldsymbol{r} = \boldsymbol{q} \boldsymbol{p} \boldsymbol{q^{-1}}\]

\[\boldsymbol{p} = (\boldsymbol{u}, 0)\]

\[\boldsymbol{q^{-1}} = (-\boldsymbol{q_v}, q_w)\]

Using the quaternions multiplication equation:

\[\boldsymbol{q} \boldsymbol{r} = (\boldsymbol{q_v} \times \boldsymbol{r_v} + r_w \boldsymbol{q_v} + q_w \boldsymbol{r_v}, q_w r_w - \boldsymbol{q_v} \cdot \boldsymbol{r_v})\]

\(\boldsymbol{r}\) can be represented by:

\[\boldsymbol{r} = (-\boldsymbol{q_v} \times \boldsymbol{u} \times \boldsymbol{q_v} + 2q_w\boldsymbol{q_v} \times \boldsymbol{u} + q_w^2 \boldsymbol{u} + (\boldsymbol{q_v} \cdot \boldsymbol{u}) \boldsymbol{q_v}, 0)\]

Using the triple product expansion formula:

\[\boldsymbol{a} \times (\boldsymbol{b} \times \boldsymbol{c}) = \boldsymbol{b}(\boldsymbol{a} \cdot \boldsymbol{c}) - \boldsymbol{c}(\boldsymbol{a} \cdot \boldsymbol{b})\]

to replace the triple product in \(\boldsymbol{r}\):

\[\boldsymbol{r} = \boldsymbol{q_v}(\boldsymbol{q_v} \cdot \boldsymbol{u}) - \boldsymbol{u}(\boldsymbol{q_v} \cdot \boldsymbol{q_v}) + 2q_w\boldsymbol{q_v} \times \boldsymbol{u} + q_w^2 \boldsymbol{u} + (\boldsymbol{q_v} \cdot \boldsymbol{u}) \boldsymbol{q_v}\]

\[= \boldsymbol{u}(q_w^2 - \boldsymbol{q_v} \cdot \boldsymbol{q_v}) + 2q_w\boldsymbol{q_v} \times \boldsymbol{u} + 2 \boldsymbol{q_v}(\boldsymbol{q_v} \cdot \boldsymbol{u})\]

Rest steps can be found at the appendix of Self Shadow blog.

Okay move along, move along people, there’s nothing to see here!

Here is the code that creates the RNP video:

from manim import *
import math

class RNM(ThreeDScene):
    def construct(self):
        axes = ThreeDAxes()
        theta = 60 * DEGREES
        u = [math.cos(theta), 0, math.sin(theta)]
        t = [0, 1, 0]
        self.set_camera_orientation(phi=75 * DEGREES, theta=30 * DEGREES)
        s = Vector([0, 0, 1], color=BLUE)
        u_mo = Vector(u, color=GREEN)
        t_mo = Vector(t, color=RED)
        self.begin_ambient_camera_rotation()
        s_label = MathTex(r's: (0, 0, 1)', color=BLUE).to_corner(UR).scale(0.6)
        u_label = MathTex(r'u: ({1 \over 2}, 0, {\sqrt 3 \over 2})', color=GREEN).next_to(s_label, DOWN).scale(0.6)
        t_label = MathTex(r't: (0, 1, 0)', color=RED).next_to(u_label, DOWN).scale(0.6)
        r_label = MathTex(r'r: ({1 \over 2}, {\sqrt 3 \over 2}, 0)', color=PINK).next_to(t_label, DOWN).scale(0.6)
        self.add(axes, s, t_mo, u_mo)
        self.add_fixed_in_frame_mobjects(s_label, u_label, t_label, r_label)
        self.wait(2)

        text = MarkupText(
            f'Find the transformation that rotates <span fgcolor="{BLUE}">s</span> to <span fgcolor="{RED}">t</span>',
            font='CMU Serif')
        self.add_fixed_in_frame_mobjects(text.to_edge(DOWN))
        self.play(Transform(s, t_mo))
        self.wait(3)
        self.remove(text)

        text = MarkupText(
            f'Apply the same transformation on <span fgcolor="{GREEN}">u</span> to obtain <span fgcolor="{PINK}">r</span>',
            font='CMU Serif')
        u_prime = np.multiply(u, [-1, -1, 1])
        t_prime = np.add(t, [0, 0, 1])
        r = np.subtract(np.divide(np.multiply(t_prime, np.dot(t_prime, u_prime)), t_prime[2]), u_prime)
        r_mo = Vector(r, color=PINK)
        self.add_fixed_in_frame_mobjects(text.to_edge(DOWN))
        self.play(Transform(u_mo, r_mo))
        self.wait(3)