OpenGL 3D Game Tutorial 29: Mouse Picking

LWJGL tutorial series on how to create a 3D Java game with OpenGL!

Using ray casting this week to implement 3D mouse picking!

End of video music- Kai Engel, "Waking Stars":

[MrGameengineer]

A nice addition to this series that could be another step forward toward an in-game editor is projected textures or graphics onto the terrain. Say that you clicked on an object in the world and this effect could draw a ring around the object projected onto the terrain.

[djamparo]

haha love it. "From normalised divice space, to .... ... this one, to eye space..."

[AdelaarGD]

This is awesome. I made a first person engine out of these tutorials and this technique let's me do things like have the player look at something and interact with it.
These tutorials are always so useful!

[tomg]

Nice. I usually do mouse picking by reading the depth buffer at the mouse's location and then calculating the world space coordinate that the mouse is pointing to and then using a spatial hash grid to locate any nearby entities efficiently and find the closest to the mouse point. This has the advantage of not needing any bounding volumes for entities or the need for intersection tests, and handles more complex objects better.

[ironfire]

Made sure to go back and like any videos I missed

[DemonicGaming]

That was the same website you set me to! Kills me that I can't carry on with these tutorials until after exams... Hope you keep them up!

[OnionRingFromSaturn]

Thanks for the tutorials, you really are a great help and have actually thought me a lot over the past week and a bit that I have been working on this. Best of luck with your game I hope I can be one of the first to support it. 

Also to anyone else, I have figured out incredibly simple sphere collision detection that doesn't require too much coding to implement(took a while to figure out though). Simply find the centerpoint(avg of largest and smallest XYZ values) and radius(largest magnitude from centerpoint to another point in the model) of a model in the obj loader. Then check if the magnitude of the distance between the centerpoints subtracted by the radius1+radius2 squared is the less than or equal to 0. If people reading this would like help to implement it I am all for it, just reply. 

Proper collision with AABB and narrow phase is something that would take a while and hopefully in the future there is a tutorial but right now best to figure it out slowly.

[jackgogvail]

Thank you this is exactly what I needed for my game
Stay awesome thinmatrix

[jasonjepson]

Wasn't necessarily interested in this topic but I love watching these regardless of topic. They sometimes give me ideas or just help me learn new things.

The problem with the terrain midpoint algorithm you were talking about is kind of like problems with culling really large objects interacting with the view frustum. They might get culled although they shouldn't be. Darn those large things in games.

[sverrebjering]

Thanks! Yes it was the projection matrix I were looking for.

[stnger]

Oooh yeah! Finally it happen! 
A Super Mega Thank you, man, you awesome! :)

[vladsbaking]

Great video! Was waiting for this. Thanks a bunch mate.

[morskipaskenny]

Love the vid! Omg the game is actually starting to look like something! ;D

[pavelp]

Nice, thank you. Almost what i was looking for. I guess I have to elaborate how to get point of near clipping plane as a ray origin and try it with both perspective and orthogonal projection. Reading depth buffer at mouse position and getting position is sometimes handy as well and converting screen x, y, z to world x, y, z is more versatile imho.
Anyways, thanks ... this seems like a good starting point.

[kumu]

One thing that is not clear here, if you have an object in a 3D world which has no limits (1, -1) ie. the middle of the screen is (0, 0, 0) and you have endless points to the axis xyz .. how would you change this normalized point to the 3d objects position which might be for example (10, 10, 100)?.
To understand my question, think about a program like FreeCAD that you have 3D world with xyz unlimited and 3D models are everywhere. You need to check which object is clicked by the mouse.
Thanks.

[antiHUMANDesigns]

I calculated the ray cast very differently.

I know the camera position, the camera "target" position, camera "up" vector (simply set to [0, 1, 0]), and the camera's vertical FOV. These are used for gluLookAt() and GluPerspective().
So, I did this:
1. camera target minus camera position gives me the vector that the camera is pointing in ("target vector"). I normalize this vector.
2. I get the right-angle vector of the camera by the cross product of camera target vector and camera up vector.
3. I re-calculate the camera's "up" vector by cross product between the camera target vector and the right-angle vector I just calculated. It's no longer simply straight up, but calculated properly.
4. I multiply the right-angle vector by the X/Y ratio of the screen (about 1.77...).
5. I multiply both the up vector and right-angle vectors by the tan() of the vertical FOV divided by 2.
6. I get the ratio of the mouse's X and Y coordinates of the screen compared to the size of the screen. I also offset the coordinate so that the center screen is [0, 0].
7. I multiply the up vector by the mouse's Y offset ratio, and likewise multiply the right-angle vector by the mouse's X offset ratio.
8. The vector for the ray is now the target vector plus the up vector plus the right-angle vector.

What I'm doing, conceptually, is that I'm constructing a model of the screen 1 unit in front of the camera, facing the camera directly, and "scaled" to exactly cover the FOV of the camera. I'm then simply calculating where on that screen the mouse is. Then I draw a line between the camera and where the mouse is calculated to be on that model screen. It's all done in world-space using vectors.

Member function of the Camera:

XYZf calculateRay(const Mouse& mouse, XY<unsigned short> screenResolution)
{
screenResolution /= 2;
XYi offset = XYi{ mouse.pos.x, mouse.pos.y } - screenResolution;

XYZf camV = target - pos;
camV.normalize();

XYZf camSide = camV.cross(up);
XYZf camUp = camV.cross(camSide);

float t = tan((fovY / 2) * 0.0174532925);

camSide *= ratio;
camUp *= t;
camSide *= t;
camUp *= float(offset.y) / screenResolution.y;
camSide *= float(offset.x) / screenResolution.x;

XYZf rayVec = camV + camUp + camSide;
rayVec.normalize();
return rayVec;
}

XY and XYZ are 2D and 3D vectors. The suffix is the data type.

[MohammedPlaysMC]

Iv fixed my problem straight after i posted it anyway i better get back to studies because i ant really supposed to be working on the game but its ever 2 weeks so it gives a lot of time to work on GCSE's anyway great tutorial :D

[haxic]

I thought of an alternative way to do mouse picking (although not sure if possible or exactly how to do it):
- Each relevant entity that is able to be "picked" with the mouse should have an unique id.
- A shader (I don't know which one) should have a uniform variable "entity id" which is set by each individual mouse-pickable entity.
- Using the depth buffer, the shader renders the scene to an "entity id texture", where each pixel represents an entity id (where e.g. rgb(0, 0, 0) means null / no entity).
- An entity is picked by the mouse by looking up the pixel in the entity id texture matching the mouse screen coordinate.

If the above is possible at all, it most certainly would reduce performance, since it requires rendering the scene one more time. I thought of some optimizations for it:
- The shader should not render any special effects, like lighting, normal mapping etc. Also, don't render any objects that you want to be able to click through e.g. some, if not all, particles.
- Reduce the resolution of the entity id texture. It may reduce per-pixel accuracy, but should improve performance.
- Only render to a small texture, say 1*1 or 3*3 pixels, at the mouse screen coordinate. All vertex calculations still would be performed, but the shader would only render to 1*1 (1) or 3*3 (9) pixels rather than whatever resolution you are using.

[kudodev]

Your method of finding the intersection on terrain is cool. But what I would do is loop through the terrains triangles first checking if the ray intersects the triangle's plane, this will give you a point. Then check whether that point is within the triangle.

[GieneqAD]

I really appreciate your hard work, its just amaizing! Thanks, and keep going :)