FXGL 3D capabilities question

[Suicolen]

Hi, I have been using JavaFX for my GUI applications for almost 5 years now, I remember hearing about FXGL a few years ago and by then i still remember it mentioned 'Experimental 3D' in it's readme, i also checked out some of the FXGL 3D videos.
I'm not sure how much FXGL's 3D capabilities have improved since then though.

Recently i had the idea to start a new project, which was my own very simple 3D model editor and i wanted to use JavaFX for it.
The main features i would want it to have are:

• Ability to create new triangles with the mouse, either by clicking any point on the screen to create a new vertex or by clicking an existing vertex and then a point on the screen to create an edge
• Ability to manipulate existing vertices and triangles with a transformation gizmo (I know how to do this already if i can figure out the first feature)

However even creating the first feature is quite painful with 'pure' JavaFX, because to do that i would need to somehow figure out how to convert a 2D mouse position to 3D, the hardest part has been figuring out the correct depth.
The best way I've found to do this is by using the internal CameraHelper class and it's pickProjectPlane(camera, x, y) method
But i haven't even been able to get that to work nicely, it seems like if i use a PerspectiveCamera with fixedEyeAtCameraZero set to true it never gives the correct result, it is close to some extent tho, if i have it as false then it works better and the only issue is if the camera is zoomed in/out, then the depth values are not what I'd expect, but it also messes with the scale of everything, the docs also say that if i transform the PerspectiveCamera (which i do, as i want to move freely in space with my mouse) then it's recommended for it to be true.

My question is, does FXGL offer anything out of the box that makes projecting 2D mouse coordinates to the 3D scene coordinates any easier? I've also looked at FXyz but couldn't find anything related to that i could easily use for my purpose.
Here is an example GIF (from existing 3d modeling software) of the first feature i mentioned: https://imgur.com/a1txiO0

[AlmasB]

Hi, I'm afraid there is currently nothing out-of-the-box that supports the first feature you talk about. FXGL's 3D is limited to creating a few extended primitives, loading .obj and being able to translate vertices of a mesh (this last one could be somewhat helpful?).

I wonder if this problem (2D mouse to 3D environment) has a generic engine-agnostic solution somewhere on the web. If so, then it becomes a problem of figuring out how to translate that to JavaFX, which I'm hoping is the easier problem.

P.S. FXyz devs have a lot of experience with 3D, I wonder if they can provide some helpful hints.

[Suicolen]

Hi, unfortunately this problem of translating it to JavaFX is the hardest part, i know exactly how to do it in raw opengl for example, essentially the steps are:

1. unproject screen coordinates at z = 0 and z = 1 for the origin and direction, then just subtract the origin from the direction and normalize, that gets the ray
2. intersect the ray with some plane and the intersection point is the 3D position

From what i know this is not easy to do in JavaFX at all because to unproject, you would need access to the cameras view/projection matrix (or the combined one) which JavaFX doesn't expose, there is an internal CameraHelper#pickProjectPlane but like i already mentioned, this doesn't seem to work correctly with cameras that have their eye position fixed at 0

I'll create an issue in the FXyz repo and hopefully they can help