A question about Simple3D color blending (and maybe ray tracing?)

[Brackets-Coder]

Hello @Xeltalliv,

This question is mainly for you. Yes, I know the reference to RT in the title seems out of place, but have patience and bear with me as I attempt to express my insanity with words and explanatory context. I'm here in hopes that I will be able to communicate my concerns with you and more easily reach you than on your Scratch profile. The Simple3D documentation says that the extension can be used for 2D projects due to instancing, that "despite lack of programmable shaders, it is still quite powerful," and "rendering into textures allowing multi-pass rendering and more advanced clipping than Clipping & Blending extension." So this is where the fun begins. Having years of prior experience with raytracing and path tracing in Blender, Unreal Engine 5, Three.js, and other softwares--although never having previously written a ray tracer--and having a detailed familiarity with the basic concepts of how raytracing is computed, I decided I would attempt (whether successfully or unsuccessfully I do not know) to write a screen-space-2d-limited-ray-tracing-like-post-processing(ish) effect in Simple3D with some very primitive global illumination (and possibly reflection features). The idea was both to enhance my programming abilities and as a proof of concept for the true power of "Simple"3D, not because it would be easy or a good idea, but actually because it was a bad idea. Considering that the development of #1974 has brought the possibility of large-scale, performant triangle raycasting to Turbowarp, and with Simple3D's transformation and color capabilities, I thought it might be possible (although definitely not feasible) to make some sort of global illumination effect in Simple3D. All that to say is, here's my problem:

Additive blending in Simple3D. Either I'm doing something wrong, or drawing/instancing a mesh with additive blending doesn't blend additively with other draw calls/instances of itself, only other meshes. Additionally, the pen extension, although it can blend additively in Clipping & Blending, does not appear to interact with Simple3D or its blending features in any way, and I'm pretty sure this is confirmed by looking at its source code. So my question is: Is self-mesh additive blending in Simple3D possible, and if so, how? Is Simple3D blending with the pen extension possible? Alternatively, I've considered creating a hybrid rasterization and ray tracing rendering engine utilizing Pen+ V7's shaders, which would probably be much easier, but why do easy when you can do hard? Yes, it's stupid, and yes, it probably shouldn't be done, but that's why it hasn't been done yet.

[yuri-kiss]

insanity

[yuri-kiss]

u should join da disc btw

[Brackets-Coder]

Yes, it's insanity, but why not? Also, @CubesterYT asked me to join discord but I'm not really a social media person because people on the internet are stupid. I don't have a problem with joining the Turbowarp discord community, but it's the other communities that are being weird and immoral that I don't want to get into and Github seems to target a more specific, professional audience that doesn't often get uncomfortable, so I'd rather just stay here on Github if that's okay with ya'll

[Xeltalliv]

Is self-mesh additive blending in Simple3D possible

It should work. Perhaps you forgot to disable Z-buffer testing, so only the first thing gets drawn, and the latter ones fail because they aren't closer. Or maybe it's some weird quirk of your GPU.

Is Simple3D blending with the pen extension possible?

Not currently. You have additive blending within Simple3D layer and additive blending within pen layer. But the thing that composites different layers together doesn't have a way to control blending.

[Brackets-Coder]

That clears things up. No pen, good to know.

So essentially what you're telling me is I'd have to set up a 2D orthographic camera for rendering the additive dots (light rays) that make up the global illumination? Also, do looks affects change how re-applied textures appear? E.g., switch costume to [texture], set color effect to [50] set [my mesh] texture to [(texture from costume [texture])] would make the texture appear with a changed color? Basically everything involving the raycasting for RT would work, it's the pixel/ray color computation that's the main holdup.

Also, wouldn't it be a good idea to start with a global color multiplier of the clear color (so there's minimal ambient lighting) and then apply GI so you don't have the rasterizer conflicting with the GI lighting?

Here's my current setup:

I probably forgot to disable the depth testing...

[Xeltalliv]

So essentially what you're telling me is I'd have to set up a 2D orthographic camera for rendering the additive dots (light rays) that make up the global illumination?

Also, wouldn't it be a good idea to start with a global color multiplier of the clear color (so there's minimal ambient lighting) and then apply GI so you don't have the rasterizer conflicting with the GI lighting?

You haven't explained how are you planning to implement a screen-space-2d-limited-ray-tracing-like-post-processing(ish) effect in Simple3D with some very primitive global illumination (and possibly reflection features), so I have no idea what those questions are referring to

Also, do looks affects change how re-applied textures appear?

no

[Brackets-Coder]

If you have Blender 4.2 or higher, Eevee-next's raytracing system is essentially what I'm going for. Since I only have the colors on the screen to work with, I could technically cache the vertex colors of off-screen meshes for non-screen-space path tracing, but that would be impractical and very unperformant and almost impossible to implement off-screen textures, so I'm going to stay screen space. I've not written any kind of advanced ray cast based graphics yet so I was planning on starting with something simple. While I'm familiar with the backwards process of ray tracing, I was planning on something similar to modern ray tracing methods or going in a backwards way. Basically, I was just planning on applying some GI and possibly reflections on top of the rasterized frame.

Based on modern standards, I could do an algorithm like:

1. Cast rays from the pixels of the camera into the environment to bounce and collect the rasterized diffuse color information from the vertices and textures. The number of rays per pixel is the number of samples used for ray tracing. 
2. Let the rays bounce around with a limited bounce amount and "stamp" a computed color (based on where lights are, distance from faces, etc. etc.) onto surrounding faces for global illumination. For perfectly reflective surfaces (mirrors), use both diffuse colors and GI as the projected surface colors to make it appear to reflect. (this is the complex step with all the color management)
3.  Possibly some kind of blended-based denoising (I doubt it) then move on to the next frame and restart without accumulative progression. 

Or, I could do it an easier (but unconventional) way:

1. Cast a limited number of rays from the lights into the scene, direction depending on the type of light.
2. Let the rays bounce around the scene a limited number of times, collecting diffuse color information as they go and "stamping" the computed color onto the faces each bounce
3. next frame

Both of these are highly simplified processes of real ray tracing and there's a lot of other factors and biases that could be introduced (like resolution, precision, etc.) and if I do the ambient lighting and global illumination perfectly there should be no need for shadow maps or ambient occlusion...

Basically it comes down to the fact that I can't decide where I want my rays to start casting, either the camera or light sources, but there's also a light count limitation depending on which algorithm I want to use.... things just get really tricky and insane but that's what makes it fun :)

[Brackets-Coder]

Also, do looks affects change how re-applied textures appear?

no

This is what is going to make it really tricky, so I'd have to use vertex colors for everything if I want changing colors. That's probably better anyway but the testing versions used a red transparent circle texture that I can discard in favor of vertex colors