🎉 Celebrating 25 Years of GameDev.net! 🎉

Not many can claim 25 years on the Internet! Join us in celebrating this milestone. Learn more about our history, and thank you for being a part of our community!

Back to Graphics and GPU Programming

Stencil buffer strange result

mllobera · 2024-06-05T16:40:40

My ultimate aim is to determine a sequence of (stencil) operations so that regardless of the order in which objects are rendered I can end up with the visible (non-occluded) part of any object. To this end, I have written some code where I am displaying two cubes one in front of the other. The smaller one is blocking partially the larger one. What I want to retrieve is the part of the larger cube that is visible. At the moment I am not able to accomplish this. The code is written in python using PyOpengl. The code shown here is used to create two renderings and save them on disk. The first rendering shows the distance from the camera and the second one is just the stencil buffer. I generate both renderings side by side in order to make sure that my work is okay. Here is the code I am using (except for the shader class which is a generic class that handles compilation and passing data to a shader), from OpenGL.GL import * from glfw.GLFW import * from glfw import _GLFWwindow as GLFWwindow import glm from shader_m import Shader import ctypes import numpy as np # settings SCR_WIDTH = 800 SCR_HEIGHT = 800 def main() -> int: # glfw: initialize and configure # ------------------------------ glfwInit() glfwWindowHint(GLFW_CONTEXT_VERSION_MAJOR, 3) glfwWindowHint(GLFW_CONTEXT_VERSION_MINOR, 3) glfwWindowHint(GLFW_OPENGL_PROFILE, GLFW_OPENGL_CORE_PROFILE) # glfw window creation # -------------------- window = glfwCreateWindow(SCR_WIDTH, SCR_HEIGHT, "LearnOpenGL", None, None) if (window == None): print("Failed to create GLFW window") glfwTerminate() return -1 glfwMakeContextCurrent(window) # configure global opengl state # ----------------------------- glEnable(GL_DEPTH_TEST) glEnable(GL_STENCIL_TEST) # build and compile shaders # ------------------------- shaderDepth = Shader("2.stencil_depth.vs", "2.stencil_depth.fs") # set up vertex data (and buffer(s)) and configure vertex attributes # ------------------------------------------------------------------ cubeVertices = glm.array(glm.float32, # positions # texture Coords -0.5, -0.5, -0.5, 0.0, 0.0, 0.5, -0.5, -0.5, 1.0, 0.0, 0.5, 0.5, -0.5, 1.0, 1.0, 0.5, 0.5, -0.5, 1.0, 1.0, -0.5, 0.5, -0.5, 0.0, 1.0, -0.5, -0.5, -0.5, 0.0, 0.0, -0.5, -0.5, 0.5, 0.0, 0.0, 0.5, -0.5, 0.5, 1.0, 0.0, 0.5, 0.5, 0.5, 1.0, 1.0, 0.5, 0.5, 0.5, 1.0, 1.0, -0.5, 0.5, 0.5, 0.0, 1.0, -0.5, -0.5, 0.5, 0.0, 0.0, -0.5, 0.5, 0.5, 1.0, 0.0, -0.5, 0.5, -0.5, 1.0, 1.0, -0.5, -0.5, -0.5, 0.0, 1.0, -0.5, -0.5, -0.5, 0.0, 1.0, -0.5, -0.5, 0.5, 0.0, 0.0, -0.5, 0.5, 0.5, 1.0, 0.0, 0.5, 0.5, 0.5, 1.0, 0.0, 0.5, 0.5, -0.5, 1.0, 1.0, 0.5, -0.5, -0.5, 0.0, 1.0, 0.5, -0.5, -0.5, 0.0, 1.0, 0.5, -0.5, 0.5, 0.0, 0.0, 0.5, 0.5, 0.5, 1.0, 0.0, -0.5, -0.5, -0.5, 0.0, 1.0, 0.5, -0.5, -0.5, 1.0, 1.0, 0.5, -0.5, 0.5, 1.0, 0.0, 0.5, -0.5, 0.5, 1.0, 0.0, -0.5, -0.5, 0.5, 0.0, 0.0, -0.5, -0.5, -0.5, 0.0, 1.0, -0.5, 0.5, -0.5, 0.0, 1.0, 0.5, 0.5, -0.5, 1.0, 1.0, 0.5, 0.5, 0.5, 1.0, 0.0, 0.5, 0.5, 0.5, 1.0, 0.0, -0.5, 0.5, 0.5, 0.0, 0.0, -0.5, 0.5, -0.5, 0.0, 1.0) # cube VAO cubeVAO = glGenVertexArrays(1) cubeVBO = glGenBuffers(1) glBindVertexArray(cubeVAO) glBindBuffer(GL_ARRAY_BUFFER, cubeVBO) glBufferData(GL_ARRAY_BUFFER, cubeVertices.nbytes, cubeVertices.ptr, GL_STATIC_DRAW) glEnableVertexAttribArray(0) glVertexAttribPointer(0, 3, GL_FLOAT, GL_FALSE, 5 * glm.sizeof(glm.float32), None) glEnableVertexAttribArray(1) glVertexAttribPointer(1, 2, GL_FLOAT, GL_FALSE, 5 * glm.sizeof(glm.float32), ctypes.c_void_p(3 * glm.sizeof(glm.float32))) glBindVertexArray(0) # shader configuration # -------------------- shaderDepth.use() # create framebuffers fbo = create_framebuffer() # bind the framebuffer glBindFramebuffer(GL_FRAMEBUFFER, fbo) # define basic viewing parameters camera_zoom = 45 camera_position = glm.vec3(0.0, 0.0, 4.0) camera_front = glm.vec3(0.0, 0.0, -1.0) camera_up = glm.vec3(0., 1., 0) view = glm.lookAt(camera_position, camera_position + camera_front, camera_up) projection = glm.perspective(glm.radians(camera_zoom), SCR_WIDTH / SCR_HEIGHT, 0.1, 100.0) # set uniforms on shader shaderDepth.setMat4("view", view) shaderDepth.setMat4("projection", projection) # clear colors and buffers glClearColor(0.1, 0.1, 0.1, 1.0) glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT) # Setting stencil buffer # -------------------------------------------------------------------- glStencilMask(0xFF) # where am I writing glStencilFunc(GL_ALWAYS, 1, 0xFF) # All fragments will pass glStencilOp(GL_KEEP, # stencil fails GL_REPLACE, # stencil passes, depth fails GL_KEEP) # stencil and depth passes # cube 1 glBindVertexArray(cubeVAO) model = glm.mat4(1.0) model = glm.translate(model, glm.vec3(0.0, 0.0, -0.75)) shaderDepth.setMat4("model", model) glDrawArrays(GL_TRIANGLES, 0, 36) # set it so that we do not draw the next to the stencil glStencilMask(0x00) # cube 2 scale = 0.25 model = glm.mat4(1.0) model = glm.scale(model, glm.vec3(scale, scale, scale)) model = glm.translate(model, glm.vec3(0.0, 0, 2.0)) shaderDepth.setMat4("model", model) glDrawArrays(GL_TRIANGLES, 0, 36) glBindVertexArray(0) # OUTPUTS #depth depth = glReadPixelsf(0,0, SCR_WIDTH, SCR_HEIGHT, GL_RED, GL_FLOAT) np.save('depth', np.flipud(depth)) # stencil stencil = glReadPixels(0,0, SCR_WIDTH, SCR_HEIGHT, GL_STENCIL_INDEX, GL_UNSIGNED_BYTE) stencil = np.frombuffer(stencil, dtype=np.uint8).reshape((800,-1)) np.save('stencil', np.flipud(stencil)) # unbind framebuffer back to default glBindFramebuffer(GL_FRAMEBUFFER,0) glfwSwapBuffers(window) glfwPollEvents() glDeleteVertexArrays(1, (cubeVAO,)) glDeleteBuffers(1, (cubeVBO,)) glfwTerminate() return 0 The framebuffer is generated using the following code, def create_framebuffer(): # create depth buffer fbo= glGenFramebuffers(1) glBindFramebuffer(GL_FRAMEBUFFER, fbo) # COLOR DEPTH color_depth_tex = glGenTextures(1) glBindTexture(GL_TEXTURE_2D, color_depth_tex) glTexImage2D(GL_TEXTURE_2D, 0, GL_R32F, SCR_WIDTH, SCR_HEIGHT, 0, GL_RED, GL_FLOAT, None) glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST) glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST) glBindTexture(GL_TEXTURE_2D, 0) # Attach depth to framebuffer glFramebufferTexture(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, color_depth_tex, 0) # DEPTH depth_tex = glGenTextures(1) glBindTexture(GL_TEXTURE_2D, depth_tex) glTexImage2D(GL_TEXTURE_2D, 0, GL_DEPTH_COMPONENT32F, SCR_WIDTH, SCR_HEIGHT, 0, GL_DEPTH_COMPONENT, GL_FLOAT, None) glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST) glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST) glBindTexture(GL_TEXTURE_2D, 0) # Attach depth to framebuffer glFramebufferTexture(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, depth_tex, 0) # STENCIL stencil_tex = glGenTextures(1) glBindTexture(GL_TEXTURE_2D, stencil_tex) glTexImage2D(GL_TEXTURE_2D, 0, GL_STENCIL_INDEX8, SCR_WIDTH, SCR_HEIGHT, 0, GL_STENCIL_INDEX, GL_UNSIGNED_BYTE, None) glBindTexture(GL_TEXTURE_2D, 0) # Attach stencil to framebuffer glFramebufferTexture2D(GL_FRAMEBUFFER, GL_STENCIL_ATTACHMENT, GL_TEXTURE_2D, stencil_tex, 0) # check for errors (after attaching a texture) if (glCheckFramebufferStatus(GL_FRAMEBUFFER) != GL_FRAMEBUFFER_COMPLETE): print(glCheckFramebufferStatus(GL_FRAMEBUFFER)) raise RuntimeError("ERROR.FRAMEBUFFER. Framebuffer is not complete!") # unbind framebuffer glBindFramebuffer(GL_FRAMEBUFFER, 0) return fbo The shader I use is very simple, *shaderDepth* #version 330 core layout (location = 0) in vec3 aPos; uniform mat4 model; uniform mat4 view; uniform mat4 projection; out vec4 pos; void main(){ pos = view * model * vec4(aPos, 1.0); gl_Position = projection * view * model * vec4(aPos, 1.0); } #version 330 core in vec4 pos; out float depth; void main(){ depth = length(pos.xyz); } Here are two renderings of what gets saved. As mentioned, the one on the left shows distance to the two cubes. The one to the right shows what is the result of the stencil operations above. (left) distance away from camera r(right) stencil output The small cube that we can see on the left is clearly in front of the large cube yet the output I get on the right does not seem to correspond at all with what I would expect based on the left image. There is definitely a disconnect between what I am rendering and the result I am getting from the stencil buffer. My expectation is that all fragments from the large cube pass the stencil test but those blocked by the small cube fail the depth test, however, this is not what I am getting. I am trying to figure out what I am doing wrong?

Graphics and GPU Programming Programming OpenGL Python pyopengl

Started by mllobera May 28, 2024 08:50 PM

15 comments, last by mllobera 3 weeks, 2 days ago

_Silence_

1,076

June 01, 2024 01:02 AM

mllobera said:
I also have to check to see how this would extend if there were several objects that I was interested in.

Just give different values to each in StencilFunc.

mllobera said:
I am still uncertain what was wrong with my previous thinking

From your first post, you were setting a value to the stencil buffer each time the fragment fails the depth test. I'm not sure if that's what you was wanting to do. Plus, since you was doing this in a single pass, stencil buffer was filled with 1 for both failed depth-test pixels from both objects.

mllobera

Author

June 03, 2024 01:03 AM

Thanks @_Silence_ !

From your first post, you were setting a value to the stencil buffer each time the fragment fails the depth test. I'm not sure if that's what you was wanting to do. Plus, since you was doing this in a single pass, stencil buffer was filled with 1 for both failed depth-test pixels from both objects.

Hmm… I need to think about this. From your suggestion it is clear that it works because whatever I draw will fail due to the depth test not passing. In my case, I was relying on the part that had to do with the stencil test passing and the depth test not passing. I guess I do not understand why this was not happening. I get a sneaky feeling that what I am not understanding is at what point the stencil test kicks in. I also cannot understand the original result I get given my original stencil test set up. What I managed to understand is what is the result I get from that original test, What gets recorded in the stencil buffer, are the sides of the cube that are not facing the camera (i.e. that is what is creating that square). Again, but I still do not understand why?

In any case thanks for the help!!!

_Silence_

1,076

June 03, 2024 09:11 PM

You just got the pixels that did not succeed the depth test, that is, the ones that are not visible, the ones that are hidden by previous treated pixels (ie pixels lying on the visible sides of the cube). This depends a lot on the order you sent the vertices.

Face culling happens before stencil, so you should not have such pixels at all.

mllobera

Author

June 04, 2024 11:52 PM

@_Silence_ Continuing our conversation (just trying to figure this out). This is what I had originally,

Clear color+depth+stencil
Set stencil mask to 255
Set the stencil test to ALWAYS pass, use as reference value 1 and function mask 255
Set the stencil operation to replace the stencil buffer value with the reference value whenever it passes the stencil test but fails the depth test
Draw a large cube (or following my example a house)
Set the stencil mask to 0 (so nothing passes the stencil test)
Draw a small cube in front of the large cube (or draw the tree), that is closer to the viewer.

So… the way I am thinking about it is that the StencilOp(GL_KEEP, GL_REPLACE, GL_KEEP)will only kick in after step 7?

Also,… if I wanted to record in the stencil buffer the visible parts of more than one object. How would one solve for the possible occlusion amongst these? (Hope this make sense).

Thanks again for everything.

_Silence_

1,076

June 05, 2024 11:54 AM

mllobera said:
So… the way I am thinking about it is that the StencilOp(GL_KEEP, GL_REPLACE, GL_KEEP)will only kick in after step 7?

No. Each time fragments that succeed the stencil test (you set it to always, so all fragments), GL will compare them to the depth buffer value at their coordinates. And each time these fragment depth value will fail the depth depth (basically being greater or equal), the stencil buffer will be updated, and a 1 will be set at their location.

Depending on the order you draw your cubes or send the vertices (and faces) of your cubes, results can be different from what you think. The sides of your first cube might be seen as front winded faces. And if you render them after the close face of your cube, they will fill the stencil buffer. This seems to be what's happening.

Since you render the closer cube after the far one, then nothing happens in the stencil operation for the seconde cube: all fragments succeed the depth buffer.

I think that if you move to a perspective instead of an orthographic projection, you'll have different values. My guess is that the stencil buffer will mostly remain empty (full of 0).

mllobera

Author

June 05, 2024 04:40 PM

Hi @_Silence_,

Okay, this is more complicated than I first imagined. I think that the main mistake I was making was thinking in terms of entire 3D objects when in reality I should be thinking in terms of primitives.

Might be worth restating what I would like to achieve and see whether this is possible at all. I am essentially looking at the building a terrain viewer on which the user can put 3D objects (there is no way of knowing what number of objects or in which order these are added to a scene). Imagine being able to wonder around this terrain. I would like the viewer to be able to select any object/s (better still parts of object/s if this was possible) and determine how much these parts are occluded by the terrain and/or other objects. My hope is to come up with a sequence of commands will achieve this.

Thanks again for your explanation, at least, now I understand better the results I was getting.

🎉 Celebrating 25 Years of GameDev.net! 🎉

Stencil buffer strange result

Popular Topics

Recommended Tutorials

🎉 Celebrating 25 Years of GameDev.net! 🎉

Stencil buffer strange result

Popular Topics

Recommended Tutorials

Reticulating splines