shaders, questions questions questions

advanced
#1

Question 1 particle system, uniforms, and the frag shader

I wrote a simple program I called glowBoids that used uniforms to supply the fragment shaders with the positions of the particles. The of course the fragment shader would iterate over all the particles (in this, n=32) and generally augment the brightness based on whatever force expression equation I was using.

The first limitations I encounter are obviously the hard limit on uniforms I can supply the shader, and the per-pixel number of operations that are feasible (it is surprisingly high, but limited).

Loading up the information in a texture is clearly the method for large data mass transfer, is there a float texture type I should be using?

I have been rendering to multiple framebuffers in series to cut down on the length of time each pixel takes to calculate in the fragment shader, and then combining the resulting textures using yet another shader.

What do I need to be doing in the fragment shader so I’m not doing 1024 operations per pixel?

Another thing I would love to do is manage the particle system on the GPU. Storing velocity and position vectors in textures and letting fragment shader do all the dirty work of calculating time+1 for me for everybody in parallel. I can do this, but how am I going about turning those 1M positions into pixels on my screen? In my dreams the fragment shader can write to a VBO. I just want to park things on the GPU and not have to rely on OpenCL within OF.

Question 2 what is that line?!?

Anything I do that is at all computationally expensive and full-screen will always introduce a nasty tracking scanline about an inch from the top of my screen. It will dance all around the screen if I don’t set the vertical sync, but the line stays put with vertical sync. This might be my video card, it might be because I use Linux. I’m getting a nice modern card this weekend, I will see what happens. It happens on multiple monitors, and my projector.

Question 3 why?

Besides “morphing” objects (understandably powerful) and doing per-vertex lighting calculations, what is the purpose of the vertex shader? I understand how powerful it is to animate a static model using just a few supplied vectors, but it is there a way to use the vertex shader to update individual particles in a simulation? Can the vertex shader read from other positions in the VBO? Can it write to a new VBO like the frag shader can ostensibly write to a texture if you use a FBO?

Besides “sparklies” and more morphing, what am I not getting about the geometry shader? Is the purpose of the geometry shader to provide highlights and increased detail at vertices?

I imagine drawing a greek column where just a single two point line is draw on the screen, but the vertices that make up the cylinder and detailed volutes at the top and bottom are drawn using new vertices provided by the geometry shader interpolated from just the single vector supplied to it.

Question 4 crosstalk

Aside from lighting information, what am I using the “varying” type for? I see no real interesting use for this, but I know I am wrong.

#2

q1 you can use ofFloatImage to load floating point textures onto the GPU. the input into the shader is still a sampler2DRect, normalized between 0-1, it just has higher resolution. alternatively, if you’re willing to break out into raw GL you can upload your own textures that are unnormalized floating point. card manufacturers are still coming to terms on the specs for these kinds of formats.

What do I need to be doing in the fragment shader so I’m not doing 1024 operations per pixel?

this really depends on your shader. different shaders are optimized in different ways. if you read about separable filters, you’ll find that certain blurs in interesting ways that would surprise you.

i’d also like to see a non-opencl particle-particle system. i was talking with memo about this a year ago, but we never made it happen. one way to do it is using a vbo where every vertex has a 2d coordinate corresponding to the location of a pixel in a texture. that texture stores the actual location (yes, you can do texture lookups in the vertex shader). if you want that point (particle) to have other properties that change over time, you can use extra textures.

q2 that line is called tearing. because your different displays have different refresh rates, or are out of phase with each other, your graphics card locks onto one of them and ignores the other. depending on your OS, your graphics card driver might have some options for deciding which screen to lock to. in general, if you are running faster than your refresh rate you will always get tearing, even with a single monitor. this is because the framebuffer is half updated but the graphics card sends it out anyway. waiting until it’s completely updated to send it out is called vertical sync.

q3 see the answer to q1. afaik you can’t write to a vbo the same way you can write to an fbo, but i would love to be proven wrong. also important about the vertex shader: sometimes the per-fragment operations you want to do are actually just per-vertex operations that can be linearly interpolated between the vertices. these interpolated variables are called ‘varyings’. a classic example is textured coordinates. you don’t compute the texcoords in every fragment (unless you’re doing bump mapping or something), you just compute them at the vertices and interpolate. another use of the vertex shader is geometry modification. consider the displacement map: one of the fastest ways to render a kinect-style depth mesh is to upload a grid of triangles once, and then upload a depth texture every frame. inside the vertex shader, you displace the vertices position based on the depth texture value. there are a number of other relevant vertex shader operations…

geometry shaders i’m still trying to understand. the way i think about them is as allowing you to describe a ‘skeleton’ of what you want, and have the rest computed on the gpu. for example, if you want to create ribbons, you could just describe the center of the ribbon and then extrude triangles from the center using the normals of the ribbon.

q4 see q3. varying is important for anything you can compute in the vertex shader that can be interpolated for every pixel in the fragment shader.

#3

I can get the higher resolution float data into the shader with floatImage.getTextureReference() but how am I getting the float texture back out?

  
  
  
ofFloatImage image  
ofTexture texUpload; texUpload = image.getTextureReference();  
ofTexture download;  
  
floatfbo.begin();  
  
shader.begin();  
  
shader.setUniformTexture(tex, 0);  
  
drawQuad();  
  
shader.end();  
  
floatfbo.end();  
  
download = floatfbo.getTextureReference();

(psuedocode)

Is download a high resolution texture?

#4

I can’t stress enough how much I appreciate y’alls help. I work day and night on my openframeworks programs and the help makes my efforts immensely more rewarding.

#5

use ofFbo::readToPixels into an ofFloatPixels. it should work, in theory, but i haven’t tested it rigorously.

also, i’ve attached an app i wrote for you last week. i meant to post it a while ago but forgot. it demonstrates the “gpu particle system” idea.

GLSLParticleSystem.zip

#6

For your q3:

Besides “morphing” objects (understandably powerful) and doing per-vertex lighting calculations, what is the purpose of the vertex shader? I understand how powerful it is to animate a static model using just a few supplied vectors, but it is there a way to use the vertex shader to update individual particles in a simulation?

This is what an attribute is http://www.lighthouse3d.com/tutorials/glsl-core-tutorial/attribute-variables/ so you can set properties of each vertex.

Can the vertex shader read from other positions in the VBO?

Using attributes you can pass the other positions into the vertex shader, but otherwise, no.

Can it write to a new VBO like the frag shader can ostensibly write to a texture if you use a FBO?

No. You can generate geometry using a geometry shader, but you’re still basing positions off of the vertices that you’ve been given. New VBOs require that you allocate buffers for them, which you can’t do on the graphics card.

#7

Damn Kyle, that’s fast. I get 10.000.000 particles at 60 fps… :slight_smile:

#8

10M particles at 60 fps sounds about right. if you get rid of velocity and just work with position it can get even faster, but it’s less interesting.

doing things like particle-particle interaction on the GPU with high numbers of particles is a little more complicated, but i think this kind of interaction between a small number of forces (just the mouse) and a lot of particles can be very optimized.

maybe the next step for that code i posted would be to upload a second texture that encodes multiple forces – one force per pixel. maybe RGBA = x,y, radius and strength.

the really annoying thing here is just working with the clamped floating point values. you always have to offset and scale your data after reading/before writing to a texture.

#9

I’ve been playing with this force-idea of Kyle so I thought I’d share. I added an additional ofFloatImage with force values, calculated using a number of static forcepoints. I also included a small shader to draw the resulting forces. I attached the project for anyone to play with.

GLSLParticleSystem.zip

#10

I was working on a GPU Flocking system for my project Efecto Mariposa ( http://www.patriciogonzalezvivo.com/butterfly.html ) and I arrive to this implementation of Daniel Shiffman flocking system on GLSL.
I´m using two textures, one for positions and other for velocities. The other 3 shaders ( frag, vert and geom) are for rendering the particles with images ( load it as a texture)

I think could be VERY optimize. Any body want to help with it?

The code it´s at: http://github.com/patriciogonzalezvivo/ofxFX

Here is the video
http://vimeo.com/30684308