How do I transfer a very large array to the shader program?

In OpenGL ES graphics and image processing, we often encounter a situation: how to pass a large array to the shader program?

At present, there are three common methods:

• Use the method of loading the array into the 2D texture, and then use textelfetch to get the data;
• Use the uniform buffer object, that is, UBO;
• Use the texture buffer object, TBO.

Load array into texture

It is the easiest way to think of to transfer large arrays by loading arrays into textures.

In order to accurately exchange the value of each pixel, the sampling function texture cannot be used at this time, because the sampling function will involve complex operations such as normalization, filtering and interpolation, and it is basically impossible to obtain the value of an exact pixel.

At this time, we need to use the texture element acquisition function texlFetch. texlFetch is an API introduced by OpenGL ES 3.0. It regards the texture as an image and can accurately access the content of pixels. We can obtain the value of an element of the array by analogy through index.

vec4 texelFetch(sampler2D sampler, ivec2 P, int lod);

vec4 texelFetch(sampler3D sampler, ivec3 P, int lod);

vec4 texelFetch(samplerBuffer sampler, int P);

texelFetch uses non normalized coordinates to directly access the texture pixels without performing any form of filtering and interpolation. The coordinate range is the width and height of the actually loaded texture image.

Textelfetch is easy to use. In the fragment shader, the following two lines of code can be interchanged, but the final rendering results will be slightly different. Why are there slight differences? You taste, you taste!

gl_FragColor = texture(s_Texture, v_texCoord);
gl_FragColor = texelFetch(s_Texture,  ivec2(int(v_texCoord.x * imgWidth), int(v_texCoord.y * imgHeight)), 0);

Using a uniform buffer object

We often use variables of uniform type to pass some vectors to the shader program to participate in rendering operations.

However, OpenGL ES has a limit on the number of uniform variables that can be used. We can use the glGetIntegerv function to obtain the maximum supported number of uniform variables.

int maxVertexUniform, maxFragmentUniform;
glGetIntegerv(GL_MAX_VERTEX_UNIFORM_COMPONENTS, &maxVertexUniform);
glGetIntegerv(GL_MAX_FRAGMENT_UNIFORM_COMPONENTS, &maxFragmentUniform);

At present, the mainstream mobile phones generally support 1024 uniform variables (vector s). When using large arrays, it is easy to break through this limit, and the uniform variables are not easy to manage. You need to set the uniform variables again and again.

So how can we break through the limit of the number of uniform variables?

The answer is to use UBO (Uniform Buffer Object).

UBO, as its name implies, is a buffer object that loads uniform variable data. It is essentially no different from other buffer objects in OpenGL ES, and the creation method is roughly the same. It is an area on the video memory used to store specific data.

When the data is loaded into UBO, these data will be stored on UBO instead of being handed over to the shader program, so they will not occupy the uniform storage space of the shader program itself. UBO is a new data transfer method from memory to video memory. In addition, UBO generally needs to be used in conjunction with the uniform block.

In this example, the MVP transformation matrix is set as a uniform block, that is, three matrices will be saved in the UBO we create later.

#version 310 es
layout(location = 0) in vec4 a_position;
layout(location = 1) in vec2 a_texCoord;

layout (std140) uniform MVPMatrix
{
    mat4 projection;
    mat4 view;
    mat4 model;
};

out vec2 v_texCoord;
void main()
{
    gl_Position = projection * view * model * a_position;
    v_texCoord = a_texCoord;
}

Set the binding point of the uniform block to 0 to generate a UBO.

GLuint uniformBlockIndex = glGetUniformBlockIndex(m_ProgramObj, "MVPMatrix");
glUniformBlockBinding(m_ProgramObj, uniformBlockIndex, 0);


glGenBuffers(1, &m_UboId);
glBindBuffer(GL_UNIFORM_BUFFER, m_UboId);
glBufferData(GL_UNIFORM_BUFFER, 3 * sizeof(glm::mat4), nullptr, GL_STATIC_DRAW);
glBindBuffer(GL_UNIFORM_BUFFER, 0);

//Define the range of buffer with binding point 0
glBindBufferRange(GL_UNIFORM_BUFFER, 0, m_UboId, 0, 3 * sizeof(glm::mat4));

When drawing, update the data of the Uniform Buffer, update the data of the three matrices, and pay attention to the offset.

glBindBuffer(GL_UNIFORM_BUFFER, m_UboId);
glBufferSubData(GL_UNIFORM_BUFFER, 0, sizeof(glm::mat4), &m_ProjectionMatrix[0][0]);
glBufferSubData(GL_UNIFORM_BUFFER, sizeof(glm::mat4), sizeof(glm::mat4), &m_ViewMatrix[0][0]);
glBufferSubData(GL_UNIFORM_BUFFER, 2 *sizeof(glm::mat4), sizeof(glm::mat4), &m_ModelMatrix[0][0]);
glBindBuffer(GL_UNIFORM_BUFFER, 0);

Using texture buffer objects

Texture Buffer Object, i.e. TBO (Texture Buffer Object), is a concept introduced by OpenGL ES 3.2. Therefore, the version of OpenGL ES should be checked first when using. For Android, API > = 24 should be guaranteed.

TBO needs to be used together with Buffer Texture. Buffer Texture is a one-dimensional texture, which stores data from texture buffer object (TBO) to allow shaders to access large memory tables managed by buffer object.

In GLSL, only the textelfetch function can be used to access the buffer texture. The sampler type of the buffer texture is samplerBuffer.

The method of generating a TBO is similar to that of VBO, which only needs to be bound to GL_TEXTURE_BUFFER, and the buffer texture is generated in the same way as an ordinary 2D texture.

//Generate a Buffer Texture 
glGenTextures(1, &m_TboTexId);

float *bigData = new float[BIG_DATA_SIZE];
for (int i = 0; i < BIG_DATA_SIZE; ++i) {
    bigData[i] = i * 1.0f;
}

//Generate a TBO and upload a large array to TBO 
glGenBuffers(1, &m_TboId);
glBindBuffer(GL_TEXTURE_BUFFER, m_TboId);
glBufferData(GL_TEXTURE_BUFFER, sizeof(float) * BIG_DATA_SIZE, bigData, GL_STATIC_DRAW);

delete [] bigData;

For fragment shaders using texture buffer, the extended texture buffer needs to be introduced. Note that the version is declared as #version 320 es.

#version 320 es
#extension GL_EXT_texture_buffer : require
in mediump vec2 v_texCoord;
layout(location = 0) out mediump  vec4 outColor;
uniform mediump samplerBuffer u_buffer_tex;
uniform mediump sampler2D u_2d_texture;
uniform mediump int u_BufferSize;
void main()
{
    mediump int index = int((v_texCoord.x +v_texCoord.y) /2.0 * float(u_BufferSize - 1));
    mediump float value = texelFetch(u_buffer_tex, index).x;
    mediump vec4 lightColor = vec4(vec3(vec2(value / float(u_BufferSize - 1)), 0.0), 1.0);
    outColor = texture(u_2d_texture, v_texCoord) * lightColor;
}

How do I use buffer textures and TBO when painting?

glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_BUFFER, m_TboTexId);
glTexBuffer(GL_TEXTURE_BUFFER, GL_R32F, m_TboId);
GLUtils::setInt(m_ProgramObj, "u_buffer_tex", 0);

It is roughly the same as normal textures, except that you need to use glTexBuffer to bind TBO to buffer textures.

In this example, we take the value of the buffer texture in the range of [0~size-1] and normalize the value result as the sampling result of the light color superimposed on the 2D texture.

As shown in the above figure, the effect is that the color intensity increases from the upper left corner to the lower right corner.

reference resources

https://www.khronos.org/opengl/wiki/Buffer_Texture https://www.khronos.org/registry/OpenGL-Refpages/es3/

-- END --