从GPUImage
开始学习OpenGL ES
OpenGL ES
- iOS使用的OpenGL ES版本是2.0
- GLKit的大部分方法已经被废弃掉,推荐使用Metal
坐标系
屏幕坐标系
- OpenGL ES 的屏幕坐标系
- 设备屏幕坐标系
参考
🌟🌟🌟🌟🌟(讲解的非常详细):OpenGL ES 2.0 (iOS)
🌟🌟🌟🌟🌟(总结很全):OpenGL ES, 初学者的自我总结
🌟🌟🌟🌟🌟(主要是实际运用):Lyman’s Blog
🌟🌟🌟🌟🌟(OpenGL ES 2.0 文档):OpenGL ES Software Development Kit
GPUImage
GLProgram
主要负责着色器程序的编译链接。
GPUImageInput
定义输入协议。
GPUImageOutput
定义输出对象,下面都是其子类。
- GPUImageVideoCamera (for live video from an iOS camera)
- GPUImageStillCamera (for taking photos with the camera)
- GPUImagePicture (for still images)
- GPUImageMovie (for movies)
GPUImageContext
直接使用单例,管理GPUImage上下文,核心使用了EAGLContext
。
GPUImageVideoCamera
availableVideoCVPixelFormatTypes
Format | YUV | Discussion |
---|---|---|
kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange |
✅ | Bi-Planar Component Y’CbCr 8-bit 4:2:0 video-range (luma=[16,235] chroma=[16,240]). baseAddr points to a big-endian CVPlanarPixelBufferInfo_YCbCrBiPlanar struct. |
kCVPixelFormatType_420YpCbCr8BiPlanarFullRange |
✅ | Bi-Planar Component Y’CbCr 8-bit 4:2:0 full-range (luma=[0,255] chroma=[1,255]). baseAddr points to a big-endian CVPlanarPixelBufferInfo_YCbCrBiPlanar struct. |
kCVPixelFormatType_32BGRA |
❌ | 32 bit BGRA. |
设置yuvConversionProgram
runSynchronouslyOnVideoProcessingQueue
,由于切换了线程,使用useImageProcessingContext
来重新设置当前上下文。
GPUImageVideoCameraDelegate
原始帧回调,在这里获取的是系统的原始帧回调,优先于target处理。
kCVImageBufferYCbCrMatrixKey
Describes the color matrix used for YCbCr to RGB conversion
CVBufferGetAttachment
获取颜色模式
Type | Detail |
---|---|
kCVImageBufferYCbCrMatrix_ITU_R_601_4 | The conversion matrix for standard digital television images (following the ITU R 601 standard). |
YUV
GPUImage默认获取的格式就是YUV的。
RGBA
OpenGL ES
GLProgram
Function | Detail |
---|---|
glCreateProgram |
创建一个着色器程序对象 |
glCreateShader |
创建一个空着色器对象 |
glShaderSource |
替换一个着色器对象的源程序 |
glCompileShader |
编译一个着色器对象 |
glGetShaderiv |
1. GL_COMPILE_STATUS 检查编译状态 2. GL_INFO_LOG_LENGTH 获取编译信息Log的长度 3. GL_LINK_STATUS 检查链接状态 |
glGetShaderInfoLog |
获取编译信息Log |
glAttachShader |
关联一个着色器对象到一个着色器程序对象上面 |
glLinkProgram |
链接一个着色器程序对象 |
glDeleteShader |
删除一个着色器对象 |
glUseProgram |
安装一个着色器程序对象 |
glDeleteProgram |
删除一个着色器程序对象 |
Function | Detail |
---|---|
glActiveTexture |
选择一个活跃的纹理单元GL_TEXTURE[i] |
glBindAttribLocation |
关联一个顶点属性到顶点变量 |
glEnableVertexAttribArray |
enable一个定点属性数组 |
glGetUniformLocation |
返回唯一变量的位置 |
glDeleteProgram |
删除一个着色器程序对象 |
glDeleteProgram |
删除一个着色器程序对象 |
glDeleteProgram |
删除一个着色器程序对象 |
glDeleteProgram |
删除一个着色器程序对象 |
glDeleteProgram |
删除一个着色器程序对象 |
glDeleteProgram |
删除一个着色器程序对象 |
OpenGL Shading Language
OpenGL着色器语言。
There are four main types: float, int, bool and sampler.
- For the first three types, vector types are available:
Type | Detail |
---|---|
vec2, vec3, vec4 | 2D, 3D and 4D floating point vector |
ivec2, ivec3, ivec4 | 2D, 3D and 4D integer vector |
bvec2, bvec3, bvec4 | 2D, 3D and 4D boolean vectors |
- For floats here are also matrix types:
Type | Detail |
---|---|
mat2, mat3, mat4 | 2x2, 3x3, 4x4 floating point matrix |
- Samplers are types representing textures. They are used for texture sampling.
Type | Detail |
---|---|
sampler1D, sampler2D, sampler3D | 1D, 2D and 3D texture |
samplerCube | Cube Map texture |
sampler1Dshadow, sampler2Dshadow | 1D and 2D depth-component texture |
Sampler types have to be uniform. They are not allowed to be declared as a non-uniform type. Here are the different sampler types
- There are three types of inputs and outputs in a shader: uniforms, attributes and varyings.
Type | Detail |
---|---|
uniform |
Uniforms are values which do not change during a rendering, for example the light position or the light color. Uniforms are available in vertex and fragment shaders. Uniforms are read-only. |
attribute |
Attributes are only available in vertex shader and they are input values which change every vertex, for example the vertex position or normals. Attributes are read-only. |
varying |
Varyings are used for passing data from a vertex shader to a fragment shader. Varyings are (perspective correct) interpolated across the primitive. Varyings are read-only in fragment shader but are read- and writeable in vertex shader (but be careful, reading a varying type before writing to it will return an undefined value). If you want to use varyings you have to declare the same varying in your vertex shader and in your fragment shader. |
All uniform, attribute and varying types HAVE to be global. You are not allowed to specify a uniform/attribute/varying type in a function or a void.
- Shader output:
Type | Detail |
---|---|
gl_Position |
4D vector representing the final processed vertex position. Only available in vertex shader. |
gl_FragColor |
4D vector representing the final color which is written in the frame buffer. Only available in fragment shader. |
gl_FragDepth |
float representing the depth which is written in the depth buffer. Only available in fragment shader. |
- There are also many built-in function which can (and should) be used:
Type | Detail |
---|---|
dot |
a simple dot product |
cross |
a simple cross product |
texture2D |
used for sampling a texture |
normalize |
normalize a vector |
clamp |
clamping a vector to a minimum and a maximum |
VertexShader
🌰 YUV => BGRA && BGRA => YUV
1 | /// kGPUImageVertexShaderString |
转换的过程中在定点着色器上没有进行任何操作,只是将纹理坐标专递给片元着色器使用。
FrameShader
🌰 YUV => BGRA
- kCVPixelFormatType_420YpCbCr8BiPlanarFullRange => kCVPixelFormatType_32BGRA
1 | GLfloat kColorConversion601FullRangeDefault[] = { |
1 | varying highp vec2 textureCoordinate; |
- kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange => kCVPixelFormatType_32BGRA
1 | GLfloat kColorConversion601Default[] = { |
1 | varying highp vec2 textureCoordinate; |
🌰 BGRA => YUV
参考文章:
未完待续。。。。