Can you program GPUs and do you want to become a HERO? #linuxphone
community needs your help.
We are trying record video, and have most pieces working, but one is
missing: fast enough debayering. That means about 23MB/sec on #librem5.
Debayering is not hard; camera images have subpixels split on two
lines, which need to be corrected. They also use different color
representation, but that's fixable by some table lookup and two matrix
multiplies.
Librem 5 has Vivante GPU, 4 in-order CPU cores and 3GB RAM. My feeling
is that it should be fast enough for that. If task is for some reason
impossible, that would be good to know, too.
Image data looks like this
RGRGRG...
xBxBxB...
.........
.........
Task is to turn that into usual rgbrgb.... format. rgb = RGB * color
matrix, with table lookups for better quality. I can fix that once I
get an example.
I'm looking for example code (#pinephone would work, too), reasons it
can not be done... and boosts if you have friends that can program
GPUs. #gpu #opensource
community needs your help.
We are trying record video, and have most pieces working, but one is
missing: fast enough debayering. That means about 23MB/sec on #librem5.
Debayering is not hard; camera images have subpixels split on two
lines, which need to be corrected. They also use different color
representation, but that's fixable by some table lookup and two matrix
multiplies.
Librem 5 has Vivante GPU, 4 in-order CPU cores and 3GB RAM. My feeling
is that it should be fast enough for that. If task is for some reason
impossible, that would be good to know, too.
Image data looks like this
RGRGRG...
xBxBxB...
.........
.........
Task is to turn that into usual rgbrgb.... format. rgb = RGB * color
matrix, with table lookups for better quality. I can fix that once I
get an example.
I'm looking for example code (#pinephone would work, too), reasons it
can not be done... and boosts if you have friends that can program
GPUs. #gpu #opensource
@pavel do you have a single frame of raw pixel data? What is the target API (OpenGL, -ES, Vulkan)?
@datenwolf Example of frame is here: https://gitlab.com/tui/tui/-/blob/master/ucam/bayer2rgb.rs?ref_type=heads (I also have frame generator and real frames captured from libobscura).
Anything that works on Librem 5 is fine, bonus points if I can understand it. Robot generated code using -lEGL -lGLESv2 -lm ... and that builds and does something. Librem 5 reports:
Vendor: etnaviv
Renderer: Vivante GC7000 rev 6214
OpenGL Version: OpenGL ES 2.0 Mesa 21.2.6
GLSL Version: OpenGL ES GLSL ES 1.0.16
Anything that works on Librem 5 is fine, bonus points if I can understand it. Robot generated code using -lEGL -lGLESv2 -lm ... and that builds and does something. Librem 5 reports:
Vendor: etnaviv
Renderer: Vivante GC7000 rev 6214
OpenGL Version: OpenGL ES 2.0 Mesa 21.2.6
GLSL Version: OpenGL ES GLSL ES 1.0.16
@pavel are you limited to OpenGL ES 2.0 or can you use a more modern version? ES-2.0 is very bare bones in its image format and shader capabilities and efficiently converting 10 bpp will be a PITA, due to lack of texelFetch function.
Anyway, spent the day finding a nice polynomial to linearize the sensor values (LUTs should be avoided if possible, memory access has latency and costs energy, if you can calculate in a few instr. prefer that).
@datenwolf I believe I'm limited to OpenGL ES 2.0. Presumably hardware can do more but our current drivers can not, so we are stuck there.
On the other hand... #librem5 main sensor can not do 10bpp at the moment, due to missing drivers. So maybe we can focus on 8bpp, first. Probably ineffecient conversion is "good enough" too, as GPU is a bit overpowered for this job.
On the other hand... #librem5 main sensor can not do 10bpp at the moment, due to missing drivers. So maybe we can focus on 8bpp, first. Probably ineffecient conversion is "good enough" too, as GPU is a bit overpowered for this job.
Follow
@pavel @datenwolf Current Mesa can do bunch of GLES3 stuff already, including texelFetch, once you force it with MESA_GLES_VERSION_OVERRIDE.
