@pavel No, it's
RGRGRG
GBGBGB
You lose meaningful data if you ignore half of green pixels.
I see no reason why it couldn't be done. Just take care not to introduce needless copies in your processing path. dmabufs are your friends.
@pavel I'm confused. V4L lets you stream to a CMA dmabuf which should be importable as GL_TEXTURE_EXTERNAL_OES, right? Or am I missing something?
@pavel Megapixels is not an example of how to do things in the most performant way :) OpenGL operates in a VRAM-centric model, it's very copy-heavy. We don't need to copy things around, as our GPUs operate on the exact same memory CPUs do.
See GL_OES_EGL_image_external and https://docs.kernel.org/userspace-api/media/v4l/dmabuf.html
@pavel After eliminating glReadPixels and having the output buffer mmaped instead: "18.9 MB in 0.08s = 244.4 MB/s"
After putting glTexImage2D out of the loop to emulate zero-copy import from V4L as well:
"18.9 MB in 0.05s = 400.1 MB/s"
@pavel I left the memcpy line commented out for a reason - with it uncommented, the result is exactly the same as with glReadPixels (which is effectively a memcpy on steroids). The point is to pass that buffer to the encoder directly, so it can read the data straight from the output buffer without waiting for memcpy to conclude.
I've also verified that the approach is sound by having the shader output different values each frame and accessing it via hexdump_pixels inside the loop. Still fast ;)
@pavel > I can't easily connect gstreamer to that
Why not? I quickly hacked up passing dma-bufs to GStreamer and even though I'm glFinishing and busy-waiting on a frame to get encoded sequentially it still manages to encode a 526x390 h264 stream in real time on L5.
@pavel Yes, of course.
BTW. Turns out that streaming to YouTube instead of a local file is just a matter of using rtmpsink instead of filesink 😁
@pavel I'm playing with GStreamer now (which is new for me) and it seems like most of this code could be replaced with GStreamer elements, and the rest should neatly plug in as custom elements 😂
@pavel You've got a dma-buf handle, already mapped buffer and even GStreamer with all its sinks available, so... however you want? Pretty much anything will be able to consume it easily.
@pavel For GTK: either https://docs.gtk.org/gdk4/class.DmabufTextureBuilder.html or https://gstreamer.freedesktop.org/documentation/gtk4/index.html
For SDL with GL: just import it the same way V4L buffers are imported.
Frankly, it's flexible enough that your choice of toolkit should only depend on other factors.
@pavel Not sure what you mean. GStreamer is internally multi-threaded, but its API is thread-safe and there's only one thread in this code. Of course any kind of production-quality code will use some mainloop and enqueue buffers based on callbacks rather than while(!processed){} loop, but it's not exactly rocket science.
@pavel That one line is the only thing that runs from another thread and it's neither scary nor requires any locking 😁
But there are several other smelly things in this code and lots of missing error handling, so I'd rather start with that when looking for suspects.
@pavel Passing the right buffer size to gst_dmabuf_allocator_alloc helps it to not crash and not have garbage at the end of the frame 😂
Do I guess correctly that shaders can do arbitrary resolutions, such as 800x600?
I like the v4l+shaders integration. I'm not sure if I like the v4l+shaders+gstreamer integration.