Reading a Steam Controller overlay and rendering it onto a VR scene in SteamVR

Question

Background

When using the Steam Controller, you can configure the touch panels as complex menus that appear on the screen when you start touching them. These menus are very convenient for games with many controls. They look like this:

However, when playing on SteamVR, none of these overlays appear in VR. They appear only on the desktop, which is displayed with it.

This is a bug or missing feature that is known. My goal was to write a fix. But it turned out to be quite complicated.

My problem

I know how to make an overlay in OpenVR / SteamVR. For me, enough documentary documentation. But I did not come up with a good way to capture this overlay, which is drawn on top of the game window.

There are two ideas in my mind. But I do not know how to implement them correctly.

Overlay capture

I don't know anything about the technical details of the Overlay password controller. However, I think this is not a bad assumption that it has the same or very similar implementation as the overlay on the panel.

According to the Steamworks API documentation, Steam listens for Current / SwapBuffers calls that the application launches and updates the overlay accordingly. (See https://partner.steamgames.com/doc/api/ISteamUtils#BOverlayNeedsPresent ) I also assume that Steam receives hWnd windows, creates an OpenGL context with its DC and passes it each time it receives a SwapBuffers call, which he is listening.

But this is the end of the story. I don’t know how to actually read what Steam is having on it. I thought about connecting to a drawing message queue. Steam uses and captures what is displayed in the window before and after overlay. I also thought about tricking Steam into thinking that another window with a green background is the one that it should display, and then just grab its contents. But this is just speculation in the wild. I do not know how to implement this.

Using Chroma Keying

The selected button has a very specific flat and opaque blue tone. I could take screenshots from the window, and then take only pixels with that color. The disadvantage is quite obvious. All other areas are translucent and therefore unsuitable for color processing. The same goes for smoothed text. Fortunately, you can change the icon and its color, as in the picture below. So it would be some kind of work, but it would be extremely ugly.

Does anyone have a better idea or solution to my problem?