Android UI in 3D: Rendering Jetpack Compose on OpenGL Surfaces via VirtualDisplay

While the use cases for this technique are fairly specialized, I want to demonstrate how to render the Android UI onto OpenGL surfaces using the Virtual Display mechanism.

Key Concepts:

  • Display — An object that describes a logical display. It contains the characteristics of real, networked, or virtual screens connected to an Android device. Documentation.

  • VirtualDisplay — A type of display that renders images to a specific memory buffer, called a Surface, rather than a physical panel. Documentation.

  • DisplayManager — An Android system service that manages all available displays, allowing you to query their characteristics and create virtual ones. Documentation.

  • Presentation — A specialized component based on the Dialog class. It doesn’t have its own full lifecycle (unlike an Activity), which simplifies management, and it is specifically designed to output content to any secondary display. Documentation.

Rendering with OpenGL

To begin, let’s create a simple example that renders only a rectangle. Only after we have verified that everything works will we start complicating the code and add the virtual display output to this object.

The easiest way to use OpenGL in Jetpack Compose is to wrap a GLSurfaceView using the AndroidView component.

In the initialization block of our GLSurfaceView, we specify the OpenGL ES 3.0 standard. I also overrode the surfaceDestroyed method to track when the view disappears from the screen—this is the exact moment when resources must be released.

interface PreviewRenderer: GLSurfaceView.Renderer {
    fun onSurfaceDestroyed()
}

@Composable
fun OpenGLPreview(
    modifier: Modifier = Modifier,
    renderer: PreviewRender,
) {
    AndroidView(
        factory = { context ->
            object:GLSurfaceView(context){
                init {
                    setEGLContextClientVersion(3)
                    setRenderer(renderer)
                }
                override fun surfaceDestroyed(holder: SurfaceHolder) {
                    super.surfaceDestroyed(holder)
                    renderer.onSurfaceDestroyed()
                }
            }
        },
        modifier = modifier,
    )
}

Our single MainActivity will look like this. Here, we initialize the renderer and set our OpenGLPreview as the main content:

class MainActivity : ComponentActivity() {

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        enableEdgeToEdge()
        val renderer = BoxRenderer()
        setContent {
            AndroidUiOnGlTheme {
                Scaffold(modifier = Modifier.fillMaxSize()) { innerPadding ->
                    OpenGLPreview(
                        modifier = Modifier
                            .fillMaxSize()
                            .padding(innerPadding),
                        renderer = renderer
                    )
                }
            }
        }
    }
}

We separate the rendering logic from the GLSurfaceView — it should reside within the GLSurfaceView.Renderer implementation. In our case, we use the PreviewRenderer interface because it allows us to handle the shutdown process (via the onSurfaceDestroyed method). At this early stage, it might seem redundant; however, proper resource cleanup will become critical later when we add the virtual display functionality.

class BoxRenderer(
    private val rotationDurationMs: Long = 15000L
) : PreviewRenderer {
    private val vertexShaderCode = """
        attribute vec4 aPosition;
        attribute vec2 aTexCoord;
        varying vec2 vTexCoord;
        uniform mat4 mvpMatrix;
        void main() {
            gl_Position = mvpMatrix * aPosition;
            vTexCoord = aTexCoord;
        }
    """.trimIndent()

    private val fragmentShaderCode = """
        precision mediump float;
        varying vec2 vTexCoord;
        void main() {
            gl_FragColor = vec4(vTexCoord.x, vTexCoord.y, 0.0, 1.0);
        }
    """.trimIndent()

    private val vertices = floatArrayOf(
        -1.0f,  1.0f, 0f,  // Top left
        -1.0f, -1.0f, 0f,  // Bottom left
        1.0f,  1.0f, 0f,  // Top right
        1.0f, -1.0f, 0f   // Bottom right
    )

    private val texCoords = floatArrayOf(
        0f, 0f, // Top left
        0f, 1f, // Bottom left
        1f, 0f, // Top right
        1f, 1f  // Bottom right
    )

    private var programId: Int = 0
    private lateinit var vertexBuffer: FloatBuffer
    private lateinit var texCoordBuffer: FloatBuffer
    private var positionHandle: Int = 0
    private var texCoordHandle: Int = 0
    private var matrixHandle: Int = 0
    private val projectionMatrix = FloatArray(16)
    private val viewMatrix = FloatArray(16)
    private val vpMatrix = FloatArray(16)
    private val modelMatrix = FloatArray(16)
    private val mvpMatrix = FloatArray(16)

    private fun createFloatBuffer(data: FloatArray): FloatBuffer {
        return ByteBuffer.allocateDirect(data.size * 4).run {
            order(ByteOrder.nativeOrder())
            asFloatBuffer().apply {
                put(data)
                position(0)
            }
        }
    }

    private fun loadShader(type: Int, shaderCode: String): Int {
        val shader = GLES20.glCreateShader(type)
        GLES20.glShaderSource(shader, shaderCode)
        GLES20.glCompileShader(shader)
        val compileStatus = IntArray(1)
        GLES20.glGetShaderiv(shader, GLES20.GL_COMPILE_STATUS, compileStatus, 0)
        if (compileStatus[0] == 0) {
            val error = GLES20.glGetShaderInfoLog(shader)
            GLES20.glDeleteShader(shader)
            throw RuntimeException("Compile error ($type): $error")
        }
        return shader
    }

    fun createProgram(vertexCode: String, fragmentCode: String): Int {
        val vertexShader = loadShader(GLES20.GL_VERTEX_SHADER, vertexCode)
        val fragmentShader = loadShader(GLES20.GL_FRAGMENT_SHADER, fragmentCode)
        val program = GLES20.glCreateProgram()
        GLES20.glAttachShader(program, vertexShader)
        GLES20.glAttachShader(program, fragmentShader)
        GLES20.glLinkProgram(program)
        val linkStatus = IntArray(1)
        GLES20.glGetProgramiv(program, GLES20.GL_LINK_STATUS, linkStatus, 0)
        if (linkStatus[0] == 0) {
            val error = GLES20.glGetProgramInfoLog(program)
            GLES20.glDeleteProgram(program)
            throw RuntimeException("Program link error: $error")
        }
        return program
    }

    override fun onSurfaceCreated(gl: GL10?, config: EGLConfig?) {
        programId = createProgram(vertexShaderCode, fragmentShaderCode)
        // prepare geometry
        vertexBuffer = createFloatBuffer(vertices)
        texCoordBuffer = createFloatBuffer(texCoords)
        // get attributes
        positionHandle = GLES20.glGetAttribLocation(programId, "aPosition")
        texCoordHandle = GLES20.glGetAttribLocation(programId, "aTexCoord")
        matrixHandle = GLES20.glGetUniformLocation(programId, "mvpMatrix")
        GLES20.glEnable(GLES20.GL_DEPTH_TEST)
    }

    override fun onSurfaceChanged(gl: GL10?, width: Int, height: Int) {
        GLES20.glViewport(0, 0, width, height)
        val ratio = width.toFloat() / height.toFloat()
        // lens
        Matrix.perspectiveM(projectionMatrix, 0, 53.1f, ratio, 1f, 10f)
        // camera
        Matrix.setLookAtM(viewMatrix, 0, 0f, 0f, 4f, 0f, 0f, 0f, 0f, 1f, 0f)
        // multiply
        Matrix.multiplyMM(vpMatrix, 0, projectionMatrix, 0, viewMatrix, 0)
    }

    override fun onDrawFrame(gl: GL10?) {
        GLES20.glClearColor(0.1f, 0.3f, 0.5f, 1.0f)
        GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT)
        GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT or GLES20.GL_DEPTH_BUFFER_BIT)
        GLES20.glUseProgram(programId)

        val angleInDegrees = 360.0f * (SystemClock.uptimeMillis() % rotationDurationMs).toFloat() / rotationDurationMs.toFloat()
        Matrix.setIdentityM(modelMatrix, 0)
        // rotation animation around X and Y axes
        Matrix.rotateM(modelMatrix, 0, angleInDegrees, 0.5f, 1f, 0f)
        Matrix.multiplyMM(mvpMatrix, 0, vpMatrix, 0, modelMatrix, 0)

        GLES20.glUniformMatrix4fv(matrixHandle, 1, false, mvpMatrix, 0)
        GLES20.glEnableVertexAttribArray(positionHandle)
        GLES20.glVertexAttribPointer(positionHandle, 3, GLES20.GL_FLOAT, false, 0, vertexBuffer)

        GLES20.glEnableVertexAttribArray(texCoordHandle)
        GLES20.glVertexAttribPointer(texCoordHandle, 2, GLES20.GL_FLOAT, false, 0, texCoordBuffer)

        GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4)

        GLES20.glDisableVertexAttribArray(positionHandle)
        GLES20.glDisableVertexAttribArray(texCoordHandle)
    }

    override fun onSurfaceDestroyed() {
        Log.d("GL", "Surface destroyed, releasing resources")
    }
}

If everything is implemented correctly, an animated, rotating colored rectangle will appear on the screen.

Creating a Virtual Display and Setting Up Presentation

Now, let’s move on to the practical implementation. At this stage, the following steps are required:

  • Prepare the Presentation object - this is the class responsible for the content we want to show on the virtual display.

  • Create an OES texture and a Surface object - a SurfaceTexture is created based on the texture ID, which is then used to instantiate a Surface. Everything rendered onto this Surface will automatically be directed into the OES texture.

  • Create a VirtualDisplay - using the obtained Surface, we register a new logical display in the system via the DisplayManager.

  • Update the rendering loop - frame-drawing logic must be added to read data from the OES texture and render it onto the main screen.

Creating a Presentation Subclass

Let’s start by implementing the presentation class. We will place a few simple, standard View elements on it:

class ViewPresentation(
    context: Context,
    display: Display,
) : Presentation(context, display) {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)

        val layout = LinearLayout(context).apply {
            orientation = LinearLayout.VERTICAL
            addView(TextView(context).apply {
                text = "This is a TextView on a virtual screen"
                textSize = 24f
                setTextColor(Color.BLACK)
                gravity = Gravity.CENTER
            }, LinearLayout.LayoutParams(
                ViewGroup.LayoutParams.MATCH_PARENT,
                ViewGroup.LayoutParams.WRAP_CONTENT
            ))

            addView(Button(context).apply {
                text = "And this is a button"
            }, LinearLayout.LayoutParams(
                ViewGroup.LayoutParams.WRAP_CONTENT,
                ViewGroup.LayoutParams.WRAP_CONTENT
            ))

            addView(SeekBar(context).apply {
                max = 100
                progress = 50
            }, LinearLayout.LayoutParams(
                ViewGroup.LayoutParams.MATCH_PARENT,
                ViewGroup.LayoutParams.WRAP_CONTENT
            ))
        }
        
        setContentView(
            layout,
            ViewGroup.LayoutParams(
                ViewGroup.LayoutParams.MATCH_PARENT,
                ViewGroup.LayoutParams.MATCH_PARENT
            )
        )
    }
}

We cannot create the presentation directly in MainActivity because a Display object is required. However, hardcoding this logic directly into the virtual display initialization code isn’t ideal either, as it violates the principle of separation of concerns. Therefore, we will move the object creation into a separate factory interface:

interface PresentationFactory {
    fun create(
        displayContext: Context,
        display: Display,
    ): Presentation
}

class ViewPresentationFactory : PresentationFactory {
    override fun create(
        displayContext: Context,
        display: Display,
    ): Presentation =
        ViewPresentation(displayContext, display)
}

Creating the Surface

To create the Surface, we will make changes to the BoxRenderer class from the previous example.

**Tip**: It is best to create a copy of the class with a different name and simply switch the 
reference to it in **MainActivity**. This allows you to quickly revert to a working version of 
the code for debugging or logic comparison if any issues arise.
    // Shader for OES textures
    private val fragmentShaderCodeOES = """#extension GL_OES_EGL_image_external : require
        precision mediump float;
        varying vec2 vTexCoord;
        uniform samplerExternalOES uTexture;
        void main() {
            gl_FragColor = texture2D(uTexture, vTexCoord);
        }
    """.trimIndent()

    private var textureOesId: Int = 0
    private var textureHandle: Int = 0
    private var surfaceTexture: SurfaceTexture? = null
    private var surface: Surface? = null

    private fun setupOutputSurface(width: Int, height: Int) {
        val textures = IntArray(1)
        GLES20.glGenTextures(1, textures, 0)
        textureOesId = textures[0]
        GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, textureOesId)
        GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR)
        GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR)
        GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE)
        GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE)
        surfaceTexture = SurfaceTexture(textureOesId)
        surfaceTexture?.setDefaultBufferSize(width, height)
        surface = Surface(surfaceTexture)
    }

    private fun releaseOutputSurface() {
        surface?.release()
        surface = null
        surfaceTexture?.release()
        surfaceTexture = null
        if (textureOesId != 0) {
            GLES20.glDeleteTextures(1, intArrayOf(textureOesId), 0)
            textureOesId = 0
        }
    }

In the setupOutputSurface method, we create an OES texture (GL_TEXTURE_EXTERNAL_OES). This is a special type of texture designed to handle image data from external sources (relative to the GPU), such as a camera or a video decoder. This specific texture type is required to create the Surface onto which our virtual display content will be rendered.

Since working with OES textures differs from standard ones, we need a different fragment shader. Therefore, when creating the shader program, we specify the fragmentShaderCodeOES:

    override fun onSurfaceCreated(gl: GL10?, config: EGLConfig?) {
        // Use the new shader to support external textures
        programId = createProgram(vertexShaderCode, fragmentShaderCodeOES)
        ...
        matrixHandle = GLES20.glGetUniformLocation(programId, "mvpMatrix")
        // Get the handle for the texture uniform variable
        textureHandle = GLES20.glGetUniformLocation(programId, "uTexture")

The setupOutputSurface method should be called within the onSurfaceChanged callback. This is where we receive the current dimensions of our OpenGL context and can calculate the dimensions of the Surface for the virtual display. In this example, I am setting its size to half of the minimum side of the screen:

    override fun onSurfaceChanged(gl: GL10?, width: Int, height: Int) {
        GLES20.glViewport(0, 0, width, height)
        val virtualSurfaceSize = min(width, height) / 2
        // Release previously allocated resources and create a new Surface
        releaseOutputSurface();
        setupOutputSurface(virtualSurfaceSize, virtualSurfaceSize)

Creating a Virtual Display

It is practical to move the virtual display management logic into a separate controller class. This allows for decoupling the preparation of graphic content from the direct management of the display’s lifecycle.

class VirtualDisplayController(
    private val context: Context,
    private val presentationFactory: PresentationFactory
) {
    private var virtualDisplay: VirtualDisplay? = null
    private var presentation: Presentation? = null
    private var width: Int = 0
    private var height: Int = 0

    fun createVirtualDisplay(
        width: Int,
        height: Int,
        surface: Surface) {
        if (virtualDisplay != null) {
            throw IllegalStateException("Virtual display already exists")
        }
        this.width = width
        this.height = height
        val mainHandler = Handler(Looper.getMainLooper())
        val displayManager = context.getSystemService(Context.DISPLAY_SERVICE) as DisplayManager
        
        // Register a DisplayListener to detect when the display is created
        displayManager.registerDisplayListener(object : DisplayManager.DisplayListener {
            override fun onDisplayAdded(displayId: Int) {
                // Check if this is indeed our new virtual display
                if (displayId == virtualDisplay?.display?.displayId) {
                    try {
                        val newDisplay = displayManager.getDisplay(displayId)
                        if (newDisplay != null) {
                            // Create a specialized context for this display
                            val displayContext = context.createDisplayContext(newDisplay)
                            val presentation = presentationFactory.create(displayContext, newDisplay)
                            presentation.show()
                            this@VirtualDisplayController.presentation = presentation
                        }
                    } catch (e: Exception) {
                        Log.e("GL", "Not found: $displayId", e)
                    }
                    // Unregister after successful addition
                    displayManager.unregisterDisplayListener(this)
                }
            }
            override fun onDisplayRemoved(displayId: Int) {}
            override fun onDisplayChanged(displayId: Int) {}
        }, mainHandler)

        // Configure flags: display for presentations and 
        // showing only app own content
        val flags = DisplayManager.VIRTUAL_DISPLAY_FLAG_PRESENTATION or
                DisplayManager.VIRTUAL_DISPLAY_FLAG_OWN_CONTENT_ONLY
        virtualDisplay = displayManager.createVirtualDisplay(
            "WebViewDisplay", width, height, 240, surface, flags
        )
    }

    fun destroyVirtualDisplay() {
        presentation?.dismiss()
        presentation = null
        virtualDisplay?.release()
        virtualDisplay = null
    }
}

The controller implements two main methods: createVirtualDisplay and destroyVirtualDisplay.

The process of creating a virtual display is asynchronous and takes some time. Therefore, we need to subscribe to state updates in the DisplayManager and wait for the onDisplayAdded callback with our new display’s ID. Once this happens, we can initialize and show the Presentation,` and then immediately unregister the listener.

Updating the Drawing Code and Connecting Everything

Let’s return to our BoxRenderer class (or its copy). We need to pass the virtual display controller via the constructor and initialize the display itself within the onSurfaceChanged method.

class BoxRenderer(
    private val virtualDisplayCtrl: VirtualDisplayController,
...
    override fun onSurfaceChanged(gl: GL10?, width: Int, height: Int) {
        ...        
        // Remove the old display and surface
        releaseOutputSurface()
        virtualDisplayCtrl.destroyVirtualDisplay()

        // Create new ones
        setupOutputSurface(virtualSurfaceSize, virtualSurfaceSize)
        surface?.let {
            virtualDisplayCtrl.createVirtualDisplay(virtualSurfaceSize, virtualSurfaceSize, it)    
        }

Now, let’s look at the rendering process itself (the onDrawFrame loop):

    override fun onDrawFrame(gl: GL10?) {
        ...

        GLES20.glUniformMatrix4fv(matrixHandle, 1, false, mvpMatrix, 0)
        GLES20.glEnableVertexAttribArray(positionHandle)
        GLES20.glVertexAttribPointer(positionHandle, 3, GLES20.GL_FLOAT, false, 0, vertexBuffer)

        GLES20.glEnableVertexAttribArray(texCoordHandle)
        GLES20.glVertexAttribPointer(texCoordHandle, 2, GLES20.GL_FLOAT, false, 0, texCoordBuffer)

        // Critical point: you must call updateTexImage(). 
        // This "pulls" the latest frame from our Presentation.
        surfaceTexture?.updateTexImage()

        // Activate and bind the OES texture
        GLES20.glActiveTexture(GLES20.GL_TEXTURE0)
        GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, textureOesId)
        GLES20.glUniform1i(textureHandle, 0)

        GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4)
        ...
    }

A vital part here is the call to surfaceTexture?.updateTexImage(). Without it, OpenGL will continue using the old frame from the buffer, and the image will not update. Also, pay close attention to the texture type GL_TEXTURE_EXTERNAL_OES — a standard GL_TEXTURE_2D will not work in this scenario.

Finally, let’s update the code in our MainActivity to correctly assemble the object hierarchy:

    val displayController = VirtualDisplayController(
        context = this@MainActivity,
        presentationFactory = ViewPresentationFactory()
    )
    val renderer = BoxRenderer(
        virtualDisplayCtrl = displayController
    )

If everything is configured correctly, we will have an animated object with a fully functional Android UI rendered on its surface.

Rendering Compose on a Presentation

Switching to Jetpack Compose doesn’t require any changes to the OpenGL code. You only need to modify the implementation of the Presentation class. Since Compose builds its own component tree, we use ComposeView as a bridge between classic Views and the declarative UI.

class ComposePresentation(
    context: Context,
    display: Display,
    private val lifecycleOwner: LifecycleOwner,
    private val viewModelStoreOwner: ViewModelStoreOwner,
    private val savedStateRegistryOwner: SavedStateRegistryOwner
) : Presentation(context, display) {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)

        val composeView = ComposeView(context).apply {
            // Set lifecycle owners (this is critical for Compose to work)
            setViewTreeLifecycleOwner(lifecycleOwner)
            setViewTreeViewModelStoreOwner(viewModelStoreOwner)
            setViewTreeSavedStateRegistryOwner(savedStateRegistryOwner)

            // Set the composition strategy (important for windows without an Activity)
            setViewCompositionStrategy(ViewCompositionStrategy.DisposeOnViewTreeLifecycleDestroyed)

            setContent {
                MaterialTheme {
                    Surface(
                        modifier = Modifier.fillMaxSize(),
                        color = MaterialTheme.colorScheme.background
                    ) {
                        Column(
                            modifier = Modifier.fillMaxSize().padding(8.dp),
                            verticalArrangement = Arrangement.Top,
                            horizontalAlignment = Alignment.CenterHorizontally
                        ) {
                            Text(
                                text = "This is Compose on a virtual screen",
                                style = MaterialTheme.typography.headlineMedium
                            )
                            Spacer(modifier = Modifier.height(16.dp))
                            var clickCounter by remember { mutableIntStateOf(0) }
                            Button(onClick = { clickCounter = clickCounter.inc() }) {
                                Text("Counter: $clickCounter")
                            }
                            Slider(
                                value = 0.5f,
                                onValueChange = { /* Action */ },
                                modifier = Modifier.padding(horizontal = 16.dp)
                             )
                        }
                    }
                }
            }
        }

        // Set ComposeView as the main content of the presentation
        setContentView(
            composeView,
            ViewGroup.LayoutParams(
                ViewGroup.LayoutParams.MATCH_PARENT,
                ViewGroup.LayoutParams.MATCH_PARENT
            )
        )
    }
}

For standard Views, having a LifecycleOwner is not strictly mandatory. However, Compose cannot function without access to the lifecycle, ViewModel store, and saved state registry. Therefore, we will use our Activity as a “donor” for these parameters and pass them through the factory into the presentation’s constructor.

class ComposePresentationFactory(
    private val lifecycleOwner: LifecycleOwner,
    private val viewModelStoreOwner: ViewModelStoreOwner,
    private val savedStateRegistryOwner: SavedStateRegistryOwner
) : PresentationFactory {
    override fun create(
        displayContext: Context,
        display: Display,
    ): Presentation =
        ComposePresentation(
            displayContext,
            display,
            lifecycleOwner,
            viewModelStoreOwner,
            savedStateRegistryOwner)
}
...
class MainActivity : ComponentActivity() {
    override fun onCreate(savedInstanceState: Bundle?) {
        ...
        val renderer = BoxRenderer(
            virtualDisplayCtrl = VirtualDisplayController(
                    context = this@MainActivity,
                    presentationFactory = ComposePresentationFactory(
                        lifecycleOwner = this,
                        viewModelStoreOwner = this,
                        savedStateRegistryOwner = this
                    )
                )
    )

This time, you should see something like this on the screen:

Dispatching Events to the Virtual Display

Let’s try not just rendering the content, but also interacting with it by passing touch events to the UI on the virtual display. To do this, we will modify the VirtualDisplayController by adding a sendEvent method:

    ...
    fun sendEvent(event: MotionEvent, x: Float, y: Float) {
        presentation?.let {
            // Create a new event, scaling the relative coordinates (0..1) 
            // to the actual pixels of the virtual display
            val scaledEvent = MotionEvent.obtain(
                event.downTime,
                event.eventTime,
                event.action,
                x * width,
                y * height,
                event.metaState
            )
            // Dispatch the event directly to the presentation's decor view
            it.window?.decorView?.dispatchTouchEvent(scaledEvent)
            scaledEvent.recycle()
        }
    }

In MainActivity, we will launch a coroutine that simulates a click (a Down/Up cycle) every three seconds:

    override fun onCreate(savedInstanceState: Bundle?) {
        ...
        lifecycleScope.launch {
            while (isActive) {
                delay(3000)
                val downTime = SystemClock.uptimeMillis()
                val eventTime = SystemClock.uptimeMillis()

                // Simulate a press (ACTION_DOWN)
                val downEvent = MotionEvent.obtain(
                    downTime, downTime, MotionEvent.ACTION_DOWN, 0f, 0f, 0
                )
                displayController.sendEvent(
                    event = downEvent,
                    x = 0.5f,
                    y = 0.3f
                )
                downEvent.recycle()

                // Simulate a release (ACTION_UP) 50ms later
                val upEvent = MotionEvent.obtain(
                    downTime, downTime + 50, MotionEvent.ACTION_UP, 0f, 0f, 0
                )
                displayController.sendEvent(
                    event = upEvent,
                    x = 0.5f,
                    y = 0.3f
                )
                upEvent.recycle()
            }
        }
        ...
    }

As a result, the text on the button will start changing with each cycle trigger. You can also notice the characteristic ripple effect in the video, confirming that the touch was successfully processed:




Comments

Post a Comment

Popular posts from this blog

Orange Pi Zero 3 - exploring GPIO

Image
GPIO (General-Purpose Input/Output, or General-Purpose Pins) is a fundamental concept that came into the world of powerful SoCs (System-on-a-Chip) directly from the realm of microcontrollers. It is the main bridge for your single-board computer to interact with simple external devices, in addition to more complex hardware interfaces (such as I2C, SPI, UART, or I2S). While specialized protocols transmit data streams, GPIO allows you to work at the lowest level -controlling individual digital signals. With each pin you can: Read the state (input): For example, check if a button is pressed, or if the signal is “1” or “0”. Control the state (output): For example, turn on an LED, activate a relay, or send a signal to another component. In this article, I want to provide a quick overview of the methods for working with GPIO on the Orange Pi Zero 3 (which is based on the Allwinner H618 SoC ). Of course, these methods are not unique and can be applied to other single-board computers a...
Read more

YD-RP2040 Module

Image
I want to tell you about the rather inexpensive YD-RP2040 module and show some examples of working with it. Photos and Schematics Top view: Bottom view: The module’s electrical schematic can be found here https://github.com/initdc/YD-RP2040/blob/master/YD-2040-2022-V1.1-SCH.pdf Differences from the original Raspberry Pi Pico This board has several key hardware differences from the standard Raspberry Pi Pico module. Key changes:  - Flash memory : The module can be fitted with a larger Flash memory capacity — 4MB (32Mbit) or even 16MB (128Mbit) instead of the 2MB in the original.  - Voltage regulator : The DC-DC step-down converter is replaced with a cheaper and less efficient linear regulator. This has freed up pin GPIO23, which was used to control the DC-DC converter in the original Pico.  - RGB LED : A WS2812 addressable LED (Neo-pixel) is connected to the freed GPIO23 pin. Unlike the original Pico, this board has two additional buttons:  - RESET — for a ...
Read more

Modifying the INA226: From 0.8A to High-Power Current Sensing

Image
For my DIY battery build (4S LiFePO4 40Ah), I decided to add a “smart” monitoring system. Sure, I could have just bought an off-the-shelf Smart BMS with Bluetooth or Wi-Fi, but that’s not my style. I want to understand the entire process and build something of my own. First and foremost, I need to track the charge and discharge current. Based on this data, I’ll later be able to develop an algorithm for accurate State of Charge calculation. First and foremost, I need to track the charge and discharge current. Based on this data, I will eventually be able to develop an algorithm to calculate the State of Charge (SoC) . I’m not going to claim that this will be a high-precision instrument - it’s not. It is more of a ‘rough gauge’ to help me stay oriented: to see if everything is fine or if the power consumption has become abnormally high and something has gone wrong. To measure the current, we’ll use the INA226 module. Typically, these boards come with a 0.1 Ohm (100 mOhm) shunt resist...
Read more