Part 2 of a technical guide on how to migrate away from TextureView to the much preferred SurfaceView focused on Color Washout with HDR tone-mapping

Intro

Hello again! This is Part 2 of the Android HDR series focused on migrating away from TextureView to SurfaceView! This part of the series will focus on dealing with “color washout.” We will talk about how to correctly transcode 10-bit HDR video to 8-bit SDR video to ensure support for devices that can’t display HDR content. This also ensures that the colors of your content look correct on TextureView.

What is “Color washout?”

In Part 1, we were able to display 10-bit HDR videos on a SurfaceView. We strongly recommend moving to SurfaceView for performance and fidelity, but you may have other reasons to stay on TextureView, being able to set the alpha value to blend with the background. What should you do in this case?

Well first, let’s see what happens when you attempt to display 10-bit HDR content on a TextureView. In our graphics samples repository, we created an activity that contains one SurfaceView and two TextureViews. One TextureView uses our Custom Decoder and the other uses MediaPlayer. Here is the resulting image comparing TextureView and SurfaceView on three different devices running different versions of Android.

For Android 13 (API 33) on the Pixel 6 Pro, they all look identical. This is because on API 33, conversion from 10-bit HDR to 8-bit SDR happens implicitly. This means under the hood, Android converts your content from 10-bit HDR to 8-bit SDR without you needing to do anything. The quality of the HDR to SDR conversion depends on the device’s underlying tone-mapping implementation.

This is great for Android 13 (API 33), but on Android 12 (API 32), this is not the story. You can see all of the videos play, but the TextureViews seem to lack a lot of color. The image looks bland compared to the SurfaceView. This is especially apparent in the color of the sky. This is because the 10-bit HDR content is not being converted correctly to 8-bit SDR for TextureView.

Converting 10-bit HDR content to 8-bit SDR for API 32 or lower devices

When sharing 10-bit content to a device that is API 32 or lower, you must convert the 10-bit content to 8-bit content. This ensures the receiving device that is API 32 or below plays back the footage with the correct colors.

To transcode 10-bit content to 8-bit, Google recommends using the Transformer API from Media3. The Transformer API can be used to convert media streams. It takes an input media stream, applies changes to it as configured by the app, and produces the corresponding output file.

There are several use cases for the Transformer API. We will be focusing on its tone-mapping functionality, which does the conversion from HDR 10-bit color to SDR 8-bit color.

At the time of writing, the conversion is dependent on the underlying codec tone-mapping capability, which is only guaranteed on devices able to capture 10-bit HDR videos on API 33. Media3 APIs will soon be updated to include its own tone-mapping algorithm that can be used on other devices.

To transform media, add the following dependency from media3 to your app’s build.gradle file:

def media3_version = "1.0.0-beta03"
implementation "androidx.media3:media3-common:$media3_version"
implementation "androidx.media3:media3-transformer:$media3_version"

In your class, create a TransformationRequest object and call the setEnableRequestSdrToneMapping() method, which we set to true.

// Set up transformer using Transformer.Builder
val request = TransformationRequest.Builder()
              .setEnableRequestSdrToneMapping(true)
              .build()

We now can build our Transformer object using the TransformationRequest and a listener.

val transformer = Transformer.Builder(applicationContext)
              .setTransformationRequest(request)
              .addListener(myListener)
              .build()

The addListener() function needs a Transformer.Listener. This class allows you to override and implement two functions:

• onTransformationCompleted(): Called when the transformation is completed successfully.
• onTransformationError(): Called if an exception occurs during the transformation.

In our case, we want to ensure that the converted 10-bit file correctly displays its color on a TextureView. If there is an error, we want to display what happens during the transformation attempt.

/** Transformer.Listener Overrides */
override fun onTransformationCompleted(
    inputMediaItem: MediaItem, 
    transformationResult: TransformationResult) 
{
    textureViewDecoder = CustomVideoDecoder.buildWithFilePath(encodedFile)
    textureViewDecoder?.setSurface(Surface(binding.textureView.surfaceTexture))
    textureViewDecoder?.start(loop = true)
}

override fun onTransformationError(
    inputMediaItem: MediaItem, 
    exception: TransformationException) 
{
    val message = "Transformation Failed... ${exception.cause}"
    Log.e(MultiViewVideoPlayerHDRTransformer::class.java.name, message)

val mainHandler = Handler(applicationContext.mainLooper)
    mainHandler.post { binding.textureViewTitle.text = message }

Toast.makeText(applicationContext, message, Toast.LENGTH_LONG).show()
}

To start the transformation of the media, we need to call on our Transformer object, pass it the location of the video file, and call the startTransformation() function. We can do this once we know the TextureView’s SurfaceTexture is available via the onSurfaceTextureAvailable() function.

/** TextureView.SurfaceTextureListener Overrides */
override fun onSurfaceTextureAvailable(surface: SurfaceTexture, width: Int, height: Int) {
    val firstVideoUri = Uri.parse("asset:///" + Constants.HDR_VIDEO_PLAYER_ASSET)
    val item = MediaItem.fromUri(firstVideoUri)
    transformer.startTransformation(item, encodedFile)

// Set up to update UI of progress
    val progressHolder = ProgressHolder()
    val mainHandler = Handler(applicationContext.mainLooper)
    mainHandler.post(object : Runnable {
        override fun run() {
            val progressState: @ProgressState Int = transformer.getProgress(progressHolder)
            if (progressState != PROGRESS_STATE_NO_TRANSFORMATION) {
                binding.textureViewTitle.text =
                    "Transformation Progress = ${progressHolder.progress}"
                mainHandler.postDelayed( /* r = */ this,  /* delayMillis = */16)
            }
        }
    })
}

With this, we now have the ability to transform 10-bit content to 8-bit content.

The full sample code can be found in our graphics samples repository.

Going on to Part #3 — Lifecycle management and SurfaceView Transformations

With everything above, you should now be able to transform 10-bit HDR content to 8-bit SDR content. This ensures that when sharing content with other devices, colors are represented correctly and are no longer washed out.

In Part #3, we will discuss some lifecycle differences between TextureView and SurfaceView and transformations with SurfaceView (this means moving a surface across the screen or how SurfaceView interacts with a scrollable container such as a RecycleView or ViewPager).