aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • Mobile
  • Programming

Convert YUV To RGB For CameraX ImageAnalysis

  • aster.cloud
  • November 22, 2021
  • 4 minute read

CameraX is a Jetpack support library that is built to help make camera app development easier. It supports different use cases such as ImageCapture, Preview and ImageAnalysis, which could seamlessly combine with ML Kit or TensorFlow Lite. This opens up a lot of possibilities to create applications like text recognition, image labeling, and even detecting and recognizing objects using your own trained TensorFlow Lite model. However, image format conversion between CameraX and these libraries is non-trivial and sometimes time-consuming. In this blog, we will talk about a new feature we recently built to support YUV to RGB conversion for CameraX ImageAnalysis, including why we built it and how to use it with just a small amount of code.

Background

CameraX produces images with YUV420_888 with Luma (Y), Chroma (U, V) and Paddings (P) with 8 bits for each channel. YUV is a generic and flexible format, it allows for OEM variations on different devices, which covers a lot of ImageAnalysis use cases. However, many apps still rely on the RGB format. In our developer community, YUV to RGB conversion is one of the most highly-requested features, because RGB format is popular, easy to work with, and is sometimes required, such as with TensorFlow Lite models). Let’s take a look at the YUV and RGB formats first.


Partner with aster.cloud
for your next big idea.
Let us know here.



From our partners:

CITI.IO :: Business. Institutions. Society. Global Political Economy.
CYBERPOGO.COM :: For the Arts, Sciences, and Technology.
DADAHACKS.COM :: Parenting For The Rest Of Us.
ZEDISTA.COM :: Entertainment. Sports. Culture. Escape.
TAKUMAKU.COM :: For The Hearth And Home.
ASTER.CLOUD :: From The Cloud And Beyond.
LIWAIWAI.COM :: Intelligence, Inside and Outside.
GLOBALCLOUDPLATFORMS.COM :: For The World's Computing Needs.
FIREGULAMAN.COM :: For The Fire In The Belly Of The Coder.
ASTERCASTER.COM :: Supra Astra. Beyond The Stars.
BARTDAY.COM :: Prosperity For Everyone.

YUV_420_888 format

YUV format can also be referred to as “YCbCr”. It includes planar (e.g. I420), semi-planar (e.g. NV21/NV12) and packed formats (e.g. UYVY). YUV_420_888 is a generic YCbCr format, capable of describing any 4:2:0 chroma-subsampled planar or semi-planar buffer (but not fully interleaved), with 8 bits per color sample. The Y-plane is guaranteed not to be interleaved with the U/V planes (in particular, pixel stride is always 1). The U/V planes are guaranteed to have the same row stride and pixel stride.

Read More  How Many Software Developers Are There in the World?

RGBA_8888 format

RGBA_8888 is a standard RGB format with red, green, blue, and alpha channels, and each channel has 8 bits. The major goal of the conversion is RGB color space. RGB is simpler with fewer variations.

API Implementation

We have evaluated 3 approaches for YUV to RGB conversion

  1. Java/Kotlin
  2. Renderscript
  3. Native (C/C++ and NDK)

A Java/Kotlin implementation of image processing comes with long computation time and garbage collection pressure. Renderscript is a candidate for computationally-intensive tasks such as YUV to RGB. However, it has been deprecated starting from Android 12.

Considering future extensibility and compatibility, we decide to use a native approach (libyuv + NDK). Libyuv is an open source project that includes YUV scaling, conversion, and rotation functionality. With all things considered, CameraX color conversion pipeline can be viewed at a high level as follows:

Our pipeline still produces ImageProxy as the output for backward compatibility. ImageProxy is a wrapper class for media.Image, which is an image buffer produced by the Android framework. To produce a converted Image, the Java/Kotlin layer can get an input Image from a Surface via dequeueInputImage() and then write Image data into it using ImageReader and ImageWriter. Since ImageWriter was added in API 23, we use ANativeWindow and its buffer to produce output images in RGBA format to support more API levels.

For input, we support different variations of YUV_420_888 formats (I420, NV12, NV21, etc.) internally in CameraX. For output, we support RGBA format now but could extend support to other RGB formats in the future.

Since we use libyuv as a new dependency, the size of our library increases by about 50 KB.

API Usage

Starting with CameraX 1.1.0-alpha08, apps can choose the output image in YUV_420_888 or RGBA_8888 format by using setOutputImageFormat in an ImageAnalysis configuration.

Once RGBA_8888 is selected, the output image format will be PixelFormat.RGBA_8888, which has only one image plane (R, G, B, A pixel by pixel) with paddings. Traditionally, the Android Framework supports image buffer formats with a subset of both PixelFormat and ImageFormat.

By comparison, if YUV_420_888 is selected, the output image will be ImageFormat.YUV_420_888, which has 3 separate image planes (Y, U, V).

Performance

We measured the speed and compared the results with those of Renderscript on different Android versions and devices. Overall, our pipeline with libyuv outperforms our reference Renderscript implementation on different devices for different resolutions and Android OS versions.

Summary

We support YUV to RGB conversion in the CameraX ImageAnalysis pipeline. Users can now simply select the output formats (YUV_420_888 or RGBA_8888) for an ImageAnalysis use case to connect with other libraries. This is the beginning of an exciting journey, and we are considering adding more image processing functionalities in our CameraX ImageAnalysis pipeline and extending it to other user cases (ImageCapture or Preview etc.). Let us know if you have any feature requests.

Sample code for YUV to RGB conversion is available on GitHub. To learn more about CameraX, refer to the official documentation. To keep up with the latest CameraX development, join the CameraX Discussion Group. Feedback is extremely valuable and welcome: feel free to leave a response here, talk to us on the CameraX Discussion group, or create CameraX issues through the official issue tracker.

Stay tuned!

By Kailiang Chen
Source Medium


For enquiries, product placements, sponsorships, and collaborations, connect with us at [email protected]. We'd love to hear from you!

Our humans need coffee too! Your support is highly appreciated, thank you!

aster.cloud

Related Topics
  • Android
  • CameraX
  • Jetpack
  • Medium
You May Also Like
View Post
  • Architecture
  • Data
  • Engineering
  • People
  • Programming
  • Software Engineering
  • Technology
  • Work & Jobs

Predictions: Top 25 Careers Likely In High Demand In The Future

  • June 6, 2023
View Post
  • Programming
  • Software Engineering
  • Technology

Build a Python App to Alert You When Asteroids Are Close to Earth

  • May 22, 2023
View Post
  • Programming

Illuminating Interactions: Visual State In Jetpack Compose

  • May 20, 2023
View Post
  • Computing
  • Data
  • Programming
  • Software
  • Software Engineering

The Top 10 Data Interchange Or Data Exchange Format Used Today

  • May 11, 2023
View Post
  • Gears
  • Mobile
  • Technology

Apple Watch Pride Edition Celebrates The LGBTQ+ Community

  • May 10, 2023
View Post
  • Architecture
  • Programming
  • Public Cloud

From Receipts To Riches: Save Money W/ Google Cloud & Supermarket Bills – Part 1

  • May 8, 2023
View Post
  • Programming
  • Public Cloud

3 New Ways To Authorize Users To Your Private Workloads On Cloud Run

  • May 4, 2023
View Post
  • Programming
  • Public Cloud

Buffer HTTP Requests With Cloud Tasks

  • May 4, 2023

Stay Connected!
LATEST
  • 1
    Just make it scale: An Aurora DSQL story
    • May 29, 2025
  • 2
    Reliance on US tech providers is making IT leaders skittish
    • May 28, 2025
  • Examine the 4 types of edge computing, with examples
    • May 28, 2025
  • AI and private cloud: 2 lessons from Dell Tech World 2025
    • May 28, 2025
  • 5
    TD Synnex named as UK distributor for Cohesity
    • May 28, 2025
  • Weigh these 6 enterprise advantages of storage as a service
    • May 28, 2025
  • 7
    Broadcom’s ‘harsh’ VMware contracts are costing customers up to 1,500% more
    • May 28, 2025
  • 8
    Pulsant targets partner diversity with new IaaS solution
    • May 23, 2025
  • 9
    Growing AI workloads are causing hybrid cloud headaches
    • May 23, 2025
  • Gemma 3n 10
    Announcing Gemma 3n preview: powerful, efficient, mobile-first AI
    • May 22, 2025
about
Hello World!

We are aster.cloud. We’re created by programmers for programmers.

Our site aims to provide guides, programming tips, reviews, and interesting materials for tech people and those who want to learn in general.

We would like to hear from you.

If you have any feedback, enquiries, or sponsorship request, kindly reach out to us at:

[email protected]
Most Popular
  • Understand how Windows Server 2025 PAYG licensing works
    • May 20, 2025
  • By the numbers: How upskilling fills the IT skills gap
    • May 21, 2025
  • 3
    Cloud adoption isn’t all it’s cut out to be as enterprises report growing dissatisfaction
    • May 15, 2025
  • 4
    Hybrid cloud is complicated – Red Hat’s new AI assistant wants to solve that
    • May 20, 2025
  • 5
    Google is getting serious on cloud sovereignty
    • May 22, 2025
  • /
  • Technology
  • Tools
  • About
  • Contact Us

Input your search keywords and press Enter.