In this post, we will learn about Proto DataStore, one of two DataStore implementations. We will discuss how to create it, read and write data and how to handle exceptions, to better understand the scenarios that make Proto a great choice.
Proto DataStore uses typed objects backed by Protocol Buffers, to store smaller datasets while providing type safety. It removes the need for using key-value pairs, making it structurally different from its
SharedPreferences predecessor and its sibling implementation, Preferences DataStore. However, that’s not all — DataStore brings many other improvements over
SharedPreferences. Feel free to quickly jump back to our first post in the series and take a look at the detailed comparison we’ve made there. Going forward, we will refer to
Proto DataStore as just
Proto, unless specified otherwise.
To sum up:
- Provides a fully asynchronous API for retrieving and saving data, using the power of Kotlin coroutines
- Does not offer ready-to-use synchronous support — it directly avoids doing any work that blocks the UI thread
- Relies on Flow’s inner error signalling mechanism, allowing you to safely catch and handle exceptions when reading or writing data
- Handles data updates safely in an atomic read-modify-write operation, providing strong ACID guarantees
- Allows easy and simple data migrations
- Need full type safety and your data requires working with more complex classes, like enums or lists? This isn’t possible with Preferences, so choose Proto instead
Intro to Protocol Buffers
To use Proto DataStore, you need to get familiar with Protocol Buffers — a language-neutral, platform-neutral mechanism for serializing structured data. It is faster, smaller, simpler and less ambiguous than XML and easier to read than other similar data formats.
You define a schema of how you want your data to be structured and specify options such as which language to use for code generation. The compiler then generates classes according to your specifications. This allows you to easily write and read the structured data to and from a variety of data streams, share between different platforms, using a number of different languages, like Kotlin.
Example schema of some data in a
How to use the generated Kotlin code for constructing your data model:
Or you can try out the newly announced Kotlin DSL support for protocol buffers for a more idiomatic way of building your data model:
Investing a bit more time into learning this new serialization mechanism is definitely worth it as it brings type safety, improved readability and overall code simplicity.
Proto DataStore dependency setup
Now let’s look at some code and learn how Proto works.
We will use the Proto DataStore codelab sample. If you’re interested in a more hands-on approach with implementation, we really encourage you to go through the Working with Proto DataStore codelab on your own.
This sample app displays a list of tasks and the user can choose to filter them by their completed status or sort by priority and deadline. We want to store their selection — a boolean for displaying completed tasks and a sort order enum in Proto.
We will firstly add Proto dependencies and some of the basic protobuf settings to your module’s
build.gradle. If you’re interested in a more advanced customisation of the protobufs compilation, check out the Protobuf Plugin for Gradle notes:
💡 Quick tip — if you want to minify your build, make sure to add an additional rule to your
<em class="hp">proguard-rules.pro</em>file to prevent your fields from being deleted:
Protobuf setup for Proto DataStore
Our journey with Proto starts by defining the structure of your persisted data in a
.proto file. Think of it as a readable schema for you and a blueprint for the compiler. We will name ours
user_prefs.proto and add it to the
Following the Protobuf language guide, in this file we will add a message for each data structure we want to serialize, then specify a name and a type for each field in the message. To help visualize this, let’s look at both a Kotlin data class and a corresponding protobuf schema.
UserPreferences — Kotlin data class:
If you haven’t used protobufs before, you might also be curious about the first few lines in the schema. Let’s break them down:
syntax— specifies that you’re using
<a class="au jk" href="https://developers.google.com/protocol-buffers/docs/proto3" target="_blank" rel="noopener ugc nofollow">proto3</a>syntax
java_package— file option that specifies package declaration for your generated classes, which helps prevent naming conflicts between different projects
java_multiple_files— file option that specifies whether only a single file with nested subclasses will be generated for this
.proto(when set to false) or if separate files will be generated for each top-level message type (when set to true); it is false by default
Next is our message definition. A message is an aggregate containing a set of typed fields. Many standard simple data types are available as field types, including
float, double, and
string. You can also add further structure to your messages by using other message types as field types, like we did with
= 2 markers on each element identify the unique “tag” that the field uses in the binary encoding — like an ID of sort. Once your message type is in use, these numbers should not be changed.
When you run the protocol buffer compiler on a
.proto, the compiler generates the code in your chosen language. In our specific case, when the compiler is run, this leads to the generation of the
UserPreferences class, found in your app’s
💡 Quick tip — You can also try out the newly announced Kotlin DSL support for protocol buffers to use a more idiomatic way of building your data model.
Now that we have
UserPreferences, we need to specify the guidelines for how Proto should read and write them. We do this via the DataStore
Serializer that determines the final format of your data when stored and how to properly access it. This requires overriding:
defaultValue— what to return if no data is found
writeTo— how to transform the memory representation of our data object into a format fit for storage
readFrom— inverse from the above, how to transform from a storage format into a corresponding memory representation
To keep your code as safe as possible, handle the
<strong class="io hq">CorruptionException</strong> to avoid unpleasant surprises when a file cannot be de-serialized due to format corruption.
💡 Quick tip — If at any point your AS is unable to find anything
<em class="hp">UserPreferences</em>related, clean and rebuild your project to initiate the generation of the protobuf classes.
Creating a Proto DataStore
You interact with Proto through an instance of
DataStore is an interface that grants access to the persisted information, in our case in the form of the generated
To create this instance, it is recommended to use the delegate
dataStore and pass mandatory
fileName is used to create a
File used to store the data. This is why the
dataStore delegate is a Kotlin extension property whose receiver type must be an instance of
Context, as this is needed for the File creation via
<a class="au jk" href="https://developer.android.com/training/data-storage/app-specific#internal-access-store-files" target="_blank" rel="noopener ugc nofollow">applicationContext.filesDir</a>. Avoid using this file in any way outside of Proto, as it would break the consistency of your data.
dataStore delegate, you can pass one more optional argument —
corruptionHandler. This handler is invoked if a
CorruptionException is thrown by the serializer when the data cannot be de-serialized.
corruptionHandler would then instruct Proto how to replace the corrupted data:
You shouldn’t create more than one instance of DataStore for a given file, as doing so can break all DataStore functionality. Therefore, you can add the delegate construction once at the top level of your Kotlin file and use it throughout your application, in order to pass it as a singleton. In later posts, we will see how to do this with dependency injection.
To read the stored data, in
UserPreferencesRepository we expose a
userPreferencesStore.data. This provides efficient access to the latest saved state and emits with every change. This is one of the biggest strengths of Proto — your
Flow’s values already come in the shape of the generated
UserPreferences. This means you don’t have to do any additional transformations from the saved data into a Kotlin data class model, like you would with
The Flow will always either emit a value or throw an exception when attempting to read from disk. We will look at exception handling in later sections. DataStore also ensures that work is always performed on
<strong class="io hq">Dispatchers.IO</strong> so your UI thread isn’t blocked.
🚨 Do not create any cache repositories to mirror the current state of your Proto data. Doing so would invalidate DataStore’s guarantee of data consistency. If you require a single snapshot of your data without subscribing to further Flow emissions, prefer using
<strong class="io hq">userPreferencesStore.data</strong>.<a class="au jk" href="https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/first.html" target="_blank" rel="noopener ugc nofollow">first()</a>:
For writing data, we will use a suspend
DataStore<UserPreferences>.updateData(transform: suspend (t: T) -> T) function.
Let’s break that down:
DataStore<UserPreferences>interface — we’re currently using
userPreferencesStoreas the concrete Proto implementation
transform: suspend (t: T) -> T)— a suspend block used to apply the specified changes to our persisted data of type T
Again, you might notice a difference to
Preferences DataStore which relies on using
MutablePreferences, similar to
MutableMap, as the default data representation.
We can now use this to change our
showCompleted boolean. Protocol buffers simplify this as well, removing the need for any manual transformation from and to data classes:
There’s a few steps to analyze:
toBuilder()— gets the
Builderversion of our
currentPreferenceswhich “unlocks” it for changes
.setShowCompleted(completed)— sets the new value
.build()— finishes the update process by converting it back to
Updating data is done transactionally in an atomic read-modify-write operation. This means that the specific order of data processing operations, during which the data is locked for other threads, guarantees consistency and prevents race conditions. Only after the
updateData coroutines complete successfully, the data will be persisted durably to disk and
userPreferencesStore.data Flow will be reflecting the update.
🚨 Keep in mind that this is the only way of making changes to the DataStore state. Keeping a
UserPreferences reference and mutating it manually after
transform completes will not change the persisted data in Proto, so you shouldn’t attempt to modify
UserPreferences outside of the
If the writing operation fails for any reason, the transaction is aborted and an exception is thrown.
If you’ve previously used
SharedPreferences in your app and would like to safely transfer its data to Proto, you can use
<a class="au jk" href="https://developer.android.com/reference/kotlin/androidx/datastore/migrations/SharedPreferencesMigration" target="_blank" rel="noopener ugc nofollow">SharedPreferencesMigration</a>. It requires a context,
SharedPreferences name and an instruction on how to transform your
SharedPreferences key-value pairs to
UserPreferences within the
migrate parameter. Pass this via the
produceMigrations parameter of the
dataStore delegate to migrate easily:
In this example, we go through the process of building the
UserPreferences and setting its
sortOrder to what was previously stored in the corresponding
SharedPreferences key-value pair, or simply defaulting to NONE.
produceMigrations will ensure that the
migrate() is run before any potential data access to DataStore. This means your migration must have succeeded before DataStore emits any further values and before it begins making any new changes to the data. Once you’ve successfully migrated, it’s safe to stop using
SharedPreferences, as the keys are migrated only once and then removed from
produceMigrations accepts a list of
<a class="au jk" href="https://developer.android.com/reference/kotlin/androidx/datastore/core/DataMigration" target="_blank" rel="noopener ugc nofollow">DataMigration</a>. We will see in later episodes how we can use this for other types of data migrations. If you don’t need to migrate, you can ignore this as it has a default
<strong class="io hq">listOf()</strong> provided already.
One of the main advantages of DataStore over
SharedPreferences is its neat mechanism for catching and handling exceptions. While
SharedPreferences throws parsing errors as runtime exceptions, leaving room for unexpected, uncaught crashes, DataStore throws an
<strong class="io hq">IOException</strong> when an error occurs with reading/writing data.
We can safely handle this by using the
catch() Flow operator and emitting
Or with a simple
try-catch block on writing:
If a different type of exception is thrown, prefer re-throwing it.
To be continued
We’ve covered Protocol Buffers and DataStore’s Proto implementation — when and how to use it for reading and writing data, how to handle errors and how to migrate from
SharedPreferences. In the next and final post, we will go a step further and look at how DataStore fits in your app’s architecture, how to inject it with Hilt and of course, how to test it. See you soon!
By Simona Stojanovic