# De-identify and re-identify sensitive data
This document shows you how to use Sensitive Data Protection to de-identify and re-identify sensitive data in text content. In the process, it guides you through creating a wrapped key using Cloud Key Management Service. You need this key in your de-identify and re-identify requests.
The process described in this document is called *pseudonymization*
(or *tokenization*). In this process, Sensitive Data Protection uses
a cryptographic key to convert (de-identify) sensitive text into a token. To
restore (re-identify) that text, you need the cryptographic key
used during de-identification and the token.
Sensitive Data Protection supports both reversible and non-reversible
cryptographic methods. To re-identify content, you need to choose a
reversible method.
The cryptographic method described here is called [deterministic encryption
using AES-SIV](https://docs.cloud.google.com/sensitive-data-protection/docs/pseudonymization#aes-siv)
(Advanced Encryption Standard in Synthetic Initialization Vector mode). We
recommend this method, because it provides the highest level of security among
all the reversible cryptographic methods that Sensitive Data Protection
[supports](https://docs.cloud.google.com/sensitive-data-protection/docs/pseudonymization.md.txt#supported-methods).
You can complete the steps in this document in 10 to 20 minutes, excluding the
[Before you
begin](https://docs.cloud.google.com/sensitive-data-protection/docs/inspect-sensitive-text-de-identify#before-you-begin)
steps.
> [!IMPORTANT]
> **Note:** When you use Cloud KMS for cryptographic operations, [charges apply](https://cloud.google.com/kms/pricing). In addition, when you use a [Cloud HSM](https://docs.cloud.google.com/kms/docs/hsm.md.txt) key or a [Cloud EKM](https://docs.cloud.google.com/kms/docs/ekm.md.txt) key, you consume Cloud KMS quota on the project that contains the key. In contrast, if you're using a software Cloud KMS key with Sensitive Data Protection, no quota values apply. For more information, see [Quotas](https://docs.cloud.google.com/kms/quotas.md.txt) in the Cloud KMS documentation.
## Before you begin
### Required roles
To get the permissions that
you need to create a wrapped AES key, de-identify sensitive data, and re-identify it,
ask your administrator to grant you the
following IAM roles on the project:
- [Cloud KMS Admin](https://docs.cloud.google.com/iam/docs/roles-permissions/cloudkms.md.txt#cloudkms.admin) (`roles/cloudkms.admin`)
- [Cloud KMS CryptoKey Encrypter](https://docs.cloud.google.com/iam/docs/roles-permissions/cloudkms.md.txt#cloudkms.cryptoKeyEncrypter) (`roles/cloudkms.cryptoKeyEncrypter`)
- [DLP User](https://docs.cloud.google.com/iam/docs/roles-permissions/dlp.md.txt#dlp.user) (`roles/dlp.user`)
For more information about granting roles, see [Manage access to projects, folders, and organizations](https://docs.cloud.google.com/iam/docs/granting-changing-revoking-access.md.txt).
You might also be able to get
the required permissions through [custom
roles](https://docs.cloud.google.com/iam/docs/creating-custom-roles) or other [predefined
roles](https://docs.cloud.google.com/iam/docs/roles-overview#predefined).
## Create a key ring and a key
Before you start this procedure, decide where you want Sensitive Data Protection
to process your de-identification and re-identification requests. When you
create a Cloud KMS key, you must store it in either `global` or in the
same region that you will use for your Sensitive Data Protection requests.
Otherwise, the Sensitive Data Protection requests will fail.
You can find a list of supported locations in [Sensitive Data Protection
locations](https://docs.cloud.google.com/sensitive-data-protection/docs/locations). Note the name of your
chosen region (for example, `us-west1`).
This procedure uses `global` as the location for all API requests. If you want
to use a different region, replace `global` with the region name.
1. Create a key ring:
```bash
gcloud kms keyrings create "dlp-keyring" \
--location "global"
```
2. Create a key:
```bash
gcloud kms keys create "dlp-key" \
--location "global" \
--keyring "dlp-keyring" \
--purpose "encryption"
```
3. List your key ring and key:
```bash
gcloud kms keys list \
--location "global" \
--keyring "dlp-keyring"
```
You get the following output:
```bash
NAME: projects/PROJECT_ID/locations/global/keyRings/dlp-keyring/cryptoKeys/dlp-key
PURPOSE: ENCRYPT_DECRYPT
ALGORITHM: GOOGLE_SYMMETRIC_ENCRYPTION
PROTECTION_LEVEL: SOFTWARE
LABELS:
PRIMARY_ID: 1
PRIMARY_STATE: ENABLED
```
In this output, `PROJECT_ID` is the ID of your project.
The value of `NAME` is the full resource name of your Cloud KMS
key. Note this value because the de-identify and re-identify requests
require it.
## Create a base64-encoded AES key
This section describes how to create an Advanced Encryption Standard (AES) key
and encode it in base64 format.
> [!NOTE]
> **Note:** These steps use the `openssl` and `base64` commands, but there are [many other ways](https://docs.cloud.google.com/storage/docs/encryption/using-customer-supplied-keys.md.txt#generate-csek) to perform this task.
1. Create a 128-, 192-, or 256-bit AES key. The following command uses `openssl`
to create a 256-bit key in the current directory:
```bash
openssl rand -out "./aes_key.bin" 32
```
The file `aes_key.bin` is added to your current directory.
2. Encode the AES key as a base64 string:
```bash
base64 -i ./aes_key.bin
```
You get an output similar to the following:
```bash
uEDo6/yKx+zCg2cZ1DBwpwvzMVNk/c+jWs7OwpkMc/s=
```
> [!WARNING]
> **Warning:** Don't use this example key to protect actual sensitive workloads. This key is provided only to serve as an example. Because it's shared here, this key is not safe to use.
## Wrap the AES key using the Cloud KMS key
This section describes how to use the Cloud KMS key that you created in
[Create a key ring and a key](https://docs.cloud.google.com/sensitive-data-protection/docs/inspect-sensitive-text-de-identify.md.txt#create-kms-key) to wrap the base64-encoded AES
key that you created in [Create a base64-encoded AES key](https://docs.cloud.google.com/sensitive-data-protection/docs/inspect-sensitive-text-de-identify.md.txt#create-base64-key).
To wrap the AES key, use `curl` to send the following request to the
Cloud KMS API
[`projects.locations.keyRings.cryptoKeys.encrypt`](https://docs.cloud.google.com/kms/docs/reference/rest/v1/projects.locations.keyRings.cryptoKeys/encrypt.md.txt)
method:
```bash
curl "https://cloudkms.googleapis.com/v1/projects/PROJECT_ID/locations/global/keyRings/dlp-keyring/cryptoKeys/dlp-key:encrypt" \
--request "POST" \
--header "Authorization:Bearer $(gcloud auth application-default print-access-token)" \
--header "content-type: application/json" \
--data "{\"plaintext\": \"BASE64_ENCODED_AES_KEY\"}"
```
Replace the following:
- `PROJECT_ID`: the [ID of your project](https://docs.cloud.google.com/resource-manager/docs/creating-managing-projects.md.txt#before_you_begin).
- `BASE64_ENCODED_AES_KEY`: the base64-encoded string returned in [Create a base64-encoded AES key](https://docs.cloud.google.com/sensitive-data-protection/docs/inspect-sensitive-text-de-identify.md.txt#create-base64-key).
The response that you get from Cloud KMS is similar to the following
JSON:
```json
{
"name": "projects/PROJECT_ID/locations/global/keyRings/dlp-keyring/cryptoKeys/dlp-key/cryptoKeyVersions/1",
"ciphertext": "CiQAYuuIGo5DVaqdE0YLioWxEhC8LbTmq7Uy2G3qOJlZB7WXBw0SSQAjdwP8ZusZJ3Kr8GD9W0vaFPMDksmHEo6nTDaW/j5sSYpHa1ym2JHk+lUgkC3Zw5bXhfCNOkpXUdHGZKou1893O8BDby/82HY=",
"ciphertextCrc32c": "901327763",
"protectionLevel": "SOFTWARE"
}
```
In this output, `PROJECT_ID` is the ID of your project.
Note the value of `ciphertext` in the response.
That is your wrapped key.
## Send a de-identify request to the DLP API
This section describes how to de-identify sensitive data in text content.
To complete this task, you need the following:
- The full resource name of the Cloud KMS key that you created in [Create a key ring and a
key](https://docs.cloud.google.com/sensitive-data-protection/docs/inspect-sensitive-text-de-identify#create-kms-key).
- The wrapped key that you created in [Wrap the AES key using the
Cloud KMS
key](https://docs.cloud.google.com/sensitive-data-protection/docs/inspect-sensitive-text-de-identify#wrap-aes-key).
You must save the sample request in a JSON file. If you use
Cloud Shell, use the Cloud Shell Editor to create the
file. To launch the editor, click **Open Editor** on the Cloud Shell toolbar.
To de-identify sensitive data in text content, follow these steps:
1. Create a JSON request file called `deidentify-request.json` with the
following text.
```json
{
"item": {
"value": "My name is Alicia Abernathy, and my email address is aabernathy@example.com."
},
"deidentifyConfig": {
"infoTypeTransformations": {
"transformations": [
{
"infoTypes": [
{
"name": "EMAIL_ADDRESS"
}
],
"primitiveTransformation": {
"cryptoDeterministicConfig": {
"cryptoKey": {
"kmsWrapped": {
"cryptoKeyName": "projects/PROJECT_ID/locations/global/keyRings/dlp-keyring/cryptoKeys/dlp-key",
"wrappedKey": "WRAPPED_KEY"
}
},
"surrogateInfoType": {
"name": "EMAIL_ADDRESS_TOKEN"
}
}
}
}
]
}
},
"inspectConfig": {
"infoTypes": [
{
"name": "EMAIL_ADDRESS"
}
]
}
}
```
Replace the following:
- `PROJECT_ID`: the ID of your project.
- `WRAPPED_KEY`: the wrapped key that you created in [Wrap the AES key using the Cloud KMS
key](https://docs.cloud.google.com/sensitive-data-protection/docs/inspect-sensitive-text-de-identify#wrap-aes-key).
Make sure that the resulting value of `cryptoKeyName` forms the full resource
name of your Cloud KMS key.
For more information about the components of this JSON request, see
[`projects.locations.content.deidentify`](https://docs.cloud.google.com/sensitive-data-protection/docs/reference/rest/v2/projects.locations.content/deidentify.md.txt).
After you complete this task, experiment with different inputs
for this request. You can use `curl` as described here. Alternatively, use
the API Explorer on that API reference page under **Try this method**.
2. Use `curl` to make a `projects.locations.content.deidentify` request:
```bash
curl -s \
-H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
-H "Content-Type: application/json" \
https://dlp.googleapis.com/v2/projects/PROJECT_ID/locations/global/content:deidentify \
-d @deidentify-request.json
```
Replace `PROJECT_ID` with the ID of your project.
To pass a filename to `curl`, use the
`-d` option (for data) and precede the filename with an
`@` sign. This file must be in the same directory where you
run the `curl` command.
> [!NOTE]
> **Note:** This example request explicitly targets the `global` location. Making this request is the same as calling the [`projects.content.deidentify`](https://docs.cloud.google.com/sensitive-data-protection/docs/reference/rest/v2/projects.content/deidentify.md.txt) API, which defaults to the `global` location.
The response that you get from Sensitive Data Protection is similar to the
following JSON:
```json
{
"item": {
"value": "My name is Alicia Abernathy, and my email address is EMAIL_ADDRESS_TOKEN(52):AVAx2eIEnIQP5jbNEr2j9wLOAd5m4kpSBR/0jjjGdAOmryzZbE/q."
},
"overview": {
"transformedBytes": "22",
"transformationSummaries": [
{
"infoType": {
"name": "EMAIL_ADDRESS"
},
"transformation": {
"cryptoDeterministicConfig": {
"cryptoKey": {
"kmsWrapped": {
"wrappedKey": "CiQAYuuIGo5DVaqdE0YLioWxEhC8LbTmq7Uy2G3qOJlZB7WXBw0SSQAjdwP8ZusZJ3Kr8GD9W0vaFPMDksmHEo6nTDaW/j5sSYpHa1ym2JHk+lUgkC3Zw5bXhfCNOkpXUdHGZKou1893O8BDby/82HY=",
"cryptoKeyName": "projects/PROJECT_ID/locations/global/keyRings/dlp-keyring/cryptoKeys/dlp-key"
}
},
"surrogateInfoType": {
"name": "EMAIL_ADDRESS_TOKEN"
}
}
},
"results": [
{
"count": "1",
"code": "SUCCESS"
}
],
"transformedBytes": "22"
}
]
}
}
```
In the `item` field, the email address is replaced with a token like
`EMAIL_ADDRESS_TOKEN(52):AVAx2eIEnIQP5jbNEr2j9wLOAd5m4kpSBR/0jjjGdAOmryzZbE/q`.
Note the value of the token in the response. To
re-identify the de-identified content, you pass the entire token in the
re-identify request.
## Send a re-identify request to the DLP API
This section describes how to re-identify tokenized data in text content.
To complete this task, you need the following:
- The full resource name of the Cloud KMS key that you created in [Create a key ring and a
key](https://docs.cloud.google.com/sensitive-data-protection/docs/inspect-sensitive-text-de-identify#create-kms-key).
- The wrapped key that you created in [Wrap the AES key using the
Cloud KMS
key](https://docs.cloud.google.com/sensitive-data-protection/docs/inspect-sensitive-text-de-identify#wrap-aes-key).
- The token that you received in [Send a de-identify request to the
DLP API](https://docs.cloud.google.com/sensitive-data-protection/docs/inspect-sensitive-text-de-identify#deidentify).
To re-identify tokenized content, follow these steps:
1. Create a JSON request file called `reidentify-request.json` with the
following text.
```json
{
"reidentifyConfig":{
"infoTypeTransformations":{
"transformations":[
{
"infoTypes":[
{
"name":"EMAIL_ADDRESS_TOKEN"
}
],
"primitiveTransformation":{
"cryptoDeterministicConfig":{
"cryptoKey":{
"kmsWrapped": {
"cryptoKeyName": "projects/PROJECT_ID/locations/global/keyRings/dlp-keyring/cryptoKeys/dlp-key",
"wrappedKey": "WRAPPED_KEY"
}
},
"surrogateInfoType":{
"name":"EMAIL_ADDRESS_TOKEN"
}
}
}
}
]
}
},
"inspectConfig":{
"customInfoTypes":[
{
"infoType":{
"name":"EMAIL_ADDRESS_TOKEN"
},
"surrogateType":{
}
}
]
},
"item":{
"value": "My name is Alicia Abernathy, and my email address is TOKEN."
}
}
```
Replace the following:
- `PROJECT_ID`: the ID of your project.
- `WRAPPED_KEY`: the wrapped key that you created in [Wrap the AES key using the Cloud KMS
key](https://docs.cloud.google.com/sensitive-data-protection/docs/inspect-sensitive-text-de-identify#wrap-aes-key).
- `TOKEN`: the token that you received in [Send a
de-identify request to the
DLP API](https://docs.cloud.google.com/sensitive-data-protection/docs/inspect-sensitive-text-de-identify#deidentify)---for example, `EMAIL_ADDRESS_TOKEN(52):AVAx2eIEnIQP5jbNEr2j9wLOAd5m4kpSBR/0jjjGdAOmryzZbE/q`.
Make sure that the resulting value of `cryptoKeyName` forms the full resource
name of your Cloud KMS key.
For more information about the components of this JSON request, see
[`projects.locations.content.reidentify`](https://docs.cloud.google.com/sensitive-data-protection/docs/reference/rest/v2/projects.locations.content/reidentify.md.txt).
After you complete this task, experiment with different inputs
for this request. You can use `curl` as described here. Alternatively, use
the API Explorer on that API reference page under **Try this
method**.
2. Use `curl` to make a `projects.locations.content.reidentify` request:
```bash
curl -s \
-H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
-H "Content-Type: application/json" \
https://dlp.googleapis.com/v2/projects/PROJECT_ID/locations/global/content:reidentify \
-d @reidentify-request.json
```
Replace `PROJECT_ID` with the ID of your project.
To pass a filename to `curl`, use the
`-d` option (for data) and precede the filename with an
`@` sign. This file must be in the same directory where you
run the `curl` command.
> [!NOTE]
> **Note:** This example request explicitly targets the `global` location. Making this request is the same as calling the [`projects.content.reidentify`](https://docs.cloud.google.com/sensitive-data-protection/docs/reference/rest/v2/projects.content/reidentify.md.txt) API, which defaults to the `global` location.
The response that you get from Sensitive Data Protection is similar to the
following JSON:
```json
{
"item": {
"value": "My name is Alicia Abernathy, and my email address is aabernathy@example.com."
},
"overview": {
"transformedBytes": "70",
"transformationSummaries": [
{
"infoType": {
"name": "EMAIL_ADDRESS"
},
"transformation": {
"cryptoDeterministicConfig": {
"cryptoKey": {
"kmsWrapped": {
"wrappedKey": "CiQAYuuIGo5DVaqdE0YLioWxEhC8LbTmq7Uy2G3qOJlZB7WXBw0SSQAjdwP8ZusZJ3Kr8GD9W0vaFPMDksmHEo6nTDaW/j5sSYpHa1ym2JHk+lUgkC3Zw5bXhfCNOkpXUdHGZKou1893O8BDby/82HY=",
"cryptoKeyName": "projects/PROJECT_ID/locations/global/keyRings/dlp-keyring/cryptoKeys/dlp-key"
}
},
"surrogateInfoType": {
"name": "EMAIL_ADDRESS_TOKEN"
}
}
},
"results": [
{
"count": "1",
"code": "SUCCESS"
}
],
"transformedBytes": "70"
}
]
}
}
```
In the `item` field, the email address token is replaced with the actual
email address from the original text.
You have now de-identified and re-identified sensitive data
in text content using deterministic encryption.
## Clean up
To avoid incurring charges to your Google Cloud account for
the resources used on this page, delete the Google Cloud project with the
resources.
### Destroy your key version
If you no longer want to use the key you created in this task, destroy
its version.
> [!WARNING]
> **Warning:** When you destroy a key version, you can no longer decrypt content that was encrypted using that version of the key. Make sure that you no longer need a key version before you destroy it.
List the versions available for your key:
```bash
gcloud kms keys versions list \
--location "global" \
--keyring "dlp-keyring" \
--key "dlp-key"
```
To destroy a version, run the following command:
```bash
gcloud kms keys versions destroy KEY_VERSION \
--location "global" \
--keyring "dlp-keyring" \
--key "dlp-key"
```
Replace `KEY_VERSION` with the number of the version to be
destroyed---for example, `1`.
### Delete the project
If you created a new project for this task, the easiest way to prevent
additional charges is to delete the project.
> [!CAUTION]
> **Caution** : Deleting a project has the following effects:
>
> - **Everything in the project is deleted.** If you used an existing project for the tasks in this document, when you delete it, you also delete any other work you've done in the project.
> - **Custom project IDs are lost.** When you created this project, you might have created a custom project ID that you want to use in the future. To preserve the URLs that use the project ID, such as an `appspot.com` URL, delete selected resources inside the project instead of deleting the whole project.
>
>
> If you plan to explore multiple architectures, tutorials, or quickstarts, reusing projects
> can help you avoid exceeding project quota limits.
1. Delete a Google Cloud project:
```
gcloud projects delete PROJECT_ID
```
### Revoke your credentials
Optional: Revoke credentials from the gcloud CLI.
```bash
gcloud auth revoke
```
## What's next
- For more information about de-identifying sensitive content, see [De-identifying sensitive
data](https://docs.cloud.google.com/sensitive-data-protection/docs/deidentify-sensitive-data).
- For information about how a de-identification workflow fits into real-life deployments, see [De-identification and re-identification of PII in
large-scale datasets using Sensitive Data Protection](https://docs.cloud.google.com/architecture/de-identification-re-identification-pii-using-cloud-dlp).
- For conceptual information about tokenizing data with a cryptographic key, see [Pseudonymization](https://docs.cloud.google.com/sensitive-data-protection/docs/pseudonymization.md.txt).