Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(rdb): encryption at rest api + bench MTA-5369 #4132

Merged
merged 8 commits into from
Dec 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
---
meta:
title: Setting up Encryption at Rest for your Managed Databases with the Scaleway API
description: This page explains how to set up Encryption at Rest for your Managed Databases with the Scaleway API.
content:
h1: Setting up Encryption at Rest for your Managed Databases with the Scaleway API
paragraph: This page explains how to set up Encryption at Rest for your Managed Databases with the Scaleway API.
tags: managed-database postgresql mysql encryption at rest
dates:
validation: 2024-12-19
posted: 2024-12-19
categories:
- managed-databases
- postgresql-and-mysql
---

Encryption at rest allows you to permanently encrypt your database data. The data is encrypted at volume level using [LUKS](https://en.wikipedia.org/wiki/Linux_Unified_Key_Setup). The management of the encryption key is done by Scaleway.

The feature can be activated upon Database Instance creation via the console and the API, or after creation exclusively through the API with the [upgrade endpoint](https://www.scaleway.com/en/developers/api/managed-database-postgre-mysql/#path-database-instances-upgrade-a-database-instance).

<Message type="important">
- Once activated on a Database Instance, encryption at rest cannot be disabled.
- All databases, data (including logs), and snapshots will be encrypted. Logical backup encryption is not currently available.
- Enabling encryption means your database data will be copied into a new, encrypted block volume. This takes approximately 1 hour per 100 GB of storage. When switching volumes, expect a few seconds of downtime towards the end of the copying process. Refer to the [Encryption at rest performance benchmark on Database Instances](/managed-databases/postgresql-and-mysql/reference-content/encryption-at-rest-performance-benchmark) page.
</Message>

## Creating a Database Instance with encryption at rest

1. Edit the POST request payload you will use to create your Database Instance. Make sure you include the `encryption` parameter and set the `enabled` attribute to `true`.
<Message type="note">
In all examples below the UUIDs, passwords and IP addresses displayed are not real. For more details about the parameters shown, refer to the [Managed Databases for PostgreSQL and MySQL API documentation](https://www.scaleway.com/en/developers/api/managed-database-postgre-mysql/)
</Message>

```json
'{
"project_id": "d8e65f2b-cce9-40b7-80fc-6a2902db6826",
"name": "myDB",
"engine": "PostgreSQL-15",
"tags": ["foo", "bar"],
"is_ha_cluster": true,
"node_type": "db-pro2-xxs",
"disable_backup": false,
"user_name": "my_initial_user",
"password": "thiZ_is_v0ry_s3cret",
"volume_type": "sbs_5k",
"volume_size": "30000000000",
"encryption": {
"enabled": true
}
}'
```

2. Create a Database Instance by running the following command. Make sure you include the payload you edited in the previous step.
```bash
curl -X POST \
-H "X-Auth-Token: $SCW_SECRET_KEY" \
"Content-Type: application/json" \
https://api.scaleway.com/rdb/v1/regions/$SCW_REGION/instances \
-d '{
"project_id": "d8e65f2b-cce9-40b7-80fc-6a2902db6826",
"name": "myDB",
"engine": "PostgreSQL-15",
"tags": ["foo", "bar"],
"is_ha_cluster": true,
"node_type": "db-pro2-xxs",
"disable_backup": false,
"user_name": "my_initial_user",
"password": "thiZ_is_v0ry_s3cret",
"volume_type": "sbs_5k",
"volume_size": "30000000000",
"encryption": {
"enabled": true
}
}'
```
You should get a response like the following confirming that the Database Instance was created, and encryption at rest is enabled.

```json
{
"id": "f5122f66-fb50-4cef-aa02-487ef4fc1af0",
"name": "myDB",
"organization_id": "895693aa-3915-4896-8761-c2923b008be7",
"project_id": "d8e65f2b-cce9-40b7-80fc-6a2902db6826",
"status": "ready",
"engine": "PostgreSQL-15",
"endpoint": {
"ip": "198.51.100.0",
"port": 22245,
"name": null
},
"tags": [
"foo",
"bar"
],
"settings": [],
"backup_schedule": {
"frequency": 24,
"retention": 7,
"disabled": true
},
"is_ha_cluster": true,
"read_replicas": [],
"node_type": "db-pro2-xxs",
"volume": {
"type": "sbs_5k",
"size": 30000000000
},
"encryption": {
"enabled": true
},
"created_at": "2019-04-19T16:24:52.591417Z",
"region": "fr-par"
}
```

## Enabling encryption at rest in an existing Database Instance

To enable encryption at rest after a Database Instance has already been created, you can use the upgrade endpoint of the Managed Databases API.

Run the following command. Make sure you replace the `instance_id` in the endpoint, and the `enable_encryption` parameter set to `true`

```json
curl -X POST \
-H "X-Auth-Token: $SCW_SECRET_KEY" \
-H "Content-Type: application/json" \
-d '{
"enable_encryption": true
}' \
"https://api.scaleway.com/rdb/v1/regions/fr-par/instances/{instance_id}/upgrade"
```

If the operation is successful, you see an output containing all the details of your Database Instance, including `"encryption":{"enabled":true}`.

Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
---
meta:
title: Encryption at rest performance benchmark on Database Instances
description: This page shows benchmarks of different Database Instances' performance when encryption at rest is enabled.
content:
h1: Encryption at rest performance benchmark on Database Instances
paragraph: This page shows benchmarks of different Database Instances' performance when encryption at rest is enabled.
tags: managed-database postgresql mysql encryption at rest
dates:
validation: 2024-12-19
posted: 2024-12-19
categories:
- managed-databases
- postgresql-and-mysql
---

In December 2024, the Scaleway Managed Databases team ran tests on DB-POP2-4C-16G Instances in a Private Network using `pgbench`. Several workloads (including read, oltp and large loads) were simulated with several scale factors (from 1M rows to 200M rows) and concurrent database connections set to 10. They measured the impact of encryption at rest on transactions, per second and latency.

<Message type="important">
For more information on pgbench transactions, refer to the "What is the 'Transaction' actually performed in pgbench?" section of the official [pgbench documentation](https://www.postgresql.org/docs/current/pgbench.html).
</Message>

<Message type="note">
The workloads applied simulate heavy usage scenarios for the purposes of testing. During the beta of encryption at rest, no performance issues were raised by users. If you would like to share your own benchmarks with our team, feel free to reach out to us in the #database channel of the [Scaleway Slack community](/tutorials/scaleway-slack-community/).
</Message>

The table below compares the different workload and setup combinations and indicates the performance results of each:

| Workload Type | Setup | Scale Factor | Performance Impact |
|----------------------|-----------------|--------------|------------------------------------------------------------------------------------|
| **Read Workloads** | Standalone | 10 | No impact |
| | Standalone | 2000 | No real impact with or without encryption at rest |
| | HA | 2000 | 20-30% impact, likely due to disk writes being replicated. Refer to [this blogpost](https://blog.okmeter.io/postgresql-exploring-how-select-queries-can-produce-disk-writes-f36c8bee6b6f) to learn more about disk writes in read operations. |
| **OLTP Workloads** | Standalone | Any | 15% performance drop with encryption at rest activated |
| | HA | 10 | 15% performance drop |
| | HA | 2000 | Up to 30% performance drop |
| **Large Data Load (29 GB)** | Standalone | Any | 10% additional latency |
| | HA | Any | 40% additional latency due to semi-sync replication and encryption overhead |

8 changes: 8 additions & 0 deletions menu/navigation.json
Original file line number Diff line number Diff line change
Expand Up @@ -2198,6 +2198,10 @@
"label": "PostgreSQL extensions reference",
"slug": "postgresql-extensions"
},
{
"label": "Encryption at rest performance benchmark",
"slug": "encryption-at-rest-performance-benchmark"
},
{
"label": "Understanding the autohealing feature",
"slug": "autohealing"
Expand All @@ -2212,6 +2216,10 @@
"label": "Managed Database for PostgreSQL and MySQL API Reference",
"slug": "https://www.scaleway.com/en/developers/api/managed-database-postgre-mysql/"
},
{
"label": "Setting up encryption at rest",
"slug": "setting-up-encryption-at-rest"
},
{
"label": "Setting up and using the pgaudit extension",
"slug": "pg-audit"
Expand Down
Loading