Docs for v17 incremental backup and recovery #1524

shlomi-noach · 2023-07-05T10:56:08Z

This PR adds (late) v17 documentation for incremental backups and recoveries, following these changes in v17:

v17 is already released, and this is a late documentation. For the purposes of review, I've only updated docs/17.0, but once approved, I will copy+paste the changes into docs/18.0 as well. These will be hopefully shortly updated with the new functionality introduced in vitessio/vitess#13270 and some expected followups.

RFC: vitessio/vitess#11227

Please review but do not merge yet

Signed-off-by: Shlomi Noach <[email protected]>

netlify · 2023-07-05T10:57:11Z

✅ Deploy Preview for vitess ready!

Name	Link
🔨 Latest commit	`c6bdede`
🔍 Latest deploy log	https://app.netlify.com/sites/vitess/deploys/64ae3fe38279000008720ac2
😎 Deploy Preview	https://deploy-preview-1524--vitess.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

shlomi-noach · 2023-07-05T11:04:02Z

Previews for affected pages:

mattlord · 2023-07-05T16:42:35Z

content/en/docs/17.0/reference/features/recovery.md

@@ -6,6 +6,23 @@ aliases: ['/docs/recovery/pitr','/docs/reference/pitr/']

 ## Point in Time Recovery

+Vitess supports incremental backup and recoveries, aka point in time recoveries. `v17` offers restore-to-position functionality, and `v18` is slated to support restore-to-timestamp functionality.


aka should be capitalized as it's an initialism.

Will v18 support both restore to position AND restore to timestamp, or restore to timestamp will replace restore to position?

v18 will support both. Clarified in text.

mattlord · 2023-07-05T16:43:53Z

content/en/docs/17.0/reference/features/recovery.md

@@ -6,6 +6,23 @@ aliases: ['/docs/recovery/pitr','/docs/reference/pitr/']

 ## Point in Time Recovery

+Vitess supports incremental backup and recoveries, aka point in time recoveries. `v17` offers restore-to-position functionality, and `v18` is slated to support restore-to-timestamp functionality.
+
+Point in time recoveries are based on Full and Incremental backups. It is possible to recover a database to a position that is _covered_ by some backup.


Why are full and incremental capitalized here?

Decapitalized.

mattlord · 2023-07-05T16:45:30Z

content/en/docs/17.0/reference/features/recovery.md

+## Point in Time Recovery: legacy functionality based on binlog server
+


Should we remove this entire section in the 18.0 docs?

Per @deepthi , we should keep it for a while.

mattlord · 2023-07-05T16:48:06Z

content/en/docs/17.0/user-guides/operating-vitess/backup-and-restore/bootstrap-and-restore.md

+Example:
+
+```shell
+vtctlclient -- RestoreFromBackup --restore_to_pos "MySQL56/0d7aaca6-1666-11ee-aeaf-0a43f95f28a3:1-60" zone1-0000000102


It would be a nice enhancement in 18.0 if we expected a valid MySQL GTID set value. i.e. not requiring MySQL56/. That is unnecessary anyway as we only support MySQL (5.7 and 8.0) today. An argument to leave that thought would be that we could re-add support for MariaDB at some point. We can also infer this based on the format of the value though.

Catching up on email after vacation and I now see that you're ahead of me 🙂

vitessio/vitess#13415

See vitessio/vitess#13474

content/en/docs/17.0/user-guides/operating-vitess/backup-and-restore/creating-a-backup.md

content/en/docs/17.0/user-guides/operating-vitess/backup-and-restore/overview.md

mattlord · 2023-07-05T16:59:42Z

content/en/docs/17.0/user-guides/operating-vitess/backup-and-restore/overview.md

+
+Restores are the counterparts of backups. A restore uses the engine utilized to create a backup. One may run a restore from a full backup, or a point-in-time restore (PITR) based on additional incremental backups.
+
+A Vitess restore operates on a tablet. The restore process completely wipes out the data in the tablet's MySQL server and repopulates the server with the backup(s) data. The MySQL server is shutdown durign the process. As a safety mechanism, Vitess by default prevents a restore onto a `PRIMARY` tablet.


typo: durign

content/en/docs/17.0/user-guides/operating-vitess/backup-and-restore/overview.md

mattlord · 2023-07-05T17:01:14Z

content/en/docs/17.0/user-guides/operating-vitess/backup-and-restore/overview.md

+Vitess supports full restores and incremental (aka point in time) restores. The two serve different purposes.
+
+* A full restore loads the dataset from a full backup onto a non-`PRIMARY` tablet. Once the data is loaded, the restore process starts the MySQL service and makes it join the replication stream. It is expected that a freshly restored server will lag behind the shard's `PRIMARY` for a period of time.
+  The full restore flow is useful for seeding new replica tablets. It may also be used to fix replicas that have been corrupted.


Do we mean REPLICA tablet types here or really non-PRIMARY tablets?

You can use any non-PRIMARY type to restore a backup.

I'd say that then to avoid any confusion (in particular for RDONLY tablets).

content/en/docs/17.0/user-guides/operating-vitess/backup-and-restore/overview.md

mattlord · 2023-07-05T18:08:37Z

Should this PR fix #1490 ?

shlomi-noach · 2023-07-06T03:11:57Z

Indeed.

Fixes #1490.

Signed-off-by: Shlomi Noach <[email protected]>

…estore/overview.md Co-authored-by: Matt Lord <[email protected]> Signed-off-by: Shlomi Noach <[email protected]>

Signed-off-by: Shlomi Noach <[email protected]>

mattlord · 2023-07-10T20:59:35Z

content/en/docs/17.0/user-guides/operating-vitess/backup-and-restore/overview.md

+Vitess supports full restores and incremental (aka point in time) restores. The two serve different purposes.
+
+* A full restore loads the dataset from a full backup onto a non-`PRIMARY` tablet. Once the data is loaded, the restore process starts the MySQL service and makes it join the replication stream. It is expected that a freshly restored server will lag behind the shard's `PRIMARY` for a period of time.
+  The full restore flow is useful for seeding new replica tablets. It may also be used to fix replicas that have been corrupted.


I'd say that then to avoid any confusion (in particular for RDONLY tablets).

Signed-off-by: Shlomi Noach <[email protected]>

shlomi-noach · 2023-07-12T05:51:14Z

I will now copy all changes over to v18 and merge.

Signed-off-by: Shlomi Noach <[email protected]>

shlomi-noach added 6 commits June 29, 2023 14:06

Explaining full/incremental backups and restores

1d66e5f

Signed-off-by: Shlomi Noach <[email protected]>

clarify the prevention of a backup on PRIMARY

014e04a

Signed-off-by: Shlomi Noach <[email protected]>

Creating an incremental backup

43adafe

Signed-off-by: Shlomi Noach <[email protected]>

bootstrap and restore: document --restore_to_pos

3c511f0

Signed-off-by: Shlomi Noach <[email protected]>

more notes on incremental restore

044d0a4

Signed-off-by: Shlomi Noach <[email protected]>

Elaborating on the new PITR, with links to user guides

d5678c7

Signed-off-by: Shlomi Noach <[email protected]>

shlomi-noach requested a review from a team July 5, 2023 10:56

mattlord reviewed Jul 5, 2023

View reviewed changes

shlomi-noach and others added 9 commits July 6, 2023 06:57

capitalize. Clarify

495687b

Signed-off-by: Shlomi Noach <[email protected]>

Decapitalize

6b91dcb

Signed-off-by: Shlomi Noach <[email protected]>

Decapitalize

09697fa

Signed-off-by: Shlomi Noach <[email protected]>

Decapitalize

d80e273

Signed-off-by: Shlomi Noach <[email protected]>

typo

9499d20

Signed-off-by: Shlomi Noach <[email protected]>

Update content/en/docs/17.0/user-guides/operating-vitess/backup-and-r…

a062690

…estore/overview.md Co-authored-by: Matt Lord <[email protected]> Signed-off-by: Shlomi Noach <[email protected]>

Update content/en/docs/17.0/user-guides/operating-vitess/backup-and-r…

514210c

…estore/overview.md Co-authored-by: Matt Lord <[email protected]> Signed-off-by: Shlomi Noach <[email protected]>

Update content/en/docs/17.0/user-guides/operating-vitess/backup-and-r…

795ec4c

…estore/overview.md Co-authored-by: Matt Lord <[email protected]> Signed-off-by: Shlomi Noach <[email protected]>

typo

49d9642

Signed-off-by: Shlomi Noach <[email protected]>

shlomi-noach requested a review from a team July 9, 2023 12:42

mattlord approved these changes Jul 10, 2023

View reviewed changes

clarify non-PRIMARY is eligible to restore

c895545

Signed-off-by: Shlomi Noach <[email protected]>

shlomi-noach added the documentation label Jul 12, 2023

shlomi-noach added 2 commits July 12, 2023 08:51

Merge branch 'prod' into v17-docs-backup-pitr

8acd838

applying doc changes to 18.0 docs

c6bdede

Signed-off-by: Shlomi Noach <[email protected]>

shlomi-noach merged commit b24d21b into prod Jul 12, 2023

shlomi-noach deleted the v17-docs-backup-pitr branch July 12, 2023 05:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Docs for v17 incremental backup and recovery #1524

Docs for v17 incremental backup and recovery #1524

shlomi-noach commented Jul 5, 2023

netlify bot commented Jul 5, 2023 •

edited

Loading

shlomi-noach commented Jul 5, 2023 •

edited

Loading

mattlord Jul 5, 2023

shlomi-noach Jul 6, 2023

mattlord Jul 5, 2023

shlomi-noach Jul 6, 2023

mattlord Jul 5, 2023

shlomi-noach Jul 6, 2023

mattlord Jul 5, 2023

mattlord Jul 5, 2023

shlomi-noach Jul 6, 2023

shlomi-noach Jul 11, 2023

mattlord Jul 5, 2023

shlomi-noach Jul 6, 2023

mattlord Jul 5, 2023

shlomi-noach Jul 6, 2023

mattlord Jul 10, 2023

shlomi-noach Jul 11, 2023

mattlord commented Jul 5, 2023

shlomi-noach commented Jul 6, 2023

mattlord Jul 10, 2023

shlomi-noach commented Jul 12, 2023

		@@ -6,6 +6,23 @@ aliases: ['/docs/recovery/pitr','/docs/reference/pitr/']

		## Point in Time Recovery

		Vitess supports incremental backup and recoveries, aka point in time recoveries. `v17` offers restore-to-position functionality, and `v18` is slated to support restore-to-timestamp functionality.

		## Point in Time Recovery: legacy functionality based on binlog server


		Restores are the counterparts of backups. A restore uses the engine utilized to create a backup. One may run a restore from a full backup, or a point-in-time restore (PITR) based on additional incremental backups.

		A Vitess restore operates on a tablet. The restore process completely wipes out the data in the tablet's MySQL server and repopulates the server with the backup(s) data. The MySQL server is shutdown durign the process. As a safety mechanism, Vitess by default prevents a restore onto a `PRIMARY` tablet.

Docs for v17 incremental backup and recovery #1524

Docs for v17 incremental backup and recovery #1524

Conversation

shlomi-noach commented Jul 5, 2023

Please review but do not merge yet

netlify bot commented Jul 5, 2023 • edited Loading

✅ Deploy Preview for vitess ready!

shlomi-noach commented Jul 5, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mattlord commented Jul 5, 2023

shlomi-noach commented Jul 6, 2023

Choose a reason for hiding this comment

shlomi-noach commented Jul 12, 2023

netlify bot commented Jul 5, 2023 •

edited

Loading

shlomi-noach commented Jul 5, 2023 •

edited

Loading