Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assorted sync metadata storage refactoring #7329

Merged
merged 6 commits into from
Feb 12, 2024
Merged

Assorted sync metadata storage refactoring #7329

merged 6 commits into from
Feb 12, 2024

Conversation

tgoyne
Copy link
Member

@tgoyne tgoyne commented Feb 8, 2024

This is pulling some of the incidental changes out of #7300 since that PR is growing far too large. None of this has meaningful functional changes on its own and is mostly just pushing code around.

One of the upcoming changes is making SyncUsers lazily loaded and reloaded when the data in the metadata store changes, which benefits from it being relatively cheap to validate that the stored user is valid. Changing RealmJWT to operate on a string view had cascading effects on base64 and exjson parsing, which I also updated to use Span rather than a pointer+size while I was touching it.

Checking if the data in the metadata realm is valid was done incompletely in three different places, and now is just in SyncMetadataManager. File action handling is also there now as that's going to be required for it to be multiprocess safe. Most of SyncMetadataManager's API is now unused outside of tests, but is left in place for now as removing it requires rewriting the tests (which is done in #7300).

CheckedMutex::lock() is const, so it doesn't have to be declared as mutable.

@tgoyne tgoyne self-assigned this Feb 8, 2024
@cla-bot cla-bot bot added the cla: yes label Feb 8, 2024
@tgoyne tgoyne force-pushed the tg/jwt-validation branch 2 times, most recently from 6d879f9 to ded9c66 Compare February 8, 2024 17:58
Base automatically changed from tg/xcode-15.3 to master February 8, 2024 18:16
@tgoyne tgoyne force-pushed the tg/jwt-validation branch from ded9c66 to 2be351d Compare February 8, 2024 19:22
Copy link

coveralls-official bot commented Feb 8, 2024

Pull Request Test Coverage Report for Build thomas.goyne_158

  • 219 of 246 (89.02%) changed or added relevant lines in 23 files are covered.
  • 82 unchanged lines in 9 files lost coverage.
  • Overall coverage increased (+0.005%) to 91.863%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/realm/object-store/c_api/app.cpp 0 1 0.0%
src/realm/sync/noinst/protocol_codec.cpp 0 1 0.0%
src/realm/sync/noinst/server/access_token.cpp 1 3 33.33%
src/realm/object-store/sync/impl/sync_metadata.cpp 94 97 96.91%
src/realm/util/file.cpp 3 6 50.0%
src/realm/object-store/sync/jwt.cpp 49 57 85.96%
src/realm/object-store/sync/jwt.hpp 3 12 25.0%
Files with Coverage Reduction New Missed Lines %
src/realm/index_string.cpp 1 87.85%
src/realm/query_expression.hpp 2 93.51%
src/realm/sync/network/websocket.cpp 2 74.44%
src/realm/util/serializer.cpp 2 90.91%
src/realm/sync/transform.cpp 4 63.01%
test/fuzz_tester.hpp 5 58.95%
src/realm/bplustree.cpp 7 75.72%
src/realm/sync/noinst/server/server.cpp 14 76.62%
test/fuzz_group.cpp 45 51.75%
Totals Coverage Status
Change from base Build 2021: 0.005%
Covered Lines: 235332
Relevant Lines: 256176

💛 - Coveralls

@tgoyne tgoyne force-pushed the tg/jwt-validation branch from 2be351d to 241668b Compare February 8, 2024 20:46
@tgoyne tgoyne marked this pull request as ready for review February 9, 2024 03:45
try {
SyncUserMetadata data(m_user_schema, realm, user);
file_manager.remove_user_realms(data.identity(), data.realm_file_paths());
realm->begin_transaction();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we delete all users in a single transaction? The downside would be that in case of an exception (other than FileAccessError), no user is removed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am currently undecided on what level of batching is correct. This PR is matching the existing behavior and it's just much more obvious that we're performing write transactions in a loop due to the code movement.

@danieltabacaru
Copy link
Collaborator

If I understand these changes correctly, we got rid of AppMetadata table right?

@tgoyne
Copy link
Member Author

tgoyne commented Feb 12, 2024

The AppMetadata table won't be deleted from existing files (that requires a schema version bump and a migration), but it won't be created in newly initialized metadata realms.

@danieltabacaru
Copy link
Collaborator

The AppMetadata table won't be deleted from existing files (that requires a schema version bump and a migration), but it won't be created in newly initialized metadata realms.

Yes, that's what I meant.

@tgoyne tgoyne merged commit 6459d4b into master Feb 12, 2024
37 checks passed
@tgoyne tgoyne deleted the tg/jwt-validation branch February 12, 2024 18:33
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 21, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants