Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SHARD-44: GlobalTx fixes #103

Merged
merged 8 commits into from
Jan 8, 2025
Merged

SHARD-44: GlobalTx fixes #103

merged 8 commits into from
Jan 8, 2025

Conversation

S0naliThakur
Copy link
Member

@S0naliThakur S0naliThakur commented Nov 21, 2024

Linear: https://linear.app/shm/issue/SHARD-44/globaltx-validation

Summary: Implement global receipt validation and enhance receipt verification logic

  1. Put archiver robust query behind useRobustQueryForReceipt flag
  2. Integrated AJV schema validation for global receipts across multiple modules.
  3. Improved error handling and logging for receipt validation processes.
  4. Updated AJV schema enumeration to include GlobalTxReceipt.
  5. Modified storeReceipt method to use verifyArchiverReceipt directly.
  6. Added functions for verifying global receipts, including verifyGlobalTxreceiptOffline and verifyReceiptMajority for better code readability.
  7. Refactored existing receipt verification methods to support both global and non-global receipts.

@@ -73,6 +73,11 @@
executionGroupNodes: ConsensusNodeInfo[],
minConfirmations: number = config.RECEIPT_CONFIRMATIONS
): Promise<{ success: boolean; newReceipt?: Receipt.ArchiverReceipt }> => {
// If robustQuery is disabled, do offline verification
if (!config.useRobustQueryForReceipt) {

Check failure

Code scanning / CodeQL

User-controlled bypass of security check High

This condition guards a sensitive
action
, but a
user-provided value
controls it.
@S0naliThakur S0naliThakur force-pushed the globalTx-fixes branch 2 times, most recently from 793f574 to c131ed4 Compare November 26, 2024 16:39
@S0naliThakur S0naliThakur marked this pull request as ready for review November 26, 2024 16:41
src/Data/Collector.ts Outdated Show resolved Hide resolved
src/Data/Collector.ts Outdated Show resolved Hide resolved
src/Data/Collector.ts Show resolved Hide resolved
src/Data/Collector.ts Outdated Show resolved Hide resolved
@S0naliThakur S0naliThakur force-pushed the globalTx-fixes branch 2 times, most recently from f29133e to cf6a9e3 Compare December 12, 2024 14:07
src/Data/Collector.ts Dismissed Show dismissed Hide dismissed
if (nestedCountersInstance)
nestedCountersInstance.countEvent(
'receipt',
'Invalid_receipt_globalModification_valid_signs_count_less_than_votingGroupCount'

Check failure

Code scanning / CodeQL

User-controlled bypass of security check High

This condition guards a sensitive
action
, but a
user-provided value
controls it.
aniketdivekar
aniketdivekar previously approved these changes Dec 13, 2024
@S0naliThakur S0naliThakur changed the base branch from itn4 to dev December 30, 2024 07:55
@S0naliThakur S0naliThakur dismissed stale reviews from aniketdivekar and devendra-shardeum December 30, 2024 07:55

The base branch was changed.

@jintukumardas
Copy link
Contributor

@S0naliThakur Since we have disabled the workers. We can also comment out the code where we are initializing and setting up the worker process. Please refer to these lines -

  1. https://github.com/shardeum/archive-server/blob/780ed888bb31bba232e5c79ac02ae8db50bf9a6d/src/server.ts#L96
  2. https://github.com/shardeum/archive-server/blob/780ed888bb31bba232e5c79ac02ae8db50bf9a6d/src/server.ts#L500

CC: @aniketdivekar

Copy link
Contributor

@jintukumardas jintukumardas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please comment out the initWorkerProcess and setupWorkerProcesses methods.

aniketdivekar
aniketdivekar previously approved these changes Jan 7, 2025
jintukumardas
jintukumardas previously approved these changes Jan 7, 2025
aniketdivekar
aniketdivekar previously approved these changes Jan 8, 2025

const nodeMap = new Map<string, P2PTypes.NodeListTypes.Node>()
// Fill the map with nodes keyed by their public keys
cycleShardData.nodes.forEach((node) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be ok to just put the nodeMap on the cycleShardData the first time we compute it (lazy init)
that does not need to change to merge this. I suspect we will do a more general perf pass later as time permits.

}
// Check if the node is in the execution group
if (!cycleShardData.parititionShardDataMap.get(homePartition).coveredBy[node.id]) {
Logger.mainLogger.error(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think in the future we should generally be putting log lines behind if checks like in the validator. This is because even if you don't log it you still pay the string format cost.
(this is not a blocker though)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do have some logger pattern upgrades to share later anyhow.

for (const signature of normalReceipt.signaturePack) {
if (!signature || !signature.owner) continue

const node = executionGroupNodes.find((n) => n.publicKey.toLowerCase() === signature.owner.toLowerCase())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when we later do a perf pass a map would avoid O(n^2)

afostr
afostr previously approved these changes Jan 8, 2025
Copy link
Contributor

@afostr afostr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made a few small comments but they are not show stoppers to get into dev. I think the testing and roll out plans will be critical for this

aniketdivekar
aniketdivekar previously approved these changes Jan 8, 2025
Copy link
Contributor

@devendra-shardeum devendra-shardeum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good now

@urnotsam urnotsam merged commit b06b9fa into dev Jan 8, 2025
5 of 7 checks passed
@aniketdivekar aniketdivekar deleted the globalTx-fixes branch January 8, 2025 16:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants