-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Fix block cache deserialization for old ethereum blocks #6330
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+234
−78
Merged
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
1c5d89e
chain/ethereum: add json_patch module for unified type field patching
incrypto32 f2dd5f9
chain/ethereum: refactor PatchingHttp to use json_patch module
incrypto32 c52a748
chain/ethereum: patch missing type field in cached blocks
incrypto32 7fa4a54
chain/ethereum: add EthereumJsonBlock newtype for cached block handling
incrypto32 5f52206
chain/ethereum: use concrete types in EthereumJsonBlock methods
incrypto32 b1a9cc6
chain/ethereum: add doc comments to EthereumJsonBlock methods
incrypto32 9d5203b
chain/ethereum: remove unused From<Value> impl for EthereumJsonBlock
incrypto32 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| use graph::prelude::serde_json::{self as json, Value}; | ||
| use graph::prelude::{EthereumBlock, LightEthereumBlock}; | ||
|
|
||
| use crate::json_patch; | ||
|
|
||
| #[derive(Debug)] | ||
| pub struct EthereumJsonBlock(Value); | ||
|
|
||
| impl EthereumJsonBlock { | ||
| pub fn new(value: Value) -> Self { | ||
| Self(value) | ||
| } | ||
|
|
||
| /// Returns true if this is a shallow/header-only block (no full block data). | ||
| pub fn is_shallow(&self) -> bool { | ||
| self.0.get("data") == Some(&Value::Null) | ||
| } | ||
|
|
||
| /// Returns true if this block is in the legacy format (direct block JSON | ||
| /// rather than wrapped in a `block` field). | ||
| pub fn is_legacy_format(&self) -> bool { | ||
| self.0.get("block").is_none() | ||
| } | ||
|
|
||
| /// Patches missing `type` fields in transactions and receipts. | ||
| /// Required for alloy compatibility with cached blocks from older graph-node versions. | ||
| pub fn patch(&mut self) { | ||
incrypto32 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| if let Some(block) = self.0.get_mut("block") { | ||
| json_patch::patch_block_transactions(block); | ||
| } | ||
| if let Some(receipts) = self.0.get_mut("transaction_receipts") { | ||
| json_patch::patch_receipts(receipts); | ||
| } | ||
| } | ||
|
|
||
| /// Patches and deserializes into a full `EthereumBlock` with receipts. | ||
| pub fn into_full_block(mut self) -> Result<EthereumBlock, json::Error> { | ||
| self.patch(); | ||
| json::from_value(self.0) | ||
| } | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. When I suggested the type parameter |
||
|
|
||
| /// Extracts and patches the inner block, deserializing into a `LightEthereumBlock`. | ||
| pub fn into_light_block(mut self) -> Result<LightEthereumBlock, json::Error> { | ||
| let mut inner = self | ||
| .0 | ||
| .as_object_mut() | ||
| .and_then(|obj| obj.remove("block")) | ||
| .unwrap_or(self.0); | ||
| json_patch::patch_block_transactions(&mut inner); | ||
| json::from_value(inner) | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,123 @@ | ||
| //! JSON patching utilities for Ethereum blocks and receipts. | ||
| //! | ||
| //! Some cached blocks are missing the transaction `type` field because | ||
| //! graph-node's rust-web3 fork didn't capture it. Alloy requires this field for | ||
| //! deserialization. These utilities patch the JSON to add `type: "0x0"` (legacy | ||
| //! transaction) where missing. | ||
| //! | ||
| //! Also used by `PatchingHttp` for chains that don't support EIP-2718 typed transactions. | ||
|
|
||
| use graph::prelude::serde_json::Value; | ||
|
|
||
| pub(crate) fn patch_type_field(obj: &mut Value) -> bool { | ||
| if let Value::Object(map) = obj { | ||
| if !map.contains_key("type") { | ||
| map.insert("type".to_string(), Value::String("0x0".to_string())); | ||
| return true; | ||
| } | ||
| } | ||
| false | ||
| } | ||
|
|
||
| pub(crate) fn patch_block_transactions(block: &mut Value) -> bool { | ||
| let Some(txs) = block.get_mut("transactions").and_then(|t| t.as_array_mut()) else { | ||
| return false; | ||
| }; | ||
| let mut patched = false; | ||
| for tx in txs { | ||
| patched |= patch_type_field(tx); | ||
| } | ||
| patched | ||
| } | ||
|
|
||
| pub(crate) fn patch_receipts(result: &mut Value) -> bool { | ||
| match result { | ||
| Value::Object(_) => patch_type_field(result), | ||
| Value::Array(arr) => { | ||
| let mut patched = false; | ||
| for r in arr { | ||
| patched |= patch_type_field(r); | ||
| } | ||
| patched | ||
| } | ||
| _ => false, | ||
| } | ||
| } | ||
|
|
||
| #[cfg(test)] | ||
| mod tests { | ||
| use super::*; | ||
| use graph::prelude::serde_json::json; | ||
|
|
||
| #[test] | ||
| fn patch_type_field_adds_missing_type() { | ||
| let mut obj = json!({"status": "0x1", "gasUsed": "0x5208"}); | ||
| assert!(patch_type_field(&mut obj)); | ||
| assert_eq!(obj["type"], "0x0"); | ||
| } | ||
|
|
||
| #[test] | ||
| fn patch_type_field_preserves_existing_type() { | ||
| let mut obj = json!({"status": "0x1", "type": "0x2"}); | ||
| assert!(!patch_type_field(&mut obj)); | ||
| assert_eq!(obj["type"], "0x2"); | ||
| } | ||
|
|
||
| #[test] | ||
| fn patch_type_field_handles_non_object() { | ||
| let mut val = json!("not an object"); | ||
| assert!(!patch_type_field(&mut val)); | ||
| } | ||
|
|
||
| #[test] | ||
| fn patch_block_transactions_patches_all() { | ||
| let mut block = json!({ | ||
| "hash": "0x123", | ||
| "transactions": [ | ||
| {"hash": "0xabc", "nonce": "0x1"}, | ||
| {"hash": "0xdef", "nonce": "0x2", "type": "0x2"}, | ||
| {"hash": "0xghi", "nonce": "0x3"} | ||
| ] | ||
| }); | ||
| assert!(patch_block_transactions(&mut block)); | ||
| assert_eq!(block["transactions"][0]["type"], "0x0"); | ||
| assert_eq!(block["transactions"][1]["type"], "0x2"); | ||
| assert_eq!(block["transactions"][2]["type"], "0x0"); | ||
| } | ||
|
|
||
| #[test] | ||
| fn patch_block_transactions_handles_empty() { | ||
| let mut block = json!({"hash": "0x123", "transactions": []}); | ||
| assert!(!patch_block_transactions(&mut block)); | ||
| } | ||
|
|
||
| #[test] | ||
| fn patch_block_transactions_handles_missing_field() { | ||
| let mut block = json!({"hash": "0x123"}); | ||
| assert!(!patch_block_transactions(&mut block)); | ||
| } | ||
|
|
||
| #[test] | ||
| fn patch_receipts_single() { | ||
| let mut receipt = json!({"status": "0x1"}); | ||
| assert!(patch_receipts(&mut receipt)); | ||
| assert_eq!(receipt["type"], "0x0"); | ||
| } | ||
|
|
||
| #[test] | ||
| fn patch_receipts_array() { | ||
| let mut receipts = json!([ | ||
| {"status": "0x1"}, | ||
| {"status": "0x1", "type": "0x2"} | ||
| ]); | ||
| assert!(patch_receipts(&mut receipts)); | ||
| assert_eq!(receipts[0]["type"], "0x0"); | ||
| assert_eq!(receipts[1]["type"], "0x2"); | ||
| } | ||
|
|
||
| #[test] | ||
| fn patch_receipts_handles_null() { | ||
| let mut val = Value::Null; | ||
| assert!(!patch_receipts(&mut val)); | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let me know if we should call it just
JsonBlocknamed it this because we already have aJsonBlockin chainstore and this one is very ethereum specificThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am fine with either, but since we are in the ethereum crate, the
Ethereumprefix isn't strictly necessary. But not a big deal either way.