Skip to content

feat: dynamic array encode #2184

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

xj894976677
Copy link

Dynamic Array Encoding Fix in Web3j

Problem Statement

The original implementation did not correctly handle dynamic arrays according to Solidity's ABI encoding specification. In Solidity, when using abi.encode() with array parameters, the output includes dynamic offsets pointing to the actual array data.

Example in Solidity

// Solidity contract
function encodeExample(uint256[] memory tokenIds, uint256[] memory amounts) public pure returns (bytes memory) {
    return abi.encode(tokenIds, amounts);
}

When calling this function with:

tokenIds = [1, 2]
amounts = [100, 200, 300]

The encoded output structure should be:

[offset_1][offset_2][array1_length][array1_data...][array2_length][array2_data...]

Issue

The original Web3j implementation was concatenating array data directly without proper offset handling:

// Original problematic implementation
ByteArrayOutputStream concatenatedArrayEncodingBuffer = new ByteArrayOutputStream();
for (Object arrayItem : arrayItems) {
    // Directly writing array data without offsets
    concatenatedArrayEncodingBuffer.write(arrayItemEncoding, 0, arrayItemEncoding.length);
}

Solution

Modified the implementation to follow Solidity's ABI encoding specification:

// Fixed implementation
// 1. Calculate and write offsets in the header
encValues.add(BigInteger.valueOf(headSize + dynamicDataSize));

// 2. Store dynamic data separately
ByteArrayOutputStream dynamicBuffer = new ByteArrayOutputStream();
// Write array length
byte[] lengthBytes = Numeric.toBytesPadded(BigInteger.valueOf(arrayItems.size()), 32);
dynamicBuffer.write(lengthBytes, 0, lengthBytes.length);
// Write array elements
for (Object arrayItem : arrayItems) {
    byte[] itemBytes = Numeric.toBytesPadded(convertToBigInt(arrayItem), 32);
    dynamicBuffer.write(itemBytes, 0, itemBytes.length);
}

Impact

  • Ensures compatibility with Solidity's abi.encode() function
  • Fixes signature verification issues in contracts using EIP-712
  • Correctly handles multiple dynamic arrays in structured data

This change aligns Web3j's implementation with Ethereum's ABI specification for dynamic array encoding.

Signed-off-by: joseph.xiang <xjun0311@gmail.com>
Copy link

@subhramit subhramit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, I am not a maintainer (in fact, I just started exploring this project) and was having a hard time finding good first issues to solve myself, so thought of beginning to contribute by reviewing code.

I don't know if the PR is correct or not, as I don't have a lot of context - so pretty sure there will be many more changes when the maintainers take a look, but I left some comments below on code quality which would help you regardless of whether the PR goes in.

@@ -307,16 +306,21 @@ public byte[] encodeData(String primaryType, HashMap<String, Object> data)

List<String> encTypes = new ArrayList<>();
List<Object> encValues = new ArrayList<>();
List<byte[]> dynamicData = new ArrayList<>(); // Store dynamic data

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redundant comment

Comment on lines +315 to +316


Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the extra newlines?

Comment on lines -319 to +323
if (value == null) continue;
if (value == null) {
continue;
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unnecessary change?

Comment on lines +342 to +343
// Calculate header size (static part)
int headSize = 32;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are not calculating the header size, but hardcoding it - so the comment is misleading.
It's better to convert this into a constant like static final HEADER_SIZE just after the class declaration above the constructor at and use it wherever needed.
If you name the constant descriptively enough, the comment can also be let go of.

Comment on lines +344 to +345
// Track total size of dynamic data
int dynamicDataSize = 0;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redundant comment

Comment on lines +353 to +354
// Handle dynamic array
encTypes.add(baseTypeName); // Use base type instead of array type

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be AI-generated.
Nevertheless, if this is correct (I don't have much context), the comment here is useless as it talks about a change, which makes sense in the context of this PR, but won't once merged. Should be removed.
Think - how would the code look like once merged?

Comment on lines +358 to +359
// Prepare dynamic data
ByteArrayOutputStream dynamicBuffer = new ByteArrayOutputStream();

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redundant comment.

ByteArrayOutputStream dynamicBuffer = new ByteArrayOutputStream();
// Write array length
byte[] lengthBytes =
Numeric.toBytesPadded(BigInteger.valueOf(arrayItems.size()), 32);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extract magic number to a constant

// Write array elements
for (Object arrayItem : arrayItems) {
BigInteger itemValue = convertToBigInt(arrayItem);
byte[] itemBytes = Numeric.toBytesPadded(itemValue, 32);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here ^

Comment on lines +374 to +375
// Update total size of dynamic data
dynamicDataSize += dynamicBytes.length;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redundant comment - possibly because of AI

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants