Skip to content

feat(benchmark): add clz opcode test case #1845

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

LouisTsai-Csie
Copy link
Collaborator

@LouisTsai-Csie LouisTsai-Csie commented Jul 2, 2025

πŸ—’οΈ Description

Add benchmark test for CLZ opcode.

πŸ”— Related Issues or PRs

Issue #1795

βœ… Checklist

  • All: Ran fast tox checks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:
    uvx --with=tox-uv tox -e lint,typecheck,spellcheck,markdownlint
  • All: PR title adheres to the repo standard - it will be used as the squash commit message and should start type(scope):.
  • All: Considered adding an entry to CHANGELOG.md.
  • All: Considered updating the online docs in the ./docs/ directory.
  • All: Set appropriate labels for the changes (only maintainers can apply labels).
  • Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.
  • Tests: For PRs implementing a missed test case, update the post-mortem document to add an entry the list.
  • Ported Tests: All converted JSON/YML tests from ethereum/tests or tests/static have been assigned @ported_from marker.

sender=pre.fund_eoa(),
)

if (tx_gas_limit is None) or (tx_gas_limit > attack_gas_limit):
Copy link
Collaborator Author

@LouisTsai-Csie LouisTsai-Csie Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This if-statement is for EIP-7852 for transaction limit cap, more details could be found in this issue.

We should have a wrapper for all the benchmark test cases, but i will leave it in a separate PR.

code_seq = Bytecode()

for i in range(2 * iteration):
value = i if i % 2 else 2**256 - 1 - i
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the second test case, I aim to add some randomness. Using a simple range from 1 to iteration results in many values being too similar.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get the idea but I would set this up differently. We have code_prefix and code_suffix so we know that we can add at most max_code_size - len(code_prefix) - len(code_suffix) bytes. I'd keep incrementing some number here and then let CLZ work on 2^256 - 1 >> i % 256. This i increments each round and will thus right shift the values such that CLZ yields a different value. Note that for each CLZ you add you should check Op.CLZ(value) + Op.POP if this length does not exceed the max size (else break). This length is not constant because it will depend on how many bytes we are pushing (to push 2^256-1 need PUSH32, and for 1 need PUSH1)

@LouisTsai-Csie LouisTsai-Csie marked this pull request as ready for review July 3, 2025 12:26
@LouisTsai-Csie LouisTsai-Csie self-assigned this Jul 3, 2025
@LouisTsai-Csie LouisTsai-Csie marked this pull request as draft July 4, 2025 14:45
@LouisTsai-Csie LouisTsai-Csie marked this pull request as ready for review July 4, 2025 14:45
Copy link
Member

@jochem-brouwer jochem-brouwer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments πŸ˜„ πŸ‘

Need to also think on how to use these benchmarks, i.e. how to test these against clients and how to interpret the results. The end goal could be to determine the gas price of the opcode. For that we also need a baseline in order to interpret the results and thus also to price the opcode. This is non-trivial because these benchmarks use other opcodes as well, and we thus have to find a way to measure the costs for CLZ only (and not the cost of the other opcodes). Food for thought!

Left a comment on one of the tests. The 248 trick is beautiful πŸ˜„ πŸ‘

value = i if i % 2 else 2**256 - 1 - i
code_seq += Op.CLZ(value) + Op.POP

code_address = pre.deploy_contract(code=code_prefix + code_seq + code_suffix)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will deploy a contract too large. Could you add the sanity check?

Suggested change
code_address = pre.deploy_contract(code=code_prefix + code_seq + code_suffix)
code_address = pre.deploy_contract(code=code_prefix + code_seq + code_suffix)
if len(attack_code) > max_contract_size:
raise ValueError(
f"Code size {len(attack_code)} exceeds maximum code size {max_contract_size}"
)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might go with the following approach for consistency now, since the remaining benchmarks use it as well.

However, we could add this refactoring for all the assertions later in this issue

 assert len(attack_code) <= max_code_size

code_seq = Bytecode()

for i in range(2 * iteration):
value = i if i % 2 else 2**256 - 1 - i
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get the idea but I would set this up differently. We have code_prefix and code_suffix so we know that we can add at most max_code_size - len(code_prefix) - len(code_suffix) bytes. I'd keep incrementing some number here and then let CLZ work on 2^256 - 1 >> i % 256. This i increments each round and will thus right shift the values such that CLZ yields a different value. Note that for each CLZ you add you should check Op.CLZ(value) + Op.POP if this length does not exceed the max size (else break). This length is not constant because it will depend on how many bytes we are pushing (to push 2^256-1 need PUSH32, and for 1 need PUSH1)

@LouisTsai-Csie
Copy link
Collaborator Author

@jochem-brouwer Thanks for your review, I've updated accordingly! Please let me know if the change is good to you.

Copy link
Member

@marioevz marioevz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing implementation, thanks!

@marioevz
Copy link
Member

I think this needs a careful rebase since I tried rebasing locally and the merge messed up the test, @LouisTsai-Csie could you try rebasing to latest main and see that it works? thanks!

@LouisTsai-Csie
Copy link
Collaborator Author

@marioevz I did not see any conflict from my side, but I still rebase to the latest changes. Let's see if this works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants