Bisection search causes inconsistency in DCV values

[trizin]

Background / Motivation

We're using bisection search to find start and end block numbers for the DF week in the networks other than ETH mainnet. As these numbers are not 100% consistent between different runs we might get different results.

There are two possible paths:

• (a) Specify a tolerance level, that's say 1/00 of a cent worth of OCEAN. Eg 0.0001 OCEAN
• (b) Specify a tolerance level that's within Solidity's or Python's numerical precision

Roughly, I'm ok with either. And, (b) is less risky so I recommend it, unless it's too slow.

We should be able to readily get these numbers consistent within a target tolerance level

TODOs

My suggestion to proceed: (highest level: go for b first, and if needed go for a)

1. Write unit tests that expose the inconsistency
2. Update the unit tests to allow a target tolerance
3. Update bisection code to hit target tolerance
4. If needed, update other code to allow the target tolerance too

[idiom-bytes]

I think this is fixed due to the new implementation to get blocks.
@trizin can we close this issue?

[trizin]

No, this is a different one. The new implementation is only for mainnet, we still have this issue on other networks when getting the volumes.

[trizin]

#793

This PR verifies that the identified block number is within a 15-minute range of the target timestamp. Throws an error otherwise.

[trentmc]

#793

This PR verifies that the identified block number is within a 15-minute range of the target timestamp. Throws an error otherwise.

@trizin do you see an opportunity to "Update bisection code to hit target tolerance"? Eg by simply tightening the target tolerance, without making runtimes much longer?

[trizin]

@trentmc xtol is set to 0.4, which is already very low for this case. Last round the identified block numbers for Sapphire were exactly correct.

I think our timestamp to block algorithm is very efficient, doesn't take long to find the block number and finds the right block numbers too.

If something goes terribly wrong with the RPC or inputs, the 15 min check shall protect datafarming.

---

Getting back to the issue, this problem is particularly noticeable on networks like Polygon, where the block time is relatively short. In such cases, the bisection search might yield slightly varying results. For instance, it could return block 1001 during one execution and block 1002 in another, especially if these two blocks are almost equally close to the target timestamp.

This variability can impact the reward distribution, especially if a substantial consume occurred in block 1001. It also poses a challenge for accurate historical data generation.

Imo, the most reliable solution to mitigate these is maintaining a record of the start and end blocks for a given week (week x). For the subsequent week (week x+1), the starting block should be set as the ending block of the previous week (week x).

Or we could try:

• xtol: Setting xtol to a much smaller value, like 0.01, would help with these inconsistencies.
• Rounding of block numbers: Currently, the code returns 'int(block_i)' which effectively rounds down the block number. In the context of the bisection search, which may return a decimal value, using the 'round()' function to round to the nearest integer would provide more consistent results across different runs.

[trentmc]

Imo, the most reliable solution to mitigate these is maintaining a record of the start and end blocks for a given week (week x). For the subsequent week (week x+1), the starting block should be set as the ending block of the previous week (week x).

Where are the candidate places that we would store the record of start & end blocks for a given week? What do you recommend. If this is reliable and repeatable, that sounds like a good choice. (Ocean Data NFT with ERC721?)

[trizin]

Ocean Data NFT with ERC721

Sounds brilliant! I'd propose deploying on Polygon Mainnet, it'd cost us 1 tx/week (probably around 0.01-0.03 MATIC)

[trentmc]

Ocean Data NFT with ERC721

Sounds brilliant! I'd propose deploying on Polygon Mainnet, it'd cost us 1 tx/week (probably around 0.01-0.03 MATIC)

Works for me.

Here's the README in ocean.py for set/get on ERC721. But you don't need an ocean.py dependency, instead, just use its implementation as input to doing it directly.