Partnered With:

Summary of the latest meeting of Ethereum core developers: Devnet #12 launch and upgrade planning process.

Read this article in 16 Minutes

On November 30th, all core developers of Ethereum participated in the 123rd All Core Developers Consensus (ACDC) conference call.

Original Title: "Ethereum All Core Developers Consensus Call #123 Writeup"

Author: Christine Kim

Translated by: Luccy, BlockBeats

Editor's Note:

The Ethereum All Core Developers Consensus Call (ACDC) is held every two weeks to discuss and coordinate changes to the Ethereum consensus layer (CL). This is the 123rd ACDC call, which mainly covered the evaluation of the Cancun/Deneb and Devnet #12 test updates, as well as the solution to the validator exit issue in Devnet #11. In addition, developers also had in-depth discussions on network specifications clarification, upgrade plan process, and EIP status.

In the discussion of Cancun/Deneb, developers emphasized stability and discussed whether to launch the Goerli shadow fork. However, due to the Prysm client not being ready for testing yet, it was decided to wait until it is ready. The discussion on testing tools and chain reorganization emphasized the need for more comprehensive test coverage and suggested the use of Hive and Kurtosis test suites. On the issue of EIP status and CFI tags, Beiko raised concerns about whether these tags should be retained, and developers have not yet reached a clear consensus on this issue.

Christine Kim, Vice President of Research at Galaxy Digital, made a detailed record of the key points of this meeting, and BlockBeats translated the original article as follows:

On November 30th, 2023, Ethereum developers gathered on Zoom for the All Core Developers Consensus (ACDC) call #122 meeting. The ACDC conference call is a bi-weekly series of meetings hosted by Ethereum Foundation researcher Danny Ryan, where developers discuss and coordinate changes to the Ethereum consensus layer (CL). This week, developers focused on discussing the latest progress of the Cancun/Deneb test, including:

· Launch of Devnet #12: Currently testing Teku, Lodestar, and Lighthouse client software as well as all execution layer (EL) client software on Devnet #12. The Prysm team expects to have their software ready for testing on the latest Devnet within two to three weeks.

· Validator exit issue on Devnet #11: Developers have discovered an issue related to validator exits on Devnet #11, which is currently being resolved by the Nimbus client team. Devnet #11 will continue to operate normally until the issue is resolved.

· Network specification clarification: Developers have discussed the specification for "BlockByRoot" and "BlobSidecarsByRoot" requests to clarify whether consensus layer (CL) nodes should respond to these requests in a specific order.

In addition to the updates on Cancun/Deneb, developers also discussed some process issues proposed by Tim Beiko, the protocol support lead at the Ethereum Foundation, to improve coordination for Ethereum upgrades.

Devnet #12

On Wednesday, November 30, 2023, the Cancun/Deneb upgrade officially launched on Devnet #12. Mario Vega from the Ethereum Foundation testing team stated that "no major issues have been found" among the three CL clients currently running on the test network. Barnabas Busa, a DevOps engineer at the foundation, mentioned that a total of 225 validators are distributed across three nodes between Lighthouse, Teku, and Lodestar. Due to the stability of Devnet #12, Parithosh Jayanthi, another DevOps engineer at the foundation, asked developers whether they should start planning for the Goerli shadow fork to further test Cancun/Deneb. However, considering that the most popular CL client, Prysm, has not yet joined Devnet #12, developers have decided to temporarily shelve plans for the Goerli shadow fork until the Prysm client software is ready for testing. Beiko suggested starting the Goerli shadow fork at some point before the end of the year. Due to the stability of Devnet #12, plans are temporarily on hold until the Prysm client software is ready for testing.

Devnet #11

The developer of Nimbus, known as "Dustin" online, introduced the details of the issues his team is currently addressing in Devnet #11. These issues were first discovered when one-third of the validators for Devnet #11 dropped out and were intended to be used in Devnet #12. Ryan asked Dustin if additional testing could be done to discover these issues. Dustin responded that in his view, the nature of these errors was caused by implementation details outside the scope of the consensus specification. However, he also acknowledged that there was a "clear tension" between strictly adhering to the CL specification when writing client software for testing coverage and venturing into the gray area of the specification for better node performance.

"I mean, more testing is always good, but it's about finding a more systematic way to incorporate more client-side functionality into the tests, perhaps through more automation, whether using Hive or Kurtosis or some other means. If these issues can be resolved or things can be broken down well enough to incorporate more of these tasks, I think that would definitely be useful," Dustin added. Another challenge that CL developers should consider addressing is figuring out the level of detail at which the CL specification should dictate and standardize node behavior. "Another challenge here is the level of standardization, which is actually not just a software engineering problem. How standardized should behavior be?" Dustin asked. All CL clients have passed tests for basic behavior, but behavior outside the scope of these tests is ambiguous. "Are these intentionally left ambiguous? What should be clearly defined by the specification and what should intentionally remain ambiguous?" Dustin asked.

At least, developers agree to add more testing for the large number of validators exiting Cancun/Deneb on future Devnets and test networks. Ryan also acknowledges that there is room for stricter and more comprehensive test coverage when it comes to implementing the CL client and CL specification. Vega suggests that Dustin create a HackMD post detailing his concerns and discuss the topic at the next Cancun/Deneb test call. Jayanthi adds that based on some issues recently discovered on the Cancun/Deneb Devnets, developers clearly need tools to systematically perform a certain amount of on-chain integration-related behaviors, such as staking ETH withdrawals and validator exits. To this end, Jayanthi proposes using a mix of Hive and Kurtosis testing suites to build such tools.

Speaking of the new testing tool for the Cancun/Deneb upgrade, Jayanthi pointed out that developers are working on a reliable tool to activate chain reorganization on Devnets and test networks. To ensure the tool works properly, Jayanthi requested that developers share detailed information about known bugs that trigger chain reorganization on Ethereum. Jayanthi explained that he will use these bugs to test the tool and ensure it can reliably identify when reorganization occurs and when it is resolved.

Network Standards Clarification

Developers briefly discussed an open pull request regarding the response order that CL clients should return when receiving "BlockbyRoot" or "BlobSidecarsByRoot" requests, proposed by Justin Traglia, a researcher at the Ethereum Foundation. Ryan asked how different CL client teams have implemented this feature. No developer answered this question during the conference call. Ryan suggested reopening the discussion on the Ethereum research and development Discord channel to involve a wider range of client developers. Ryan acknowledged the ambiguity of the specifications for these two requests, which "may be the cause of problems and strange edge cases" and "worth clarifying and organizing," Ryan affirmed.

Ryan also mentioned that he will be releasing a new version of the CL specification in the coming days. The latest version significantly increases the specification for when CL clients can provide blocks and blocks via "byRoot" RPC requests. For more background on the discussion of retrieving missing blocks and blocks via "byRoot" RPC requests, please refer to the previous conversation record. Ryan emphasized that the new additions to the CL specification included in the latest version will not have any impact on validators running on Devnet #11 or #12, and there will be no destructive changes to the code.

Upgrade Planning Process

Next, developers discussed various process matters proposed by Beiko. Beiko stated that a blog post warning Goerli testnet users to activate the upgrade on Goerli within 3 months after Cancun/Deneb upgrade or within 1 month after activating the upgrade on the Ethereum mainnet (whichever comes later) will be published on the Ethereum Foundation blog on November 30th. Since the end of the conference call, the aforementioned blog post has been published and can be read here.

Beiko suggests creating a dedicated upgrade folder under the Ethereum "pm" repository to organize various files related to specific upgrades for future use. The developers on the conference call agreed with Beiko's suggestion.

Beiko's second proposal is about creating a "Meta EIP" document to track all the completed network upgrades on Ethereum. "Currently, there is no good place to track the full scope of network upgrades before deployment and announcement in blog posts," Beiko wrote in a post about his Meta EIP proposal. "For Dencun, we have EL EIPs in a hard-to-find markdown file 7, while CL EIPs are part of Beacon Chain specification 3. This is not ideal because both are a bit hard to find, they use different 'formats,' and can lead to duplication. Since ERC and EIP are now separate, I suggest (going back to) using Meta EIPs to track EIPs included in network upgrades." Developers support the creation of these documents. Beiko said he will draft a document for Cancun/Deneb upgrade review starting this week.

Finally, Beiko raised a practical issue regarding the usefulness of marking proposed code changes as "Considered for Inclusion" (CFI) in Ethereum Improvement Proposals (EIPs). According to Beiko, CFI is a historical "soft signal" used by developers to indicate that the author of an EIP should continue working on proposals that may be included in future hard forks. It is only used for EL-focused code changes and upgrades. "[CFI] is higher than random thoughts from random people, but before being accepted in a fork," Beiko said.

In the past, the CFI tag had little effect on indicating which EIPs on EL would be implemented or when they would be implemented in upgrades. It has been applied to various EIPs with different levels of code readiness and client team support. In the case of the EVM Object Format (EOF) proposal, developers used this tag to indicate their commitment to implementing EOF in the upcoming Cancun/Deneb upgrade. However, EOF and several other proposals marked as CFI were rejected from Cancun/Deneb, and it is currently unclear what the status of these CFI-tagged EIPs will be in the next Prague/Electra upgrade.

Beiko stated that if this state is not helpful, developers should remove it, but if they intend to keep it, CL developers should also adopt the same tag for code changes to be implemented in CL upgrades. It is currently unclear to what extent the CL EIP review process reflects the EL EIP review process and whether they should develop in the same way in the future. Typically, proposed code changes to the CL specification are discussed in ACDC conference calls, while proposed EIPs to the EL specification are discussed in ACDE conference calls.

Swirlds Labs' Danno Ferrin also proposed an idea to create a placeholder field for all EIPs, whether CL or EL related, to identify their status during the review process, including CFI status. There was no clear decision on the topic of EIP status and CFI tags during this conference call.