New Open-Source Tool for Testing Client API Conformance: eth2-comply

Ethereum developers today benefit from the ubiquity and uniformity of the Ethereum JSON-RPC API and the flexibility it provides by letting them swap Ethereum client implementations without breaking code. At Infura we see this API interoperability as a valuable public good. To support Ethereum 2.0 client teams working towards compatibility with the Ethereum 2.0 API specification, Infura is pleased to be open-sourcing an Apache 2.0-licensed tool for testing client API conformance: eth2-comply. Client teams can integrate it into their CI pipelines starting today.

eth2-comply is a simple CLI tool which can run against any HTTP appliance:

eth2-comply --target http://localhost:5051

eth2-comply requires only that its target implements the /v1/node/health and /v1/node/syncing endpoints from the Ethereum 2.0 API spec. With those endpoints available, developers may integrate eth2-comply into their CI pipelines while continuing to achieve API conformance incrementally using the subset and failSilent flags. subset enables specifying subsets of the full test suite to run; for example, only the tests for routes you have implemented. failSilent configures eth2-comply to return a 0 code even if tests are failing.

An important goal in developing eth2-comply was to make it as easy as possible for anybody to contribute test cases to eth2-comply's standard test suite. Test cases are specified in JSON, using a simple syntax. Here is an example of a minimal test, which will pass for any response that satisfies the Ethereum 2.0 API specification for the /v1/beacon/genesis operation. Here is a more complex test case, with a request body and an expected response status code; responses for this test must satisfy both the API spec, and the test's explicit response expectations. Expected response bodies may be specified as well.

We are eager to implement features that make it easier to integrate eth2-comply into your development workflow or CI pipeline, and which help test contributors to write more expressive scenarios. Please do not hesitate to open issues communicating the features you need, the bugs you discover, and the questions you have.

In the meantime, our Eth2 API is live and ready for testing! If you want the chance to help shape our Eth2 testnet API, apply now and register your interest. Documentation will be sent out directly to users who are accepted into the beta.