Progressive decentralization part 2: Evolving DIN through federation

In this blog, we discuss DIN's shift from centralized to federated, championing Web3's vision of digital sovereignty through decentralization.

If you have been following Infura’s journey of creating the decentralized infrastructure network, which we’ve termed DIN, you are aware that in order to progressively decentralize the RPC layer, we are moving from a centralized architecture to a federated one. You can learn more about this work here.

But first, let’s take a step back to revisit some core concepts of web3, the challenges and opportunities each presents and where the DIN fits in as well as the progress the team has made to date. 

Web3 is tricky to define. Is it a blockchain? Cryptocurrency? More privacy and freedom? The answer to these questions is “yes and…”. Web3 is about enabling digital sovereignty and developing new ways to connect and transact with each other using blockchain technology that removes the need for intermediaries. And this is what it looks like at a high level: 

Progressive decentralization is a journey

Decentralization can be thought of as a spectrum along 3 axes: technical, economic, and legal. By legal, what we really mean is governance. On the left, you can see examples of how centralization materializes over these axes. As we move right toward decentralized mechanisms, visibility, participation, and inclusion become increasingly enabled. Governance moves toward DAOs and decentralized roadmaps. Funding and economic incentives move more toward fostering community contributions. The technical side moves along a path that increasingly incentivises collaboration.

Different projects start out in different places along this spectrum and typically have a roadmap of how they want to progressively decentralize. We believe that because we are developing new models, new economics, and new ways of organizing, it can be risky to start at the far right of the spectrum.

Centralization can help stabilize projects that are early in their lifecycle to maintain high benchmarks of quality, ensure consistency of funding, and provide stronger governance. A good compromise to start new projects is through Federation. We start closer to the centralized side of the spectrum, but with core decisions to put the project on a decentralized path. While key elements of decentralization may be absent, there is a clear path on how to progressively move toward the right side. And this is precisely how the DIN is evolving.

The current state of the RPC layer

Now let’s narrow our focus to JSONRPC providers. While there are some providers that take a more decentralized approach, the vast majority of RPC providers follow Infura’s model of RPC as a service. It’s a useful and convenient service but has its challenges as a business. We think of RPC as a commodity business because the RPC is a standardized interface across providers, it starts to become an increasingly commoditized service as more providers enter the market. It has very low switching costs making providers interchangeable. The reality of a commoditized business is great for the customer as providers must drop prices to razor-thin margins in order to compete with each other.

In commodity businesses, the only way to succeed is to consolidate the market. A greater share of the market allows for economies of scale to continue driving down the price to win and maintain more market share. It’s a vicious cycle of centralization.

Looking at the provider space with the lens of the web3 ethos, it suffers from all of the common centralized issues: they tend to be single points of failure and are susceptible to availability issues, whether that’s due to technical issues, regulatory issues, or specific provider limitations. 

There is also an adversarial relationship between all of us providers. We are all doing very similar things to provide service, but there is a lot of wasted effort recreating our respective stacks. Infrastructure providers limit themselves by avoiding collaborative efforts to deliver a key service to the ecosystem.

So, given all of that talk on web3 ethos and decentralization, where should the infrastructure provider community go?

DIN: The path forward for a decentralized RPC layer 

For the past 7 years, Infura has been providing RPC as a service, but through the traditional mechanism, we are all familiar with - giving tools to developers, which is Saas.

Over the last few years, we really felt like it was a challenge for us to keep up with the amount of networks that were being added and at the same time the way that developers were interacting with us was changing. In the beginning, people were running their own nodes and maybe they were splitting traffic at some point and doing a combination of using Infura and running their own infrastructure. Later down the line,  they might have just decided that was not worth the pain and effort to run their own infrastructure and they were solely relying on us.

Over time, a whole ecosystem or industry of RPC providers popped up where there were many other providers serving the same networks that Infura was providing and people started to go the route of  - instead of running their own infrastructure they would use multiple providers.

Our goal has always been and will continue to be ensuring that any developer can gain access to production-ready and reliable Web3 infrastructure. Achieving 100% uptime requires a wider network of node providers to handle requests in the kind of extreme volumes we saw in past congestion events.

So to go back to the initial question - where should the infrastructure provider community go? Towards decentralization. A decentralized infrastructure network could help prevent things like outages while also providing shared uptime, wider access to new networks and a more reliable experience overall. It could also mean zero downtime.

Our approach to decentralized RPC is creating a system or a network that allows providers to generate revenue and customers can get an endpoint that is similar to their current experience - here is one endpoint that you can go to that is always on, connecting you to many different blockchains while behind that the traffic gets served by a network of providers as opposed to a single source.

Network roles and functions on DIN 

Right now we are still in the early stages of bringing this vision to life, but we have a clear path set ahead of us. The primary roles that we have envisioned for the federated state:

Ingress Operator - one or more agents who accept payment from a consumer and take responsibility for optimally routing requests to the network.

Node Provider or Operator - a reliable and robust service provider who can serve high-throughput requests at an agreed-upon quality of service.

The Watcher - reports degraded service with evidence and periodically tests node operators' performance anonymously. The goal of the watcher is to maintain the quality of service standards consistent across the network.

Let’s talk about what the current node operator journey looks like in the federated state. Bear in mind that as we are growing and progressing in this journey, this might change in order to accommodate any new requirements. 

Just to recap, when we say node operator, we refer to a participant in the federated network that is a service provider that can serve high throughput requests. They are in this for profit and they are amicably competing with the other DIN operators on quality of service.

As a prospective DIN partner, the first step you will go through is an onboarding session with some of the DIN team members where we will dive deeper into what DIN is, and what your responsibilities and requirements to participate in the network would be.

Next, we need to benchmark your capabilities in order to understand what kind of loads you can handle. The DIN team has put together a set of acceptance criteria for prospective partners which we will talk in more detail about further down in this blog. In a nutshell, there will be a set of tests run on your endpoints in order to determine what your current capabilities are. This helps us understand what are your strengths and weaknesses and where you would best fit within the federation.

Once benchmarking is done and the service agreements between the node operator and the federation are in place, you can start receiving failover traffic from other partners. If all goes well, the next step will be to optimise traffic from other partners. What we define as optimized traffic is the following: instead of having to scale up your own infrastructure when surges in traffic happen, you can just route the surplus traffic to one of the DIN partners, ideally maintaining your resources at a baseline level constantly. This can have so many benefits: easier to predict running costs, increased quality of service, minimized risk of production incidents.

Network acceptance criteria for infrastructure operators in a federated state 

Now, let’s talk about acceptance criteria. You might have noticed there that this is the second iteration of the acceptance criteria set. That is because, 2 weeks ago, we had the second in-person DIN partner meetup in New York. Let me just say that being in the same room with so many different and talented people all aligned on a vision has been truly inspiring and energizing. 

One of the topics we discussed was ways in which we can improve the initial acceptance criteria set we had in order to better understand a partner’s capabilities and to get a more robust understanding of their strengths and weaknesses.

As a result, we have agreed to benchmark prospect partners on 6 different criteria.

Availability of methods - in this federated state, we want to ensure that you can support the same methods that the majority of the partners can support. This is so that you can receive the failover or optimized traffic that I was talking about previously.

Latency - we will do a stress test approach where we will send different loads incrementally to determine where your system stops performing optimally. Of course, the rule of thumb here is that lower is better.

Consistency - there are different levels of testing consistency but for this initial federated state, we just want to verify that your nodes are consistent between themselves.

Max Gas Cap - we want to make sure you can support eth_calls at a certain threshold over the EVM block gas limit.

Sensible error response - we want to ensure consistency in error handling across partners for a seamless user experience.

Correctness - we are checking block number monotonicity. Again there are different levels of correctness, but for this stage, we just want to make sure that the new block number >= previous block number queries.

What’s next for the DIN team?

In conclusion, whether you're a seasoned Infura user or simply following our journey, the future holds exciting prospects. For our valued customers, a seamless experience remains our top priority, with enhanced reliability, improved uptime, and consistent performance on the horizon.

We're committed to demonstrating how this network will bring immense value to both customers and providers. The Federation will persistently welcome new partners, refining our acceptance criteria, and fortifying failover mechanisms, while advancing optimized traffic and inter-partner routing. As we continue to evolve, make sure to stay tuned for the innovative developments that lie ahead in the world of DIN.

If you're eager to delve deeper into the topics discussed in this blog, we invite you to check out our Smartcon 2023 talk. It provides an in-depth exploration of these subjects. You can view it here.