A Brief History of Infura

With the new year kicking off, we wanted to bring you on a journey of how we started Infura and some of the problems we’ve solved since our inception.

Infura has always been focused on improving the developer experience and making it easier for developers to connect to Web3. In this dynamic space, we continuously improve and innovate, adapting to the evolving problems blockchain developers face. What sparked the thought process that created what we know as Infura today?

A Simple Solution for Web3 Builders

When we launched Infura in 2016 at DevCon 2, our goal was to simplify the developer experience and make an impact on the growth of the ecosystem. We wanted to enable developers to bootstrap their Web3 applications and to experiment, without having to take on the burden of running their own infrastructure. Our hope was that more people would build innovative projects on Web3 and drive the rapid growth of the Ethereum ecosystem.

Getting your smart contracts onto Ethereum may seem daunting, especially if you are new to Web3. To build an application on Ethereum, you will need a connection to the Ethereum network. This necessitates operating a blockchain client (node software), which can be costly. It also requires a long initialization period to download the latest data from other peers and may require full-time personnel just to manage the software deployment.

This is why Infura created a zero-maintenance, easy-to-use API for Web3. And the best part? Developers get to choose whether they wish to use just Infura’s API to connect to the network or a hybrid infrastructure model with Infura as their primary provider, alongside an in-house node.

To make things even sweeter, we offered Infura completely free to use, up until around a year ago, when we finally implemented paid pricing for users that require 100,000 API requests per day. Even now, we have a forever-free Core plan. Why do we do all this, you may well ask. Because we genuinely want developers to have the freedom to choose, as well as the freedom to experiment with Web3, without having to first drop a large investment in time and money.

Solving the Problem of Monolithic Ethereum

The Ethereum client has historically been monolithic in its architecture. It has several subsystems that do specific things and are limited in how they can scale in the interest of minimizing operating requirements for users. These subsystems include:

• Query Interface (JSON-RPC, WebSocket, IPC)
• P2P Connection
• Ethereum Virtual Machine (EVM)
• Chaindata Storage
• Transaction System
  

In the beginning, the above limitation in an Ethereum client’s architecture was not a problem. Then, along came the huge explosion of popular dApps such as CryptoKitties and massive token launches, both of which made running a single node insufficient and unsustainable in the long term.

So, what happens when you reach the limit of what a single node can do for you?

Due to the monolithic nature of the client, your only options are to run it on a larger machine or add more nodes to your deployment. Vertical scaling only works to a certain extent. The approach is not only expensive but due to the large chain database, it doesn’t always solve the problem.

If there is a bug discovered in the underlying node software, your whole system may become unstable and the fact that your application is bug-free won’t matter. It only takes one part of your sub-system failing, to potentially affect the health of your entire node. Case in point: in 2018, we encountered an issue where, if we received a deluge of websocket subscriptions to one particular node, the node would go into a partially deadlocked state, where it would receive blocks but not publish it to the subscribers.

The only way to deal with that problem at the time was to restart the node. This was not ideal, as we would disconnect not just from subscribers but also from everyone else who was connected to the node. So, Infura and other providers had to come up with custom solutions to stabilize the node software.

To address this inefficiency, Infura created a “cloud client” with a cloud-native architecture. The modularity of this allows us to be adaptable and horizontally scale to fit the needs of the growing Web3 ecosystem. We also developed a variety of microservices that offloaded dependency on the underlying client, and this allowed us to be more resilient to errors, as well as scale better.

Syncing Nodes: Faster, Better, More Efficient

Fast forward to today, the Ethereum client has become more modular, there are light nodes, stateless clients and a huge focus on trying to limit the growth of chain data. Even with the advancements, it still takes quite a while to sync an archive node or a full node. When you are growing rapidly and need more access to these nodes, you will find scaling increasingly difficult.

Back in 2016, it only took a couple of hours to sync our nodes and no additional tooling was required to maintain our infrastructure. As time went on and chain data grew, we switched to using snapshots of Amazon’s Elastic Block Store (EBS) chain data because these periodic snapshots were valuable for failure recovery. Later, we adapted again and started using fast Solid State Drives (SSD) we synced directly from object storage for efficiency and speed gains.

If you are interested, our implementation is actually based on MyCrypto’s implementation. There is more technical detail in the link, if you wish to give creating your own set-up a go.

On top of the fast SSDs, we are always looking for new solutions to help you scale faster.

We have a Source of Truth node that creates a snapshot of the latest chain data and pushes that snapshot to our Amazon S3 object storage. After which, a validator service verifies the chain data, ensuring there is no corruption and that all the required data is there. Then, the validator service marks the chain data as “verified”. This is all very useful if we have to scale up and add more nodes because the new nodes can easily grab this verified snapshot of the chain data, to speed up the syncing process.

What next?

The continued explosion of Ethereum users and demand for connectivity to other chains and services have pushed growth of our product suite further.

Today, Infura is powered by a cutting-edge, microservice-driven architecture that is optimized for scalability, reliability and efficiency. More than 350,000 developers connect to the Ethereum and IPFS networks through HTTPS and WebSocket. This year, we also expanded our support for Layer 2 networks, such as Polygon, Optimism and Arbitrum. And it all started with a simple premise: to make it easier to build on Web3.

In closing, we wish to say a big thank you to Tom Hay, Senior Product Manager and Jee Choi, Senior Systems Engineer, for their presentation on the history and evolution of Infura, at ETHPortland (see video below).

As we look forward to 2022 unfolding, what problems would you like us to help you solve? Holler at us on Twitter. Or, sign up for a forever-free account on Infura and start building your dream project today.