When there are too many pending transactions

Once up a time, Ethereum blocks had plenty of room. Usually they had a few or even zero transactions, and once in a while you’d see a block with over a hundred. If you sent a transaction, it was usually included in the next block.

In the token launch era, however, things have changed. Now we see thousands of transactions flooding the network around the start of a launch. I say “around the start” because some people send their contributions even before the start block arrives — a suboptimal strategy.

The large number of pending transactions is testing the limits of Geth’s and Parity’s mempool settings. The mempool (the term is borrowed from Bitcoin) is the set of data structures inside an Ethereum client to store pending transactions before they are mined (Geth calls it the “transaction pool”; Parity calls it the “transaction queue”).

Typically, a transaction received by an Ethereum client is held in the mempool and also forwarded to other clients in the network. It eventually reaches a miner, who adds it to a new block. When a client later receives a block that includes a transaction already in its mempool, it removes it from its mempool, freeing up space.

In Geth 1.6.5, the basic config for the mempool is in a Go structure called TxPoolConfig. One of the interesting fields in that structure is GlobalSlots ("Maximum number of executable transaction slots for all accounts"). In the default config (DefaultTxPoolConfig), GlobalSlots is set to 4,096. Once that limit is reached, Geth starts freeing up space by evicting some of the transactions in its mempool according to a fairly complex set of rules.

In the current token launch environment, 4,096 slots is tragically low. This may be why token launch contributors sometimes see their transactions disappear after sending them to a client (via the JSON-RPC method eth_sendRawTransaction), even if the client's response (a transaction hash) appears to indicate success. The transactions are simply purged from the mempool while the network is overloaded. They may never reach a miner.

Geth 1.6.2 mercifully added several CLI options to tweak the mempool config: they are the ones with the --txpool prefix. Use --txpool.globalslots value for the global slots setting. However, it's doubtful that many clients are using this setting today. That needs to change, pro bono publico.

With tens of thousands of pending transactions sometimes floating around the network, you can set this setting quite high. Make sure to test with it to make sure your machine finds it agreeable.

Similarly, Parity 1.6.8 has a setting called --tx-queue-size LIMIT. The default is 1,024. Crank it.

At INFURA, we're working on a feature that will help your transactions get mined irrespective of the mempool situation. We'll announce it soon, so keep an eye on our Twitter @infura_io .

Thanks to INFURA intern Eric Tu for helping with research for this post.