Back In February we asked for help testing Ether Cattle, a project we built, in part with funding from a 0x Ecosystem Acceleration Grant, that uses streaming replication to make Ethereum clients easier to manage at scale. At the time, we had a new replica server that we needed to test for correctness, but we had a lot of work left to do making Ether Cattle easy for anyone to manage and deploy. Today Ether Cattle is ready for you.

Design

Ether Cattle runs as a cluster of servers. A cluster consists of one “master” node, which generally behaves as a conventional Geth node, connecting to peers and validating blocks, but it also writes its changes to a Kafka broker. At the other end of the cluster are one or more “replica” nodes, which pull the master’s changes via Kafka and store them locally. The replica nodes then act as RPC servers, which can be load balanced to meet an application’s need for RPC capacity. Because Replicas don’t have to deal with peer-to-peer connectivity and block validation, we’ve found that a single replica can handle about 4x the RPC request volume of a full Geth node on equivalent hardware.

The technical design of Ether Cattle is documented in detail on ReadTheDocs.org.

Running In AWS

We spent a lot of time developing out a set of CloudFormation templates for running an Ether Cattle Cluster in AWS. At a high level, it consists of three CloudFormation Stacks:

Generating a Chaindata Snapshot

The first CloudFormation stack will start a Geth node, sync it with the network, and create an EBS snapshot of the chaindata folder. We’ve simplified this process down to filling out a form in CloudFormation, but it can still take days for a fresh Ethereum node to sync, and Ether Cattle doesn’t change the time for an initial sync.

Standing up an Infrastructure Stack

Once you have your Chaindata Snapshot, the second CloudFormation stack will stand up a cluster of Kafka servers on your behalf, and set up an internal Application Load Balancer that will soon handle requests to your replicas. Again, we’ve boiled this down to a pretty simple CloudFormation form, and this time it only takes a few minutes to deploy.

Standing up one (or more) Ether Cattle Clusters

Once you have your Infrastructure Stack, you’re ready to deploy your master and replicas. You’ll need the Snapshot ID from generating your Chaindata Snapshot (which should have been e-mailed to you by the first stack). Fill out the CloudFormation form, and it will deploy a master and replicas for you, along with a monitoring dashboard, alarms, and an automated snapshotting process.

Each cluster will take a chaindata snapshot once a day. If a master or replica fails, the autoscaler will automatically replace it based on the latest chaindata snapshot, it will pull the latest data from Kafka, and resume normal operation quickly.

Let’s Get Started

Step-by-step instructions for running Ether Cattle in AWS via CloudFormation are available on ReadTheDocs.org.

Running On Your Own Hardware

If AWS isn’t where you run infrastructure, you’ll be more interested in our Infrastructure Agnostic Setup Documentation. Obviously this will be a bit more complicated than step-by-step instructions and filling out a form, but we’re hoping to make it easy for people to get up and running with different providers. We’d also welcome contributions to stand up clusters using other automation suites, such as Terraform, Kubernetes, HEAT, or other provider-specific toolkits.

Announcing Rivet

Rivet Logo

As we developed Ether Cattle, we came to realize that the costs of running a highly-available cluster get a bit steep for a single project. To reach our target level of uptime, we’ve determined that we need at least two masters and four replica servers, on top of a highly available Kafka cluster. Based on our load testing, 4 replica servers can conservatively handle 60,000 RPC requests per second, while OpenRelay needs less than 10 RPC requests per second. Needless to say, we’ll have some extra capacity.

As a result, we’ve decided to launch Rivet, a hosted service based on Ether Cattle. With Rivet you will be able to drop an endpoint URL into your Web3 application without having to manage your own RPC servers. Rivet will offer self-service signup, clear up-front pricing, and a strong privacy policy protecting not just our customers (dApp developers and businesses) but also their customers (the end-user).

We’re planning a month long, invite-only soft launch starting in July, followed by general availability starting in August. If you’d like to get on our list for the soft-launch, signups are at rivet.cloud.

As a final note, just as api.openrelay.xyz is based on the open source project OpenRelay, rivet.cloud will be based on the open source Ether Cattle Initiative. The fact that we plan to monetize through a hosted service won’t deter our commitment to open source software.