Today we’re pleased to announce Massive, a command line toolkit for managing 0x orders. One challenge we’ve had at OpenRelay is figuring out how to create and manipulate orders in a streamlined fashion, without having to write new code for each task. Along the way we’ve built some tooling to help, and today we’re opening it up to the community. The Massive Toolkit is far from complete - what we’re releasing today is a set of tools we’ve needed internally. We’ll add more to it as the need arises, and we’d be happy to take outside contributions.

You can get Massive for Linux on Github, or run

go get github.com/notegio/massive/cmd/massive

if you already have the go toolchain installed.

Example

Massive is meant to be used on the command line, and it follows the philosophy that each tool should do one thing and do it well. It makes heavy use of Unix pipelines so that a bunch of individual tools that each do one thing well can be recombined in many different ways to accomplish many different tasks.

One task Massive helps with is going from a CSV with basic order information, and submitting those orders to a relayer.

Reading CSVs

Massive includes a tool for reading CSVs. It’s fairly flexible, and can read any CSV that has column headers corresponding to the values in a 0x Order. This allows us to manage order lists as spreadsheets, and easily upload them to OpenRelay. A CSV could contain all of the fields involved in an order, or just a subset so that other tools can fill in the rest. For this example we’ll start with a CSV that only includes the token pairs and quantities to trade of each token.

makerTokenAddress,makerTokenAmount,takerTokenAddress,takerTokenAmount
0xa1df88ea6a08722055250ed65601872e59cddfaa,1000000000000000000,0xc778417e063141139fce010982780140aa0cd5ab,1000000000000000000
0xc778417e063141139fce010982780140aa0cd5ab,1000000000000000000,0xa1df88ea6a08722055250ed65601872e59cddfaa,1000000000000000000

If we pass this into Massive, it will output the orders in a JSON format for further processing:

massive 0x csv --input=sample.csv

The result will be a JSON stream of 0x Orders. By default it will go to standard out, but most Massive commands offer an --output flag that lets you send it straight to a file instead.

Adding Expiration Dates

If your CSV included expiration dates, great. Otherwise, you’ll want to add an expiration date. Massive can help.

The Massive Expiration subcommand lets you specify either an expiration timestamp (as a Unix timestamp in seconds), or a duration for the order (the amount of time after the order gets processed before it expires). To set all of your orders to expire in ten days, you could run:

massive 0x csv --input=sample.csv | \
massive 0x expiration --duration 864000

The first command reads your orders out of a CSV, while the second command sets an expiration on each order, 10 days in the future.

Adding Fees

If you’re going to submit your orders to a relayer, chances are they will require you to include a fee on your order. Massive can help. Massive can look up fees from any relayer implemeting the 0x Standard Relayer API. It also lets you indicate what portion of the fee you want the maker to pay - this aligns with OpenRelay’s flexible fee structure, but not all relayers will allow the maker to change the allocation of fees.

For example:

[...] | \
massive 0x getFees --maker-share .5 --target https://api.openrelay.xyz/

Would get the orders from our earlier example, query OpenRelay.xyz to find out the required fee, and allocate 50% of that fee to the maker, and 50% of the fee to the taker.

Setting Other Properties

In addition to expiration dates and fees, there are a few other properties to think about. Massive can help.

The setExchange subcommand knows about the Exchange contract deployments on each network, and can set the corresponding attribute on each order.

The setSalt subcommand will set a unique salt on your orders. By default, it sets the salt equal to the current time in seconds. Alternatively, you can pass the --random flag to set a random 256 bit salt.

To extend our earlier example:

[...] | \
massive 0x setExchange --ropsten | \
massive 0x setSalt

Would set the exchange contract address for the Ropsten network, and set a salt based on the current timestamp in seconds.

Signing Orders

Even if you started out with a perfectly valid order, if you used any of the previous commands to manipulate the orders, you’ll need to sign them again. Massive can help.

By default Massive’s sign subcommand will only sign orders if the order’s maker address matches the provided private key. If we had included the maker address in our CSV, this would allow us to have multiple makers and pipe the orders through multiple instances of the sign subcommand. Each instance of the sign subcommand would sign orders corresponding to its private key, and the rest would pass through unsigned.

But in our earlier example we didn’t specify a maker. If we specify the --replace-on-mismatch flag, Massive will update the maker of each order to correspond to the address of the provided private key.

Extending our example:

[...] | \
massive 0x sign --replace-on-mismatch /path/to/private-key

In this case the private key must be in a hex encoded file on the local filesystem. Right now Massive doesn’t support encrypted wallets, hardware wallets, signing via RPC requests, etc. but we’d be open to contributions to help make this possible.

Setting Allowances

Now we have orders with all of the necessary attributes set, but before a relayer will consider these orders valid, we must set allowances for the appropriate tokens on the blockchain. Massive can help.

Massive can send transactions to an Ethereum node to set allowances for each order. As of this writing it only supports unlimited allowances, and must have the private key available (it does not currently support having the Ethereum node’s wallet sign the transaction).

Massive will only set allowances for each token once, only if allowances are not already set, and it will hold onto any orders depending on those transactions until the corresponding Ethereum transactions have been confirmed. Once the transactions are confirmed, they will be written to the output stream.

NOTE: Right now Massive does not support specifying the gas price for this transaction. It follows the recommendation of Ethereum’s Gas Price Oracle. Be aware that this can be large at times. You may want to set allowances outside of massive until we support specifying gas prices.

Extending our example:

[...] | \
massive 0x setAllowance --unlimited http://localhost:8545 /path/to/private/key

As allowances are set on the blockchain, the orders will be written to the output stream.

Uploading to a Relayer

Now we have orders set exactly as we want them, allowances are set on the blockchain so the relayers will consider them valid. All we have left to do is upload them to a relayer. Massive can help.

The massive upload subcommand can submit orders to any relayer that implements the 0x Standard Relayer API.

Extending our example:

[...] | \
massive 0x upload --target https://api.openrelay.xyz/

Putting It All Together

Putting all of our steps together into a single pipeline:

massive 0x csv --input=sample.csv | \
massive 0x expiration --duration 864000 | \
massive 0x getFees --maker-share .5 --target https://api.openrelay.xyz/ | \
massive 0x setExchange --ropsten | \
massive 0x setSalt | \
massive 0x sign --replace-on-mismatch /path/to/private-key | \
massive 0x setAllowance --unlimited http://localhost:8545 /path/to/private/key | \
massive 0x upload --target https://api.openrelay.xyz/

There’s a lot going on here, but if allowances are already set the entire pipeline should complete in seconds or less.

Under The Hood

Massive is written entirely in Go, leaning heavily on the OpenRelay code base for its order interaction. For most subcommands, the input and output will be a line-delimited JSON stream. That means you could pretty easily write code in Python or JavaScript to manipulate objects. In Python, a basic example for dealing with JSON streams looks like:

import sys
import json

for line in sys.stdin:
    item = json.loads(line)
    # manipulate item
    json.dump(item, sys.stdout)
    sys.stdout.write("\n")

You could also use existing command line tools like sed and grep to filter for and alter orders based on your needs. With this flexibility you could write tools in any language you like so long as they can read and write JSON. In theory you could write a C# tool that decides what orders you want to create, then prints them on standard out for Massive to sign and upload to the order book.

Future Work

Right now, most of the Massive subcommands are specific to 0x. We’re working on some additional tools that get into other aspects of interacting with an Ethereum blockchain.

Additionally, Massive is currently limited to creating orders and getting them in to the order book. In the future, we plan to have Massive support getting orders out of the order book and filled on the blockchain, as well as validating orders against the blockchain to ensure they will be fillable. Stay tuned for more details.