dType - Decentralized Type System for EVM | Ethereum Improvement Proposals

Simple Summary

The EVM and related languages such as Solidity need consensus on an extensible Type System in order to further evolve into the Singleton Operating System (The World Computer).

Abstract

We are proposing a decentralized Type System for Ethereum, to introduce data definition (and therefore ABI) consistency. This ERC focuses on defining an on-chain Type Registry (named dType) and a common interface for creating types, based on structs.

Motivation

In order to build a network of interoperable protocols on Ethereum, we need data standardization, to ensure a smooth flow of on-chain information. Off-chain, the Type Registry will allow a better analysis of blockchain data (e.g. for blockchain explorers) and creation of smart contract development tools for easily using existing types in a new smart contract.

However, this is only the first phase. As defined in this document and in the future proposals that will be based on this one, we are proposing something more: a decentralized Type System with Data Storage - ERC-2158. In addition, developers can create libraries of pure functions that know how to interact and modify the data entries - dType Functions Extension. This will effectively create the base for a general functional programming system on Ethereum, where developers can use previously created building blocks.

To summarize:

We would like to have a good decentralized medium for integrating all Ethereum data, and relationships between the different types of data. Also, a way to address the behavior related to each data type.
Functional programming becomes easier. Functions like map, reduce, filter, are implemented by each type library.
Solidity development tools could be transparently extended to include the created types (For example in IDEs like Remix). At a later point, the EVM itself can have precompiled support for these types.
The system can be easily extended to types pertaining to other languages. (With type definitions in the source (Swarm stored source code in the respective language))
The dType database should be part of the System Registry for the Operating System of The World Computer

Specification

The Type Registry can have a governance protocol for its CRUD operations. However, this, and other permission guards are not covered in this proposal.

Type Definition and Metadata

The dType registry should support the registration of Solidity’s elementary and complex types. In addition, it should also support contract events definitions. In this EIP, the focus will be on describing the minimal on-chain type definition and metadata needed for registering Solidity user-defined types.

Type Definition: TypeLibrary

A type definition consists of a type library containing:

the nominal struct used to define the type
additional functions:
- isInstanceOf: checks whether a given variable is an instance of the defined type. Additional rules can be defined for each type fields, e.g. having a specific range for a uint16 amount.
- provide HOFs such as map, filter, reduce
- structureBytes and destructureBytes: provide type structuring and destructuring. This can be useful for low-level calls or assembly code, when importing contract interfaces is not an efficient option. It can also be used for type checking.

A simple example is:

pragma solidity ^0.5.0;
pragma experimental ABIEncoderV2;

library myBalanceLib {

    struct myBalance {
        string accountName;
        uint256 amount;
    }

    function structureBytes(bytes memory data) pure public returns(myBalance memory balance)

    function destructureBytes(myBalance memory balance) pure public returns(bytes memory data)

    function isInstanceOf(myBalance memory balance) pure public returns(bool isInstance)

    function map(
        address callbackAddr,
        bytes4 callbackSig,
        myBalance[] memory balanceArr
    )
        view
        internal
        returns (myBalance[] memory result)
}

Types can also use existing types in their composition. However, this will always result in a directed acyclic graph.

library myTokenLib {
    using myBalanceLib for myBalanceLib.myBalance;

    struct myToken {
        address token;
        myBalanceLib.myBalance;
    }
}

Type Metadata: dType Registry

Type metadata will be registered on-chain, in the dType registry contract. This consists of:

name - the type’s name, as it would be used in Solidity; it can be stored as a string or encoded as bytes. The name can have a human-readable part and a version number.
typeChoice - used for storing additional ABI data that differentiate how types are handled on and off chain. It is defined as an enum with the following options: BaseType, PayableFunction, StateFunction, ViewFunction, PureFunction, Event
contractAddress - the Ethereum address of the TypeRootContract. For this proposal, we can consider the Type Library address as the TypeRootContract. Future EIPs will make it more flexible and propose additional TypeStorage contracts that will modify the scope of contractAddress - ERC-2158.
source - a bytes32 Swarm hash where the source code of the type library and contracts can be found; in future EIPs, where dType will be extended to support other languages (e.g. JavaScript, Rust), the file identified by the Swarm hash will contain the type definitions in that language.
types - metadata for subtypes: the first depth level internal components. This is an array of objects (structs), with the following fields:
- name - the subtype name, of type string, similar to the above name definition
- label - the subtype label
- dimensions - string[] used for storing array dimensions. E.g.:
  - [] -> TypeA
  - [""] -> TypeA[]
  - ["2"] -> TypeA[2]
  - ["",""] -> TypeA[][]
  - ["2","3"] -> TypeA[2][3]

Examples of metadata, for simple, value types:

{
  "contractAddress": "0x0000000000000000000000000000000000000000",
  "typeChoice": 0,
  "source": "0x0000000000000000000000000000000000000000000000000000000000000000",
  "name": "uint256",
  "types": []
}

{
  "contractAddress": "0x0000000000000000000000000000000000000000",
  "typeChoice": 0,
  "source": "0x0000000000000000000000000000000000000000000000000000000000000000",
  "name": "string",
  "types": []
}

Composed types can be defined as:

{
  "contractAddress": "0x105631C6CdDBa84D12Fa916f0045B1F97eC9C268",
  "typeChoice": 0,
  "source": <a SWARM hash for type source code files>,
  "name": "myBalance",
  "types": [
    {"name": "string", "label": "accountName", dimensions: []},
    {"name": "uint256", "label": "amount", dimensions: []}
  ]
}

Composed types can be further composed:

{
  "contractAddress": "0x91E3737f15e9b182EdD44D45d943cF248b3a3BF9",
  "typeChoice": 0,
  "source": <a SWARM hash for type source code files>,
  "name": "myToken",
  "types": [
    {"name": "address", "label": "token", dimensions: []},
    {"name": "myBalance", "label": "balance", dimensions: []}
  ]
}

myToken type will have the final data format: (address,(string,uint256)) and a labeled format: (address token, (string accountName, uint256 amount)).

dType Registry Data Structures and Interface

To store this metadata, the dType registry will have the following data structures:

enum TypeChoices {
    BaseType,
    PayableFunction,
    StateFunction,
    ViewFunction,
    PureFunction,
    Event
}

struct dTypes {
    string name;
    string label;
    string[] dimensions;
}

struct dType {
    TypeChoices typeChoice;
    address contractAddress;
    bytes32 source;
    string name;
    dTypes[] types;
}

For storage, we propose a pattern which isolates the type metadata from additional storage-specific data and allows CRUD operations on records.

// key: identifier
mapping(bytes32 => Type) public typeStruct;

// array of identifiers
bytes32[] public typeIndex;

struct Type {
  dType data;
  uint256 index;
}

Note that we are proposing to define the type’s primary identifier, identifier, as keccak256(abi.encodePacked(name)). If the system is extended to other programming languages, we can define identifier as keccak256(abi.encodePacked(language, name)). Initially, single word English names can be disallowed, avoiding name squatting.

The dType registry interface is:

import './dTypeLib.sol';
interface dType {
    event LogNew(bytes32 indexed identifier, uint256 indexed index);
    event LogUpdate(bytes32 indexed identifier, uint256 indexed index);
    event LogRemove(bytes32 indexed identifier, uint256 indexed index);

    function insert(dTypeLib.dType calldata data) external returns (bytes32 identifier);

    function remove(bytes32 identifier) external returns(uint256 index);

    function count() external view returns(uint256 counter);

    function getTypeIdentifier(string memory name) pure external returns (bytes32 identifier);

    function getByIdentifier(bytes32 identifier) view external returns(dTypeLib.dType memory dtype);

    function get(string memory name) view external returns(dTypeLib.dType memory dtype);

    function isRegistered(bytes32 identifier) view external returns(bool registered);
}

Notes:

To ensure backward compatibility, we suggest that updating types should not be supported.

The remove function can also be removed from the interface, to ensure immutability. One reason for keeping it would be clearing up storage for types that are not in use or have been made obsolete. However, this can have undesired effects and should be accompanied by a solid permissions system, testing and governance process. This part will be updated when enough feedback has been received.

Rationale

The Type Registry must store the minimum amount of information for rebuilding the type ABI definition. This allows us to:

support on-chain interoperability
decode blockchain side effects off-chain (useful for block explorers)
allow off-chain tools to cache and search through the collection (e.g. editor plugin for writing typed smart contracts)

There is one advantage that has become clear with the emergence of global operating systems, like Ethereum: we can have a global type system through which the system’s parts can interoperate. Projects should agree on standardizing types and a type registry, continuously working on improving them, instead of creating encapsulated projects, each with their own types.

The effort of having consensus on new types being added or removing unused ones is left to the governance system.

After the basis of such a system is specified, we can move forward to building a static type checking system at compile time, based on the type definitions and rules stored in the dType registry.

The Type Library must express the behavior strictly pertinent to its defined type. Additional behavior, required by various project’s business logic can be added later, through libraries containing functions that handle the respective type. These can also be registered in dType, but will be detailed in a future ERC.

This is an approach that will separate definitions from stored data and behavior, allowing for easier and more secure fine-grained upgrades.

Backwards Compatibility

This proposal does not affect extant Ethereum standards or implementations. It uses the present experimental version of ABIEncoderV2.

Test Cases

Will be added.

Implementation

An in-work implementation can be found at https://github.com/pipeos-one/dType/tree/master/contracts/contracts. This proposal will be updated with an appropriate implementation when consensus is reached on the specifications.

A video demo of the current implementation (a more extended version of this proposal) can be seen at https://youtu.be/pcqi4yWBDuQ.