Weekly Updates:
- Did I mention I was a father again? Not a whole lot of coding this week as I’m trapped in the whirlpool of 3 hour feeding cycles. Never fear though we do explore something of value today.
- My book Immortality is up to #123 in the Kindle Store > Kindle eBooks > Nonfiction > Politics & Social Sciences > Philosophy > Metaphysics category on the kindle store. Watch out world.
- I purchased my various tickets to DevCon3. Look out Mexico, Here I come!
The Code
Today’s post isn’t very long but I think it will help answer some questions for others who have been confused by the storage system in Ethereum.
There are a number of good pieces of info out there that talk about how you should handle storage for more complicated contracts. The basic idea is that you create a storage contract to hold all of your data so that if your master contract needs to be replaced you can just deploy a new contract and point the storage to a new place. Here are a few articles on how some of this works:
https://blog.colony.io/writing-upgradeable-contracts-in-solidity-6743f0eecc88
https://ethereum.stackexchange.com/questions/13167/are-there-well-solved-and-simple-storage-patterns-for-solidity
https://ethereum.stackexchange.com/questions/15930/pushing-pulling-data-from-data-storage-contract
http://solidity.readthedocs.io/en/develop/miscellaneous.html
The colony article is really interesting and they end up setting a place for all different kinds of data. As it is difficult to predict what kinds of data you are going to have I thought it would be interesting to think about how to make this more generic. This gets especially difficult when you start doing mappings of mappings. This is going to be important for our decaying tokens as we are going to need to keep track of transaction inputs in a mapping(address => mapping(address => amount)) such that we can get data you like prefs[reciver][payer]
My little example looks at allocating ERC20 allowances and how to store those:
contract MapTest {
mapping(address => mapping(address => uint256)) public approvals;
mapping(bytes32 => uint256) public approvalsKeccak256;
mapping(bytes32 => uint256) public genericStore;
function MapTest(){
}
function AddAllowanceMap(address sender, address recipient, uint256 amount){
approvals[sender][recipient] = amount;
}
function AddAllowanceKeccak256(address sender, address recipient, uint256 amount){
approvalsKeccak256[keccak256(sender,recipient)] = amount;
}
function AddAllowanceGeneric(address sender, address recipient,uint256 amount){
genericStore[keccak256('allowance', sender, recipient)] = amount;
}
}If you load this up into Remix and run each function with "0x1","0x3",100 you will see that the gas cost is just about the same for each instance.
This is pretty powerful as we can now keep all of our mappings in one place. This violates all kinds of separations of concerns and may be a bad idea, but it will make the code in our stage contract much cleaner. So if we are going to violate the separation of concerns, let's go all the way and build a generic storage contract that can be ETLed if we need to copy data.
Here is the basic generic storage contract.
contract StorageTest {
//keep track of if we have seen this key before or not
mapping(bytes32 => bool) public genericStoreExists;
//a place to put our data
mapping(bytes32 => bytes32) public genericStore;
//a place to keep track of our keys. Out of order...but still, we
//have them
bytes32[] public genericIterator;
//keep track of the number of keys we are storing
uint256 public genericCount;
function StorageTest(){
}
//store our data
// We assume that your variable name has been convered to
// bytes32. You can do this via a keccak or converting string to bytes
// We assume you have keccaked your variable path
// We assume that all values are cast to bytes32
function PutValue(bytes32 _dataGroup, bytes32 _kecKey, bytes32 _value){
bytes32 key = keccak256(_dataGroup, _kecKey);
return PutValueRaw(key, _value);
}
function PutValueRaw(bytes32 key, bytes32 _value){
genericStore[key] = _value;
if(genericStoreExists[key] == false){
genericStoreExists[key] = true;
genericIterator.push(key);
genericCount = genericCount + 1;
}
}
//get the raw bytes
function getBytes(bytes32 _dataGroup, bytes32 _kecKey) constant returns(bytes32){
bytes32 key = keccak256(_dataGroup, _kecKey);
return genericStore[key];
}
//get integers
function getInt(bytes32 _dataGroup, bytes32 _kecKey) constant returns(uint256){
bytes32 key = keccak256(_dataGroup, _kecKey);
return uint256(genericStore[key]);
}
//get addresses
function getAddress(bytes32 _dataGroup, bytes32 _kecKey) constant returns(address){
bytes32 key = keccak256(_dataGroup, _kecKey);
return address(genericStore[key]);
}
//get bools
function getBool(bytes32 _dataGroup, bytes32 _kecKey) constant returns(bool){
bytes32 key = keccak256(_dataGroup, _kecKey);
bytes32 value = genericStore[key];
if(value == 0x0){
return false;
}
else{
return true;
}
}
//not sure about arrays
//not sure about enums
}There is more work to do here, but the basics are here. Try putting in some data by creating a contract in remix and then submitting PutValue with “0x0000000000000000000000000000000000000000000000000000000000000001","0x0000000000000000000000000000000000000000000000000000000000000001",”"0x0000000000000000000000000000000000000000000000000000000000000014".
Once that is complete, call getInt with “0x0000000000000000000000000000000000000000000000000000000000000001","0x0000000000000000000000000000000000000000000000000000000000000001" and you should get out 20. It works!
We can also ETL(Extract, Transform, Load) this contract. Here I’ve built a small data copy contract that will move data from this storage to another storage contract of the same type:
contract StorageCopy {
StorageTest public Source;
StorageTest public Target;
uint256 public CurrentItem;
function StorageCopy(address _source, address _target){
Source = StorageTest(_source);
Target = StorageTest(_target);
}
function DoCopy() returns (uint256){
uint256 SourceCount = Source.genericCount();
for(uint256 i = CurrentItem; i < SourceCount; i++){
Target.PutValueRaw(Source.genericIterator(i),
Source.genericStore(Source.genericIterator(i)));
//keep track of where we are in the process
CurrentItem = i;
//todo: scheme to check remaining gas and bail if gas gets too low
//next call will start with Current item
}
}
}Create two StorageTest contracts in Remix. Call PutValue on the first contract with "0x1","0x1","0x0000000000000000000000000000000000000000000000000000000000000001" and "0x2","0x2","0x0000000000000000000000000000000000000000000000000000000000000002". Now create a StorageCopy contract and pass in the addresses of your two created cotracts like this: "0x0971b5d216af52c411c9016bbc63665b4e6f2542","0xde6a66562c299052b1cfd24abc1dc639d429e1d6" where the addresses are what Remix generated for you. Call the DoCopy Function and the data should copy over.
You should now be able to call getInt with "0x2","0x2" on the second StorageTest contract and get out the 2 value you put in for that dataGroup / mapping address pair. It works!
There is a lot left undone here. You’d want to secure the data contract. You also are going to have to deal with gas limits as the comments in the comments say. There are some data types that I’m not sure how to handle yet. Arrays could be particularly difficult. You might be able to save a ton of gas by writing some native assembly in here as well. All in all, though, this is a decent footprint for a basic storage contract.
If this is interesting to you and you'd like to see where we are going with Catallax, please pick up my book Immortality (Purchase of a physical or kindle copy helps support this project).
Donations always accepted at:
BTC: 1AAfkhg1NEQwGmwW36dwDZjSAvNLtKECas
ETH and Tokens: 0x148311c647ec8a584d896c04f6492b5d9cb3a9b0
If you would like more code articles like this please consider becoming a patron on patreon.