DeepBrew: Deeplink v1
This Beer game is to demonstrate a proof of concept to Deeplink’s core vision—combining off-chain machine learning into on-chain environments e.g. smart contracts.
The Beer Game PoC proposes a framework for the augmentation of smart contract execution with artificial intelligence to enable dynamism in elaborate situations that opens an array of new applications. The aim is to develop a system of knowledge relating to the possibilities, methodologies, and limitations of this concept.
To illustrate this, a blockchain-native adaptation of ‘The Beer Game’, a macroeconomic supply chain scenario, has been devised to provide a dynamic optimization problem involving transactions and the management of complex systems. A soft actor-critic deep reinforcement learning algorithm is then trained against rule-based agents. Finally, variables from this model and environment are passed on to prompt the execution of an Ethereum smart contract via an oracle, demonstrating an intelligent ‘on-chain agent.’
This game does not have any commercial application or utility other than displaying Beer game deep learning training results and interactions between on-chain and off-chain. Summary as part of Deeplink’s research towards the amalgamation of deep reinforcement learning (DRL) and on-chain execution (something which we refer to as “on-chain agents”), a blockchain native version of The Beer Game has been created using Web3.py and a private Ganache testnet.
Agents in this game trade ETH for BEER (a custom ERC20 token deployed for the game). An agent-based model was written to play the game following simple mathematical instructions. Afterwhich, a DRL (soft actor-critic (SAC)) model was trained to play the game against these agent-based players.
The Beer Game provided a transaction-based environment that could be easily represented on the Ethereum blockchain This provides a gamified platform for testing out machine learning approaches that involve both reading from and transacting on Ethereum in which a model’s effectiveness can be easily evaluated with little-to-no industry or task expertise required, I.e., any other crypto environments are more complicated, and it is less immediately obvious to determine whether or not a model is performing well.
The Beer Game is known to be effectively optimized by reinforcement learning approaches, in particular, Q-learning and deep Q-learning Deep Q-learning was identified by our researchers (Priyanka) to be a likely candidate for the best approach to optimizing SOR with deep learning Finally, a large portion of the research project focuses on the integration of machine learning models with smart contract execution, something which will likely come in handy in phases 3-4 of our Eta project.
The Beer Game research proposes a framework for and investigates the viability of bringing machine learning models into the execution of Ethereum smart contracts via API and Chainlink, oracles-effectively facilitating machine learning keepers, or ‘on-chain agents’.