Evolutionary meta reinforcement learning for portfolio optimization

Abstract

Portfolio optimization is a control problem to find the optimal strategy for the process of selecting the proportions of assets that can provide the maximum return. Conventional approaches formulate the problem as a stationary Markov decision process and apply reinforcement learning methods to provide solutions. However, it is well-known that the financial markets are non-stationary processes which can lead to violations of this assumption in these methods. In this work, we reformulate the portfolio optimization problem to deal with the non-stationary nature of the financial markets. In our approach, we divide a non-stationary process into multiple short-term stationary processes and consider the portfolio optimization problem as a multitask control problem. We propose an evolutionary meta reinforcement learning approach to search for an initial policy that can quickly adapt to the upcoming target tasks. We model the policies as convolutional networks that can score the match of the patterns in charts of the market data. We test our approach on real-world cryptographic currency data and show that it adapts well to the changes in the market and leads to better profitability.

Myoung Hoon Ha

Postdoctoral Researcher

My research interests include deep learning, reinforcement learning, representation learning, and adversarial robustness.