High performance serverless architecture for developing Stock Market Deep Learning models

Resumen

Several studies reported in the literature have analyzed the usefulness of various sources of information to predict the price of stock markets. Some of them have focused attention on the predictive power of information from social networks, which by their nature is unstructured and present a large volume of data. However, capturing and processing this information to obtain a mineable view to build predictive models can require vast amounts of computing power and storage, cost-prohibitive for an academic project. Additionally, training deep learning models requires a large consumption of computation, since hundreds of thousands of parameters must be calculated iteratively. The present work proposes a cloud-native architecture that scales elastically at a low cost using serverless technologies, which allows for capturing Twitter messages, performing natural language processing, and training Long-Short-Term- Memory (LSTM) and Convolutional Neural Networks. For each stage of the processing, the computing power used, the cloud storage capacity used, and the costs associated with the execution of each experiment are detailed. The cloud services used are also described, as well as the frameworks and libraries used both for data capture and for the training of Deep Learning models.