Running models on the cloud

Jupyter Notebook is the most popular tool for doing data science in Python. It is powerful, flexible, and easy to use. 

There are online platforms available where you can upload your Jupyter notebooks and run your code on the cloud. It is better that you run your code on any of these and not use your own computer, unless you are very experienced in Linux administration and handling GPU drivers. 

• Colab: A popular free service from Google. 
• Gradient: Works like real Jupyter notebook and you can save your models and notebooks. 
• DataCrunch.io: no setup required, extremely good value and extremely fast GPUs, or 
• Google Cloud: extremely popular service, very reliable, but the fastest GPUs are expensive.
• JarvisLabs.ai:- With Jarvis Cloud you get a GPU powered Jupyter notebook pre-configured with all the necessary software in less than 30 seconds.
• Azure:- You can use Azure Data Science Virtual Machine (DSVM) which is a family of Azure Virtual Machine images, pre-configured with several popular tools that are commonly used for data analytics, machine learning and AI development.
   

I have been using Colab till now and it has worked for most of the tasks. It is better to optimize your code to work on the given machine configuration as there are number of ways to reduce compute requirements. Keep experimenting!

Numerai - The Financial Machine Learning Competition

Numerai is the hardest data science tournament in the world as it aims to build the world's last hedge fund. 

You have to apply machine learning to predict the stock market on the data shared by Numerai. 

This is a very good comprehensive guide for the competition: 

https://tit-btcqash.medium.com/a-comprehensive-guide-to-competing-at-numerai-70b356edbe07

Couple of Kaggle notebooks also give a quick start to the competition as you aim to build a model and then stake real dollars to earn money.

https://www.kaggle.com/code1110/numerai-tournament

https://www.kaggle.com/carlolepelaars/how-to-get-started-with-numerai

I will aim to share my Numerai journey in the future blogs. 

Stocks Data

 Let's look at various service providers for data for equities:

a) Tiingo- From $0 a month to  $10 a month. It provides an API to access the data.

b) FMP Cloud:- $14/month wit unlimited requests. 

c) Zerodha:- Rs. 4000/m which is approx $ 55/month provides the historical data API linked to Kite Connect. 

Stripe Atlas - Incorporating company in the US

 Recently heard of Stripe Atlas, a service by Stripe to incorporate your company in the US from anywhere on the globe. 

Services provided include almost everything required to set up a startup and costs $500 currently: 

• Formation of a company in Delaware 
• Delaware state filing fees 
• Signed documents to establish company bylaws and protect IP 
• Tool to issue stock to founders 
• First year of registered agent fees 
• Tax ID (EIN) filing 
• Stripe Atlas Community membership
• Free templates for post-formation legal needs 

For more, you can refer to https://stripe.com/atlas

LSTM Model

 This is a pretty good guide to LSTM: 

https://towardsdatascience.com/illustrated-guide-to-lstms-and-gru-s-a-step-by-step-explanation-44e9eb85bf21

Youtube video: https://www.youtube.com/watch?v=8HyCNIVRbSU

A research paper from Stanford looks at using the model for market prediction: 

http://cs230.stanford.edu/projects_winter_2020/reports/32066186.pdf

Depth First Learning

If you haven’t heard of it, Depth First Learning is a wonderful resource for learning about machine learning.

Grown out of a Google AI residency, the DFL program builds curricula around specific ML papers.  DFL has built an entire self-paced class, with background reading, lectures, and practice problems, that culminates in the paper itself. So far, they’ve built guides like this for DeepStack, InfoGAN, and TRPO.

This is  advanced stuff in ML but if you go that far, you should surely have a look at DFL!

Kudos to the team! 

L2 Regularization and Batch Norm

So to a first-order approximation, once you are using batch normalization in a neural net, an L2 objective penalty term or weight decay no longer contribute in any direct manner to the regularization of layers that precede a batch norm layer. Instead, they take on a new role as the unique control that prevents the effective learning rate from decaying over time.

This could of course itself result in better regularization of the final neural net, as maintaining a higher learning rate for longer might result in a broader and better-generalizing optimum. But this would be a result of the dynamics of the higher effective learning rate, rather than the L2 objective penalty directly penalizing worse models.

Of course, this analysis does not hold for any layers in a neural net that occur after all batch normalization layers, for example typically the final fully-connected layers in common architectures. In those layers, obviously the normal regularization mechanism applies. Other variations on architecture might also affect this analysis. And as mentioned near the start of this post, if you are using an optimizer other than stochastic gradient descent (or stochastic gradient decent with momentum - the analysis is very similar), things might also be a little different.

Source: https://blog.janestreet.com/l2-regularization-and-batch-norm/