Found a great book called Grokking Deep Learning from Andrew Trask. The hole point in to make your own Deep Learning framework from scratch. Using python, but I think I'm going to try to go at it using Typescript. Lets see how that goes.