As I read Deep Learning (Goodfellow, Bengio & Courville) I am reminded of Prof Ghosh who taught me Physics in my undergrad.
Every once in a while, he would have a bemused expression on his face & throw a question to the class. I had learned with experience to decode his expression. The auto-encoderwas perpetually on! A curve-ball like that with an accompanying gotcha look from him meant the answer was a Tensor … perhaps of rank 3. So, I would promptly raise my hand donning the so-called intelligent look & respond appropriately.
We played this game a few times. But, he was too smart a professor to not know what was happening. Drat, he was learning with experience too! His questions got more unstructured & abstract & I could feel I was regressing. Funny, those days regression to me was a negative term (it only meant I was slipping back) but the supervised learningcontinued! With time, happily, it turned into un-supervised learning (pun completely intended)!
Much like Prof Ghosh, I have learnt to throw questions (with as much drama), targeted at myself though. How deep is deep where learning is concerned, I wonder?
I look to Artificial Intelligence to answer this.
When you can take simple concepts & use them to construct more complex concepts in a kind of hierarchy or when you take a representation of data & construct more complex representations, you can attain great powers of wisdom & deep insight.
But with wisdom comes responsibility. Use your new-found wisdom wisely. I can almost see Prof Ghosh’s quizzical smile. Because he knows that I have figured out that deep learning is not merely a tensor of rank 3!