Notes from Bay Area Deep Learning School
- Deep Learning is the king, but only if you have enough data, computing power and proper algorithms.
- Amount of data required could be enormous (e.g. billion of parameters).
- Comparing to human, Deep Learning still needs many improvements, e.g. to learn to understand the speech it requires tens of thousands hours of data to produce the results comparable to the young child. Comparing to human ‘AI’ (current implementations of machine learning algorithms) can’t generalise based on the small amount of data.
- In multiple areas Deep Learning can produce the results with better accuracy comparing to single human expert +5-10%, comparing to group of experts +2-4% in overall accuracy.
- It’s noticeably harder to improve the model after it reached the human level.
- GPU and infrastructure (High performance systems/computations) is an important problem for Machine Learning, many prominent companies have a separate devision inside AI labs dedicated to infrastructure for machine learning and high performance calculations.
- Labeled Data is still required, although the unsupervised deep learning showed a great advancements recently, the big amount of properly labelled data will play an important role in most implementations of machine learning systems in the near future.
- No universal and scalable solution for GPU in the cloud or Machine Learning in the cloud is presented at the moment. Some attempts are being made to create the universal platform, but many researchers (e.g. Baidu AI Lab [SVAIL]; Google Brain; DeepMind; OpenAI) are using their own software and/or hardware solutions.
- Deep Learning have a wide range of applications, multitude of new business applications will emerge soon.
- The current implementation of Deep Learning may not solve the problem of AI by itself. New algorithms which can learn on a smaller amount of data shall emerge.
- Multiple great advancements in speech recognition, object recognition, visual question answering, and many other fields were discussed at Bay Area Deep Learning School .
[Video Day 1] Bay Area Deep Learning School Day 1 at CEMEX auditorium, Stanford:
[Video Day 2] Bay Area Deep Learning School Day 2 at CEMEX auditorium, Stanford:
Official site: http://www.bayareadlschool.org/