Kyunghyun Cho, current Assistant Professor at NYU, has kindly shared his own vision on doing research in Machine Learning field. Cho is making a huge impact on the development of Machine Translation, as well as being extremely efficient in leading his research group and teaching at NYU.
Are there any differences in how professors and PhD students should read research papers?
Well, the style of reading differs from one person to another. Some people want to read all recent papers, other people read mostly the abstracts and so on.
At the same time, for the PhD students it is of great importance to read and understand key papers in detail. Key papers are essentially those closely related to their narrowed-down topic and most probably selected together with their supervisors. Those several papers should be understood deeply. Not understanding one sentence, that contains an important point, can lead to missing out the whole advantage, or disadvantage of the proposed approach. So details matter, since one wants to become an expert or scientist in the chosen field.
Do you have red flags when not to read a publication?
It pretty much depends on what you want to achieve. Generally, there should be a balance between reading the papers and doing your own research.
If you spend too much time reading others’ papers, there is less time to do your own research. I see sometimes people swamped with reading papers. Moreover, excessive reading can create additional boundaries, holding a researcher away from trying out new ideas. Hence a researcher might step back from trying a model, because someone did not succeed previously in the same direction. So, pursue your direction, try yourself, no matter if someone was successful or not in this direction so far.
Excessive reading can create additional boundaries, holding a researcher away from trying out new ideas
For a PhD student it is extremely important to read textbooks. There are so many details and fundamentals that are assumed to be known while writing the papers. Indeed, recently published papers are written assuming the basics are known. Therefore many important details needed, e.g., for implementation are missing. So the first step is always understanding the fundamentals from courses and textbooks.
Pursue your direction, try yourself, no matter if someone was successful or not in this direction so far.
Which books would you recommend to read, except for the “Deep Learning” book?
The first book I read was Chris Bishop’s book “Pattern Recognition and Machine Learning” that provides good background and detailed explanation. I was told Kevin Murphy’s “Machine Learning: A Probabilistic Perspective” is a nice book as well, but have not read it thoroughly yet. Then, depending on your specific topic, more books can be found.
If you had chance to do your PhD studies again, what would you do differently?
I would spend more time exploring and studying various topics within machine learning while doing my PhD studies.
You did your Master’s and PhD in Aalto, now you are in New York University. What is the biggest difference you have noticed between the education systems?
PhD study process is influenced mainly by the supervisor, rather than the university. So it is difficult to generalize. One thing I can see in NYU in terms of the environment is a high level of diversity. I supervise 7-9 students and most of them have diverse cultures and backgrounds, like physics, electrical engineering, computer science, mathematics, etc. In Finland I saw less diversity in terms of student/faculty background and the countries they came from. I am worried that the tuition fee introduced recently for the Master’s programs may further decrease the diversity.
Who are the researchers you admire?
Let us say there is a distinction between the visionaries+scientists and research scientists. Let me think about the people, who are visionary. I am not a visionary, though perhaps a good research scientist.
The visionaries are the ones who believe or at least have a clear justification why their idea works, at least for themselves. Geoff Hinton started from 1970s and has continued to work on neural networks despite the “Neural Net Winters“.
Yoshua Bengio did his PhD on deep learning for speech recognition in the late 80’s and continued to work with neural networks because he believed in them.
Similarly, Yann LeCun has started working on convolutional networks already from mid 80’s and continued working in this direction along, introducing those technologies to the industry, such as handwritten digits recognition for checks at Bell Labs.
Erkki Oja together with Teuvo Kohonen from Finland have as well continuously researched neural networks and pattern recognition. They had known they needed to do pattern recognition, despite the fact that there was neither relevant industry nor huge popularity in the field then. Erkki did research all the way until his recent retirement. He probably still continues to do research…
On the contrary, the majority pursue whatever is popular. This is not a “visionary” way.
So the visionaries are the ones from whom I learn and want to learn.
The visionaries are the ones who believe or at least have a clear justification why their idea works, at least for themselves.
How to get that vision?
I don’t know and I want to know. Solid scientific foundations help to build those logical steps and to convince oneself that the direction to pursue is the right one.
What matters other than working hard? Personality traits?
Personality traits? I don’t believe in being born to do something or having a particular personality for doing something. You just need to do whatever you like or are passionate about. It is not what other people or the society think is right. This is what visionaries do. They find something right and work on it, because they are convinced logically and scientifically. This is not that easy to do technically. Besides, it is quite hard to resist if the society does not accept your idea. So, it is crucial not to be swayed by the opinion of others. I hope to be on the side of visionaries at some point soon.
Visionaries find something right and work on it, because they are convinced logically and scientifically.
Interview was made by Luiza Sayfullina, NYU visiting PhD student from Aalto University