The Professor's lab moved to a beautiful modern designed building with a lot of Big Logos on the door. There were layers of security checks. Several new people were hired. I personally felt alienated in that environment though I visited several HQs of Big Tech and Unicorn in the Valley.
My stay was short. I needed computing power to continue research. The most critical pain point was a supply of GPUs. This resource was scarce. Better GPUs would give an advantage to run a larger model. Therefore it would make a better result. The grant from Professor would give me access to V100s - the top notch at this moment with technical support from a Canadian computing platform. This would help a lot to run the experiment at the moment.
To explain the success of reincarnation of AI under the name deep learning, it would point to a convergence of three things, e.g. data, computing power, and an open culture of sharing knowledge. The explosion of the internet and digitization brought us a massive amount of data, e.g. big data. Also, the rise of the game industry made an initiative to make better chips and GPUs to run better graphics that serve customers. In 2006, a core group of AI researchers had started to notice this kind of hardware and started to use it to boost the performance of algorithms. V100 was the next successor of K80s with a promise to reduce the amount of training models and increase models’ performance. TPU was created but specialized for TensorFlow. That enables a generation of graphical processing units.
I was working on a reinforcement learning based generative model and synthesis data. The high level idea is to combine two architectures called autoencoder and GAN. It was traced back to a research group in France working on optimal theory. I am quite familiar with the key paper cited in the reference. It was my frustration in undergraduate research. Indeed, there were only a few papers to read.
GAN at the early stage was trained by 20 machines in the Professor’s lab. Despite proposing an idea, the first author still doubts about the ability of training a good model. Theoretically we would say that yeah it would be possible, however, the reality was quite different. The first early interesting result of GAN indeed came from industry where companies could buy extra GPUs to run experiments. In a rich lab, probably we would have access to up to 4 GPUs with a very humble technical team. In industry, this would be a very different situation with hundreds and thousand cutting edge GPUs with a well-trained engineering team.
Somebody did not believe that I was an expert. They kicked me out of the border because I look poor and mis organized. I did agree because I mis-read the US time system as if it was in the Quebec system. I did not have enough proper preparation and just ran out of the research institution to shortly attend a seminar and say goodbye to a few. However, the officer who took my cases also had a problem. She worked in the only dark room in the whole area. Despite holding my passport for several hours, she picked up the wrong pages and crossed my old visa which then resulted in my second escort at the airport two weeks later. I didn't even know what happened. Flight attendants did not know what happened until I showed up again at the gate and showed them all my documents before I went straight to a complaint office at the airport. Because of this, I could not return to San Francisco but this was how I kicked my ass with a world trip. What an epic scale.
I was kicked out of the research institution by trying to state my relationship with an institution. My new grant was canceled. Luckily, this was not the first time I had experienced this situation. I expect to see this happen especially when doing something new that a very few people could understand. The common reaction would be on an extreme of emotion on both sides of reactions. Excitement and anxiety. Joy and fear. I have seen this so frequently that I could tell what was behind the scene and driven by what. I personally don’t see myself fitting into an image of a positive figure. Very often to escape that hype cycle of AI, I just don’t wanna say anything at all and just do.
All of those made me stay in Montreal for more than a week. I suddenly became a mentor of a hackathon of DeepMind, at least last time my team won $10K of healthcare in the biggest hackathon in a city. And indeed it was not the first healthcare related event I joined, so I had something decent to say. Indeed, talking about a case study like Theranos, I have stories of real people working in healthcare to explain what happened rather than reading a book of a Wall Street Journalist or HBO Documentary, then believing that I knew it all.
In a hackathon, we were talking about how to apply AI4Good by picking up case studies from the local non profit organization and charities.
Sometimes I think Data Science is more like an art than science when your job is more telling an interesting story after mining a bunch of messy data from multiple sources. Not only that is about communication to inform a decision maker at the level they would understand and know how to take a direction. During a conversation with other mentors, somebody talked about climate change and then I started to remember an organization of culture and environment.
This happened because I moved around and talked to people which then made a bunch of interesting projects outside my original interest. In the past ten years, there were several many different things, e.g magazines, knowledge bases, awards, charities, organizations, and companies. One of them was an environment and culture project I worked on several years ago, starting as a small project of solving problems caused by climate change.
I don’t think competitive advantage is by hiding information, but finding the right problem to solve. This is a lesson I learned a long time ago with mathematics. It is not about memorizing all fancy solutions to ace the exam by making it look nice. It is about an ability to find the right problem to solve given resources and timeframe. I personally believe that this is how I spotted GAN at a very early stage when most experienced researchers would kill an idea when it was incubated. I did not rely on textbooks but my own experience working on several frontier of research, especially math to tell my own version.