Philosophy of work
Research Philosphy
“When art critics get together they talk about Form and Structure and Meaning. When artists get together, they talk about where you can buy cheap turpentine” - Pablo Picasso
I want to work at a place where we can talk turpentine. I’m interested in training neural networks not because it’s a science, but because it’s an art! Getting to become a numerical detective and whisper to the ghost in the machine makes battling nvidia drivers well worth it. Whether its designing a machine learning system from 0 to 1, or squeezing the last drop of performance out of the latest SoTA architecture, I’m in it for love of the game.
If real estate is all about location location location, I’d posit that research is about collaboration collaboration collaboration. Working with other researchers is important to me not just for sharing the hard-won tricks of the trade, but also for sharing all the negative results from trying things that just didn’t work. There’s so much ground to cover in modern ML systems, touching on everything from compression, information theory, linear algebra, projective geometry, computer architecture, inferenence optimization, etc. I’ve done my best to become at least aware of the different levels of abstraction, and since I can’t be an expert in all of them, I love working with people who are. In particular, despite my math background and appreciation for theory, I have a tremendous respect for the CS-minded engineers that understand all the levels that I’ve been lucky enough to be abstracted away from, like how a USB keyboard works or how an ISA interfaces with the output of an LLVM program.
On Shape Rotators
Building geometric intuitions for the types of transformations neural networks do has been a cornerstone of my approach. I try to stay grounded with solid fundamentals and a willingness to ask dumb questions. I find the often times if the problem is framed correctly, the rest of the work just becomes project management.
In college, I learned about the Vajrayana Buddhist meditation practice that involves picturing complex geometrically detailed, fractal-like mandalas. The practicioner sits for minutes or even hours, holding the image in full 3d dimensions in their head, examining and relaxing into perfect concentrative witnessing, effortless examination that lets one see all the angles simultaneously.
As the complexity of the figure grows with the skill of the meditator, a certain type of mental relaxation becomes a prerequisite for actually picturing the thing. To perceive all the subtle facets of the shape as it slowly rotates before you, you need have a familiarity with the territory. However, you also need to relax a bit to perceive the gestalt and finally hold all of its complexities in a way that isn’t so mentally taxing.
At the end of the meditation, they dissolve the complex shape in their head back to dust and open their eyes. My understanding is that the point of the exercise is not to worship the figure, but rather to practice letting your mind bend into whatever shape you ask it to in order to open yourself up to the goodwill of the universe.
Building geometric intuition for how machine learning architectures transform space is much the same. It’s very hard to rotate a ball in your head under duress. Part of letting the garden grow is just letting!
With a steady intuition and a willingness to look at problems without grasping for answers, I find that the solutions often present themselves. If Bezos espouses “disagree and commit”, I say “intuit and collaborate.”