Research Directions
July 6, 2026
Hi Tony,
This past week, I was searching a new research direction; bookmarking any and all tweets that half-interested me. I also some essays, one by Dr. Richard Hamming and another by Vlad Feinberg.
And at same time I reflected on what Edward said: do research that interests me. I realized (as I mentioned last week) that PuM had some serious problems. As Edward pointed out, if I wanted to have a model that could play Minecraft I should just “cheat.” Why can't I give a model Minecraft packets and save myself the trouble of rendering the game and having a model read pixels and so on and so forth. On the other hand, if I wanted to have a model that was could do things more “general” than Minecraft, the approach of “decomposing into blocks” or “training on how to predict inventory data” totally defeated that. Having a project in Minecraft might be “fun” but the way I went about it was not a good foundation for a paper. I think this was a great education, though: I got to train my own VLM starting from the data-collection, learned that I should use wandb (haha) and learned lots of other stuff I should do/avoid doing.
ANYWAYS over the past week I really thought about what my constraints were and --as Edward mentioned in our meeting-- what interested me. The two things that I always find myself coming back to are VLAs and interpretability. There was a passage in Hamming's essay that went like:
While I was at Los Alamos after the war, I got to thinking about the famous Buffon needle problem where you can calculate the probability of a needle tossed at random of crossing one of a series of equally spaced parallel lines. I asked myself if it was essential that the needle be a straight line segment (if I counted multiple crossing)? No. Need the parallel lines be straight? No. Need they be equally spaced or is it only the average density of the lines on the plane? Is it surprising that some years later at Bell Labs when I was asked by some metallurgists how to measure the amount of grain boundary on some micro photographs I simply said, “Count the crossings of a random line of fixed length on the picture?” I was led to it by the previous, careful thought about an interesting, and I thought important, result in probability. The result is not great, but illustrates the mechanisms of preparation and emotional involvement.
Hamming, You and Your Research
Hamming says that careful thought about a problem you find interesting helps you notice stuff later on… Well, what was my Buffon needle problem?
For a while now, the answer has been how models represent concepts. I follow Neel Nanda's work on interpretability. I keep coming back to the Goodfire Manifold Steering paper. I remember a lunch earlier this summer where a friend mentioned he was interning at a lab building on Goodfire's work, and I started talking about the manifold paper at length… he was surprised I'd already read it but it had been on my mind for weeks.
At the same time, I believe strongly that VLAs are going to be rly important. In like, 10-15 years, we're gonna have robots doing meaningful physical work irl and the models doing that will be descendants of what we're studying now. That's a compelling vision. Finally, more concretely, I love making clear visuals. For fun I'll do stuff like this recent post, where I finetuned a small model on DELTARUNE dialogue and ran some steering experiments on it. I want to do the same thing on VLAs.
The other essays (Feinberg) that I mentioned earlier talks about how I, at my age, should strive to have a visible artifact that demonstrates I can do work:
The most obvious way to get hired by a lab is to demonstrate that you have a specific skill that the lab requires.
Adding further that:
Papers are the highest-signal version of that. Well-written synthesis blog posts are a strong second -- they demonstrate reading depth, categorization ability, and taste, and they get read by more people than most first-author papers.
Anyways, here is my new idea for a research direction (and some blog post ideas). I thought of these by hunting for cool papers on Twitter and looking at their future work section and/or thinking about how I could apply them to VLAs.
Research Direction
My research taste
I'm interested in research that can have some cool visualization (think: Goodfire style) instead of benchmark tables. Harry so graciously offered 8 H100s of compute, but I'd really rather not depend on physical robot demos. Anyways I'm an undergrad and still figuring out my research taste so imo it's totally legitimate to go for something I think is “cool” (i hope) but I'm not rly interested in frameworks, systems papers, or lit reviews. (Think taste closer to a theoretical physicist doing than a roboticist.)
Main project idea: VLA activation manifold trajectories
Building on Goodfire Labs' Manifold Thesis, I want to see how a VLA's residual stream moves through activation space while it does a manipulation task, colored by phase (approach, then grasp, then lift, then move, finally place, etc.).
If it works, the trajectories should trace coherent paths through a shared manifold, with each phase living in roughly its own region. What would be even cooler is if, say, the “grasp” region ended up shared across (let's say across pick-place-cup and pick-place-block episodes). This would mean that the model actually learned a shared grasping representation instead of a separate one per task.
But even if not, that would tell me the VLA I tested on memorized tasks instead of generalizing. I think Physical Intelligence wrote a paper on how they got their model to generalize if another lab made a similar claim I could test my method on their model vs a model that was overfit. Could be fun?
I'd end up with smth like this (but for VLAs)
Goodfire Labs' activation-manifold UMAP, trajectory traced through it
Related Research
The Goodfire Meandering-on-Manifolds paper (Bigelow et al., 2026), obviously. Their claim is that LLMs represent stuff like emotions along curved manifolds in activation space, and as the model reads a story sentence by sentence, its activations trace a trajectory along that manifold tracking the story's emotional arc.
The VLA SAE paper (Swann et al., 2026) trained sparse autoencoders on $\pi_{0.5}$ and OpenVLA and found about 80% of features are interpretable and map onto task phases, sensorimotor events, that kind of thing.
According to Claude, 98% of those features on LIBERO are memorized rather than general, which is kind of a bummer honestly. I'd use their phase features as the behavioral readout for my manifold plot, sort of like how Goodfire has the LLM self-report “how much fear on a scale of 1-10.” Anyways I've read the Goodfire paper but not this one I'm gonna do that tmrw actually
Blog Idea: Verifying VLA generalization claims via features
Like I mentioned earlier, VLA papers love to claim their model “generalizes” based on success rate on some held-out objects, scenes, or instructions. I want to actually check that.
Lowkey this would be an addition to the above paper or like, a blogpost so someone skimming my site can get a feel for what I like to do.
Blog posts
Blog Idea: VLA failure detection landscape
There are like fifteen VLA failure-detection papers from the last year that all agree with each other. This idea is a blogpost that, just categorizes what everyone is doing...
Blog ideas are like, smaller stuff just for the personal site but I think getting a quick project out can be valuable just so I have something to show for what I'm doing. Do you agree?
Conclusion
Does the main project seem feasible? I have some other ideas too I can share but I don't want this to be super long! Happy to talk through any of this whenever works.
Vishnu