Theory
In doing research work of a more theoretical nature, it is easy to get caught up in technical details that are not directly related to the problem of interest, but rather a part of the "machinery" needed for doing formal analysis.
As a graduate student, I often recalled the words of Frank Spitzer, as passed on by J. Michael Steele, "It's not the theorem that is important, but the phenomenon." This maxim guided me through my doctoral studies, in which I dove into a lot of mathematical statistics and classical probability theory, struggling to weed out the deep, interesting, and important results from what might be called academic clutter. A certain degree of "maturity", or perhaps "literacy" in the technical elements of your discipline is necessary, and looking back, it took me a few years of hard reading in graduate school to get to that point.
Nearing the end of my doctoral studies, in my spare time, I came across the work of Robert Paul Wolff, including his online writings. On his website, called The Philosopher's Stone, in a March 2015 post, he wrote a short story called "The Parable of the Butcher and the Analytic Philosopher." Just a few paragraphs in length, this short parable beautifully captures how I feel about tackling papers in mathematical statistics and theoretical machine learning.
The Parable of the Butcher and the Analytic Philosopher
Robert Paul Wolff
A contest was announced to see who could do the best job of carving up a side of beef. The judge was announced as a famous chef, who had earned two Michelin stars. Attracted by the prize money, a butcher and an analytic philosopher entered the contest.
The Analytic Philosopher went first. A fresh side of beef was placed on a large wooden table, and he approached to begin. He was dressed in freshly pressed chinos and a button-down shirt. The Analytic Philosopher laid a leather case on one corner of the table and opened it, revealing a gleaming set of perfectly matched scalpels, newly sharpened. He selected one scalpel carefully and addressed the side of beef. After inspecting its surface carefully, he raised his hand and made the first cut, a precise slice in a perfectly straight line. Working steadily, but with meticulous care, he proceeded to make slices and cross slices until he had completed the carving of the beef, a task that took him the better part of an hour. When he had finished, he stepped back, wiped the scalpel clean on a piece of paper toweling, replaced it in the case, and with a bow to the judge, withdrew.
The butcher was next up. Her side of beef was on a table next to that on which the Analytic Philosopher had been working. She was dressed in overalls and a butcher's apron, on which one could see spots of blood and stains from her work. She took out a cleaver, a saw, and a sharp butcher's knife, and went to work on her side of beef, wasting no time. Bits of fat and gristle flew here and there, some ending up on her apron and even in her hair, which she had covered with a net. She whistled as she worked at the table, until with a flourish, she put down her saw, bowed to the judge, and stepped back.
The judge examined each table for no more than a moment, and then without the slightest hesitation, handed the prize to the butcher. The Analytic Philosopher was stunned. "But," he protested, "there is simply no comparison between the results on the two tables. The butcher's table is a shambles, a heap of pieces of meat, with fat and bits of bone and drops of blood all over the place. My table is pristine -- a careful display of perfectly carved cubes of meat, all with parallel sides and exactly the same size. Why on earth have you given the prize to the butcher?"
The Judge explained. "The butcher has turned her side of beef into a usable array of porterhouse steaks, T-bone steaks, sirloin steaks, beef roasts, and a small pile of beef scraps ready to be ground up for chop meat. She clearly knew where the joints were in the beef, how to cut against the grain with the tough parts, where to apply her saw. You, on the other hand, have reduced a perfectly good grade-A side of beef to stew meat."
Moral: When butchering a side of beef, it is best to know something about what lies beneath its surface."
Modifying Professor Wolff's observation only slightly: this is also not a bad idea when doing theoretical machine learning. When reading some technical material, thinking about "what lies beneath the surface" lets one look beyond the story the author is trying to tell, and to look at the facts or insights that are relevant to the phenomenon of interest to the reader. This is difficult, especially when reading work from the "masters" of the field, since their work is of such high quality and their main arguments are very powerful. That said, it seems to me that in pursuit of wide-reaching insights and useful new technologies, a top-down strategy is best: begin with a novel approach to the phenomenon of interest, and descend rapidly through the existing technical literature, hacking away the irrelevant material like the butcher in Professor Wolff's parable. With the proper equipment in hand, one is then prepared to attack the research problem of interest.