Bad research?

I got this question:
"One of my committee members was suggesting that I change the intro and lit review based on my findings. Specifically, since I found that independent_variable2 is a good predictor of dependent_variable and independent_variable1 really doesn't predict dependent_variable -- refashion the front end as if I never had the original hypothesis. I know this is bad research but I am not sure if I should just do it."

My general advice was to do whatever seemed to lead to the degree most quickly. That's what the dissertation is really about.

But what about the ethical question? OK, so the results didn't show what you expected to find, but you did find something significant. Is it "bad research" to do post hoc data analysis? It depends on what your standard of knowledge is. What theory of the creation of knowledge do you aspire to? And what level of stringency do you apply to a dissertation as a research project?

Firstly, the dissertation. What do you think the point of the dissertation is? To produce a great study that wows everyone? That would be nice. Or maybe it's a project to ensure that you know how to manage an entire research project responsibly? If the purpose is the dissertation is the second, is it a violation of this principle to do a post hoc analysis?

Secondly, standards of knowledge. How do we know and what constitutes good research? On one level we can say, as I blithely did to the questioner, that research is not about what you hope to find; it's about what you do find. Darwin didn't look for evolution. On another level, we can can argue that science and research are inherently flawed and don't live up to the high standards they often claim. I like Paul Feyerabend's Against Method, which argues: "Science is an essentially anarchistic enterprise: theoretical anarchism is more humanitarian and more likely to encourage progress than its law-and-order alternatives....This is shown both by an examination of historical episodes and by an abstract analysis of the relation between idea and action. The only principle that does not inhibit progress is: anything goes" (from the analytical index to Against Method). The emphasis is Feyerabend's and bears repeating: "anything goes." Feyerabend, then, would argue that using post hoc analysis when appropriate is merely following the best principles of research. I have never seen any convincing refutation of Feyerabend's objections to some of science's claims that it is a strictly rule-based enterprise (and one that labels post hoc analysis as "bad research"). Admittedly, I have not looked specifically for such refutations (beyond reading a little of the work of Imre Lakatos, whose opposing views motivated Feyerabend to write Against Method, and who then presented his own critique of the book). What I have seen is many convincing theories that suggest that Feyerabend's claims fit within a larger rubric of science as a social activity that is influenced by political and social interests (including Kuhn's use of the idea of paradigms in Structure of Scientific Revolutions, Foucault's Order of Things and Latour's Science in Action).

My take on it it this: yes, it is in some ways preferable to do only the analyses that you plan prior to running the study. But, in truth, the point of running a study is to see what you find. If you're looking for relationships among several variables as part of your planned analysis, and you don't find the ones you expected, then it is your responsibility to report that lack of finding. But if you find another relationship that is significant, is there any reason not to present it?

Beyond that, there is the somewhat awkward question of writing a study as if you intended to find a different result than the one you intended to find. I feel a little differently about this. I wouldn't necessarily want to rewrite without any hint of my earlier intentions. I might present the material a little differently--as if I had gone to study a general question, and as a result of looking at multiple variables of interest, I had found only one result of interest. In other words, if I was primarily interested in variable1, but had also studied variable2, I might not write "I wanted to study variable2"; I might, instead, write "I was interested in variable1 and variable2 to see if they were playing any role in dependent_variable."