One thing that I always find challenging is how to handle changes in the analytic plan. If you send the first of result to the writing group to be discussed and they come back with "wouldn't it make more sense if . . ." then what do you do?
In one sense, this sort of constructive feedback can improve our understanding of an association and improve the paper. On the other hand, this rather does make the "p-values" less clear. If you pick the association with the lowest p-value are you optimizing on how to best present an association or are you picking a result that is optimized on the distribution of noise in the data.
It is pretty clear to me that with a pre-specified test of an association that you should stick the analysis plan. But what if you are exploring? Is there a rule for exploratory analysis?
No comments:
Post a Comment