Spreading and evaluating risk in experimental project planning
We can’t control the significance of our results, and some experiments may just not work out as expected. But we can spread out the risk by working on different projects and methods, e.g. not spending too much time trying to get something to work that just isn’t working but consider other methods or approaches to test the same question. At least half of planned PhD experiments and all MSc or BSc projects should be low-risk.
In most cases I should be able to advise on whether an experiment is high risk or low risk and be able to outline potential problems that might arise. But if you are really striking off on your own with a new method for our lab, make sure to get expert advice from someone with experience in the protocol on both the likely problems which might arise as well as the expected timeline.
What is a low-risk experiment?
Low-risk means that the experiment is designed in such a way and to test such a question that even negative results should be analysable and publishable. By negative results I mean here results that disprove your original hypothesis, or that show that the effect you were looking for is not observed, but for which we still obtain quantitative, conclusive data. Although disappointing, negative results should only be a problem for publication if we cannot conclusively say that our results are not due to experimental error. If we can stand behind our negative result with confidence and it still tells us useful information, then this is a ‘low-risk’ experiment.
An example of a low-risk experiment would be to test if unreduced gamete production is elevated in interspecific hybrids relative to their parent species. This is safe because we can easily measure unreduced gamete production using several different methods, so if we already have the interspecific hybrids (or know that we can easily generate these from a specific cross combination) then we know we should be able to support or reject our hypothesis based on our experimental results. Of course, it’s less interesting if our hypothesis is not supported, but it still contributes valuable knowledge to the scientific literature. Another low-risk experiment (which I have frequently given to MSc and BSc students over the years) is to assess fertility and chromosome inheritance in small interspecific hybrid populations. If we already have the seeds, then growing a novel set of hybrids is low risk in terms of obtaining an experimental population, and methods to assess pollen fertility and count chromosomes from root tips are already well-established, easy to learn and quickly generate quantitative data which can be analysed and effectively used for thesis write-ups.
We can improve our chances of designing a low-risk experiment by reading deeply in the literature around our planned project. What have other studies found already? Is it very clear that there is a “research gap” where our experiment will definitely contribute novel information to the scientific literature? How well-supported are our hypotheses? Some hypotheses are “safer” than others, as in more likely to be supported, but the only way we can know this in advance is if we are very familiar with what has already been found out from the literature.
What is a high-risk experiment?
An example of a high-risk experiment is any experiment which aims to produce a specific result. If your experiment can be formulated as ‘we want to obtain or demonstrate x’ then the risk is that if you fail to obtain or demonstrate x then this doesn’t give us any new, useful scientific information. One example is ‘we will get floral dip transformation working in rapeseed’. I have met at least two researchers who tried to get this working and failed (think how many more people probably tried it unsuccessfully…) but there is no report of this in the literature (that I know of). It’s not inconceivable, but it’s difficult to publish a failed attempt to get a protocol working. The question is always ‘well, maybe you just needed to try different conditions?’, but this is not sustainable (or worth your time in the PhD) to try every possible experimental variation. So, in general, aims-driven research is higher risk than hypothesis-driven research (where it’s still possible to publish a result that fails to support your initial hypothesis, provided this is also not likely due to experimental error).
Another factor to consider which increases risk is if you have an experimental plan which relies heavily on the first step working to reach the second step, or the second step working to reach the third step etc. If you need all of x, y and z to work in order to test your hypothesis properly, and there is a reasonable chance that one or more of these won’t work as expected, then this is also a high-risk experiment. An example of this is any project where we want to validate gene function using knock-out mutants (where we are starting from the beginning of the process). In order to validate gene function, we need to be able to observe a phenotype in individual plants which are (usually) homozygous for loss-of-function of our gene of interest. In order to get plants which are homozygous for a loss-of-function mutation, we need to make sure we have no working copies of the gene anywhere in the genome. In polyploids (or for genes which tend to be present in multiple copies) this means we need to generate or order knock-out mutants for each gene copy, then cross between lines and self-pollinate with selection for the gene mutations we want until we have at least one individual with loss-of-function mutations in every gene copy. If even one of these mutations does not produce a loss of function as expected, or if we miss a gene copy that turns out to be functional after all, then we can’t go ahead with our phenotypic screening. So, this is an example of a higher-risk experiment, since validation of the phenotypic effect of the gene requires that we first successfully generate or screen mutant lines, and that each of the mutations has the predicted loss of function effect.
Does this all mean we should avoid high risk experiments? Not necessarily – if you have considered all the risks involved and are still keen then this is fine, you can take responsibility for your own research. But high-risk experiments should also be high gain, and for a PhD for instance the majority of the experimental work (let’s say at least half!) should fall into the ‘low risk’ category. Realistically, higher risk experiments should probably be mostly left for postdocs, who will be balancing multiple projects anyway and putatively have additional experience already with specific methods that might be required, or to technicians (for tasks like setting up and troubleshooting protocols or generating and validating experimental material, or where several experimental steps need to be done in advance before we can test any hypotheses).
PHASE Lab 