Methods vs Insights #4: The four stages of a project (and the fifth you should avoid)


Methods vs Insights is back. Today with a discussion of general research practice.

Most projects in my lab take years from start to finish. So it is important for me to manage the expectations my students and postdocs may have. Here is a plot I have developed to discuss the different stages of a scientific project with them and to prepare them for what’s ahead.

The

The four stages of a scientific project: Explore! Dig! Refine! Sell! And the stage you want to avoid: Waste! Plus the prevalent emotion in each stage and the key skill you will need to successfully navigate it. (x-axis it time, y-axis is work you’ve put in.)

Stage 1: Explore!

The first phase is the exploration phase. You need to define your research question, come up with original approaches, and work out a detailed outline of how the most promising approach to answer your question might look like.

Even if you work on a project your advisor has given you, you need to take full ownership. So use the first months to familiarize yourself with the problem and to try different approaches. You might feel very confused for a while, but after a few weeks/months you should have a good idea of (i) why the question you work on is important and (ii) how you want to approach it.

Go to seminars outside your particular focus. Every scientific problem has already been tackled from different perspectives and getting a bigger picture will help you understand your problem better. Even before you have data, taking the exploration step seriously will ensure that you work on a question worth your efforts. Make sure this phase is not too short.

The key skill in this phase is creativity to think about your project and ways to solve it. The prevalent emotion at the beginning of a project is a mix of curiosity and confusion. Hopefully the curiosity stays. If even after a few months you are still confused and unhappy with the project, you need to talk to your advisor. Don’t waste time on a project that you think is not important or unfeasible.

Stage 2: Dig!

The second stage is all about producing results.

Once you have developed a plan you can start to dig in. For most people this is the fun bit of the project, where you work hard but see progress and the first results come in. The main skill you need is just getting things done – simple as that!

Stage 3: Refine and validate!

Refinement and validation make a good project great. The third stage is the most important stage of your project and often the most frustrating one.

Maybe you have written well-documented and reproducible code already in Stage 2 (congratulations if that is the case) but most of us just have a collection of messy scripts that spit out raw results. And the figures you produced in Excel and Powerpoint for a lab meeting will almost certainly not be up to the standard of figures in Nature and Science. And just because one data set showed a significant result doesn’t mean others will. This is why you need to refine and validate your results.

Refining comes in many flavors and takes as much time as exploration and digging together. You need to refine your results, the clarity of their presentation, and proof reproducibility. Results: Do yet another case study. Test yet another data set. Compare to yet another complementary data type. Clarity: Remove jargon. Fix the colors in your plots. Align plot panels and remove white space. Make the axes labels readable. Reproducibility: Document your code. Produce an R package. Combine code and documentation in a Sweave file. Deposit everything at a public repository.

Yes, you need to do all these things on top of the results you already have. The biggest source of frustration I have seen is that people confuse the end of stage 2 with the end of the project. Understandably, they are exhausted and want to rush to the finish line. As a PI you need to stay tough and not be swayed by the excuses people might come up with to skip the refinement phase. I have heard quite a few:

`Impact factor is overrated. Let’s just submit to an easier journal!’

  • Wrong! Where is your pride in your work? Why would you want to submit half-baked results to a mediocre journal that accepts sloppily done papers?

`If something is missing the reviewers will tell us. Let’s just submit!’

  • Wrong! If something is missing, your paper will be rejected.

`It doesn’t matter how the figures look. Only the results matter. Let’s just submit!’

  • Wrong! Lazy execution of figures means that you didn’t put the right amount of effort into your paper – and the reviewers will see that. The same goes for lazy writing full of jargon and typos.

`No one will check the code. I will clean it up later. Let’s just submit!’

  • Wrong! Nothing is worse than having to admit in revision that you can’t reproduce your own results because you hurried the submission (happened to us, still hurts big time!). Documenting your code and cleaning it will help you to spot mistakes. And even if most reviewers will not check code and Sweave files, providing them shows your confidence in your work and that you value honesty, transparency and reproducibility.

In my group I have started to require Sweave files for all our papers to ensure that we can reproduce every single test and p-value we report as well as every plot we show. Code and data are bundled into R data packages, which we put on our webpage during review and upon acceptance deposit at Bioconductor (examples here, here and here).

If you are sloppy in the refinement phase you set yourself up for failure in the next stage: Selling your great piece of work to a journal!

Stage 4: Sell!

The final stage is getting your work published. This involves submitting and resubmitting it to (a series of) journals and interacting with journal editors and reviewers. From first submission to acceptance it can easily take a year.

Interacting with journals is closely linked to the refinement process, because you will need to defend your work against criticism and, in almost all cases, extend the results. Extending the results shouldn’t be too hard if you haven’t been lazy in the last phase. In my group, as soon as a paper is submitted, I work with the first author on putting together a todo list of further analyses and additional case studies that we can do while we wait for the reviewer reports. `Wait and see what the reviewers say‘ is not the best strategy; it is wiser to proactively produce more results – in my experience nothing has ever been wasted this way, we have always been able to use these additional results in some way when arguing with reviewers.

Stage 5: Don’t waste!

I hope you never reach this stage: When all the work is done and the project still drags on. Sometimes because coauthors, editors or reviewers are unresponsive, more often however, because there is still this one detail that could be fixed and this other sentence that could be stronger …

From all I’ve said so far it seems like I always push for more, more, more and you might wonder how we ever get to submit a paper. And indeed perfectionism is a great danger in science. At some point your job as a PI is to say ‘This is a story we can submit!’ — and actually do it. Finding the balance between refining and submitting can be hard. Dragging out projects can be hard on everybody involved and a disaster for the careers of postdocs who desperately need the papers their advisors are sitting on.

A long road to success …

So you can see there is quite a variety of skills you need to succeed and I have seen people fail for many different reasons. If you are not curious and creative you will have a hard time starting your own project. If you are creative but never commit to working hard on a single project, you will never dig deep enough. And even if you are very creative and hard-working but don’t have the stamina it takes to make it through refinement, you will be frustrated and demotivated very quickly. And, finally, even with highly refined work it takes confidence and salesmanship to get it published.

For a career in science you need to combine curiosity and creativity with hard work and stamina, as well salesmanship and confidence – that’s what I tell my new students and postdocs when we discuss this figure.

Florian

About these ads
Tagged , ,

3 thoughts on “Methods vs Insights #4: The four stages of a project (and the fifth you should avoid)

  1. […] B-Sides brings to you the Four Stages of a Project infographic. With pithy observations and useful […]

  2. […] a career in science you need to combine curiosity and creativity with hard work and stamina, as well salesmanship and confidence […]

  3. Reblogged this on Political Science Replication and commented:
    Reproducibility is one of the hardest stages in a research project. These are tips for writing successful papers in any field.

You gotta talk to me!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: