When I check up on how my friends are doing -- friends from all over the world -- we talk about our social lives, our families, our friends, our partners and of course our work. They talk about their jobs in the private sector, the cool projects they are working on, the network events and so forth. Because they run through projects quite quickly, they have this idea that this is how life works. So inevitably, they ask me how my projects (research) are going. And this question is always followed by one of the most dreaded in academia: "have you published yet?" I am afraid to tell you that I have in fact not published yet. My first research was run in November 2017. Yes, that is effectively one and a half years ago. That is a while. Most projects in the private sector don't take that long to run fully, let alone if we are only talking analysis and writing up. Academia, unfortunately, just seems much slower. Not only does it seem slower, it just is. And there are a few reasons for that. The most prominent ones being the setting up of the study and the process that is publishing.
Setting up a study If you have ever had an idea, any idea, and wanted it to exist in reality, you know there to be a massive difference between theory and practice. In theory things work, in practice they are more likely to fail. Testing an idea is much the same. What you will have to come up with is a plan of testing something. If you are doing "good" science, this plan will be well thought out and pre-registered. Pre-registering is the process of typing out every single detail of the study in itself: - how will people be recruited for the study
- how many people will be recruited
- what will the people have to do
- what are the stimuli they will be exposed to (and upload those if possible)
- what are the exclusion criteria for people not following the study correctly
- what analysis will be conducted on the data collected (and why)
This may seem like a rather easy and arbitrary list of things to keep in mind. However, all these things will have to be mentioned and justified in the write-up as well, so there needs to be a lot reasoning and intent behind each answer to these questions. As such, this process take a long time. It takes even longer if the study conducted is experimental and happens in the lab. Meaning the experimenter will have to code, test, re-code (repeat) the study until it works satisfactorily. This in itself can take months.
Conducting the study Now your initial set of months is up. The experiment has been coded, it runs well (doesn't always happen, bugs can cause a lot of delays). What will happen now is that the study will be published online and data collection will happen, or the study will be done in the lab. With online collection you just have to wait, with lab experiments you often have to be present when the experiment is running. You will have signups for different timeslots, not all people will turn up, they just never do. So if your study would run for 3 days, assuming that each timeslot would be filled fully, expect the study to run closer to 5 days. Duration can more than double if the experiment required people to be matched to a certain number of others (meaning the amount of people in a single timeslot has to be divisible by a certain number, say groups of 4). If the study has multiple sessions, meaning a participant would have to show up to three seperate occassions, expect a massive drop-out rate, so the study will take much longer than anticipated. It's dirty work, but someone's got to do it.
Analysis Now, however long it has taken, it is complete! You have achieved the sample size you wanted. Good stuff! Now comes the analysis. The speed of this process depends on a couple of things, starting with your own skill level. If you are good at coding, this will go a lot quicker than if you are not that good at coding (me) and need a lot of help. The pre-registration can help a lot with this, as you have essentially specified what you were going to do before you even conducted the analysis. But, this can backfire. In my first study we pre-registered an analysis that did not work. It didn't work because of how the data was distributed. We assumed a normal distrubution in responses, but that wasn't the case. The entire analysis went out the window. This part had to be started from scratch again, and as such took longer. Yet another delay. I think by this point, in study 1, we were one year in from when we started with the idea itself.
Another thing that can really complicate this process: agreement amongst all parties. Different people have different ideas as to what needs to be analysed, what needs to be graphed out, put into a table and what is not relevant. You'll have many discussions with your supervisors sorting this one out. And yes, that takes time.
Write up The writing process, just like the analysis, depends mainly on skill level. Are you a good writer? Maybe the better question is: are you a fast writer? I write fast. So I can present my supervisors with new drafts every day if I needed to. If you struggle with this a lot more, make sure you have good examples you can copy from. Where do you find those examples? Well, that depends on which journal you want to publish in. Some people write up without a journal in mind, they have a general version of the entire paper. Then they select a journal, and a category of paper within that journal. They then adapt the paper to fit the criteria of that journal and its style. Others don't have a general version, they select a journal, and then write around these criteria, they do not write beforehand. The latter might seem quicker, but there are advantages to having a more general version to go from, as we will discuss in the section on publishing. Writing does not just happen when the study and analysis are done. Most of the literature tends to be reviewed before the study is even set-up, where else do the idea and methodology come from? A lot of writing can also happen in between steps. When stuck in coding the experiment or the analysis, often people look back at the literature review, or the methodology to figure out issues. Often, to keep moving and not feel like they are doing nothing (being stuck), they will write out these sections. A lot of the paper can be written before the experiment is even conducted. Although it will require quite a lot of edits. All in all, it is difficult to estimate how long this will take. Again, you can run into the issue of your supervisors not agreeing on how things need to be put, but journal selection will cancel out a lot of these arguments. Publishing
One version of the paper is finished. Excellent! I would recommend you take a moment to appreciate how far you have come. Now the paper as it is, should be in the right shape to submit to a journal. So let's do that. Going into a submission portal, you need about 200 different University accounts, but once you're in, you're in! Upload your documents, making sure you stick with the criteria, fill in all your details (and those of co-authors) and select your reviewers and editors. Oh yeah, did I not mention that before? Another discussion to have with your supervisors: who do they know in this topic? This step requires a lot of researching and soul-searching, but if you know who to select you should be good. Select, upload cover letter (not necessary for each journal) and submit. Hallelujah! Now you wait. About 75% of papers get desk rejected. This means that the main editor looks at it, finds fault with it, and let's you know within two weeks that the paper is not up the standards of the journal. The remaining 25% goes to round 2, where they are reviewed by the other editors. They decide whether the paper will be send to the reviewers. Again, the majority does not make the cut. This process takes over another month. Then, if you are one of the lucky few, your paper will be reviewed by your selected reviewers, which can take another several months. And then, after months and months of waiting, you can still be rejected. What then? Well, depends. Some comments (by those who rejected the paper) can indicate what you need to do. Some research benefits from having more experiments in the same paper, increasing the robustness of the result. Others need a better analysis. Some are just not a good fit for that particular journal. In the latter case, you have to re-write parts of this paper to fit a different journal and go through the process again. Otherwise, you might need to start again from much higher, conducting more experiments, or doing the analysis (and as a result a lot of the write-up) again. This costs a lot of time. As you can now imagine. So far, the time passed between the conceptualisation of an idea, and actually getting published has been a long time (over a year) already. And this is normal in academia. Within the field of economics, the process of publishing itself is already multiple years, so it's even longer. Is it surprising that when academia finally puts something out, it might no longer be relevant?
Additional issues: If you thought I was done, I am not! I started this process by discussing the set-up of the study, going from idea to reality. It is a good starting point. But from there, I move to the conducting of the study. There are however, two steps in between there that I should not fail to mention:
Ethics Do you remember the study of Milgram from 1963? He made people believe that they were administering shocks to someone in a different room (a confederate). A lot of participants in that study increased the voltage to such a level that the person "died". These participants left the experiment thinking they had killed someone. These types of things are no longer possible in 2019, because of the ethics committees that we have.
Before a study can be conducted it needs to have been approved by such a committee. They go through the stimuli used, information needed from the participants, information given to the participants, how the data is stored etc. The filling in of these forms takes long, but the waiting for this committee to meet once every so often, and have them decide on your research takes longer. This can also take several weeks (or months) and again, can lead to rejection, needing resubmission. There go another several months of your life.
Funding Morals and money seem to go hand in hand. At the same time that you are thinking about ethics, you should be thinking about, and applying for funding as well. Unless you can run your study for free. Participants often expect to be paid, and the materials for the study might cost money as well. Some funding is continuous, for example grants given to your supervisors. These are easily accessible and don't take time, if they agree to invest it in your study. Some funding applications only run once or twice a year and have a clear deadline. After the deadline has passed, the decision is communicated quite quickly, but the process of application is long. Why is it long? Well, you are competing against other researchers for what is often quite a big pot of money. So your application does have to be near perfection. Perfection takes time. So that's why.
So this is why I haven't published yet, my dear friends. Please do use this article to explain this to your non-academic friends and family who just do not understand why this is taking 16 years.
What can I say? It's a process.