Reproducibility is one of the pillars of science, it allows separate groups of scientists to perform the same experiments and come to the same conclusions. Despite the importance of reproducibility, studies 1 and 2 show that oftentimes the conclusions published in high-impact science journals are not reproducible. As iGEM is a synthetic biology competition, we wanted to find out if synthetic biologists also encountered similar problems. We reached out to scientists to ask for their thoughts on reproducibility, and whether or not an online protocol database would help to increase reproducibility in science. The survey was emailed to 155 recipients, the majority of whom were iGEM advisors and student participants. A link to the survey was also posted on the iGEM facebook page. In total, 55 responses were received, and were used to aid our design of ProtoCat.
We asked participating teams about their experiences with the protocols that iGEM provides on their website. Surprisingly, 78% of teams encountered issues with, or had to make adjustments to one or more of the iGEM-provided protocols that they used. Even though the iGEM-provided protocols are classic molecular biology protocols, and among the most commonly used in the field, a large majority of users had problems with them. With this information in mind, we wanted to gauge the appeal of a software like ProtoCat. We asked scientists whether or not they could see themselves using a multidisciplinary, online protocol database, with 73% indicating 'yes', 27% indicating 'maybe', and none indicating 'no'. Given the positive responses, we felt certain the ProtoCat would be appealing to scientists searching for protocols.
Respondents provided plenty of ideas such as giving users the ability to search for, review and rate protocols. We implemented as many of these as we could, placing particular emphasis on creating an efficient and easy to use search engine as many it seemed to be the most important part in the respondents' minds. More information on the results of the project can be found here.