How credible are your research sources?

It might come as an unpleasant shock to some, but you actually need to research the research you want to cite. While "trust the science" has become an unofficial motto in a post-pandemic world, like the humans that produced them, they're not infallible.

Research usually falls into two categories – qualitative (non-numerical data collection) and quantitative (numerical data collection). The quality criteria for qualitative research are credibility, transferability, dependability, and confirmability. Quality criteria used in quantitative investigations are internal validity, external validity, reliability, and objectivity. While the factors determining their legitimacy differ, the core principles are the same.  

Purpose

The purpose of the study itself will influence its methodology, and its result by extension. This is even more applicable to qualitative research, where there's more room for human interference. 

Author

It's necessary to determine whether the people who conducted the research had the relevant expertise for it. Checking the source of funding is a good option in this case. 

Industry-sponsored studies tend to be biased towards providing favourable results of the sponsor's products. A famous example is Coca-Cola's history of funding scientists who downplay the risks of sugary beverages on obesity.

Bias

Selection bias is a common phenomenon that occurs when the researcher includes or excludes relevant groups or data while conducting the study. People volunteering (self-selecting) for the study will also lead to highly skewed results, for instance, online surveys. Interpreting information in a way that fits a preconceived hypothesis, known as confirmation bias, has been observed in political, financial, and organisational contexts and research is no different. 

This is why it's important to observe who is conducting the research, as their interests might influence their conclusions. Design bias happens when the structure of the study is flawed. You can usually see this type of bias in flawed research questions.

Sample size

A small sample size likely won't be representative of the larger population. This is why large sample sizes are heavily preferred in quantitative experiments. 

On the other hand, qualitative investigations can get away with relatively smaller samples, since each respondent will likely yield a lot of information to be analysed. 

Methodology

For qualitative experiments, you should check whether multiple data sources and methods were used or not and whether the participants had enough time to answer the questions. 

For quantitative studies, internal validity needs to be scrutinised. Internal validity refers to how accurately results reflect the studied group. This means having control groups and a sufficient sample size. 

Peer-reviewed articles are generally considered trustworthy since it adds another layer of quality control.

Transferability

When the study passes all other checkpoints, it's time to ponder the most important question – does the conclusion apply to your context? For example, the results of a study measuring stress levels of high school students in Germany cannot necessarily apply to students in Bangladesh.

It's important to note that using unreliable sources can make your work less credible by association. Aside from ethics, blindly following research without fact-checking can also influence your personal life, especially considering how quickly misinformation goes viral on social media. Healthy scepticism is celebrated even in the scientific world, so why not practise it? 

Ziba Mahdi is your resident pessimist. Cheer her up at mahdi.ziba@yahoo.com