Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion content/OS-developing-world/OS-developing-world.md
Original file line number Diff line number Diff line change
Expand Up @@ -248,7 +248,7 @@ The paper also directly informs FORRT’s ongoing work on:

## Peer-reviewed Publication

The open access **<font style="color:#0e2a38">publication</font>** can be found [here](https://doi.org/10.1177/25152459251357565), and the **<font style="color:#0e2a38">postprint</font>** can [be found here (osf.io/7ubk2)](http://osf.io/7ubk2). Download the supplementary material [here](https://journals.sagepub.com/doi/suppl/10.1177/25152459251357565/suppl_file/sj-docx-1-amp-10.1177_25152459251357565.docx?_gl=1*qdrt5e*_up*MQ..*_ga*MzI4ODQ0MDI1LjE3NjY0ODg2NjU.*_ga_60R758KFDG*czE3NjY0ODg2NjUkbzEkZzAkdDE3NjY0ODg2NjUkajYwJGwwJGgxOTI0NTEyNTAw).
The open access **<font style="color:#0e2a38">publication</font>** can be found [here](https://doi.org/10.1177/25152459251357565), and the **<font style="color:#0e2a38">postprint</font>** can [be found here (osf.io/7ubk2)](http://osf.io/7ubk2). Download the supplementary material [here](https://doi.org/10.1177/25152459251357565).

> Hu, C.-P., Xu, Z., Lazić, A., Bhattacharya, P., Seda, L., Hossain, S., Jeftić, A., Özdoğru, A. A., Amaral, O. B., Miljković, N., Ilchovska, Z. G., Lazarevic, L. B., Wu, H., Bao, S., Ghodke, N., Moreau, D., Elsherif, M., C., C., Ghai, S., ... Azevedo, F. (2025). **Open Science in the Developing World: A Collection of Practical Guides for Researchers in Developing Countries.** *Advances in Methods and Practices in Psychological Science*, *8*(3), 25152459251357565. [https://doi.org/10.1177/25152459251357565](https://doi.org/10.1177/25152459251357565)

Expand Down
2 changes: 1 addition & 1 deletion content/adopting/adopting.md
Original file line number Diff line number Diff line change
Expand Up @@ -292,7 +292,7 @@ Common structural barriers and how to address them:
* Use one of [FORRT’s lesson plans](https://forrt.org/neurodiversity-lessonbank/) that aim to promote Neurodiversity and Open Scholarship in Academia.
* Use resources such as the [BIPOC-authored Psychology Papers spreadsheet](https://docs.google.com/spreadsheets/d/1i7Eacoyv9VVg2lBbCV-KJZg4nSGvR_VZFOysOyOGG8g/edit?fbclid=IwAR1zlaZHcY1HMYYnaa5M96aC577qDagmphf_R7EH2YXBl_P1ihJcPu9zUPM&gid=666010790#gid=666010790%20), intended for use by instructors of undergraduate/ graduate-level psychology courses to help diversify their syllabi, or [the Diversity Reading list](https://www.google.com/url?q=https://diversityreadinglist.org/about/&sa=D&source=docs&ust=1749130331285358&usg=AOvVaw1xfd6fPDUWRJ2pzNFMxdf7) offering quality texts in philosophy by authors from underrepresented groups, or [FORRT’S annotated reading list](https://forrt.org/curated_resources/point-of-view-an-annotated-introductory/) supporting readers in understanding some of the key ideas and topics within neurodiversity.
* Use inclusive citation practices.
* Consider adding a [Citation Diversity Statement](https://www.sciencedirect.com/science/article/abs/pii/S1364661320301649) in your course materials or assignments, to acknowledge and intentionally include scholarship from diverse voices and underrepresented groups.
* Consider adding a [Citation Diversity Statement](https://doi.org/10.1016/j.tics.2020.06.009) in your course materials or assignments, to acknowledge and intentionally include scholarship from diverse voices and underrepresented groups.
* Use [McGill’s Citation Justice Guide](https://libraryguides.mcgill.ca/citation_justice/how_to) to help students audit their citation practices and intentionally seek out diverse voices in their research. The guide offers practical tools for analyzing whose work is being cited, identifying gaps, and finding resources to include scholars from marginalized communities.
* Read [FORRTs Manuscript](https://forrt.org/citation-politics/) on The Citational Justice Toolkit which offers practical guidance and tools to help researchers make more equitable and conscientious citation choices throughout the research process.
* Engage with *Towards Social Justice In Academia Initiatives:*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ This module is framed around a schematic version of how empirical psychological

In the schematic, you generate a hypothesis from theory, design a study to test the hypothesis, collect data based on the study design, analyze the data to test the hypothesis, interpret the results, publish the data, and begin the cycle anew. By using this process to compare discrepancies between your hypothesis and the data, you can identify flaws in your theoretical assumptions, which allows you to revise the theories and improve them.

However, a variety of events made psychologists aware that something about this cycle wasn’t working. First was the observation that only about 8% of the results published in psychology journals are negative ([Fanelli, 2010](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0010068)) – yet these negative results are necessary to identify flaws in theoretical assumptions. Second was the publication of a paper by the well-respected social psychologist Daryl Bem, who, in 2011, published a somewhat unusual paper in the world’s top journal for social psychology, the _Journal of Personality and Social Psychology_ ([Bem, 2011](https://psycnet.apa.org/record/2011-01894-001)). This paper used methods that met or exceeded the standards of rigor that were typical for the time, but advanced a claim that was patently absurd: that college students (and, by extension, everyday people) could be influenced by future events.
However, a variety of events made psychologists aware that something about this cycle wasn’t working. First was the observation that only about 8% of the results published in psychology journals are negative ([Fanelli, 2010](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0010068)) – yet these negative results are necessary to identify flaws in theoretical assumptions. Second was the publication of a paper by the well-respected social psychologist Daryl Bem, who, in 2011, published a somewhat unusual paper in the world’s top journal for social psychology, the _Journal of Personality and Social Psychology_ ([Bem, 2011](https://doi.org/10.1037/a0021524)). This paper used methods that met or exceeded the standards of rigor that were typical for the time, but advanced a claim that was patently absurd: that college students (and, by extension, everyday people) could be influenced by future events.

![Daryl's Bem "Feeling the Future"](fig3.webp "Daryl's Bem Feeling the Future")

Expand Down Expand Up @@ -91,7 +91,7 @@ In contrast, distant replications (where you vary some aspect of the method) are

I also use this exercise to highlight the importance of minimizing sampling error and fully documenting procedures; both sampling error and undocumented differences in procedure (“hidden moderators”) provide possible explanations for why replication results differ from original results. This highlights a somewhat hidden side benefit of replications: they force you to very carefully document a particular procedure.

To illustrate how to document the procedure behind a replication study, I introduce the students to the “replication recipe” ([Brandt et al., 2014](https://www.sciencedirect.com/science/article/pii/S0022103113001819)). The replication recipe provides a structured set of questions to guide the process of creating a method section for a replication study. As [an exercise](https://osf.io/j3pna/), I ask students to fill out the first section of the replication recipe with an article that I assign (I pre-selected two articles that are short and have a relatively simple research design). After the exercise, we discuss the process of using the replication recipe and identify issues that came up – including the poor reporting standards of most (but not all) psychology articles.
To illustrate how to document the procedure behind a replication study, I introduce the students to the “replication recipe” ([Brandt et al., 2014](https://doi.org/10.1016/j.jesp.2013.10.005)). The replication recipe provides a structured set of questions to guide the process of creating a method section for a replication study. As [an exercise](https://osf.io/j3pna/), I ask students to fill out the first section of the replication recipe with an article that I assign (I pre-selected two articles that are short and have a relatively simple research design). After the exercise, we discuss the process of using the replication recipe and identify issues that came up – including the poor reporting standards of most (but not all) psychology articles.

In the last part of this module, we discuss a way to choose replication targets. I teach a somewhat informal version of a framework developed by [Isager and colleagues (2020)](https://osf.io/preprints/metaarxiv/2gurz/). As [an exercise](https://osf.io/zjwcg/), the students use the framework to rate the value, uncertainty, and cost of doing the replication study that I assigned them.

Expand Down Expand Up @@ -121,7 +121,7 @@ As another example, when I illustrate the smallest-effect-size-of-interest workf
![Notes](fig9.webp "Notes")


One last issue that comes out of the simulations is the number of assumptions that one must make in the process of doing a simulation study. This includes both statistical assumptions, such as the size of the standard deviation of the outcome measure, and non-statistical assumptions, such as the length of time it takes for a typical participate in the study (a fact that is necessary to accurately estimate the number of participants who can participate in a lab-based study, for example). I argue that pilot studies are useful for developing good values for these assumptions. Pilot studies are _not_ useful for directly estimating the value of the target effect size itself ([Albers & Lakens, 2018](https://www.sciencedirect.com/science/article/pii/S002210311630230X)); in any case it is better to power to a smallest effect size of interest than the expected effect size.
One last issue that comes out of the simulations is the number of assumptions that one must make in the process of doing a simulation study. This includes both statistical assumptions, such as the size of the standard deviation of the outcome measure, and non-statistical assumptions, such as the length of time it takes for a typical participate in the study (a fact that is necessary to accurately estimate the number of participants who can participate in a lab-based study, for example). I argue that pilot studies are useful for developing good values for these assumptions. Pilot studies are _not_ useful for directly estimating the value of the target effect size itself ([Albers & Lakens, 2018](https://doi.org/10.1016/j.jesp.2017.09.004)); in any case it is better to power to a smallest effect size of interest than the expected effect size.

![Workflow 2](fig10.webp "Workflow 2")

Expand All @@ -130,7 +130,7 @@ One last issue that comes out of the simulations is the number of assumptions th

_Learning goals: To preregister something, create an OSF project & put the replication recipe in the registry. There’s evidence this helps make research more credible_

The bulk of this module is focused around completing a pre-registration for the article assigned to the students in the previous modules. Because the workshop participants have already completed the first part of the replication recipe ([Brandt et al., 2014](https://www.sciencedirect.com/science/article/pii/S0022103113001819)) for this article, they are already familiar with the article’s purpose and materials. For the first part of this module, the students complete the remainder of the replication recipe (or at least, as much as they can) as part of [the last exercise](https://osf.io/w35fp/) of the workshop.
The bulk of this module is focused around completing a pre-registration for the article assigned to the students in the previous modules. Because the workshop participants have already completed the first part of the replication recipe ([Brandt et al., 2014](https://doi.org/10.1016/j.jesp.2013.10.005)) for this article, they are already familiar with the article’s purpose and materials. For the first part of this module, the students complete the remainder of the replication recipe (or at least, as much as they can) as part of [the last exercise](https://osf.io/w35fp/) of the workshop.

The replication recipe in hand, the students can complete a replication recipe-based preregistration on the Open Science Framework ([OSF](https://osf.io)). I walk the students through this process and introduce them to the basics of uploading materials on an OSF page. But the students already completed hard parts of preregistration as part of the previous exercises.

Expand Down
2 changes: 1 addition & 1 deletion content/educators-corner/010-Neurodiversity/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ Grant, A., & Kara, H. (2021). Considering the Autistic advantage in qualitative

Grinker R R. (2010). Commentary: On being autistic and social. Ethos, 38 (1), 172-8.

Harding, S. (1992). Rethinking standpoint epistemology: What is" strong objectivity?". The Centennial Review, 36(3), 437-470. https://www.jstor.org/stable/23739232
Harding, S. (1992). Rethinking standpoint epistemology: What is" strong objectivity?". The Centennial Review, 36(3), 437-470. https://doi.org/10.1371/journal.pone.0175967

Jackson, M. (1998) Minima Ethnographica: Intersubjectivity and the Anthropological Project. Chicago: University of Chicago Press.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ There are echoes in this struggle of the dispute UCU has waged since before the

## Features, Not Bugs

Where do these analogous structural failings come from? The answer to this starts with deconstructing the idea that these are “failings,” rather than intentional components of a system that views degrees as commodities and people as cogs in machines. A system wherein the task of teaching is secondary to that of shoring up a university’s research reputation will not mind if teachers have to [live in their cars](https://www.nea.org/advocating-for-change/new-from-nea/homeless-professor-who-lives-her-car) or [move from city to city every semester](https://www.timeshighereducation.com/news/casualised-staff-dehumanised-uk-universities) in search of work. Likewise, a system where [glossy building façades](https://www.nytimes.com/2012/12/14/business/colleges-debt-falls-on-students-after-construction-binges.html) matter more than what happens inside of those buildings will not mind using historically excluded groups, particularly women of color, [as PR fodder](https://doi.org/10.1080/01419870701356015) while [exploiting their labor](https://psycnet.apa.org/record/2023-37109-001) until they burn out.
Where do these analogous structural failings come from? The answer to this starts with deconstructing the idea that these are “failings,” rather than intentional components of a system that views degrees as commodities and people as cogs in machines. A system wherein the task of teaching is secondary to that of shoring up a university’s research reputation will not mind if teachers have to [live in their cars](https://www.nea.org/advocating-for-change/new-from-nea/homeless-professor-who-lives-her-car) or [move from city to city every semester](https://www.timeshighereducation.com/news/casualised-staff-dehumanised-uk-universities) in search of work. Likewise, a system where [glossy building façades](https://www.nytimes.com/2012/12/14/business/colleges-debt-falls-on-students-after-construction-binges.html) matter more than what happens inside of those buildings will not mind using historically excluded groups, particularly women of color, [as PR fodder](https://doi.org/10.1080/01419870701356015) while [exploiting their labor](https://doi.org/10.1037/dhe0000462) until they burn out.

Others have written [far more comprehensively](https://doi.org/10.1177/1478210317719792) about how these features of higher education (not bugs) [have become ever more present](https://www.boldtypebooks.com/titles/davarian-l-baldwin/in-the-shadow-of-the-ivory-tower/9781568588919/) in the past few decades. My point is simply that they are unavoidable, regardless of country of residence. Some might claim that coming to the UK will bring reduced research pressures compared to US research-intensive universities, which may be true in terms of the number of publications required but overlooks how the UK’s massive (and growing) [grant-grubbing apparatus](https://annameier.substack.com/p/grant-culture) saps staff time and discourages counterhegemonic work. Others might note that US academic salaries are generally higher than in the UK, which may be true even at poorer institutions but masks the [massive (and growing) disparities](https://www.nature.com/articles/d41586-021-01183-9) across and within US institutions. Regardless, the “it’s better over there” narrative sidesteps the reality of the academic job market for many: that, if they are able, they will go wherever hires them.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ Nowadays, more and more researchers move away from NHST toward Bayesian inferenc

### A computationally feasible and intuitive way

In our [new tutorial](https://osf.io/preprints/psyarxiv/6zsx3_v2), we walk you through a lesser known form of Bayes Factor calculation using the Savage-Dickey approximation ([Dickey & Lientz 1970](https://www.jstor.org/stable/2239734)). This form of inference is not only conceptually intuitive and computationally efficient, but it also allows us to tackle another conceptual issue with traditional hypothesis testing: Researchers coming from NHST tend to test whether differences between conditions are smaller or greater than exactly zero (i.e. testing point-0 hypotheses). But not every difference that is not zero is meaningful for either theoretical or practical purposes. In our tutorial, we use a data set on pitch perception by Korean speakers in formal and informal contexts. The researchers want to know if speakers use meaningfully higher or lower pitch in these two social contexts.
In our [new tutorial](https://osf.io/preprints/psyarxiv/6zsx3_v2), we walk you through a lesser known form of Bayes Factor calculation using the Savage-Dickey approximation ([Dickey & Lientz 1970](https://doi.org/10.5517/ccdc.csd.cc2f5mhr)). This form of inference is not only conceptually intuitive and computationally efficient, but it also allows us to tackle another conceptual issue with traditional hypothesis testing: Researchers coming from NHST tend to test whether differences between conditions are smaller or greater than exactly zero (i.e. testing point-0 hypotheses). But not every difference that is not zero is meaningful for either theoretical or practical purposes. In our tutorial, we use a data set on pitch perception by Korean speakers in formal and informal contexts. The researchers want to know if speakers use meaningfully higher or lower pitch in these two social contexts.

To test this hypothesis, we suggest the following workflow:

Expand Down
Loading
Loading