Meeting schedule

As the major theme of AbGradCon 2021 was international collaboration, a goal of the meeting was to facilitate connection building between attendees from different countries. The conference schedule was thus divided into three major regions based on time zone (an oversimplification driven by necessity to fit within the Japanese time zone of the organizers (Japan Standard Time, JST)): East Asia/Oceania, Middle East/Europe, and The Americas. Each scientific session was scheduled to include two of these regions, so as to not marginalize attendees from specific time zones (which is an issue of non-inclusivity of many digital meetings (Jack and Glover, Reference Jack and Glover2021)). Optimally, this meant that the conference would consist of three sessions spread out over the 24 h day, but the decision was made to optimize for the organizers hosting and running the event who were based in Japan, resulting in two scientific sessions per day (Tables 1 and 2). One session would be in the morning JST, which allowed attendees from the regions of East Asia/Oceania and the Americas (and all areas in between) to attend at a ‘reasonable hour’ (‘reasonable hour’ is loosely defined, spanning 0800 to 2230 in local time of the attendees). The second session would be in the late afternoon JST, allowing attendees from the East Asia/Oceania region and the Middle East/Europe region (and all areas in between) to attend at a ‘reasonable hour’.

Table 1. 14 September session schedule

Table 2. 15–17 September session schedule

Additionally, the schedule of 14 September (Table 1) was slightly different from that of 15–17 September (Table 2), as no contributed presentations were given on 14 September. The session timetable for 15–17 September was also identical for each day, although the content was different on each day. Ample breaks were designed into the programme for health reasons, so that participants could stand up from their terminals to stretch once in-awhile, as prolonged periods of remaining sedentary are detrimental to personal health (Cowgill et al., Reference Cowgill, Perez, Gerdes, Sadda, Ly, Slusser and Leung2021). We report the event times in JST, Central European Summer Time (CEST), and Eastern Daylight Time (EST) in Tables 1 and 2 to showcase the reasonability of the events for the selected regions' time zones.

Each plenary session (labelled ‘live-streamed presentations with Q&A’) covered one topic through three presentations, and generally plenary sessions with the same topic appeared only two times throughout the course of the meeting (i.e., once during a morning session and once during an afternoon session). Session topics varied widely, including biology, geology, astronomy & physics, planetary science, chemistry, and engineering & instrument design. This session topic allocation allowed the participants to be exposed to presentations from a variety of disciplines in astrobiology, which is inherently a very interdisciplinary field (Race et al., Reference Race, Denning, Bertka, Dick, Harrison, Impey and Mancinelli2012). Previous iterations of AbGradCon have included ‘warm-up talks’ given by a more experienced attendee (senior graduate student or postdoctoral fellow) at the beginning of each scientific session to introduce the session topic. Due to time constraints (the online sessions were much shorter than past in-person AbGradCon sessions), the organizers decided not to include ‘warm-up talks’ in the main programme, but note that past warm up talks would have been available for attendees to view on Youtube. However, ‘warm-up talks’ should be considered for future on-site conferences.

Meeting attendees

260 abstracts from applicants based in 34 different countries were submitted for AbGradCon 2021, all of which were either undergraduate or graduate students, postdoctoral fellows, or other early-career researchers. After a strict peer-review process (vide infra), 146 of these abstracts were accepted in early June, and ultimately 119 attendees registered for the meeting, representing 26 countries of residence (30 countries of citizenship) (Table 3). The abstracts submitted by these attendees included diverse topics. However, the titles of all accepted abstracts included 29 mentions of ‘life’, 17 mentions of ‘Earth’, 16 mentions of ‘Mars’, nine mentions of ‘origin’ or ‘origins’, and eight mentions of ‘RNA’, which shows the areas which were more focused upon by the attendees; interestingly, ‘implication’, or ‘implications’ were also quite popular words, with nine mentions as well (Fig. 2(a)). Similarly, we analysed the word-counts of all of the abstract texts, and found that they contained 252 mentions of ‘life’, 232 mentions of words containing ‘chem’ in any part, 150 mentions of ‘Earth’, 138 mentions of words containing ‘environ’ in any part, 105 mentions of ‘RNA’, 100 mentions of words containing ‘evol’ (short for evolution-related words) in any part, 90 mentions of ‘Mars’, 89 mentions of ‘condition’ or ‘conditions’, and 80 mentions of ‘early’; ‘can’ and ‘will’ appeared 122 and 93 times, respectively (Fig. 2(b)).

Fig. 2. Wordclouds (wordclouds.com) representing the frequency of words within the attendees' (a) abstract titles and (b) abstract text.

Table 3. Regions of residence of AbGradCon 2021 participants

For 96 of the 119 attendees, AbGradCon 2021 was their first attendance at an AbGradCon conference. 114 attendees reported gender/gender identity, including 56 women, 55 men, 1 nonbinary, 1 male/non-binary, and 1 agender participant. Each participant was also asked to self-disclose up to three fields that their abstract was relevant to in order to aid in sorting speakers and abstracts in the conference programme. Abstract disciplines included such descriptions as astrobiology (which was then altered to the secondary choice of best fit), astronomy, biology, chemistry, education/outreach/diversity, engineering and instrument design, geology, physics, planetary science, and others (Table 4). Additionally, all participants were required to agree to a code of conduct, which required them to pledge to behave in a professional manner, to report observed harassment and misconduct, to protect the privacy of prospective participants and their research (by not taking screenshots of presentations and not sharing information with those not attending the meeting).

Table 4. Fields of abstracts presented at AbGradCon 2021, as self-reported by the presenters

Each abstract was required to be associated with at least a primary field, but association with a secondary and/or a tertiary field was not required (10 abstracts were not assigned a secondary field, while 45 abstracts were not assigned a tertiary field).

Pre-recorded presentations

In lieu of completely live presentations, which have considerable risk of coming upon technical errors of all sorts, all participants of AbGradCon 2021 submitted pre-recorded videos as their presentations. Participants could choose to submit up to two types of videos: a 12-minute full presentation or a 1-minute flash talk. All flash talks were streamed during a plenary session of the meeting, while a number of full presentations were selected for live-streaming during a plenary session with a short question and answer session immediately following the video. By selecting pre-recorded videos and streaming these videos from a central account, no time was lost due to presenter transition; as a result, no session went over-time. Participant presentations were also uploaded to Youtube as unlisted videos and the links shared to all participants (and not made public) so that they could stream them on-demand. About one month after the conference, videos of presenters that agreed to have them publicly released were published on the NASA Astrobiology Youtube channel, while videos of presenters who elected to not have them publicly released were deleted. In total, 36 full presentations and 44 flash talks were live-streamed during the conference, with an additional 61 full presentations available for on-demand streaming.

Keynote presentations

One keynote speaker for each session (total of two speakers) on 14 September 2021 was invited to give a presentation entailing both their current research as well as their career path in astrobiology. In particular, as the meeting theme is international collaboration, desired keynote speakers would have extensive experience both working in different countries and collaborating with researchers around the world. Two speakers, who were not early career speakers but were instead established researchers (yet still somewhat early in their independent careers), who satisfied these criteria (while also being based in Asia, which could further potentially connect attendees from outside of the Asian region with astrobiology research communities within the region) were selected.

Prof. Hikaru Yabuta of Hiroshima University in Hiroshima, Japan is an expert in cosmochemistry and the leader of the organic macromolecule sub-team of the initial analysis in JAXA's Hayabusa2 asteroid sample return mission (Watanabe et al., Reference Watanabe, Hirabayashi, Hirata, Hirata, Noguchi, Shimaki, Ikeda, Tatsumi, Yoshikawa, Kikuchi, Yabuta, Nakamura, Tachibana, Ishihara, Morota, Kitazato, Sakatani, Matsumoto, Wada, Senshu, Honda, Michikami, Takeuchi, Kouyama, Honda, Kameda, Fuse, Miyamoto, Komatsu, Sugita, Okada, Namiki, Arakawa, Ishiguro, Abe, Gaskell, Palmer, Barnouin, Michel, French, McMahon, Scheeres, Abell, Yamamoto, Tanaka, Shirai, Matsuoka, Yamada, Yokota, Suzuki, Yoshioka, Cho, Tanaka, Nishikawa, Sugiyama, Kikuchi, Hemmi, Yamaguchi, Ogawa, Ono, Mimasu, Yoshikawa, Takahashi, Takei, Fujii, Hirose, Iwata, Hayakawa, Hosoda, Mori, Sawada, Shimada, Soldini, Yano, Tsukizaki, Ozaki, Iijima, Ogawa, Fujimoto, Ho, Moussi, Jaumann, Bibring, Krause, Terui, Saiki, Nakazawa and Tsuda2019; Tachibana et al., Reference Tachibana, H, R, Y, K, YN, C, H, S, P, Y, S, F, H, T, T, H, H, A, N, K, AM, K, S, T, R, S, E, Y, K, Y, M, M, N, M, T, H, C, T, T, H, T, M, K, A, M, AI, S, K, A, K, T, M, T, TR, M, T, M, HC, Fujii, Hasegawa, Hirata, Hirata, Hirose, Hosoda, Iijima, Ikeda, Ishiguro, Ishihara, Iwata, Kikuchi, Kitazato, Lauretta, Libourel, Marty, Matsumoto, Michikami, Mimasu, Miura, Mori, Nakamura-Messenger, Namiki, Nguyen, Nittler, Noda, Noguchi, Ogawa, Ono, Ozaki, Senshu, Shimada, Shimaki, Shirai, Soldini, Takahashi, Takei, Takeuchi, Tsukizaki, Wada, Yamamoto, Yoshikawa, Yumoto, Zolensky, Nakazawa, Terui, Tanaka, Saiki, Yoshikawa, Watanabe and Tsuda2022). Her previous work experience has included appointments around the world.

Prof. André Antunes, who leads the astrobiology group at the State Key Laboratory of Lunar and Planetary Sciences at Macau University of Science and Technology (MUST) in Macau, China, is a microbiologist whose expertise is in astrobiology and exploring various extreme environments around the world, especially deep-sea brines (Antunes et al., Reference Antunes, Alam, Simões, Daniels, Ferreira, Siam, El-Dorry and Bajic2015; Antunes et al., Reference Antunes, Olsson-Francis and McGenity2020), and has had experience working in Europe, Africa, Asia, and the Middle East.

Breakout discussions

Live-streamed presentations were followed by 30-minute breakout discussions at the end of each session. Participants were assigned to discussion groups randomly to allow for the representation of various research fields and countries in each group and were then invited to join the designated Gather.town (vide infra) discussion group spaces to meet with their peers. Groups always consisted of a discussion leader (a volunteer from within the participants or organizers), one of the speakers whose presentation was live streamed in the preceding session, and other meeting participants. A total of at most six discussion groups were running in parallel, ranging from three to ten participants each.

Discussion leaders were provided with prompting questions related to the topic and talks of the session and were encouraged to use these as a basis for further dialogue. Ideas and insights addressed during the discussions were collected and uploaded to the respective discord channels (vide infra), allowing participants from other groups as well as other time zones to read through them. Breakout discussions enabled participants to meet with one another and tackle specific and open questions through an interdisciplinary approach and were also used to reflect on the topics of each session. Key points were presented in a final overall discussion with all participants before the sessions were closed.

Gather.town platform in lieu of zoom

Gather.town is an online meeting platform which connects video conferencing with (virtual) spatial positioning of participants. This is done by placing a controllable avatar for each user in a shared virtual space. Gather.town attempts to replicate real-life interactions by making users move their avatars throughout the space and initiate video/audio connections based on the proximity of their avatar to those of others, or based on designated rooms with conferencing, and creates a more natural environment for gatherings compared to conference software like Zoom by utilizing shared virtual space. For example, rooms in Gather.town can have locations which are designated as spotlight, and users standing within a spotlight will stream their video or screen sharing to the entire room. In that way, the conference chair can very naturally say: ‘Can speaker Y come to the stage?’, while on classic conferencing software (Zoom, etc.) the chair must give permission for screen sharing to speakers. The same is true for Q&A sessions with spotlight locations, acting as real-life microphones, where Gather.town users can line up and ask their questions.

The platform provides several premade spaces and the ability to design your own custom space. This provides enormous flexibility in planning a conference by allowing organizers to connect and optimize planned activities with the space they will be executed in. As an online platform, Gather.town also provides capabilities which do not necessarily exist in real life, such as portals to reach different parts of the conference space quickly, locating users by their name and following them, private bubbles for communicating without interrupting the main presentation, and others. Gather.town has thus been implemented in various online fora up until now including education (Latulipe, Reference Latulipe2021); McClure and Williams, Reference McClure and Williams2021), workshops (Samiei et al., Reference Samiei, C, L, T and F2020), and conferences (Jacobs and Lindley, Reference Jacobs and Lindley2021). However, at the time of first implementation (mid-2021), it was unclear whether Gather.town was General Data Protection Regulation (GDPR) compliant, an important regulation related to data storage and privacy for European Union (EU) or European Economic Area (EEA) residents and citizens as Gather.town is not able to save data in the EU. Thus, the organizers asked all participants to acknowledge this fact, and to sign a data processing agreement with Gather.town directly before registering for the meeting if this was a point of concern. As of January, 2022, Gather.town is now both GDPR and California Consumer Privacy Act (CCPA)-compliant. In addition, EU or EEA citizens and residents may directly make requests related to personal information or data storage, as well as make complaints or claims to local authorities.

For the purposes of AbGradCon 2021, a custom Gather.town virtual meeting space was designed by NASA staff, using the original meeting proposed space at ELSI as an inspiration for several common rooms (Fig. 3). The virtual meeting space included a plenary room, several breakout rooms for small group discussions, a media room with links to submitted presentation videos, an entertainment room containing games and social areas, and a main gathering room with multiple private spaces intended for informal discussions initiated by the participants.

Fig. 3. Comparison of Virtual Meeting Space in Gather.town versus The Original Proposed Meeting Space at ELSI, Tokyo Institute of Technology.

Discord server

To facilitate text-based discussions before, during, after, and between sessions, a Discord server was created as a mechanism to supplement the Gather.town platform. First of all, the Discord server was used by the organizers to make official conference announcements, to list the code of conduct, to list all other relevant links (such as to videos, the conference schedule, the Gather.town platform, etc.), and also contained a help channel for participants with technical difficulties. A number of channels were also created for the deposition of notes taken during the breakout discussion sessions (vide infra), while some private channels were made for exclusive discussion amongst the organizers regarding organizational issues. A number of public channels, which all participants could join and comment freely in, were also created covering a variety of topics. These topics included science communication, the proposal review panel (vide infra), a channel for participant introductions, career opportunities, upcoming meetings, general science, and a social channel. Participants were also able to propose new channels, and a number of channels dedicated to discussion on specific scientific topics were created for this purpose. Finally, Discord is both GDPR and CCPA-compliant.

Pre-conference workshops and activities

As AbGradCon is a conference for students and early career researchers, career building is normally incorporated into the meeting agenda. A proposal writing retreat (PWR), for participants to practice preparing a scientific proposal, is usually held before the meeting. However, due to time zone differences and unbalanced availability of participants and organizers, such a PWR was not held before AbGradCon 2021. Instead, a number of alternative pre-conference workshops and activities to highlight important skills for researchers were held instead.

Social media

During the organization phase of both the 2020 and 2021 meetings, the Twitter and Facebook accounts of AbGradCon were used for various purposes, such as raising interest about the conference, announcing deadlines (e.g., abstract submission), updating (prospective) participants and the community on decisions taken following the COVID-19 pandemic situation, as well as sharing job opportunities, conferences, and recent astrobiology papers. For example, the AbGradCon social media accounts were used to share the experiences of former attendees and organizers of past AbGradCon conferences. This was done as a way to motivate early-career astrobiologists to attend AbGradCon 2020/2021 by showing how taking part in the conference has helped past attendees to get in touch with other researchers in the field and to pursue careers, both within and outside of academia. Furthermore, a number of organizing team members were also featured on social media before the conference with a photo and a short description about their motivation to take part in the organization of (and/or past participation in) the conference to highlight their contribution to organizational efforts. Finally, social media was also used during the conference as a way to share information and updates about the conference (e.g., upcoming sessions, conference photos on Gather.town) with the participants and the wider community. By using the hashtag #AbGradCon21, participants highlighted the work presented at the virtual conference and shared their experience.

Conference booklet

A conference booklet was prepared to guide the participants through the meeting as a supplemental resource. This booklet contained basic information about the conference organizers, host (ELSI), and sponsors/partners. It also provided links to a variety of supplementary resources including science communication video workshop sessions produced by a partner organization, the links to the various platforms (Gather.town and Discord) along with tutorial videos/resources for both platforms, and links to video recordings (Youtube) from previous AbGradCon conferences. In addition to the programme, the abstract and biography of the two keynote speakers were provided.

The conference booklet also acted as an abstract booklet. First, all participants, their institutions, and their presentation title were listed by discipline for ease of searching by discipline. Then, grouped by discipline, each abstract was listed including the contact information of the presenter (if they chose to have this disclosed to other participants) and the link to the presentation and/or flash talk (Youtube). In this way, participants could easily access videos or contact the presenter immediately after reading the abstract. Finally, a simple table of all participants in alphabetical order (by last name) was compiled as well as any video links associated with them were presented as a means for accessing video links by presenter rather than abstract.

Conference gift package

In many conferences, a conference gift bag is distributed to each participant containing conference information and also items from and information about the sponsors. AbGradCon 2021 elected to send a small gift package before the meeting to all participants who would accept one as a means to increase anticipation before the meeting. A number of sponsors contributed gift items to the gift package, and all of the items were sent to a centralized location in the United States. Then, these items were arranged and distributed into separate packages for further distribution by domestic or international shipping to the relevant participants (with support from AbGradCon partners Breakthrough Listen, Blue Marble Space Institute of Science (BMSIS), and Greenspace). Ultimately, while many of the gift packages were delivered before the meeting, due to the current situation regarding the tardiness of international shipping, a number of gift packages were not delivered until after the meeting began or concluded.

Sponsors and partners

AbGradCon 2021 could not have been successfully held without the support of various sponsors and partners (listed in detail in the acknowledgements). Some of these sponsors and partners provided funding for the Gather.town platform, hiring workshop facilitators, and payment for conference gift package shipping, while other partners managed and distributed this funding (a number of sponsors pledged funding for both the 2020 (postponed) and 2021 meetings, but ultimately this additional funding was not needed). Other partners supported the meeting through advertisement of the meeting to their networks, leading and facilitating workshops, and packaging and sending the conference gift packages.

2020 undergraduate flash talk contest

In 2020, in lieu of the traditionally-held Undergraduate Poster Contest (which grants travel funding to the conference to the winner), the organizers decided to hold an Undergraduate Flash Talk Contest. Undergraduate students contributed a short flash talk (audio only) describing their research in five minutes or less. Each of these flash talks was evaluated based on content (clarity, quality, and how well their content supported their claims), organization (transitions, logical flow, clear thesis and supporting data, and informative and clear project summary), and delivery (professional/engaging delivery, accessibility to those outside of their field and enthusiasm). Ultimately, two co-winners and one honourable mention were selected, and these flash talks were featured on the AbGradCon website. Originally, only the co-winners would receive full travel funding to attend AbGradCon 2020, but given its postponement and the digital nature of AbGradCon 2021, all co-winners and the honourable mention were granted acceptance to AbGradCon 2021 if they chose to submit an abstract. The Undergraduate Flash Talk Contest was not held in 2021.

Organizational roles

The tasks for the organization of both the 2020 and 2021 meetings were shared among different members of the local and external organizers; each role provided support for various essential aspects for the meeting. Some roles were filled by one organizer, while others were filled by a team of organizers; all organizers also contributed to multiple roles. In the Supporting Information (SI Notes 1–3), we provide a brief overview of each of these tasks listed in somewhat chronological order, separated by meeting year, as well as the rough number of organizers that contributed to that role (in parentheses after each role). We note that while some organizers selected roles which allowed them to learn a new skill, ultimately most of the organizers performed roles which took advantage of their already-held skills and experience. After the postponement of the meeting from 2020 to 2021, many organizers needed to step down from the event, resulting in more labour falling to a smaller group of organizers. As a result, many of the roles which began as well-organized group experiences changed to the responsibility of one or two organizers taking on multiple roles, unable to easily recruit new team members to replace those who stepped down. Luckily, many background aspects of the organization from 2020 (i.e., logo, webpage, sponsors and partners, application, abstract evaluation scheme, etc.) were able to be carried over to 2021 without significant variation (although for certain aspects, like the application, lessons from the 2020 meeting application process were used to optimize the 2021 application), while other components (i.e., field excursion and undergraduate flash talk contest) were abandoned by necessity. Additionally, roles were given to professional partners to develop new interactive components for the digital meeting.

Budget

The cost for holding AbGradCon 2021 virtually totalled roughly 25 000 USD, and included payment for the Gather.town platform, pre-conference Workshops, and the shipping and handling of the conference gift packages. These costs were offset by a budget allocation from the NASA Astrobiology Program, some rollover funding from previous AbGradCon conferences, and from support by partner organizations. Using such online platforms for a virtual AbGradCon greatly decreased the budget compared to an on-site AbGradCon meeting, which routinely cost more than 100 000 USD in recent iterations (and has reached more than 120 000 USD on occasion). Additionally, in the past, some attendees could not be allocated funding for attendance to an on-site meeting due to budget constraints; we believe that the virtual AbGradCon 2021 meeting decreased the probability that someone could not attend for budget reasons.

Abstract evaluation flow

AbGradCon is a conference designed for a small community to participate in direct discussions and interactions with other participants. As such, traditionally, the total number of attendees has been limited (to around 100). Given the desire to contribute a similar conference feeling, AbGradCon 2021 would also cap the total attendance (also in order to balance the geographic distribution of attendees). Additionally, live-streamed presentations must be selected from submitted abstracts with some discernable metric. As such, an abstract evaluation process was used to assign a score to each submitted abstract; abstracts below a certain score would not be selected for the meeting, while the abstracts above this score would be accepted. Additionally, taking into consideration the fields of the abstracts, the highest-scoring abstracts in a given field were selected for live-streamed presentations.

A number of potential volunteer abstract evaluators were contacted to contribute to this organizational step. These evaluators may have been past AbGradcon attendees, AbGradCon2021 organizers, members of partners of the current meeting, and other students/postdocs/early career researchers within the astrobiology community; evaluators were not chosen in any scientific or systematic way.

Each volunteer evaluator was initially assigned to between 20 and 21 abstracts arbitrarily and not delineated by field (all personally identifiable information, including author names and affiliations, were not included to maintain strict double-blind reviewing; we provided only the title and the abstract, strictly truncated to 250 words). If an evaluator noticed any conflict of interest based only on the title and the abstract, such as the abstract being their own, of a recent collaborator, of a lab member, or any other reason (such as a competing research group), they were asked to declare their conflict of interest and were assigned a supplementary abstract to evaluate by the organizers. Then, all evaluators were to proceed to evaluate each abstract according to a standard scoring rubric (vide infra), and report these scores in a central system. In total, each abstract was to be evaluated by three or four evaluators. However, there were cases where evaluators did not do any evaluations, had to drop out from this duty due to other reasons, or evaluated the wrong abstracts (those not explicitly assigned to them), which necessitated additional reviewing of a certain number of abstracts by evaluators; as such, the organizers assigned these additional abstracts with missing evaluations to evaluators who could volunteer to evaluate more abstracts. Finally, the average scores of all abstracts were tabulated, and conference acceptance and live-streamed presentation selection was determined based on the average score, field represented and country/region represented (vide infra). All organizers, discussion leaders, and abstract evaluators were accepted to the meeting. A similar process was utilized for the evaluation of abstracts for AbGradCon 2020, although the selection step after the collation of all evaluation scores was halted due to the postponement of the meeting at that time.

Abstract evaluation scoring rubric

The questions on the scoring rubric for abstract evaluations for AbGradCon 2021 (a similar scoring rubric was used for AbGradCon 2020 evaluations) are presented in SI Note 4. The abstract score was determined based on the literacy, novelty and soundness of the abstracts. These four topics are where the total scoring of the abstract were derived from (all four questions had identical choices). We designed these categories so that Scientific Merit rated the overall significance of the research, Scientific Literacy rated the background of the research, Novelty rated how novel the research was, and Soundness rated whether the experimental or theoretical methods were the correct ones to be applied to the research as well as the validity of the conclusions. However, evaluators were free to interpret these categories freely based on their own knowledge, understanding, and experience. During the evaluation process, a number of abstract evaluators expressed concern over their ability to evaluate these categories for abstracts outside of their own field. However, as astrobiology is inherently interdisciplinary, one of the criteria for submitted abstracts was to make it as understandable as possible for those outside of one's own field, and submitted abstracts which were more easily understood by a variety of evaluators from different fields likely scored higher.

Specific questions were also included to identify conflicts of interest between a reviewer and an applicant so as not to bias the scoring, while also assessing whether the abstract was relevant to astrobiology. Although AbGradCon aims to be an open conference for those who are interested in learning astrobiology for the first time, one of the abstract submission requirements was to include something that was astrobiology-related. For those who had abstracts in fairly far-afield topics, this could be achieved by proposing how one could use one's own research and apply it to astrobiology. While this was not included in the abstract score explicitly, any abstracts with all or nearly all evaluators deeming that it was not relevant to astrobiology were not considered for the meeting. Additionally, we also asked each evaluator to determine what primary and secondary field each abstract fell under. As all evaluators self-reported their own primary and secondary fields, these two questions were included so that we could analyse any trends or biases that arise from the evaluation of abstracts within versus outside of one's own field of expertise. The fields that could be chosen are represented in Table 4, with an ‘other’ option also available.

While AbGradCon is held in English, participants of all language backgrounds are welcome as long as they demonstrate the ability to competently communicate during the meeting in English, which we included one question to evaluate. This question was not meant to discourage the participation of non-native English speakers. Rather, this question is meant to assess if the applicant's English level will result in difficulty communicating with other AbGradCon participants. If the evaluator was uncertain but had some comments, they could leave some notes. Finally, we allowed evaluators to leave any further comments or notes about the abstract that did not fit any of the evaluation questions; these comments were used in case decisions had to be made about abstracts with similar average scores.

Abgradcon 2021 submitted abstract information

260 abstracts were submitted, and 225 of these abstracts were determined to be ‘valid’; abstracts were deemed invalid due to various reasons such as not providing a proper title and/or abstract, clearly not providing a professional abstract, not properly reporting country of institute or residence (this was a requirement for abstract submission, but was only used for statistical purposes), or submitting identical abstracts as another applicant, among others. Applications were submitted from participants residing in 36 countries (region breakdown is reported in Fig. 4(a)) and with citizenship in 42 countries, while their primary fields (Fig. 4(b)) and career levels (Fig. 4(c)) were also broad, spanning from undergraduate student to early career independent researcher (i.e., non-tenure track faculty). Finally, 140 of the applicants learned of AbGradCon from their colleagues, 22 from Twitter, and 30 from the AbGradCon website.

Fig. 4. Various demographic information about the applicants. (a) Region of residence. (b) Primary Field of submitted abstract (self-reported). (c) Career level.

Abstract evaluator information

As mentioned previously, a number of evaluators, with a large breadth with respect to career level, region, discipline, and years of astrobiology experience, scored the abstracts (Table 5). For example, evaluators ranged from master's students to postdoctoral fellows or independent early career researchers; were based in Asia, Europe, North America, and Oceania; reported primary fields ranging from astronomy, biology, chemistry, geology, and planetary science; and had each been in the astrobiology field for varying lengths of time. This suggested that the evaluation process could proceed fairly in principle, although no systematic mechanism for evaluator selection (such as evenly distributed amongst regions, fields, career levels, etc.) was performed.

Table 5. Total number of evaluators in different demographic categories (self-reported)

Specifically, up until now, past AbGradCon meetings relied only on students and postdoctoral fellows as evaluators; this allows attendees to be evaluated by their peers. We originally intended to have a similar evaluator pool, but ultimately had three evaluators who were ‘out of the traditional career range’ (Table 5), although most of these evaluators still would have been eligible for AbGradCon. This was due to a number of reasons. First of all, the position and title systems in different countries vary greatly, and in some countries ‘postdoctoral scholar’ positions are not always used. Thus, independent early career researcher positions (without a traditional supervisor) could be available directly after the PhD; such applicants were considered eligible for AbGradCon 2021 as long as they obtained their PhD relatively recently. Additionally, in some countries, early-career faculty positions are not independent positions (such as non-tenured junior faculty-level positions Japan), and work under a PI; such positions can be thought of as the equivalent of a postdoctoral fellow position with student mentoring duties, and such attendees were also eligible for the meeting. Finally, as the meeting was postponed for one year (and also took a total of four years to organize), some of the organizers graduated or moved on to other positions during this time (Section ‘Organizational roles’). One of the conference chairs of AbGradCon 2021 was incidentally promoted to a non-tenured junior faculty position during the organization process (which traditionally would have resulted in ineligibility to attend the meeting), and was allowed to participate in the conference (both as an attendee and evaluator) as their participation was necessary for the successful organization of the meeting. Nevertheless, the organizers still recommend that the evaluation pool for future meetings consist of those who would normally be eligible for conference attendance so that attendees can be evaluated exclusively by their peers.

After the evaluation process, the mean number of abstracts evaluated per evaluator was 25.5, with a standard deviation of 7.9. The maximum number of abstracts that any one evaluator evaluated was 42, while the minimum was 13. The evaluators had fairly different average evaluation scores. For example, one evaluator had an average evaluation score of 9.5 (out of 16 points, maximum), while another had an average evaluation score of 15.5 (the mean average evaluation score of all evaluators was 12.1, with a standard deviation of 1.8). Additionally, each evaluator had a different evaluation scoring variance, with some showing little variance in scoring (standard deviation as low as 0.7 or as high as 5.3). These observations of statistical variability between evaluators may be due to a number of different reasons, including interdisciplinarity of their own research (as abstracts evaluated may not have been from their own fields), however it is difficult to make definitive conclusions.

Abstract evaluation results

Each of the 225 abstracts was evaluated at least three times by different evaluators, and on average by 4.1 different evaluators. However, three abstracts were evaluated by seven different evaluators, while 72 abstracts were evaluated by only three different evaluators. In total, there were 917 evaluations performed, with a global mean score (out of 16 points) and standard deviation of 12.1 and 3.1, respectively. 142 evaluations received a perfect score, three evaluations received a score of 0, and 12 evaluations were determined to be irrelevant to astrobiology. Evaluators were also asked to categorize each abstract that they evaluated into primary and secondary scientific fields (Table 6). It appeared that evaluations of abstracts determined to be in most of the ‘traditional’ scientific categories (by primary field) were similar in score, but abstracts determined to be in the philosophy of science or society category had lower mean evaluation scores. A heteroscedastic two-tailed t-test comparing the scores of evaluations of philosophy of science abstracts against all other abstracts, and society abstracts against all other abstracts, showed that while the mean evaluation score of society abstracts was lower than that of all other abstracts (p-value of 0.031), the mean evaluation score of philosophy of science abstracts was not significantly lower than that of all other abstracts (p-value of 0.064).

Table 6. Statistics of evaluations of abstracts sorted by category (as determined by evaluators)

The mean score of all evaluated abstracts was 12.1. Starred values indicate statistical significance: * = p ≤ 0.05; ** = p ≤ 0.01; *** = p ≤ 0.001; **** = p ≤ 0.0001.

However, we also discovered that physics abstracts (p-value of 0.023) were scored lower than average abstracts, while chemistry abstracts (p-value of 0.000023) were scored significantly higher than average abstracts (and both of these could be considered ‘traditional’ scientific categories). In most conferences, abstracts are evaluated by experts in that particular field. Given that astrobiology is a broad field by definition with many sub-disciplines, abstract evaluators for AbGradCon2021 likely evaluated abstracts not only within their own discipline, but also within other disciplines where they were less familiar with. Thus, it may be possible that the lack of familiarity of topics of other disciplines led to systematic or random biases in the evaluation of abstracts from out-of-discipline (especially for society-related abstracts), or perhaps even led to changes in evaluation score variation (due to potential ‘random’ evaluation).

After all of the evaluations were completed, the results were compiled and the mean evaluation score of each submitted abstract was used to determine which abstracts would be accepted to the conference (i.e., final abstract score). The maximum score for any submitted abstract was 15.67 (two abstracts), the minimum score was 5.2, and the mean and standard deviation of final abstract scores was 12.1 and 2.0, respectively. We next sorted each abstract based on various categorizations (region of applicant residence and primary field of abstract) to observe whether there were any trends related to the evaluation of abstracts from different categories (Tables 7 and 8). Abstracts from those in Asia on average appeared to receive lower evaluation scores than other abstracts, while abstracts from those in North American on average appeared to receive higher evaluation scores than others (with statistical significance). We speculate that this may be due to a number of cultural and language differences related to the education practices in each country and the maturation of astrobiology-specific scientific programs in those countries. For example, North America was one of the first regions around the world to adopt astrobiology as a research topic, and is currently world leaders in terms of astrobiology research productivity, funding, and education (with a number of astrobiology-focused programs). Although there has been the expansion of astrobiology as a research and education topic in Asia, these activities have only gained significance in very recent years, and there are fewer astrobiology research groups, education programs, and funding opportunities in Asia as compared to North America or the rest of the world. As such, perhaps the quality of the abstracts is affected by the educational and research opportunities afforded by each student or researcher's home country/region; i.e., without the proper training and opportunities, it is difficult to prepare a professional-level abstract for an international, interdisciplinary astrobiology conference. A more detailed study into those factors is beyond the scope of this report, but as the field of astrobiology matures and ages globally, such variations in abstract scores are likely to disappear.

Table 7. Demographics statistics of all submitted abstracts (mean score of all evaluated abstracts was 12.1)

A heteroscedastic two-tailed t-test was performed comparing the mean abstract evaluation scores of the abstracts in the category listed in the same row, against mean abstract evaluation scores of all other abstracts. Starred values indicate statistical significance: * = p ≤ 0.05; ** = p ≤ 0.01.

Table 8. Sub-field statistics of all submitted abstracts (mean score of all evaluated abstracts was 12.1)

A heteroscedastic two-tailed t-test was performed comparing the mean abstract evaluation scores of the abstracts in the category listed in the same row, against mean abstract evaluation scores of all other abstracts. Starred values indicate statistical significance: * = p ≤ 0.05; ** = p ≤ 0.01.

Additionally, given that AbGradCon is traditionally organized in and by residents of North America, it may be that the abstract style required for submission or the abstract evaluation criteria are biased towards those who are either from or educated in North America. This regional scoring disparity may thus mean that the evaluation schema was either flawed or unfair towards non-North Americans (as what a higher ‘quality’ abstract means could be somewhat subjective), or the organizers did not do an adequate job to inform the evaluators of potential bias. Thus, it is possible that the style and/or structure of an abstract that is generally considered ‘good’ by researchers in North America (in terms of the four evaluation categories used by AbGradCon 2021: merit, literacy, novelty and soundness) is different from those considered ‘good’ by researchers in other regions. As such, it is possible that the North American evaluators evaluated abstracts with the expectation that a ‘good’ abstract would be written in the ‘North American style’, which was not familiar to those in other countries.

While the organizers were explicit in informing the evaluators that English ability of the applicant based on perceptions from the abstract was not to be judged so that the conference would be accessible to a wide-range of English levels (as long as the participant could communicate effectively in English, they could attend the meeting), it is possible that English ability also affected the ability for abstract evaluators to understand scientific aspects of the abstract (or, evaluators from North America were not used to evaluating such abstracts, and gave lower scores subsequently despite such abstracts having no problems in terms of scientific content). In particular, while English education differs greatly by country, region, and even within each country, a number of countries in Asia do not have strict English language requirements or many English-based schools or educational opportunities en masse (as compared to those in North America, Europe or Oceania), leading to the possibility of less opportunities and experience for applicants from Asia to practice or participate in English language-based scientific exchange, programs, or events, which may have been reflected in these abstract evaluations. However, this is simply speculation. Nevertheless, we will note again that although we believe that the evaluation process was fair, the evaluator selection process was not rigorous or systematic, which may have resulted in some of the biases observed herein.

Abstract acceptance/selection

Of the applications, 146 abstracts were accepted to AbGradCon 2021 on the basis of their scores and other factors. First, as is customary for traditional in-person AbGradCon meetings, local attendees who submit a valid and relevant abstract are accepted regardless of score, and as such, all participants based in Japan were accepted to the meeting regardless of score. Additionally, all applicants who contributed to organizing the meeting (including discussion leaders and abstract evaluators) were accepted regardless of score. Finally, those who were co-winners or honourable mentions from the AbGradCon 2020 Undergraduate Flash Talk Contest were also accepted.

Next, we accepted abstracts based on evaluation scores. 115 additional applicants with a final abstract score of 12.4 (out of a maximum 16 points) or higher were accepted. This cut-off score was selected with a targeted number of attendees (a maximum of 150 total) in mind; AbGradCon is not traditionally a large conference, and the smaller conference environment appeared to allow more interpersonal connections and discussions during past AbGradCon meetings, which is why the organizers decided to aim for a similar attendance for a virtual AbGradCon. However, this resulted in an overly skewed representation of the meeting towards participants from North America (and in particular, the USA), with 62 of the 136 accepted abstracts (46%) coming from those based in the USA. As the theme of the meeting was the international collaboration, further increasing the diversity of participants (based on country of residence) was one very valid way to increase the potential for international collaboration; a meeting with almost half of the participants based in a single country would not have been conducive towards such goals. As such, all non-USA-based applicants with a final abstract score of 11.67 or above (which was 10 applicants) were also accepted to the meeting, leading to the final total number of accepted abstracts (which was still below the targeted maximum attendance of 150).

As such, of the 146 abstracts accepted, the mean score was 13.1 (standard deviation of 1.3), while the minimum score was 6.75; the maximum score was 15.67, as reported earlier. Of the abstracts submitted by organizers and of those based in Japan, 11 organizers (including discussion leaders and abstract evaluators) and six participants based in Japan would not have been accepted had we not guaranteed attendance to these groups of people. However, it is not immediately clear why so many organizers' abstracts received such low evaluation scores. Ultimately, 119 of these accepted participants registered for the meeting; those who withdrew cited different reasons for withdrawal, including conflicts with other events, conflicts with regular work/research/school duties, conflicts with field excursions, etc.

Live-streamed presentation selection

The selection of live-streamed presentation speakers was the result of sequential elimination based on criteria recorded from the participant questionnaire answers received. In the case of an in-person meeting, the oral presentations would have been chosen first by field (Biology, Geology, Planetary Science, Astronomy & Physics, Chemistry, or Engineering & Instrument Design) followed by the individual participant's ranking in the abstract evaluation. Some correction was planned to more equally divide speakers by country of residence to increase the international representation of the meeting. In the case of a digital meeting (as in AbGradCon2021), these criteria were still the general basis for selection but additional parameters needed to be included. Questions in the participant questionnaire included information about whether or not a participant would prefer to prepare a full oral presentation, a flash talk or a combination of the two. Those who selected a flash talk alone were removed from the pool of potential live speakers. Second, we asked participants if they wished to be removed from consideration for a live presentation, yielding the opportunity to a different participant. Anyone who yielded their spot was removed from the pool of potential options.

The next selection criteria was specific to an online meeting across different time zones: availability. In a traditional AbGradCon event, all of the participants would in theory be present at all times during the meeting. With the online format, we had both a morning and an afternoon session (in JST) to account for differences in time zones. Live-streamed speakers were expected to answer questions live during their session and would be required to attend the session for this reason. As such, we included one question in the participant questionnaire which asked about the participant's availability for each session on each day of the conference classified as: in full, part of the time, or none of the time. This required eight individual questions (eight sessions over four days) within the questionnaire to guarantee that the information was sorted in a useful way within the master spreadsheet. Initially, those participants who could not attend on the days selected for talks within their scientific field were removed, but exceptions were later made to include certain participants who replied ‘part of the time’. This selection required the most amount of careful consideration and time to make certain that enough speakers were available for the various times by scientific category, and the programme was changed to account for these differences in availability for fields which had far fewer participants available. An email was sent to all the selected live-streamed presenter candidates to confirm that they were still willing to provide a live-streamed presentation before the final programme was confirmed. Those who decided to withdraw at this stage were replaced by alternate candidates following the above-mentioned process and speakers were notified of their assigned presentation times. In the end, the morning sessions were populated primarily by participants in the North and South Americas with the inclusion of a few participants in Asia. Collectively, many Asia-based participants had declined to be considered for the live-stream session. The afternoon session was primarily speakers from Europe and Asia, with occasional North American speakers who had selected to remain online well-outside of the ‘normal’ working-times. The selection process itself relied heavily on carefully crafted questions in the participant questionnaire which allowed for the determination of viable live-streaming candidates for the required times of the meeting. A total of 36 different speakers were selected by this method to present in 12 different live sections divided into six different scientific fields of concentration. Each of the six scientific fields was represented twice; once in the morning session and again in the afternoon, allowing for different speakers in both session time zones.