The First Two Years of FLEET: An Active Search for Superluminous Supernovae

Type I superluminous supernovae (SLSNe) can be up to 100 times more luminous than "normal" Type Ic SNe (SNe Ic). They are identified by their high luminosity and blue spectra, and, much like their normal-luminosity SN Ic counterparts, SLSNe are thought to be the end stage of massive stripped-envelope stars (Chomiuk et al. 2011; Quimby et al. 2011). To date, we know of ∼200 spectroscopically classified SLSNe (Chen et al. 2023; S. Gomez et al. 2023, in preparation). Many open questions remain about the nature of SLSNe, what powers them, their environments, their progenitors, and their connection to other transients (e.g., Ofek et al. 2007; Pastorello et al. 2010; Dessart et al. 2012; Gal-Yam 2012; Inserra et al. 2013; Lunnan et al. 2014; Nicholl et al. 2017; Blanchard et al. 2020b; Ørum et al. 2020; Yan et al. 2020c; Könyves-Tóth & Vinkó 2021; Gomez et al. 2022a; Hosseinzadeh et al. 2022). To address these questions, it is imperative that we find SLSNe early and efficiently.

Telescope resources worldwide can only spectroscopically classify ∼10% of all transients discovered by current optical time-domain surveys. Since SLSNe only make up ∼1.5% of all discovered transients (Perley et al. 2020b), we need efficient methods to select the most likely SLSN candidates for spectroscopic follow-up and make the most efficient use of limited telescope time. Toward this end, we developed Finding Luminous and Exotic Extragalactic Transients (FLEET), a machine-learning algorithm that can determine the probability of being an SLSN, P(SLSN-I), for any transient, calculated using the properties of both its light curve and host galaxy (Gomez et al. 2020a). We presented three versions of FLEET, a rapid version trained on the first 20 days of photometry, a late-time version that uses the first 70 days of photometry, and a redshift version, the only one that includes the redshift as a required parameter. The rapid version is meant to be used for real-time classification, including additional photometry as it becomes available, before the SN fades. The late-time version, on the other hand, can provide more robust predictions and be used to create photometric samples for population studies at the expense of the SN being either dimmer or having faded. The redshift version is the most accurate of the three but requires knowing the redshift of the transient a priori.

The structure of this paper is as follows. In Section 2, we present the results from our first 2 yr of FLEET operations and confirm that we have been able to accurately and efficiently discover SLSNe. In Section 3, we describe the expanded training set used for FLEET, present updated performance metrics of purity and completeness, and explore how FLEET can be used to generate photometric samples of SLSNe. In Section 4, we compare the SLSNe from FLEET to the overall sample of SLSNe and explore possible selection effects from FLEET in terms of both physical and observational parameters. Finally, we summarize our conclusions in Section 5. FLEET is provided as a Python package on GitHub ¹⁰ and Zenodo (Gomez et al. 2020b), as well as included in the Python Package Index under the name fleet-pipe.

We began operating FLEET in 2019 November with the aim of finding SLSNe in ongoing transient alert streams. We selected candidates from transients reported to the Transient Name Server (TNS) ¹¹ and followed up the ones that were most likely to be SLSNe using the Blue Channel (Schmidt et al. 1989) and Binospec (Fabricant et al. 2019) spectrographs on the MMT 6.5 m telescope and the Low Dispersion Survey Spectrograph (Stevenson et al. 2016) and Inamori-Magellan Areal Camera and Spectrograph (Dressler et al. 2011) on the Magellan 6.5 m telescopes. Since we began running FLEET and up to 2022 January, there have been 50 SLSNe spectroscopically classified worldwide, and FLEET is responsible for classifying 21 of them (42%); the full list is shown in Table 1. For completeness, in Table 2, we include a list of 17 transients classified by FLEET that turned out to be something other than SLSNe (six SNe IIn, six SNe Ia, three SNe II, and two SLSNe II). We define an SLSN-II as an SN IIn with a peak absolute magnitude M_r < −20. The classifications and spectra of all SNe classified as part of this program were published on the TNS.

FLEET is a binary classifier that can determine if a transient either is an SLSN with a probability P(SLSN-I) ≥ 0.5 or, if the probability is lower, is not. Purity is defined as the number of true-positive SLSNe divided by the sum of true- and false-positive SLSNe. For our purity measurements, we only consider transients with P(SLSN-I) ≥ 0.5, whether they are SLSNe (N = 14) or not (N = 4). Completeness is defined as the number of true-positive SLSNe divided by the sum of true-positive and false-negative SLSNe. Since we do not know the true number of undiscovered SLSNe, we cannot measure the true completeness of the classifier. Nevertheless, we can obtain an upper limit on the completeness by considering all SLSNe discovered by FLEET with P(SLSN-I) ≥ 0.5 (N = 14) divided by the total number of SLSNe discovered worldwide (N = 46), whether classified by FLEET or by someone else in the community. We do not include the four SLSNe with P(SLSN-I) ≥ 0.5 discovered by others in our estimate of completeness to be more conservative about our estimates. We find that the on-sky performance surpassed the expected performance for almost every metric:

• 1.  
  rapid purity = 85% (predicted 50%),
• 2.  
  late-time purity = 78% (predicted 59%),
• 3.  
  rapid completeness ≲ 33% (predicted 30%), and
• 4.  
  late-time completeness ≲ 30% (predicted 44%).

The on-sky measurements of purity are particularly successful, given that FLEET was originally designed to optimize for purity as opposed to completeness. Additionally, it is reassuring to see that even when optimizing for purity, the measured completeness is consistent with the predicted level. In addition to the SLSN candidates with high P(SLSN-I) values, we observed some transients that had P(SLSN-I) < 0.5 when other methods independent of FLEET suggested that they could be possible SLSNe. For example, we targeted SN 2021jmm, SN 2021jtu, and SN 2021owc because they were part of the Zwicky Transient Facility (ZTF) SLSN candidate reports (Perley et al. 2021a; Yan et al. 2021b, 2021c). We also observed a few targets as part of an experimental attempt to target Type II SLSNe, which, although they had low P(SLSN-I) values, had P(SLSN-II) values above 0.7: SN 2019itq, SN 2019otl, SN 2019pvs, SN 2019sgh, SN 2021xfu (all spectroscopically classified as Type I SLSNe), SN 2021owc, SN 2021srg (SN IIn), and SN 2020mad (SLSN-II). We manually vetted every transient before triggering spectroscopic follow-up, removing transients with very poor or erroneous data or misidentified host galaxies or that were clearly stellar in nature. This means that we did not observe every transient with P(SLSN-I) > 0.5, which might have lowered our measured purity.

The redshift classifier was originally designed with the goal of using it with data from the Legacy Survey of Space and Time once it provides photometric redshift estimates for galaxies down to i ≈ 25 mag (Graham et al. 2018). Therefore, we do not include a comparable validation of the redshift classifier, since we have not yet tested it on active surveys.

The original version of FLEET was trained on 1813 spectroscopically classified transients. Here we retrain FLEET on an updated and more extensive list of 4780 spectroscopically classified transients with the following distinct labels from the TNS: 2983 SNe Ia, 749 SNe II, 187 SLSNe I, 157 SNe IIn, 143 CVs, 105 SNe Ic, 89 SNe IIP, 80 SNe Ib, 68 SNe IIb, 52 SLSNe II, 46 tidal disruption events (TDEs), 35 SNe Ic-BL, 26 SNe Ibc, 23 active galactic nuclei (AGNs), 19 SNe Ibn, and 18 variable stars. We retrain the algorithm using the same procedures outlined in Gomez et al. (2020a), where we optimize the depth of the random forest trees, the features included, and the number of days of photometry to consider. We find that the optimal parameters we determined for FLEET 1.0 still perform best for FLEET 2.0, with the exception that a tree depth of 11 branches now results in a higher completeness than the original depth of seven branches.

Given that the number of transients per class varies substantially, the classification would tend to be biased toward selecting the more common classes. To prevent this, we oversample each class to have a total of number of objects as the most common class (i.e., 2983 SNe Ia) using the Synthetic Minority Over-sampling Technique (Chawla et al. 2002). This algorithm draws random samples from a vector connecting every pair of objects across their parameter space until the desired number of events is reached.

In Figure 1, we show the updated purity and completeness obtained from the new FLEET 2.0 compared to those of FLEET 1.0 for all three classifiers. The uncertainties for all predictions represent the 1σ scatter of 25 different realizations of each model generated using a different initial random seed. We find that for transients with P(SLSN-I) ≥ 0.5, the expected purity is ≈50% for both the rapid and late-time classifiers. The expected completeness for the same threshold is ≈50% for the late-time classifier and ≈40% for the rapid classifier. FLEET 1.0 had a comparable purity to FLEET 2.0 but a completeness ≈10% lower for both classifiers. For the redshift classifier, the purity increased in FLEET 2.0 compared to FLEET 1.0, from ≈50% to ≈60%, and the completeness increased from ≈55% to ≈60%. The drop seen in the FLEET 1.0 rapid classifier at P(SLSN-I) > 0.9, which was due to the small sample size, is now not present in FLEET 2.0.

Zoom In Zoom Out Reset image size

3.1. Photometric Sample of SLSNe

We use FLEET 2.0 to generate a photometric sample of SLSN candidates by running it on every optical transient reported to the TNS and searching for unclassified SLSNe. Since these are mostly transients that have faded away by the time of this analysis, we use the late-time classifier that was trained on the first 70 days of photometry. For this reason, we exclude transients that were too young, discovered after 2022 March 1. At that time, there were a total of 95,729 transients reported to the TNS. Of these, 83,589 have a decl. > −32°, required for FLEET to query the PS1/3π (Chambers & Pan-STARRS Team 2018) or Sloan Digital Sky Survey (SDSS) catalogs (Alam et al. 2015; Ahumada et al. 2020). Of these, 34,805 were discovered before ZTF began operations and have either only one data point or minimal discovery photometry available. An additional 2078 transients were excluded because they only had coverage in one band, despite having tens to hundreds of detections in ZTF. Of the remaining transients, 31,922 had at least two g-band and two r-band points, required for FLEET to fit a model to their light curve. Finally, we excluded 30 transients that were located in regions with a Galactic extinction ≳1.5 mag. The final list of transients on which we run FLEET contains 31,892 transients.

In Figure 2, we show the P(SLSN-I) values from FLEET 2.0 compared to the estimates from FLEET 1.0 for all 31,892 transients. The increase in completeness of FLEET 2.0 is evident in the cluster of transients that had low P(SLSN-I) values in FLEET 1.0 that now lie at P(SLSN-I) > 0.5. In Figure 2, we also include the sample of 106 spectroscopically classified SLSNe, shown in blue. A total of 69 of these SLSNe had P(SLSN-I) < 0.2 in FLEET 1.0, 20 of which now have P(SLSN-I) > 0.5 in FLEET 2.0, representing the improvement in completeness. While 21 SLSNe now have a lower P(SLSN-I) in FLEET 2.0, 85 have a higher P(SLSN-I) in FLEET 2.0 compared to FLEET 1.0, indicating a net gain. Of the 21 SLSNe that lowered in confidence, only six decreased significantly from P(SLSN-I) > 0.5 in FLEET 1.0 to P(SLSN-I) < 0.5 in FLEET 2.0.

Zoom In Zoom Out Reset image size

There were 25 SLSNe classified by other groups, which had P(SLSN-I) < 0.5 in FLEET 1.0. Of these, nine have P(SLSN-I) > 0.5 in FLEET 2.0. Of the remaining ones, nine still have significantly low values of P(SLSN-I) < 0.25. These low P(SLSN-I) values are mostly due to sparse or noisy light curves and, in the case of SN 2021lwz, a very quickly evolving light curve.

Of the 106 spectroscopically confirmed SLSNe in the sample shown in Figure 2, 45 have P(SLSN-I) > 0.5, corresponding to a completeness of 43%. We find a total of 382 transients with P(SLSN-I) > 0.5, including the 45 known SLSNe. This corresponds to a purity of 12%, much lower than the on-sky purity presented in Section 2, for two main reasons. First, obvious failure modes still exist that pollute the sample (e.g., variable AGNs or variable stars with long light curves, and no detected host can be misclassified as an SLSN). We can visually purge these obvious failure modes to increase the purity of our sample, as was done during the first 2 yr of operations to obtain the observed purity cited in Section 2. Second, there are likely still some SLSNe in those 382 transients that have not been classified but can be considered to create a photometrically selected sample of SLSNe. Of those 382 transients, we exclude 139 from being possible SLSNe because either their light curves are too noisy or sparse or they strongly resemble a stellar transient or AGN (these are the green points in Figure 2 with P(SLSN-I) > 0.5 in FLEET 2.0). That leaves 243 transients as possible SLSNe, noting that none of the 45 spectroscopic classified SLSNe were rejected in the manual purging process. Of these, 70 transients are already classified: 2 AGNs, 2 CVs, 11 SLSNe II, 1 SN I, 13 SNe II, 2 SNe IIP, 17 SNe IIn, 16 SNe Ia, 2 SNe Ia-91T-like, 1 SN Ia-CSM, 2 SNe Ic, and 1 TDE. Finally, 128 remain unclassified SLSN candidates. We show the 128 photometrically selected SLSN candidates in orange in Figure 2. We find that 45% of these candidates also had P(SLSN-I) > 0.5 in FLEET 1.0, whereas the other half were previously below this threshold. If we were to spectroscopically classify these 128 SLSN candidates, assuming that the likelihood of any one target being an SLSN is equal to their P(SLSN-I), we expect that about 82 of them would be true SLSNe, or an ∼40% increase of the current total population size of SLSNe. A full exploration of this population will be presented in future work.

In this section, we explore whether the use of FLEET introduces biases or selection effects in terms of observed or physical SLSN parameters. For this test, we use the sample of 187 SLSNe used for training FLEET. We run these SLSNe through the late-time classifier in FLEET and compare the observational parameters of the SLSN population at large, with those of 74 "recovered" SLSN with P(SLSN-I) > 0.5. To quantify how different the parameter distributions are, we implement a two-sample Kolmogorov–Smirnov (K-S), where a K-S metric of D = 0.0 indicates that the two samples are drawn from the same distribution, and D = 1.0 means that there is no overlap between the distributions. We show how the apparent r-band magnitude, redshift, (g − r) color, and Δm compare between the two samples in the top panels of Figure 3, where Δm is defined as the host magnitude minus the transient peak magnitude. We find no obvious selection effects in terms of redshift r-band magnitude or (g − r) color, which have K-S metrics (and p values) of 0.13 (0.4), 0.07 (0.96), and 0.09 (0.3), respectively. The only observational parameter for which we do find a possible difference between the full and recovered sample is Δm, where SLSNe with Δm < 0 are generally not selected as likely SLSNe. This is reflected in its K-S metric of 0.18 (0.09). To calculate Δm, we use as reference the r-band magnitudes of the host galaxies obtained from either PS1/3π, SDSS, or values published in compilation studies of the hosts of SLSNe (Inserra et al. 2013; Lunnan et al. 2014; Angus et al. 2016, 2019; Perley et al. 2016; Schulze et al. 2018, 2021) or studies of individual SLSNe (Leloudas et al. 2012; Vreeswijk et al. 2014; Lunnan et al. 2016, 2018; Yan et al. 2017; Blanchard et al. 2018, 2021b; Yin et al. 2022).

Zoom In Zoom Out Reset image size

The power source behind SLSNe is still a debated topic, but the most likely one appears to be a magnetar central engine (Angus et al. 2016; Nicholl et al. 2017; Margalit et al. 2018). When a millisecond magnetar is born after the explosion of the progenitor, it will spin down and deposit this energy into the ejecta, powering the bright optical light curves observed in SLSNe (Kasen & Bildsten 2010; Woosley 2010). Therefore, we explore the selection effects in terms of the physical parameters of the SLSN population, assuming these are powered by magnetars. We derive the physical parameters of the 187 SLSNe from light-curve fits using MOSFiT, a Python code designed to model the light curves of transients using a variety of different power sources and derive their physical parameters (Guillochon et al. 2018). We show how the population of recovered SLSNe compares to the overall population in the bottom panels of Figure 3. We find a K-S metric (and p value) of 0.15 (0.26) for the magnetic field B_⊥, 0.09 (0.79) for the ejecta mass M_ej, 0.05 (0.99) for the ejecta velocity V_ej, and 0.12 (0.03) for the spin period P_spin. This shows that the two populations are very similar at the 5%–10% level and that FLEET is not introducing a bias in terms of physical parameters. We note that here we are concerned mostly with the relative distribution of physical parameters, and the specific choice of physical model is not the discerning factor.

In conclusion, we find that even if FLEET is less likely to select SLSNe with low values of Δm, this does not translate into a bias on the derived physical parameters.

We have provided an evaluation of the on-sky performance of FLEET during our first 2 yr of operations. We include the list of 21 SLSNe found by FLEET 1.0 from 2019 November to 2022 January, which represent 42% of all SLSNe discovered worldwide in the same time period. We show that our estimate for purity of ≈50% from FLEET 1.0 was surpassed with a measured purity of ≈80%. The measured completeness was in line with our predicted value of ≈30%.

We describe an updated version, FLEET 2.0, trained on 4780 transients, or almost three times as many as FLEET 1.0. The updated estimates for purity and completeness are both ≈50% for the late-time classifier and ≈50% purity and ≈40% completeness for the rapid classifier. The estimated purity and completeness for the redshift classifier are both ≈60%. It is of course possible that the on-sky purity will be higher, as was the case for FLEET 1.0. We test whether FLEET might be introducing selection effects into the recovered SLSN population and find no significant biases against any physical or most observational parameters.

Finally, we generate a photometric sample of 128 likely SLSN candidates selected by FLEET from ZTF archival data. We find that if we were to spectroscopically classify all of these candidates, assuming that the likelihood of any transient being an SLSN is equal to its P(SLSN-I) value, ∼80 of them would result in true SLSNe, which would represent an ∼40% increase in the current population size of SLSNe.

S.G. is supported by an STScI Postdoctoral Fellowship. The Berger Time Domain group at Harvard is supported in part by NSF and NASA grants, including support by the NSF under grant AST-2108531, as well as by the NSF under Cooperative Agreement PHY-2019786 (The NSF AI Institute for Artificial Intelligence and Fundamental Interactions; http://iafi.org/). M.N. is supported by the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation program (grant agreement No. 948381) and a fellowship from the Alan Turing Institute. This research has made use of NASA's Astrophysics Data System. This research has made use of the SIMBAD database, operated at CDS, Strasbourg, France. Operation of the Pan-STARRS1 telescope is supported by the National Aeronautics and Space Administration under grant Nos. NNX12AR65G and NNX14AM74G issued through the NEO Observation Program.

Facilities: ADS - , TNS. -

Software: Astropy (Astropy Collaboration et al. 2018), extinction (Barbary 2016), Matplotlib (Hunter 2007), emcee (Foreman-Mackey et al. 2013), NumPy (van der Walt et al. 2011), scikit-learn (Pedregosa et al. 2011), SMOTE (Chawla et al. 2002), FLEET (Gomez et al. 2020b).