Science in the modern era is increasingly reliant on enormous datasets and automated analysis. In astronomy, the Vera C. Rubin Observatory’s Legacy Survey of Space and Time (LSST) — a ten-year survey covering the entire southern sky almost a thousand times — will test the limits of this reliance.
Located on a mountaintop at Cerro Pachón in Chile, the observatory is expected to catalogue the night sky in exquisite detail. It aims to answer fundamental questions by studying supernovae, asteroids, dark matter, and the properties of our own galaxy. Yet, it also addresses a question dominating 21st-century science: how is discovery viewed in the age of big data?
Although primarily funded by the US Department of Energy and National Science Foundation, the Rubin telescope is a collaborative effort spanning six continents. Assistance in setting up data processing systems was provided by the UK, France, Spain, Italy, Japan, Brazil, Australia, South Africa, and Canada. These inkind contributions grant researchers from these nations data rights for the LSST.
Scientific alerts are forwarded to seven “brokers” worldwide — software platforms that astronomers use to access LSST data. These alerts provide information on new astronomical objects, their likelihood of being real, and changes in brightness over time. This allows astronomers to select the best candidates for follow-up research.
However, even with these brokers, there is too much transient data for any team to sift through. The final stage of processing involves machine learning and AI to identify the best data, filtering real cosmic objects from terabytes of false alerts. Astronomy is increasingly code-heavy; given the massive volume of observations, it is naturally one of the first sciences to embrace machine learning. The LSST Informatics and Statistics Science Collaboration, for example, consists of over 150 data scientists developing tools specifically for these goals.
Astronomy has led the charge in big data, with funding from companies such as Amazon and Microsoft. Indeed, the namesake of the 8.4-metre Simonyi Survey Telescope, Charles Simonyi, is known for his software development at Microsoft and his philanthropic work. The data volume will create opportunities not only for scientists and tech workers but also for volunteers via citizen science projects. A partnership with the Zooniverse platform will ask volunteers to help classify phenomena and discard “garbage” data.
What does the Rubin Observatory tell us about modern astronomy? The 20th century saw a push for international collaboration, but the sophistication of modern observatories means more astronomers now work to enable science rather than making discoveries themselves. This scale is not unique to Rubin; surveys like Euclid, the LIGO-Virgo-Kagra collaboration, and the future Square Kilometre Array all involve thousands leveraging massive datasets.
The Conversation