Nothing Special   »   [go: up one dir, main page]
Next Article in Journal
Spatiotemporal Analysis of Water Body in the Chongming Island Region over the Past Decade Based on the ISUNet Model
Next Article in Special Issue
A Question and Answering Service of Typhoon Disasters Based on the T5 Large Language Model
Previous Article in Journal
Evaluating the Impact of Human Activities on Vegetation Restoration in Mining Areas Based on the GTWR
Previous Article in Special Issue
Multi-Scale Non-Local Spatio-Temporal Information Fusion Networks for Multi-Step Traffic Flow Forecasting
You seem to have javascript disabled. Please note that many of the page functionalities won't work as expected without javascript enabled.
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJGI
Volume 13
Issue 4
10.3390/ijgi13040133
ijgi-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

An LLM-Based Inventory Construction Framework of Urban Ground Collapse Events with Spatiotemporal Locations

by
Yanan Hao
1,2,
Jin Qi
1,2,*,
Xiaowen Ma
3,
Sensen Wu
1,2,
Renyi Liu
1,2 and
Xiaoyi Zhang
4
1
School of Earth Sciences, Zhejiang University, Hangzhou 310027, China
2
Zhejiang Provincial Key Laboratory of Geographic Information Science, Hangzhou 310028, China
3
Research Center for Intelligent Technology Standardization, Zhejiang Lab, Hangzhou 311121, China
4
School of Spatial Planning and Design, Hangzhou City University, Hangzhou 310015, China
*
Author to whom correspondence should be addressed.
ISPRS Int. J. Geo-Inf. 2024, 13(4), 133; https://doi.org/10.3390/ijgi13040133
Submission received: 7 February 2024 / Revised: 1 April 2024 / Accepted: 15 April 2024 / Published: 16 April 2024
(This article belongs to the Special Issue Innovative GIS Models and Approaches for Large Environmental and Urban Applications in the Age of AI)
Browse Figures
Figure 1
<p>Map of Zhejiang province, China.</p> "> Figure 2
<p>The framework for UGC inventory construction.</p> "> Figure 3
<p>Prompt templates for the LLM model.</p> "> Figure 4
<p>Evaluation of UGC event recognition model for each city.</p> "> Figure 5
<p>Statistic characteristics of UGC events from 2005 to 2022.</p> "> Figure 5 Cont.
<p>Statistic characteristics of UGC events from 2005 to 2022.</p> "> Figure 6
<p>Probability density risk evaluation results.</p> "> Figure 7
<p>Examples of false-positive events (collapse-related events).</p> "> Figure 8
<p>Geo-hazard frequency quantification of cities in Zhejiang province.</p> ">
Versions Notes

Abstract

:
Historical news media reports serve as a vital data source for understanding the risk of urban ground collapse (UGC) events. At present, the application of large language models (LLMs) offers unprecedented opportunities to effectively extract UGC events and their spatiotemporal information from a vast amount of news reports and media data. Therefore, this study proposes an LLM-based inventory construction framework consisting of three steps: news reports crawling, UGC event recognition, and event attribute extraction. Focusing on Zhejiang province, China, as the test region, a total of 27 cases of collapse events from 637 news reports were collected for 11 prefecture-level cities. The method achieved a recall rate of over 60% and a precision below 35%, indicating its potential for effectively and automatically screening collapse events; however, the accuracy needs to be improved to account for confusion with other urban collapse events, such as bridge collapses. The obtained UGC event inventory is the first open access inventory based on internet news reports, event dates and locations, and collapse co-ordinates derived from unstructured contents. Furthermore, this study provides insights into the spatial pattern of UGC frequency in Zhejiang province, effectively supplementing the statistical data provided by the local government.

1. Introduction

In recent years, urban ground collapse (UGC) [1,2] events have become more frequent with the increased intensity of urban underground space construction, presenting a continuous expansion pattern parallel with urbanization; this has serious consequences for urban economic development and the safety of citizens [1,3]. The contradiction between underground space development and urban safety has intensified so much that it has become urgent that the government formulates better management and control strategies and conducts in-depth research on collapse risk [4,5,6]. News reporting is one of the important data sources for understanding and studying UGC events. Therefore, relevant researchers have collected different unstructured information sources, such as local news media and social network media, and then processed these to construct event databases [7]. Although the analysis of a single event can help to put forward specific measures for the prevention and control of ground collapse regarding collapse mechanisms [8,9], event databases can provide better management references through revealing the spatiotemporal patterns and complex causes of collapses [10,11]. Consequently, combining geographic information technology and text recognition technology to extract UGC events from a large number of news reports and analyzing their spatiotemporal patterns can quantitatively evaluate geological risks at a regional scale, which is of great significance [12,13,14].
Event detection and monitoring have long been the focus of government public affairs management, as the timely understanding of the beginning and evolution of popular social events helps authorities to respond quickly [15,16]. Existing event extraction methods can be generally divided into three categories: pattern matching methods [17], machine learning methods [18], and deep learning methods [19]. Traditional pattern matching methods rely on manually set rules, whereas machine learning methods are based on statistical patterns. In comparison, deep learning models for event extraction have been more widely used in recent years. Examples of these applications can be found in construction accidents [20], improvement [21], football games [22], and so on. To date, large language models (LLMs) have achieved significant breakthroughs in numerous automated learning tasks through unsupervised training on extensive corpora and data sets using generative pre-trained transformers (GPTs). These tasks include machine translation, text classification, intelligent question-answering, and more, and they have even surpassed human performance (on average) in areas such as semantic understanding, common-sense reasoning, and dialogue generation [23,24].
Considering the fact that the urban ground collapse problem involves scattered, irregular distributions and complex characterizations in terms of events, LLMs are an excellent choice for the task of automating the construction of an inventory of UGC events, along with the statistical analysis of their spatiotemporal distribution [25]. Their natural ability to understand language helps LLMs to fully consider contextual words and sentence information [26], such that the extraction of entities and entity relationship is more accurate [27,28]. Through event extraction, historical events can be analyzed more scientifically to effectively prevent future collapse events. Another challenge in UGC data collection is the precise location and time of a collapse event, which has seen growing research interest in the GIS science field [29,30]. For instance, linguistic and text information have been explored using different information extraction methods to transform narratives into spatiotemporal data [31,32,33].
Therefore, to address the challenges of UGC events, this paper presents an LLM-aided framework that is used to crawl and extract unstructured information from news reports to gather structured information for inventory construction. In China, Zhejiang province is a typical region with a significant number of UGCs and other geological hazards. As in other provinces (except for Shenzhen) [34], UGCs in Zhejiang also occur due to a lack of management and accountability, with a lack of official data on historical events. Therefore, the data-processing framework discussed in this paper takes Zhejiang as the test area, but it is also applicable to other provinces. The contributions of this research are summarized as follows: (1) we propose an applicable framework for the recognition and extraction of news information with geographic and time information; (2) based on the framework, we construct the first UGC event database for the provincial level in China, which is representative and complete as the relevant news data were comprehensively manually checked. In addition, the model’s precision and accuracy can be used as a benchmark for other researchers studying the identification of UGC events.
The rest of the paper is structured as follows: Section 2 introduces the geological background and geo-hazard events of Zhejiang province. Section 3 provides details of the methodology of our framework for UGC events extraction. In Section 4, the results of the case study of Zhejiang province are analyzed, and the precision and accuracy of our approach are calculated. Section 5 discusses the sources of error, as well as the differences between events of the same type and different types. The conclusions and future works are summarized in Section 6.

2. Study Area

Zhejiang province is located on the southeast coast of China, with mountains accounting for 74.6% of the land area and diverse landform types (Figure 1). It is a vibrant and economically dynamic province, known for its rich cultural heritage, picturesque landscapes, and significant contributions to the country’s economic growth. Zhejiang has 11 prefecture-level cities, each of which is an administrative unit between province and county. In terms of population, Hangzhou is the capital city of Zhejiang province and ranks first with more than 10 million people. Wenzhou, Ningbo, Jinhua, Taizhou, Jiaxing, and Shaoxing are cities that have a population of more than 5 million, and the remaining four cities have a population of more than 1 million.
The topography of Zhejiang province is complex, and the terrain is in the form of steps from southwest to northeast (tilt). In the southwest of Zhejiang province, the mountains are more than a kilometer high and the highest peak, Huang-mao-jian, is 1929 m above sea level. Dozens of small and large red bed basins are scattered and distributed among these mountains [13]. The northeast plain is a part of the Yangtze River Delta. The outcrop layer in the north Zhejiang Plain mainly consists of quaternary loose deposits. Sedimentary clastic rocks and carbonate rocks are widely distributed in the mountainous area of northwest Zhejiang province. Pyroclastic rock series and granites are widely distributed in the south mountain area of southwest Zhejiang province. Continental clastic rocks are widely distributed in the hilly basin of central Zhejiang province, and volcanic rock series form a low mountainous area around the basin. The hilly and mountainous areas along the southeast coast of Zhejiang province are mainly volcanic series, and the plain areas are quaternary loose sedimentary layers. The Zhoushan and coastal islands consist of volcanic rocks and granites.
The sudden geological disasters in Zhejiang province consist of four main kinds of mountain disasters [35,36]—landslides, debris flows, collapse, and ground collapse—which are mainly distributed in the hilly and mountainous areas of northwest, southwest, central, and southern Zhejiang [37]. Land subsidence is the most important geological hazard in Zhejiang Plain, and is the most serious in Hang-jia-hu Plain and Ning-Feng Plain.
Zhejiang province is one of the most economically developed provinces in China. Rapid economic development in the area has been accompanied by rapid urbanization, and the exploitation and utilization of underground space are becoming more and more frequent. At present, four cities in Zhejiang province, including Hangzhou, Ningbo, Shaoxing, and Wenzhou, have metro systems, while the other seven cities are still in the planning stages in this aspect. The scale of human engineering activities is huge, which not only brings huge benefits to humans but also produces many serious environmental engineering geological problems, such as surface subsidence and ground collapses caused by underground mining. This is also a significant feature of UGCs, which differs from collapse events in traditional geological hazards. A typical city is Hangzhou, located in the eastern coastal area, which has experienced severe UGC events [2,38]. Mountain tunnelling [39] and metro system construction [40,41] have contributed to the occurrence of UGC events in Zhejiang province.

3. Materials and Methods

In order to implement the extraction of UGC event information, we integrated LLM technologies into a framework (Figure 2). The framework primarily comprises three modules: the news report crawling module, which can recursively request a news search service and collect news content; the event recognition module, which can identify and categorize events within news reports; and the event extraction module, which extracts specific arguments or relationships from unstructured text and formulates the report into an events inventory. The LLM technology mainly participates in the latter two modules to assist in the task of structuring event information.

3.1. News Report Crawling

The news report crawling module serves as the foundational component in the UGC inventory construction framework. Its primary function is to collect relevant information from the internet and prepare the necessary content for subsequent analysis. This module functions through a web crawler designed to download web pages from pertinent links. Through employing HTTP requests with predefined keywords such as ‘daolu’ (road), ‘dimian’ (ground), ‘taxian’ (collapse), and specific city names, the news search engine effectively retrieves any pertinent web links. Each request is tailored to a specific city, initiating a targeted search to obtain a comprehensive list of web links. During the request and crawling process, we employed a recursive mechanism to traverse web links and gather comprehensive information. This iterative approach allows us to delve deeply into each web page, extracting not only the provided title, thumbnail, and abstract from the news engine but also accessing the original webpage to gather additional detailed information. This process enables us to obtain a more thorough understanding of news content, including the media sources and news supplementary details that may not be readily available through the initial news engine interface.
Upon accessing the original webpage, various elements are encountered, including the logo, news media name, more news images, advertisements, related news links, and in some cases, a discussion board. In order to extract the true content of the news report from this diverse array of elements, a content filtering algorithm must be implemented. This algorithm is crucial for discerning and isolating the core news content from the surrounding web page elements, ensuring that the retrieved information accurately represents the news report itself. In our model, a simple noise removal strategy is used according to the structure and pattern of the webpage. Due to the concentrated and lengthy nature of text content in news reports, a contextual window with a distance of n lines can be utilized to compute the average number of characters and the variance of characters in each line. In doing so, any text within the window that exhibits an average character count lower than a predetermined value and variance exceeding a specific threshold can be identified as noise and subsequently filtered out.

3.2. LLM-Aided Event Recognition

When utilizing news search engines, a wide array of web pages is returned, encompassing news reports, event briefings, rescue tracking reports, and other related content. However, among these, less relevant content, such as disaster prevention policy comments or disaster prevention product introductions, may also be included. Leveraging LLM technology, we aimed to identify the most relevant instances of UGC news reports and subsequently transform this information into a structured data format. This process aligns with the classic task in the natural language processing (NLP) field known as event extraction. Event extraction involves two primary subtasks: event-type detection and event-argument extraction. Event-type detection focuses on identifying the core words that signify the occurrence of an event.
An LLM learns human natural language logic and background knowledge from large amounts of text data through pre-training, gaining the ability to understand and generate human language. Prompt engineering is needed to guide the LLM when solving complex problems in specialized fields. It refers to the process of the strategic design of the LLM input–prompt word (prompt), which usually needs to specify the LLM task role, task conditions, and task-related expertise. The prompt has obvious characteristics, such as zero-shot or a few examples. In this framework, we used ChatGLM-3b [42], a deep learning model based on transformer architecture, to solve the event extraction task. ChatGLM-3b is widely used in the construction of natural language dialogue systems and has the characteristics of smooth dialogue. Therefore, when designing a prompt, we can take advantage of the interactivity and contextual guidance of ChatGLM-3b.
First, we asked the LLM model some questions about UGC events, and then we further refined or adjusted the extraction requirements based on the model’s answers. Second, the appropriate examples were selected for contextual information about a UGC event to help the model better locate and identify the event. Finally, prompts (Figure 3a) were designed based on (1) making the task objective clear through specifying (in the prompt) that the goal of the task was to determine whether a given news report text was a collapse event and (2) providing an example; in order to help the model better understand the task, a news report of a UGC was given to present the model with a true result. At the same time, a news report about a non-UGC was given to present the model with a false result. (3) For output formatting, we specified (in the prompt) whether the output result was true or false.

3.3. LLM-Aided Event Extraction

The final process is an event argument extraction task in order to identify and extract specific components or elements related to an event from a given text. These attributes often include entities such as the participants (subjects and objects involved in the event), time expressions, locations, and other relevant details. The task can be approached using techniques such as named entity recognition (NER), part-of-speech tagging, syntactic parsing, and semantic role labeling. In our framework, we also used ChatGLM-3b to extract the event attributes, including the name of the event, when and where it occurred, and the area and depth of the collapse. The design of the prompt is similar to the event-recognition process, where a collapse report and its associated attributes and values are provided as a demonstration case, and the model is asked to return results in a specified format (Figure 3b).
Time is an ambiguous and complex concept that can be expressed in many ways, such as absolute time (1 May 2023) and relative time (yesterday, next year, etc.), which presents a challenge for time extraction. The LLM model can better deal with the ambiguity of time extraction through understanding context, normalization, developing an understanding of a time reference frame, multi-modal information fusion, and knowledge introduction.
Another challenge is the geographical location extraction, including geographical name, location description, etc. The name of the place may be ambiguous, for example, “Beijing” can be Beijing City, a place within Beijing City, or a specific neighborhood. In our framework, we utilized a geocoding API to translate the place name into the corresponding geographic coordinate information [43].
An important post-processing task after event attribute extraction is co-reference filtering. Co-reference refers to the phenomenon of multiple news reports or multiple sentences in a news article referring to the same subject. Co-reference problems will affect the accurate extraction of event information, so it is necessary to conduct co-reference ablation. A variety of methods can be used for co-reference filtering. In this framework, we adopted a rule-based approach. As the UGC event is accidental, the co-reference relationship can be identified by judging the spatiotemporal similarity of different extraction results. The events for which the spatiotemporal similarity value is higher than the threshold can be merged through formulating a similarity calculation policy for event attributes.

4. Results and Analysis

4.1. UGC Event Recognition

By crawling the response contents using keywords from the Baidu news portal, we obtained 637 links in total (Table 1). Hangzhou had 220 news pieces, Wenzhou had 79 news pieces, and Ningbo had 83 news pieces, all ranked in the top three, while the number of response content pieces from other cities had an average number of 31. There are great differences in the number of news events reported in different cities. On the other hand, although Baidu’s news search engine was used, it can be seen that in the results returned by the search engine in Table 2, except for the news from Hangzhou, 61.3% of the proportion were reports of collapse events. For the news from the remaining 10 prefecture-level cities, less than 15.7% covered real collapse events. Only a small proportion of the news reports warranted further content extraction.
The computational power required by the LLM prompt and inferencing is not large, and a server with an NVIDIA A100 GPU was used for event recognition and extraction. In order to measure the performance of the LLM model, we calculated the recall, precision, and F1_score. Recall is a measure used to evaluate the ability of a model to identify all relevant instances within a dataset. It is also known as the true-positive rate. Precision is a measure of the accuracy of the positive predictions obtained from the model. The F1_score measures the overall correctness of the model’s predictions. They can be calculated using Equations (1)–(3):
R e c a l l = T P T P + F N
P r e c i s o n = T P T P + F P
F 1 _ s c o r e = 2 × r e c a l l × p r e c i s i o n r e c a l l + p r e c i s i o n
where true positives (TP) are the number of correctly identified positive instances; false negatives (FN) are the number of positive instances that were incorrectly classified as negative; false positives (FP) are the number of negative instances that were incorrectly classified as positive; and true negatives (TN) are the number of correctly identified negative instances.
The results of our UGC identification model were compared with the manually checked results. If the UGC identification model and the manual check marked the news as a true event, then it was regarded as TP. However, if the result after manual checking was not a UGC event, then it was regarded as FP. If the UGC identification model marked the news as a false event, and so did the manual result, then it is regarded as TN; otherwise, it is regarded as FN. If the UGC identification model did not give a formatted result, then it was labeled fail. After manually checking the results of each city in Zhejiang province, we obtained the evaluation results of our UGC identification model, as can be seen in Table 1.
The result can also be seen in Figure 4. The curves represent the recall, precision, and F1 scores in red, green, and blue, respectively. Apart from Huzhou, Jinhua, and Zhoushan, which all had no UGC events, the majority of the F1 scores and precision values are relatively low (under 35%), which means our model is not an ideal tool for recognizing UGC events directly. However, the recall curve is significantly higher than the other two curves. There are two cities with 100% and four cities with more than 60%, which indicates that the LLM model can do well in terms of figuring out relevant events.

4.2. UGC Event Extraction

The evaluation of the event attribute extraction results mainly sought to understand the correctness of the extraction of event occurrence time and place. During the precision calculations, if any attribute in time or place was extracted incorrectly, the extraction failed. In the case of time, the comparison is accurate to the day. For location, the comparison is appropriate for city names. The final results are listed in Table 2. Similarly, if the extraction result did not conform to the required JSON format, it was regarded as fail.
As there was only a single case of UGC in each city except for Hangzhou, the extraction results for Hangzhou can be considered as representative of the overall model performance. As one can see, the accuracy of time recognition is very low: it is only 34%. In contrast, location recognition was correct 49% of the time. This is due to a certain lag in news reports, and the time of the event is often expressed as a relative date, such as yesterday, the day before yesterday, or this afternoon, which leads to the incorrect extraction of the event time by the model; it is easy for the model to confuse the time of the news report with the time of the news event, resulting in extraction errors. As news reports have certain timeliness, when designing time filtering rules, events within 3 days can be defined as news entities with time similarity. Due to the sporadic nature of ground collapse events, very few events will occur in the same place and the same city (at present, only two collapse events occurred on the same day in Hangzhou). Therefore, the definition of place similarity rules according to the combination of city names and county names can effectively distinguish most co-referential events.

4.3. UGC Inventory

The inventoried UGC events correspond to the period of May 2005–May 2022 and consist of 27 events with known locations and dates derived from recognition results, attribute extraction results, and co-reference filtering results (Table 3). Annual and monthly UGC frequency (number of events per year and per month) is depicted in Figure 5a,b, respectively, and a statistical representation of the known collapse area for the UGC inventories is given in Figure 5c.
The curve in Figure 5a is characterized by an increasing trend, especially a subsequent abrupt increase in the last decade. Since 2015, UGC events have accounted for 70.4%, which is similar to the conclusion of Chen’s study [25]. Many studies have indicated that the collapse frequency in the rainy season is higher than that in the non-rainy season. Generally, the rainy season in China refers to April to September in Zhejiang province. Figure 5b shows the monthly distribution of ground collapse accidents in Zhejiang province from 2005 to 2022. Approximately 51.8% of the 27 statistical collapse events occurred from April to September, which is close to the proportion of the number of collapse events during the rainy season in China at 46.7% [2]. Figure 5c gives the statistical characteristics of the collapse area of UGC events; although most of the literature agrees that the area of urban ground collapse is smaller than that of traditional collapse [25], it can be seen that there are still large-scale collapses caused by 2000 square meters of engineering construction.
Of the 27 recorded events, 10 contained information about the collapse area, and six contained information about collapse depth. From the point of view of the collapse area, all of the events are small collapses of less than 1 square kilometer. The length of the collapse is greater than 30 m, indicating a tendency towards elongated forms. In the development and utilization of underground spaces, 0 to 3 m belong to the surface layer, and 3 to 15 m belong to the shallow layer. From the perspective of collapse depth, only one case of pavement collapse caused by heavy rain was 0.5 m deep, and the rest were shallow. Most of the collapses were caused by subway construction (six cases) and engineering foundation pits (six cases), followed by poor engineering quality (five cases) and underground pipeline leakage (three cases). These factors are all related to human activities, especially engineering construction and underground development projects. Therefore, measures to prevent UGC should focus on improving the standardization of engineering and construction projects, including strict supervision and code implementation, high-quality engineering design and construction, and comprehensive risk assessment and monitoring.
Further, we conducted a distribution analysis of UGC events within the cities of Zhejiang province, considering both the events with known co-ordinates (Figure 6 left) and the number of occurrences (Figure 6 right). Using the kernel density model, the spatial distribution characteristics were reclassified into five different levels, from green to red, using the natural breaks classification method [1]. The core of the risk is concentrated in the eastern part of Hangzhou, while the rest are scattered in different prefecture-level cities and were not gathered together. As a rapidly developing city, Hangzhou’s continuous construction activities may have had an impact on the stability of its underground structures. This effect may come from human activities such as subway construction, real estate development, and infrastructure construction. Another possible reason is that there may be a certain bias in the reports of internet news media, which pay more attention to Hangzhou than other cities, resulting in more concentrated coverage of its collapse events.

5. Discussion

5.1. Comparison with Collapse-Related Accidents

In order to figure out the causes of a low accuracy rate but a high recall rate, we further analyzed the false-positive events and found that these misjudgments are basically caused by the misclassification of events. The most commonly misidentified events include tower crane collapse (and scaffold collapse), building collapse (especially the collapse of dangerous buildings), bridge collapse, and viaduct collapse, which are shown in Figure 7. Take Wenzhou as an example: 43 out of 79 reports were judged to be UGC events, but only 5 of them were true positives, and the remaining 38 (accounting for 88.3%) were all other collapse events, most of which were old residential buildings.
These false-negative events are mainly caused by semantic ambiguity, which is also a big challenge in NLP tasks. In some cases, the term tanta is used in news reports, which is semantically similar to taxian (collapse). In some cases, the two terms may be used interchangeably or as synonyms in news reports or related content, potentially leading the search engine to return content containing the keyword tanta as relevant reports for taxian (collapse). Another reason is the relevance algorithm of the search engine might consider various accidents and disasters related to UGC, some of which may involve collapses of buildings or structures. It is a fact that ground collapse events always lead to damage to buildings, structures, municipal facilities, and traffic facilities. As a result, search results may include these related incidents.
In summary, the misjudgments of our LLM model mainly arise from the combined effects of semantic ambiguity and relevance algorithms. In order to more accurately recognize and classify UGC events, further optimization of the LLM model and processing framework is needed. As we have not yet collected a sufficient number of samples for easily confused events, fine-tuning the LLM model is not an appropriate approach at present. A possible solution to this may be to integrate it with a knowledge graph; that is, the semantic representation of the output of the language model is associated and integrated with other semantic information, such as entity, relation, and attribute, in a knowledge graph. Different fusion strategies can be adopted in this step, such as rule-based fusion and similarity-based fusion. Through integrating the language model output and knowledge graphs, the richness and accuracy of semantic representation might be further enhanced. At the same time, the relational and attribute information in a knowledge graph can be used for inference and relational inference, so as to obtain a more complete and accurate classification result.

5.2. Comparison with Other Geo-Hazards Events

Zhejiang Natural Resources and Statistical Yearbook on the Environment [44] is an annual gathering of comprehensive statistical data that are jointly compiled by the Zhejiang Provincial Bureau of Statistics and other relevant departments and bureaus. It reflects the basic situation of Zhejiang’s environment in various fields and summarizes and counts the relevant information on geological disasters, including landslides, rock collapse, debris flow, and ground collapse. In this section, we compare the sum number of geo-hazards with the results of our UGC inventory. Figure 8a–e show the hierarchical distribution of the number of geo-hazards occurring in prefecture-level cities each year from 2016 to 2020. The “natural-break” and “4-level” classification methods were used in the classification. Red indicates the highest level, orange is the second highest level, light green is the second lowest level, and dark green is the lowest level. Figure 8f shows the counting and grading results of the number of UGC events in different cities.
In 2016, the number of geological disasters in the 11 prefecture-level cities in Zhejiang was graded as seven cities at level 1, two cities at level 2, and one city at levels 3 and 4, respectively. Except for Wenzhou and Lishui, most cities are at a low-risk level. In 2017, the number of geological disasters in 11 prefecture-level cities in Zhejiang was graded as two cities at level 1, three cities at level 2, five cities at level 3, and one city at level 4. Most cities are at a moderate risk level, with Zhoushan being the highest. In 2018, the situation changed again, with most cities at low-risk levels, including three cities with a level 1 risk, five cities with a level 2 risk, and one city with a level 3 risk, with Wenzhou and Hangzhou at the highest risk level. The situation in 2019 was similar to that in 2018, except that the risk level in Zhoushan and Quzhou increased from level 2 to level 3, and in Ningbo, it increased from level 1 to level 2. In 2020, Quzhou and Jinhua were at a level 4 risk, and Hangzhou Lishui and Wenzhou were at a level 3 risk.
From the above situation over four years, the change in risk of geological disasters in different cities varies greatly between years. In summary, Wenzhou’s risk level was at level 4 for 3 years and level 3 for 1 year, making it the city with the most intense geological disasters. This is followed by Hangzhou, which had 2 years at level 4 and 2 years at level 3. The four cities of Huzhou, Jiaxing, Ningbo, and Taizhou were at a low-risk level. Other cities presented some irregularities. In terms of UGC, Hangzhou is the city with the highest frequency of occurrence, followed by Ningbo and Shaoxing at level 3, Quzhou, Jinhua, and Wenzhou at level 2, and the remaining 5 cities at level 1.
It can be seen that the distribution of UGC events in the entire province differs significantly for geo-hazards. First, in terms of coverage, the number of cities at risk of geo-hazards far exceeds the number of cities at risk of UGC. Second, there are differences in high-risk cities. The cities with the highest geo-hazard risk are Wenzhou and Hangzhou, while Wenzhou’s UGC risk is relatively low, and Hangzhou occupies a considerable proportion. This significant difference also tells us that, although UGC is closely related to adverse geological conditions, the factors contributing to its occurrence are fundamentally different from those of geo-hazards, and human activities might be a deeper cause [38]. It can also be noticed that, in the years 2018 and 2019, the high-risk cities for UGC and geo-hazards were the same. The probable reason for this might be the heavy rainfall in summertime [45].
Apart from the difference in distribution pattern, our UGC inventory has another advantage over the Zhejiang Natural Resources and Statistical Yearbook on Environment document. There were only two events recorded in the years 2016 to 2020 in the statistical data, while there are 13 events in our UGC inventory. Further, in the statistical yearbook, UGC events can only be found according to a sum count in each city, whereas we give the specific location, probable trigger factors, and the magnitudes of the events, which are all vital for disaster management.

6. Conclusions

In order to accurately identify UGC events from big data in news reports and social media and establish a database for geological disaster prevention and control, this paper discusses the construction of a working framework based on LLM models, which consists of three steps: news report crawling, UGC event recognition, and event attribute extraction. Leveraging the advantages and convenience of GPT in semantic understanding and information extraction, this framework utilized the Baidu search engine and the guidance of UGC event-related keywords, achieving the collection of 637 news reports and the identification and extraction of 27 cases of UGC in Zhejiang province, China, including the latitude, longitude, and time information within a time window of 3 days.
We obtained a UGC event inventory based on the proposed framework, which has been manually checked comprehensively (i.e., for each item). The UGC events in the inventory occurred from May 2005 to May 2022, including 27 cases. This is the first open access inventory of UGC events based on internet news reports, and event dates, locations, and collapse areas derived from unstructured content. The kernel density model shows that the core of UGC risk is concentrated in the eastern part of Hangzhou, while the rest are scattered over different prefecture-level cities and are not spatially concentrated. Compared with geo-hazard statistical data, the inventory has its specialty in reflecting the spatial pattern of UGC frequency in the cities of Zhejiang province which, in turn, proves that UGC is not only caused by adverse geological conditions, but also human activities. This historical database can be used as basic data by relevant departments for the prevention and control of urban geological hazards.
However, the framework and its utilized LLM models still need to be improved. Evaluation of the model showed that (1) the LLM model shows an advantage of being able to identify events, with a recall higher than 60%, yet it is still weak in distinguishing the collapse event type between urban ground collapse and the collapse of artificial buildings or structures. The use of an ontology-type recognition method might solve this problem; (2) the LLM model shows 34% precision in extracting the date of the UGC event and 49% precision in extracting the location of the UGC event. The frequent occurrence of time descriptors in news reports interferes with event extraction, and the time window of 3 days is an optimal choice for co-conference filtering. More work is needed to improve the accuracy of the proposed method in the future.

Author Contributions

Conceptualization, Jin Qi, Renyi Liu and Xiaoyi Zhang; Methodology, Jin Qi and Xiaoyi Zhang; Software, Yanan Hao; Validation, Jin Qi; Formal analysis, Yanan Hao; Investigation, Jin Qi and Xiaowen Ma; Resources, Renyi Liu; Writing—original draft, Yanan Hao and Xiaowen Ma; Writing—review & editing, Jin Qi and Xiaoyi Zhang; Visualization, Yanan Hao; Supervision, Xiaoyi Zhang; Project administration, Sensen Wu; Funding acquisition, Sensen Wu. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the National Key Research and Development Program of China (grant 2021YFB3900900); the Provincial Key R&D Program of Zhejiang (grant 2021C01031); the Deep-time Digital Earth (DDE) Big Science Program; the joint funds of the Zhejiang Provincial Natural Science Foundation of China (grant LHZY24A010001) and the Scientific Research Foundation of Zhejiang University City College (grant X-202208).

Data Availability Statement

The data presented in this study are available upon request from the corresponding author. The data are not publicly available due to restrictions imposed by privacy approval and informed consent agreements regarding study participants.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Zhang, X.; Ruan, Y.; Xuan, W.; Bao, H.; Du, Z. Risk Assessment and Spatial Regulation on Urban Ground Collapse Based on Geo-Detector: A Case Study of Hangzhou Urban Area. Nat. Hazards 2023, 118, 525–543. [Google Scholar] [CrossRef]
  2. Wang, X.-W.; Xu, Y.-S. Investigation on the Phenomena and Influence Factors of Urban Ground Collapse in China. Nat. Hazards 2022, 113, 1–33. [Google Scholar] [CrossRef]
  3. Pellicani, R.; Spilotro, G.; Gutiérrez, F. Susceptibility Mapping of Instability Related to Shallow Mining Cavities in a Built-up Environment. Eng. Geol. 2017, 217, 81–88. [Google Scholar] [CrossRef]
  4. Liu, R.; Sun, H.; Qin, J.; Zheng, Z. A Multi-Geophysical Approach to Assess Potential Sinkholes in an Urban Area. Eng. Geol. 2023, 318, 107100. [Google Scholar] [CrossRef]
  5. Zhang, C.; Kwon, S.; Oh, J.; Park, K. An Integrated AHP-PROMETHEE II Ranking Method to Evaluate the Resilience of Sewer Networks Considering Urban Flood and Ground Collapse Risks. Water Sci. Technol. 2023, 87, 1438–1453. [Google Scholar] [CrossRef] [PubMed]
  6. Zhou, H.; Liu, C.; Wang, G.; Kang, K.; Liu, Y. Study on Drilling Ground Collapse Induced by Groundwater Flow and Prevention Based on a Coupled CFD-DEM Method. KSCE J. Civ. Eng. 2022, 26, 2112–2125. [Google Scholar] [CrossRef]
  7. Tufano, R.; Guerriero, L.; Annibali Corona, M.; Bausilio, G.; Di Martire, D.; Nisio, S.; Calcaterra, D. Anthropogenic Sinkholes of the City of Naples, Italy: An Update. Nat. Hazards 2022, 112, 2577–2608. [Google Scholar] [CrossRef]
  8. Tan, F.; Tan, W.; Yan, F.; Qi, X.; Li, Q.; Hong, Z. Model Test Analysis of Subsurface Cavity and Ground Collapse Due to Broken Pipe Leakage. Appl. Sci. 2022, 12, 13017. [Google Scholar] [CrossRef]
  9. He, Q.; Tan, F.; Peng, Z.; Tao, L.; Jiao, Y.; Liu, S.; Peng, H. Genesis Analysis of Ground Collapse in Wuhan Based on 3D Geological Model. Front. Earth Sci. 2022, 10, 934452. [Google Scholar] [CrossRef]
  10. Zerhouny, M.; Fadil, A.; Hakdaoui, M. Underground Space Utilization in the Urban Land-Use Planning of Casablanca (Morocco). Land 2018, 7, 143. [Google Scholar] [CrossRef]
  11. Lu, W.; Wu, Y.; Choguill, C.L.; Lai, S.-K.; Luo, J. Underground Hangzhou: The Challenge of Safety vs. Commerciality in a Major Chinese City. Cities 2021, 119, 103414. [Google Scholar] [CrossRef]
  12. Gutiérrez, F.; Parise, M.; De Waele, J.; Jourde, H. A Review on Natural and Human-Induced Geohazards and Impacts in Karst. Earth-Sci. Rev. 2014, 138, 61–88. [Google Scholar] [CrossRef]
  13. Xie, H.; Zhang, Y.; Chen, Y.; Peng, Q.; Liao, Z.; Zhu, J. A Case Study of Development and Utilization of Urban Underground Space in Shenzhen and the Guangdong-Hong Kong-Macao Greater Bay Area. Tunn. Undergr. Space Technol. 2021, 107, 103651. [Google Scholar] [CrossRef]
  14. Zhou, S.; Zhai, G. A Multi-Hazard Risk Assessment Framework for Urban Disaster Prevention Planning: A Case Study of Xiamen, China. Land 2023, 12, 1884. [Google Scholar] [CrossRef]
  15. Yu, M.; Bambacus, M.; Cervone, G.; Clarke, K.; Duffy, D.; Huang, Q.; Li, J.; Li, W.; Li, Z.; Liu, Q. Spatiotemporal Event Detection: A Review. Int. J. Digit. Earth 2020, 13, 1339–1365. [Google Scholar] [CrossRef]
  16. Wang, W.; Stewart, K. Spatiotemporal and Semantic Information Extraction from Web News Reports about Natural Hazards. Comput. Environ. Urban Syst. 2015, 50, 30–40. [Google Scholar] [CrossRef]
  17. Peuquet, D.J.; Robinson, A.C.; Stehle, S.; Hardisty, F.A.; Luo, W. A Method for Discovery and Analysis of Temporal Patterns in Complex Event Data. Int. J. Geogr. Inf. Sci. 2015, 29, 1588–1611. [Google Scholar] [CrossRef]
  18. Eligüzel, N.; Çetinkaya, C.; Dereli, T. Comparison of Different Machine Learning Techniques on Location Extraction by Utilizing Geo-Tagged Tweets: A Case Study. Adv. Eng. Inform. 2020, 46, 101151. [Google Scholar] [CrossRef]
  19. Feng, Y.; Sester, M. Extraction of Pluvial Flood Relevant Volunteered Geographic Information (VGI) by Deep Learning from User Generated Texts and Photos. ISPRS Int. J. Geo-Inf. 2018, 7, 39. [Google Scholar] [CrossRef]
  20. Zhang, F.; Fleyeh, H.; Wang, X.; Lu, M. Construction Site Accident Analysis Using Text Mining and Natural Language Processing Techniques. Autom. Constr. 2019, 99, 238–248. [Google Scholar] [CrossRef]
  21. Khalef, R.; El-adaway, I.H. Automated Identification of Substantial Changes in Construction Projects of Airport Improvement Program: Machine Learning and Natural Language Processing Comparative Analysis. J. Manag. Eng. 2021, 37, 04021062. [Google Scholar] [CrossRef]
  22. Huang, Y.; Li, Y.; Shan, J. Spatial-Temporal Event Detection from Geo-Tagged Tweets. ISPRS Int. J. Geo-Inf. 2018, 7, 150. [Google Scholar] [CrossRef]
  23. Kasneci, E.; Seßler, K.; Küchemann, S.; Bannert, M.; Dementieva, D.; Fischer, F.; Gasser, U.; Groh, G.; Günnemann, S.; Hüllermeier, E.; et al. ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education. Learn. Individ. Differ. 2023, 103, 102274. [Google Scholar] [CrossRef]
  24. Kauf, C.; Ivanova, A.A.; Rambelli, G.; Chersoni, E.; She, J.S.; Chowdhury, Z.; Fedorenko, E.; Lenci, A. Event Knowledge in Large Language Models: The Gap between the Impossible and the Unlikely. Cogn. Sci. 2023, 47, e13386. [Google Scholar] [CrossRef]
  25. Chen, Y.; Tang, R.; Fang, J.; Kong, F. Analysis on Spatio-Temporal Law and Causation of Urban Road Collapse in China from 2014 to 2018. Water Resour. Hydropower Eng. 2020, 51, 108–116. [Google Scholar] [CrossRef]
  26. Yang, S.; Feng, D.; Qiao, L.; Kan, Z.; Li, D. Exploring Pre-Trained Language Models for Event Extraction and Generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, 28 July–2 August 2019; pp. 5284–5294. [Google Scholar] [CrossRef]
  27. Liang, Y.; Liu, Y.; Wang, X.; Zhao, Z. Exploring Large Language Models for Human Mobility Prediction under Public Events. arXiv 2023, arXiv:2311.17351. [Google Scholar] [CrossRef]
  28. Huang, F.; Huang, Q.; Zhao, Y.; Qi, Z.; Wang, B.; Huang, Y.; Li, S. A Three-Stage Framework for Event-Event Relation Extraction with Large Language Model. In Proceedings of the International Conference on Neural Information Processing, Changsha, China, 20–23 November 2023; Springer: Singapore, 2023; pp. 434–446. [Google Scholar] [CrossRef]
  29. Abraham, S.; Mäs, S.; Bernard, L. Extraction of Spatio-Temporal Data about Historical Events from Text Documents. Trans. GIS 2018, 22, 677–696. [Google Scholar] [CrossRef]
  30. Kokla, M.; Guilbert, E. A Review of Geospatial Semantic Information Modeling and Elicitation Approaches. ISPRS Int. J. Geo-Inf. 2020, 9, 146. [Google Scholar] [CrossRef]
  31. Hu, X.; Zhou, Z.; Li, H.; Hu, Y.; Gu, F.; Kersten, J.; Fan, H.; Klan, F. Location Reference Recognition from Texts: A Survey and Comparison. ACM Comput. Surv. 2023, 56, 1–37. [Google Scholar] [CrossRef]
  32. Sermet, Y.; Demir, I. Towards an Information Centric Flood Ontology for Information Management and Communication. Earth Sci. Inform. 2019, 12, 541–551. [Google Scholar] [CrossRef]
  33. Wang, B.; Wu, L.; Li, W.; Qiu, Q.; Xie, Z.; Liu, H.; Zhou, Y. A Semi-Automatic Approach for Generating Geological Profiles by Integrating Multi-Source Data. Ore Geol. Rev. 2021, 134, 104190. [Google Scholar] [CrossRef]
  34. Guo, L.; Lei, C.; Xiong, C. Application Research of Ground Penetrating Radar Technology in Prevention and Control of Ground Collapse Disaster in Shenzhen. In Proceedings of the SEG 2019 Workshop: Geophysics for Smart City Development, Beijing, China, 29–31 July 2019; Society of Exploration Geophysicists: Houston, TX, USA, 2019; p. 19. [Google Scholar] [CrossRef]
  35. Liao, X.; Xu, W.; Zhang, J.; Qiao, Y.; Meng, C. Analysis of Affected Population Vulnerability to Rainstorms and Its Induced Floods at County Level: A Case Study of Zhejiang Province, China. Int. J. Disaster Risk Reduct. 2022, 75, 102976. [Google Scholar] [CrossRef]
  36. Liu, B.; Siu, Y.L.; Mitchell, G. A Quantitative Model for Estimating Risk from Multiple Interacting Natural Hazards: An Application to Northeast Zhejiang, China. Stoch. Environ. Res. Risk Assess. 2017, 31, 1319–1340. [Google Scholar] [CrossRef]
  37. Ding, J.; Zhang, G.; Gao, J.; Wang, S.; Xue, B.; Wang, K.; Yang, J.; Jiang, R.; Lian, J. The Rainfall Thresholds and Soil Characteristics of Large Geological Disasters in Zhejiang, China. Arab. J. Geosci. 2022, 15, 559. [Google Scholar] [CrossRef]
  38. Zhou, L.; Zhang, X.; Shen, H.; Zhang, D.; Bao, H. GIS Analysis of Urban Ground Collapse Susceptibility: A Case Study of Eastern Hangzhou, China. Front. Earth Sci. 2024, 11, 1338877. [Google Scholar] [CrossRef]
  39. Yan, F.; Qiu, W.; Sun, K.; Jiang, S.; Huang, H.; Hong, Y.; Hou, Z. Investigation of a Large Ground Collapse, Water Inrush and Mud Outburst, and Countermeasures during Subway Excavation in Qingdao: A Case Study. Tunn. Undergr. Space Technol. 2021, 117, 104127. [Google Scholar] [CrossRef]
  40. Gong, X.; Zhang, X. Excavation Collapse of Hangzhou Subway Station in Soft Clay and Numerical Investigation Based on Orthogonal Experiment Method. J. Zhejiang Univ. Sci. A 2012, 13, 760–767. [Google Scholar] [CrossRef]
  41. Chen, R.P.; Li, Z.C.; Chen, Y.M.; Ou, C.Y.; Hu, Q.; Rao, M. Failure Investigation at a Collapsed Deep Excavation in Very Sensitive Organic Soft Clay. J. Perform. Constr. Facil. 2015, 29, 04014078. [Google Scholar] [CrossRef]
  42. Zeng, A.; Liu, X.; Du, Z.; Wang, Z.; Lai, H.; Ding, M.; Yang, Z.; Xu, Y.; Zheng, W.; Xia, X.; et al. Glm-130b: An Open Bilingual Pre-Trained Model. arXiv 2022, arXiv:2210.02414. [Google Scholar] [CrossRef]
  43. Chen, W.; Zhou, Y.; Stokes, E.C.; Zhang, X. Large-Scale Urban Building Function Mapping by Integrating Multi-Source Web-Based Geospatial Data. Geo-Spat. Inf. Sci. 2023, 1–15. [Google Scholar] [CrossRef]
  44. Qin, C.; Tang, Z.; Chen, J.; Chen, X. The Impact of Soil and Water Resource Conservation on Agricultural Production-an Analysis of the Agricultural Production Performance in Zhejiang, China. Agric. Water Manag. 2020, 240, 106268. [Google Scholar] [CrossRef]
  45. Du, M.; Hou, Y.; Qi, P.; Wang, K. The Impact of Different Historical Typhoon Tracks on Storm Surge: A Case Study of Zhejiang, China. J. Mar. Syst. 2020, 206, 103318. [Google Scholar] [CrossRef]
Ijgi 13 00133 g001
Figure 1. Map of Zhejiang province, China.
Figure 1. Map of Zhejiang province, China.
Ijgi 13 00133 g001
Ijgi 13 00133 g002
Figure 2. The framework for UGC inventory construction.
Figure 2. The framework for UGC inventory construction.
Ijgi 13 00133 g002
Ijgi 13 00133 g003
Figure 3. Prompt templates for the LLM model.
Figure 3. Prompt templates for the LLM model.
Ijgi 13 00133 g003
Ijgi 13 00133 g004
Figure 4. Evaluation of UGC event recognition model for each city.
Figure 4. Evaluation of UGC event recognition model for each city.
Ijgi 13 00133 g004
Ijgi 13 00133 g005aIjgi 13 00133 g005b
Figure 5. Statistic characteristics of UGC events from 2005 to 2022.
Figure 5. Statistic characteristics of UGC events from 2005 to 2022.
Ijgi 13 00133 g005aIjgi 13 00133 g005b
Ijgi 13 00133 g006
Figure 6. Probability density risk evaluation results.
Figure 6. Probability density risk evaluation results.
Ijgi 13 00133 g006
Ijgi 13 00133 g007
Figure 7. Examples of false-positive events (collapse-related events).
Figure 7. Examples of false-positive events (collapse-related events).
Ijgi 13 00133 g007
Ijgi 13 00133 g008
Figure 8. Geo-hazard frequency quantification of cities in Zhejiang province.
Figure 8. Geo-hazard frequency quantification of cities in Zhejiang province.
Ijgi 13 00133 g008
Table 1. Results comparison between LLM model and manual checking.
Table 1. Results comparison between LLM model and manual checking.
City NameNews CountsLLM ResultManual ResultModel Evaluation
TrueFalseFailTruePercentageFalsePercentageRPF1
Hangzhou220127435013561.3%8538.7%74.07%78.74%76.34%
Huzhou301512300%30100%0.00%0.00%0.00%
Jiaxing40191110410.0%3690.0%100.00%21.05%34.78%
Jinhua351611100%35100%0.00%0.00%0.00%
Lishui26998311.5%2388.5%66.67%22.22%33.33%
Ningbo8332361533.6%8096.3%100.00%9.38%17.14%
Quzhou191072315.7%1684.2%66.67%20.00%30.77%
Shaoxing321113839.3%2990.6%66.67%18.18%28.57%
Taizhou49221116612.2%4387.8%50.00%13.64%21.43%
Wenzhou7943221456.3%7493.7%40.00%4.65%8.33%
Zhoushan24717000.0%24100%0.00%0.00%0.00%
Table 2. The precision of event attribute extraction on the LLM model.
Table 2. The precision of event attribute extraction on the LLM model.
City NameTrue
Positive		TimeLocationModel
Precision
RightWrongFailRightWrongFail
Hangzhou100345794942934%
Huzhou00000000
Jiaxing422040050%
Jinhua00000000
Lishui211020050%
Ningbo3300300100%
Quzhou211020050.0%
Shaoxing20202000%
Taizhou311111133.3%
Wenzhou211011050.0%
Zhoushan00000000
Table 3. UGC events that occurred in the period 2005–2022 in Zhejiang province.
Table 3. UGC events that occurred in the period 2005–2022 in Zhejiang province.
No.DateLon.Lat.CityLocationFactorLength
(m2)		Area
(m2)		Depth
(m)
118 May 2022120.31330.309HangzhouJinshahu Park Sunken PlazaPipe burst\\\
230 August 2021120.13030.271HangzhouSoutheast corner of Academy Road and Tianmushan Road\\12\
329 December 2020120.12530.269HangzhouXixi Road SidewalkSubway construction\25\
430 June 2020120.15330.161HangzhouDongguan Road Puyan Road Farmers’ Market, Binjiang DistrictHeavy rain\8~90.5
530 June 2020120.32830.318HangzhouIntersection of Shangsha Road and Xueyuan Street, XiashaHeavy rain\\\
62 October 2021120.28030.442HangzhouIntersection of Heyu Road and Xinzhou Road, Linping DistrictCollapse of foundation pit\84
715 November 2008120.23430.168HangzhouXianghu Station of Metro Line 1Collapse of foundation pit75\15
821 March 2021120.18330.311HangzhouResettlement housing site of ShenjiaNanyuan, opposite to Dongxinyuan, Changbang Road, Xiacheng DistrictCollapse of foundation pit40\\
928 August 2019120.18030.268HangzhouBao-shan-qiao to North Jianguo Road station at Metro Line 5Subway construction\\\
1016 August 2019120.18230.244HangzhouUnder the approach bridge to Hangzhou City Station Railway StationSubway construction\\\
117 May 2021120.07230.330HangzhouJinjiadu to Zijinhang Road Station Interval at Metro Line 4Subway construction\\\
1211 June 2018120.06430.347HangzhouYunhe Lake construction project site in the north of SanDunCollapse of foundation pit\2000\
137 December 2018120.18030.252HangzhouIn front of KuiXiang Community, Jianguo Middle RoaSubway construction\\\
1421 April 2016120.12930.282HangzhouIntersection of WenEr Road and XueYuan RoadRainwater pipes leaking\20\
1522 February 2016120.10230.175HangzhouYunqi HNA Hotel, No. 1, Meiling South RoadRoad construction\250\
1619 October 2015120.16230.339HangzhouChuXin Road No. 31Collapse of foundation pit\103
171 April 2014120.20730.240HangzhouChengXing Road Station at Metro Line 4Subway construction\\\
1825 February 2009120.20030.282HangzhouPu-jia-qiao section on the Canal Walking TrailEngineering quality\\\
197 January 2008120.18430.277HangzhouGenShan Canal BridgeWater pipe burst\303~4
2013 March 2012120.09630.294HangzhouProject site of Wuzhou International Plaza at the intersection of Gudun Road and Yuhangtang RoadCollapse of foundation pit\\\
2126 April 2021120.56930.045ShaoxingIntersection of Guandu Road and Henghu RoadEngineering quality\\\
2214 January 2018120.57530.047ShaoxingJiayuan PlazaEngineering quality\\\
2318 October 2010119.88828.982JinhuaFormer Dongfeng Perspex Company in Wuyi CountyAbandoned mines\\\
2410 September 2020120.61027.960WenzhouShanghui Road, Louqiao StreetEngineering quality\\\
2510 March 2019121.63129.883NingboGaoxin District Baolong PlazaEngineering quality30150\
269 May 2005120.93830.040NingboXiao-Yong Railway between Yuyao West and Yiting IntervalEarth cutting150\10
279 August 2006118.75628.831QuzhouHouxi Street, Jiangshan City, Quzhou, Zhe-Gan LineGeological cave collapse\\\
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Hao, Y.; Qi, J.; Ma, X.; Wu, S.; Liu, R.; Zhang, X. An LLM-Based Inventory Construction Framework of Urban Ground Collapse Events with Spatiotemporal Locations. ISPRS Int. J. Geo-Inf. 2024, 13, 133. https://doi.org/10.3390/ijgi13040133

AMA Style

Hao Y, Qi J, Ma X, Wu S, Liu R, Zhang X. An LLM-Based Inventory Construction Framework of Urban Ground Collapse Events with Spatiotemporal Locations. ISPRS International Journal of Geo-Information. 2024; 13(4):133. https://doi.org/10.3390/ijgi13040133

Chicago/Turabian Style

Hao, Yanan, Jin Qi, Xiaowen Ma, Sensen Wu, Renyi Liu, and Xiaoyi Zhang. 2024. "An LLM-Based Inventory Construction Framework of Urban Ground Collapse Events with Spatiotemporal Locations" ISPRS International Journal of Geo-Information 13, no. 4: 133. https://doi.org/10.3390/ijgi13040133

APA Style

Hao, Y., Qi, J., Ma, X., Wu, S., Liu, R., & Zhang, X. (2024). An LLM-Based Inventory Construction Framework of Urban Ground Collapse Events with Spatiotemporal Locations. ISPRS International Journal of Geo-Information, 13(4), 133. https://doi.org/10.3390/ijgi13040133

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop