CN108694188B - A method and related device for updating index data - Google Patents

Abstract

The embodiment of the invention discloses a method for updating index data, which can be applied to an index data updating device, wherein the index data updating device comprises a memory area and a hard disk area, and the method comprises the steps of receiving an index data updating request; updating the index data in the memory area according to the index data updating request, and obtaining first index data; and if the first index data meets the preset index data updating condition, updating the index data in the hard disk area according to the first index data, and obtaining second index data. The invention also provides an index data updating device. According to the embodiment of the invention, the index data can be stored in a grading manner, and different storage modes are respectively adopted for different index data types, so that the first index data is updated in real time, and the second index data is updated periodically, thereby effectively balancing the updating of the index data, reducing the memory consumption and the computing resource and reducing the computing cost.

Description

A method and related device for updating index data

technical field

The invention relates to the technical field of the Internet, in particular to a method for updating index data and a related device.

Background technique

With the continuous development of Internet technology, it is becoming more and more common for users to use the Internet to search. In order to search, it is usually necessary to index the search data in the database first. An index is a structure that sorts the values of one or more columns in a database table, using an index to quickly access specific information in a database table. In the process of accessing various vertical services, the retrieval platform generally adopts different index storage solutions according to the specific types of services, such as data volume, data update volume, and retrieval volume and other business characteristics.

At present, for businesses with a large amount of data and infrequent index updates, solid state drives (Solid State Drives, SSD) or disk storage methods are generally used, that is, the full index is stored in the SSD or hard disk, and the data is updated. The online effective time is almost equal to the time for full update of index data.

However, although the index storage method using SSD or hard disk reduces the cost to a certain extent, it is limited by the physical characteristics of SSD or hard disk. Simultaneous index update and query will bring simultaneous "reading" and "writing" to the hard disk. question. In order to solve the problem of "reading" and "writing" at the same time, the index generally does not allow real-time update, but adopts a full update method with a long period. At this time, in order to make the incrementally updated index data go online as soon as possible, it is necessary to reduce the cycle of full index update as much as possible, resulting in an increase in computing costs.

Contents of the invention

The embodiment of the present invention provides a method for updating index data and related devices, which can store index data hierarchically, that is, index data is divided into first index data and second index data, and different index data types can be used for different index data types. In this way, the second index data can be updated periodically while the first index data is updated in real time, so that the update of index data can be effectively balanced, memory consumption and computing resources can be reduced, and computing costs can be reduced.

In view of this, the first aspect of the present invention provides a method for updating index data, the method is applied to an index data updating device, and the index data updating device includes a memory area and a hard disk area, and the method includes:

Receive index data update request;

update the index data in the memory area according to the index data update request, and obtain first index data;

If the first index data satisfies a preset index data update condition, then update the index data in the hard disk area according to the first index data, and obtain second index data.

The second aspect of the present invention provides an index data update device, the index data update device includes a memory area and a hard disk area, and the index data update device includes:

A first receiving module, configured to receive an index data update request;

A first updating module, configured to update the index data in the memory area according to the index data update request received by the first receiving module, and obtain first index data;

A second update module, configured to update the index data in the hard disk area according to the first index data if the first index data updated by the first update module satisfies a preset index data update condition. , and get the second index data.

It can be seen from the above technical solutions that the embodiments of the present invention have the following advantages:

In an embodiment of the present invention, a method for updating index data is provided. The method can be applied to an index data updating device. The index data updating device includes a memory area and a hard disk area. First, the index data updating device receives an index data updating request, and then Update the index data in the memory area according to the index data update request, and obtain the first index data, if the first index data meets the preset index data update conditions, then update the index data in the hard disk area according to the first index data , and get the second index data. Through the above method, the index data can be stored hierarchically, that is, the index data can be divided into the first index data and the second index data, and different storage methods can be adopted for different index data types, so as to update the first index in real time At the same time, the second index data is updated periodically, so that the update of the index data can be effectively balanced, memory consumption and computing resources are reduced, and computing costs are reduced.

Description of drawings

FIG. 1 is a topological diagram of a memory index independent storage solution in an embodiment of the present invention;

FIG. 2 is a topological diagram of a hard disk index independent storage solution in an embodiment of the present invention;

FIG. 3 is a topological diagram of combined storage of memory index and hard disk index in an embodiment of the present invention;

FIG. 4 is a schematic diagram of an embodiment of a method for updating index data in an embodiment of the present invention;

Fig. 5 is a schematic flow chart of data retrieval in the embodiment of the present invention;

FIG. 6 is a schematic diagram of an embodiment of an index data updating device in an embodiment of the present invention;

FIG. 7 is a schematic diagram of an embodiment of an index data updating device in an embodiment of the present invention;

FIG. 8 is a schematic diagram of an embodiment of an index data updating device in an embodiment of the present invention;

FIG. 9 is a schematic diagram of an embodiment of an index data updating device in an embodiment of the present invention;

FIG. 10 is a schematic diagram of an embodiment of an index data updating device in an embodiment of the present invention;

Fig. 11 is a schematic diagram of an embodiment of an index data updating device in an embodiment of the present invention;

FIG. 12 is a schematic diagram of an embodiment of an index data updating device in an embodiment of the present invention;

FIG. 13 is a schematic diagram of an embodiment of an index data updating device in an embodiment of the present invention;

FIG. 14 is a schematic structural diagram of an index data updating device in an embodiment of the present invention.

Detailed ways

The embodiment of the present invention provides a method for updating index data and related devices, which can store index data hierarchically, that is, index data is divided into first index data and second index data, and different index data types can be used for different index data types. In this way, the second index data can be updated periodically while the first index data is updated in real time, so that the update of index data can be effectively balanced, memory consumption and computing resources can be reduced, and computing costs can be reduced.

The terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of the present invention and the above drawings are used to distinguish similar objects, and not necessarily Used to describe a specific sequence or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances such that the embodiments of the invention described herein are, for example, capable of practice in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", as well as any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or device comprising a sequence of steps or elements is not necessarily limited to the expressly listed instead, may include other steps or elements not explicitly listed or inherent to the process, method, product or apparatus.

It should be understood that the present invention can be applied to the memory area and the hard disk area of the server, wherein the memory area is the memory, and the hard disk area is the hard disk. At present, there are usually two ways to store index data, one is to store in the memory , while the other is stored in the hard disk. The following will briefly introduce these two independent storage solutions.

The specific method of storing index data in the memory is as follows. Please refer to Figure 1. Figure 1 is a topology diagram of the independent memory index storage solution in the embodiment of the present invention. As shown in the figure, there are incremental indexes and full indexes stored in the memory. Search The engine can call the information in the memory index. When the user searches for data, he can use the search engine to find the corresponding index data from the memory index, and determine the corresponding search result according to the index data, and then send it to the client for presentation to the user. When data is updated, the incremental index in the memory index will be updated in real time, and the full index will also be updated synchronously.

The specific method of storing index data on the hard disk is as follows. Please refer to Figure 2. Figure 2 is a topological diagram of the independent storage scheme of the hard disk index in the embodiment of the present invention. As shown in the figure, the full index is stored in the hard disk, and the search engine can call the information in the index. When the user searches for data, he can use the search engine to find the corresponding index data from the hard disk index, and determine the corresponding search result according to the index data, and then send it to the client for presentation to the user. When data is updated, the full index in the hard disk index will also be updated.

Compared with the above two storage schemes, the index data storage solution adopted by the present invention can greatly reduce the index storage cost. Please refer to Fig. 3. Fig. 3 is a topological diagram of memory index and hard disk index combined storage in the embodiment of the present invention. As shown in the figure, a search service can be divided into three layers, namely the search access layer and the search engine implementation layer and data connection layer. Among them, the retrieval access layer is mainly used to shield the implementation details of the internal search engine from the outside, and its main functions include: retrieval access, merging of retrieval results, deduplication of retrieval results, and access to data update requests. The search engine implementation layer is mainly used to divide all storage into two parts during internal implementation, that is, memory index and hard disk index. The memory index storage part keeps updated data in the latest period, and mainly receives real-time data update requests and data retrieval requests, while The hard disk index storage part keeps the latest full index data, and mainly receives data retrieval requests, not real-time data update requests. The data connection layer is mainly used to realize the connection between the memory and the hard disk index data, the regular merging of the full amount of data, and the regular update of the full amount of data.

The following will introduce the method for updating index data in the present invention from the perspective of the device for updating index data. Please refer to FIG. 4 . An embodiment of the method for updating index data in the embodiment of the present invention includes:

101. The index data update device receives an index data update request, wherein the index data update device includes a memory area and a hard disk area;

In this embodiment, the index data update device deployed in the server first receives an index data update request, and the index data update request will be synchronized to the memory area and the hard disk area. The latest update data is stored in the memory area, which is mainly used to receive index data update requests and data retrieval requests in real time, while the hard disk area stores a copy of the latest full index data, which is mainly used to receive data retrieval requests, but not in real time. Receive index data update requests.

Index data is a decentralized storage structure created to speed up the retrieval of data rows in a table. The index is established for the table, which is composed of index pages other than the data page. The rows in each index page will contain logical pointers to speed up the retrieval of physical data.

Index data mainly includes sequential data, inverted data and summary data. Due to the differences in data size, update frequency, and retrieval volume of different businesses, the index storage and update models adopted for different business types are different. For example: For a business with frequent data updates, the storage model of index data should ensure the performance of data updates and data integrity as much as possible. Among them, the sequential index refers to the mapping from a document to all the words contained in this document. An inverted index is a mapping of a term to all documents containing that term.

102. The index data updating device updates the index data in the memory area according to the index data update request, and obtains the first index data;

In this embodiment, next, the index data update device will update the index data in the memory area in real time according to the index data update request, wherein the first index data includes an incremental index, and the incremental index can specifically refer to the Add new index data in the memory area. For example, the A list in the memory area originally had index data a and index data b, but now the A list has been updated to add index data c, and the index data c is an incremental index. Then the index data a, the index data b and the index data c together constitute the first index data.

103. If the first index data satisfies the preset index data updating condition, the index data updating device updates the index data in the hard disk area according to the first index data, and obtains the second index data.

In this embodiment, if the first index data stored in the memory area satisfies the preset index data update condition, then the index data updating device will use the first index data to update the index data in the hard disk area, and obtain the second index data data, the second index data can also be referred to as full data.

Similarly, assuming that the B list in the hard disk area has index data x, index data y, and index data z, now the first index data (ie, index data a, index data b, and index data c) in the memory area is combined with the B list The second index data can be obtained by merging the data in the index data, which includes index data a, index data b, index data c, index data x, index data y and index data z.

In an embodiment of the present invention, a method for updating index data is provided. The method can be applied to an index data updating device. The index data updating device includes a memory area and a hard disk area. First, the index data updating device receives an index data updating request, and then Update the index data in the memory area according to the index data update request, and obtain the first index data, if the first index data meets the preset index data update conditions, then update the index data in the hard disk area according to the first index data , and get the second index data. Through the above method, the index data can be stored hierarchically, that is, the index data can be divided into the first index data and the second index data, and different storage methods can be adopted for different index data types, so as to update the first index in real time At the same time, the second index data is updated periodically, so that the update of the index data can be effectively balanced, memory consumption and computing resources are reduced, and computing costs are reduced.

Optionally, on the basis of the above embodiment corresponding to FIG. 4 , in the first optional embodiment of the method for updating index data provided by the embodiment of the present invention, the index data in the memory area is updated according to the index data update request , and after getting the first index data, you can also include:

Obtain the amount of data corresponding to the first index data;

It is judged whether the data amount corresponding to the first index data reaches the storage capacity threshold, and if so, it is determined that the first index data satisfies the preset index data update condition.

In this embodiment, the index data updating device may first obtain the data volume corresponding to the first index data, and then determine whether the data volume corresponding to the first index data reaches a preset storage capacity threshold, and when the storage capacity threshold is reached, Then, the first index data can be used to update the index data in the hard disk area to obtain the second index data.

Due to the limited storage resources of the memory area, it is usually necessary to set a storage capacity threshold for the occupied amount of the memory area. If the storage capacity threshold is 80%, when the storage capacity of the index data in the memory area reaches a certain storage capacity threshold (for example, 80% ), the index data in the memory area and the index data in the hard disk area need to be merged to generate a new full amount of data and reload.

Secondly, in the embodiment of the present invention, a method for judging whether the first index data satisfies the preset index data update conditions is provided. Specifically, firstly obtain the data volume corresponding to the first index data, and then judge whether the data volume has reached the storage capacity. The capacity threshold, only when the storage capacity threshold is reached, is it determined that the first index data satisfies the preset index data update condition. Through the above method, on the one hand, a reasonable storage capacity threshold can be set, which not only achieves the purpose of fully utilizing the memory space, but also does not cause waste of memory space; on the other hand, when the data volume of the first index data in the memory reaches the storage capacity threshold, The first index data can be automatically sent to the hard disk to ensure sufficient memory space.

Optionally, on the basis of the above embodiment corresponding to FIG. 4 , in the second optional embodiment of the method for updating index data provided by the embodiment of the present invention, the index data in the memory area is updated according to the index data update request , and after getting the first index data, you can also include:

Detect the time value corresponding to the first index data;

It is judged whether the time value corresponding to the first index data reaches the target time, and if so, it is determined that the first index data satisfies the preset index data update condition.

In this embodiment, in addition to the manner described in the first embodiment corresponding to FIG. 4 above, the second index data may also be updated periodically. Specifically, first detect the time value when the first index data is obtained, assuming that the time value is 12:00:00, if the time value reaches the preset target time, then it is considered that the update cycle has arrived, so it can be determined that the first index data An index data satisfies the preset index data update condition, so the first index data is sent to the hard disk area, so that the hard disk area is updated according to the first index data, and the second index data is obtained.

Secondly, in the embodiment of the present invention, another method for judging whether the first index data satisfies the preset index data update conditions is provided, specifically, first detecting the time value corresponding to the first index data, and then judging whether the time value reaches the target When the target time is reached, it is determined that the first index data satisfies the preset index data update condition. Through the above method, in the case of limited memory resources, the memory space can be used to periodically send the first index data to the hard disk, so that the hard disk can also periodically update the index data without affecting the effect of updating the index data in the memory and hard disk. Thus, the flexibility and practicability of the scheme are improved.

Optionally, on the basis of the first or second embodiment corresponding to FIG. 4 and FIG. 4 above, in the third optional embodiment of the method for updating index data provided by the embodiment of the present invention, according to the first index data The index data in the hard disk area is updated to obtain the second index data, which may include:

The first index data and the index data in the hard disk area are merged to obtain the second index data.

In this embodiment, how to update the index data in the hard disk area will be specifically introduced.

Specifically, it is assumed that the A list is stored in the current memory area, and the A list includes index data for a period of time, please refer to Table 1, which shows the index data in the A list.

Table 1

It can be seen from Table 1 that the first index data also includes a, b, c and d, and then merge the first index data with the index data stored in the hard disk area, that is, display the merged index data in the B list For the obtained second index data, please refer to Table 2, which is the index data in the B list.

Table 2

It can be seen from Table 2 that the second index data also includes a, b, c, d and x. At this point, the update of the index data in the hard disk area is completed.

Again, in the embodiment of the present invention, the index data updating device updates the index data in the hard disk area, and then obtains the second index data. This process may be to add the first index data in the memory area to the hard disk area, and The index data originally stored in the area together constitute the second index data. Through the above method, the update of the index data in the hard disk area can be completed, thereby improving the feasibility and operability of the solution, and providing a specific basis for the realization of the solution.

Optionally, on the basis of the third embodiment corresponding to FIG. 4 above, in the fourth optional embodiment of the method for updating index data provided by the embodiment of the present invention, the index in the hard disk area is updated according to the first index data. After the data is updated and the second index data is obtained, it may also include:

determining the third index data according to the retrieval frequency of the second index data, wherein the third index data is a part of the index data in the second index data;

The first index data is replaced by the third index data to store the third index data in the memory area.

In this embodiment, after the index data updating device obtains the second index data, it may further select a part of index data with high retrieval frequency from the second index data as the third index data, and then send the third index data to the memory area, at this time, the first index data in the memory area will be cleared and replaced by the third index data.

It can be understood that whether the index data is updated in the hard disk area to obtain the second index data, or the third index data is selected from the second index data, offline processing can be adopted, and the route is switched after the processing is completed. online program.

Further, in the embodiment of the present invention, after the index data update device obtains the second index data, it can also determine the third index data according to the retrieval frequency of the second index data, and then synchronize the third index data to the memory area, so that the memory area The first index data originally stored in the middle is replaced with the third index data. Through the above method, when the user performs a search, it is more likely to directly extract the corresponding search results from the memory area. Usually, it is more efficient to find information from the memory than from the hard disk. Therefore, the search frequency will be Higher third index data is stored in memory, which is more conducive to the improvement of retrieval efficiency, thereby increasing the practicability of the scheme.

Optionally, on the basis of the above-mentioned embodiment corresponding to FIG. 4 , in the fifth optional embodiment of the method for updating index data provided by the embodiment of the present invention, it may further include:

receive data retrieval requests;

Acquiring the first retrieval result corresponding to the first index data from the memory area according to the data retrieval request, and obtaining the second retrieval result corresponding to the second index data from the hard disk area;

If the first search result is consistent with the second search result, the first search result or the second search result is sent.

In this embodiment, a method of using index data for data retrieval will be introduced. Firstly, the index data updating device receives a data retrieval request initiated by the user through the client, and then, the index data updating device simultaneously sends the data retrieval request to the memory area and the hard disk area. In the memory area, the first index data is used to obtain the first search result corresponding to the data search request, and at the same time, in the hard disk area, the second index data is used to obtain the second search result corresponding to the data search request.

For example, assuming that the user carries "2016 Olympic Basketball Game" in the data retrieval request, then the directory related to "2016 Olympic Basketball Game" will be queried in the memory area and hard disk area, that is, the index data. The desired search results can be obtained.

If the first retrieval result obtained in the memory area is consistent with the second retrieval result obtained in the hard disk area, then one of the retrieval results is selected and sent to the user.

Secondly, in the embodiment of the present invention, a data retrieval process is introduced. The process is to first receive the data retrieval request sent by the user, and then obtain the first index corresponding to the first index data from the memory area according to the data retrieval request. As for the search result, the second search result corresponding to the second index data can be obtained from the hard disk area, and if it is detected that the first search result is consistent with the second search result, then one of the search results can be selected. Through the above method, the search results corresponding to the incremental index and the search results corresponding to the full index can be obtained, thereby improving the reliability and comprehensiveness of the search. In practical applications, you can choose between the first search result and the second search result One is to send, so as to ensure that the sent search results will not be repeated, thereby improving the rationality of the search.

Optionally, on the basis of the fifth embodiment corresponding to Figure 4 above, in the sixth optional embodiment of the method for updating index data provided by the embodiment of the present invention, before sending the first search result or the second search result , which can also include:

Obtain the first update duration of the first retrieval result and the second update duration of the second retrieval result, wherein the first update duration is the time length between the moment when the first retrieval result is obtained and the current moment, and the second update duration is The length of time between the moment of the second retrieval result and the current moment;

Sending the first search result or the second search result may include:

If the first update duration is less than the second update duration, then send the first retrieval result;

If the first update duration is longer than the second update duration, the second retrieval result is sent.

In this embodiment, because the search engine internally divides all index data into two storages (i.e., memory storage and hard disk storage), when a data retrieval request comes, it is very likely that the same index data will exist in the two index data at the same time. A query result. At this time, it is necessary to select the latest result according to the update time of the result and return it to the user as the final result, while the second newest result will be directly discarded.

Specifically, it is assumed that the index data update device obtains the first retrieval result at 10:15:35 on March 18, 2017, and obtains the second retrieval result at 12:38:27 on March 17, 2017. , it is not difficult to see that it takes longer to obtain the second search result, so the first search result will be sent first. This allows you to select updated search results. Conversely, if the first update duration is longer than the second update duration, the second retrieval result is sent to the user.

Again, in the embodiment of the present invention, when the first search result is consistent with the second search result, the update duration of the two can be further used to select one of the search results, that is, the first update duration of the first search result is obtained first And the second update duration of the second search result, the first update duration is the time length between the time when the first search result is obtained and the current time, and the second update time is the time between the time when the second search result is obtained and the current time The length of time, and finally select the search result with a shorter update time as the sending object. Through the above method, the index data update device can provide users with more time-sensitive search results. At the same time, the first search result is usually stored in the memory. The first retrieval result, and the time to extract the required first retrieval result from the memory is shorter, thereby improving the operability and practicability of the solution.

Optionally, on the basis of the above-mentioned embodiment corresponding to FIG. 4 , in the seventh optional embodiment of the method for updating index data provided by the embodiment of the present invention, it may further include:

receive data retrieval requests;

Acquiring the retrieval result corresponding to the second index data from the hard disk area according to the data retrieval request;

Send the retrieval result corresponding to the second index data.

In this embodiment, another way of using index data for data retrieval will be introduced. First, the index data updating device receives the data retrieval request initiated by the user through the client, and then, the index data updating device sends the data retrieval request to the hard disk area, because in many cases, the index data that can be stored in the memory area is relatively small. Therefore, in order to overcome this problem, we can store index data with a large amount of data in the hard disk area. In this way, the index data update device can directly obtain the retrieval result corresponding to the second index data from the hard disk area according to the data retrieval request, and finally send the retrieval data to the user through the client

Secondly, in the embodiment of the present invention, a data retrieval process is introduced. The process is to first receive the data retrieval request sent by the user, and then directly retrieve the retrieval result from the hard disk area according to the data retrieval request. The retrieval result is only It is stored in the hard disk area, not in the memory area, so there is no need to consume time and resources to find the retrieval results from the memory area. Through the above method, the consumption of memory resources can be reduced as much as possible, and the corresponding retrieval results can be directly extracted from the hard disk area, thereby improving the application efficiency and feasibility of the solution.

For ease of understanding, the process of realizing data retrieval in the present invention can be described in detail below in a specific application scenario, specifically:

The index data update device abstracts the search engine and develops a platform for self-service access to search services after the search engine is componentized. Using the index data updating device to build a search engine makes the realization of the search function simpler. The core of a search engine lies in index data. The operation of multi-index data mainly includes the storage, update, and retrieval of index data. Especially when the amount of business data is large, the problem of index data storage and update becomes more prominent. Therefore, in the process of building a search service, how to store and update index data has become a key issue. The specific process of data retrieval will be introduced below.

Please refer to FIG. 5. FIG. 5 is a schematic flow chart of data retrieval in an embodiment of the present invention, specifically:

In step 201, first, the index data update device receives a data retrieval request sent by the user through the client, and the request carries the content that the user wants to retrieve;

In step 202, then, the index data updating device will judge whether the first retrieval result corresponding to the first index data can be obtained from the memory area according to the data retrieval request, and also need to judge whether the second retrieval result corresponding to the first index data can be obtained from the hard disk area. For the second retrieval result corresponding to the index data, if the two retrieval results can be obtained at the same time, enter step 203, otherwise, jump to step 204;

In step 203, if the index data updating device obtains the first retrieval result and the second retrieval result at the same time, it should also select one of the two retrieval results as the final retrieval result;

In step 204, the index data update device sends the selected retrieval results to the client.

The index data updating device in the present invention will be described in detail below, please refer to FIG. 6 , the index data updating device includes a memory area and a hard disk area, and the index data updating device 30 includes:

The first receiving module 301 is configured to receive an index data update request;

The first updating module 302 is configured to update the index data in the memory area according to the index data update request received by the first receiving module 301, and obtain first index data;

The second update module 303 is configured to update the index data in the hard disk area according to the first index data if the first index data updated by the first update module 302 satisfies the preset index data update condition update, and obtain the second index data.

In this embodiment, the first receiving module 301 receives an index data update request, and the first update module 302 updates the index data in the memory area according to the index data update request received by the first receiving module 301, and Obtain the first index data, if the first index data updated by the first update module 302 satisfies the preset index data update condition, then the second update module 303 updates the hard disk area according to the first index data The index data is updated to obtain the second index data.

In an embodiment of the present invention, an index data update device is provided, the index data update device includes a memory area and a hard disk area, first the index data update device receives an index data update request, and then updates the index in the memory area according to the index data update request The data is updated to obtain first index data. If the first index data satisfies the preset index data update condition, the index data in the hard disk area is updated according to the first index data to obtain second index data. Through the above method, the index data can be stored hierarchically, that is, the index data can be divided into the first index data and the second index data, and different storage methods can be adopted for different index data types, so as to update the first index in real time At the same time, the second index data is updated periodically, so that the update of the index data can be effectively balanced, memory consumption and computing resources are reduced, and computing costs are reduced.

Optionally, on the basis of the above-mentioned embodiment corresponding to FIG. 6 , please refer to FIG. 7 , in another embodiment of the index data updating device 30 provided by the embodiment of the present invention,

The index data updating device 30 also includes:

The first acquiring module 304A is configured to acquire the first index data after the first updating module 302 updates the index data in the memory area according to the index data update request and obtains the first index data corresponding amount of data;

The first judging module 304B is configured to judge whether the amount of data corresponding to the first index data acquired by the first acquiring module 304A reaches a storage capacity threshold, and if so, determine that the first index data satisfies the preset index Data update condition.

Secondly, in the embodiment of the present invention, a method for judging whether the first index data satisfies the preset index data update conditions is provided. Specifically, firstly obtain the data volume corresponding to the first index data, and then judge whether the data volume has reached the storage capacity. The capacity threshold, only when the storage capacity threshold is reached, is it determined that the first index data satisfies the preset index data update condition. Through the above method, on the one hand, a reasonable storage capacity threshold can be set, which not only achieves the purpose of fully utilizing the memory space, but also does not cause waste of memory space; on the other hand, when the data volume of the first index data in the memory reaches the storage capacity threshold, The first index data can be automatically sent to the hard disk to ensure sufficient memory space.

Optionally, on the basis of the above-mentioned embodiment corresponding to FIG. 6 , please refer to FIG. 8 , in another embodiment of the index data updating device 30 provided by the embodiment of the present invention,

The index data updating device 30 also includes:

The detection module 305A is configured to detect the index data corresponding to the first index data after the first update module 302 updates the index data in the memory area according to the index data update request and obtains the first index data. time value;

The second judgment module 305B is used to judge whether the time value corresponding to the first index data detected by the detection module 305A reaches the target time, and if so, determine that the first index data satisfies the preset index data update condition.

Secondly, in the embodiment of the present invention, another method for judging whether the first index data satisfies the preset index data update conditions is provided, specifically, first detecting the time value corresponding to the first index data, and then judging whether the time value reaches the target When the target time is reached, it is determined that the first index data satisfies the preset index data update condition. Through the above method, in the case of limited memory resources, the memory space can be used to periodically send the first index data to the hard disk, so that the hard disk can also periodically update the index data without affecting the effect of updating the index data in the memory and hard disk. Thus, the flexibility and practicability of the scheme are improved.

Optionally, on the basis of the above-mentioned embodiments corresponding to FIG. 6, FIG. 7 or FIG. 8, please refer to FIG.

The second update module 303 includes:

The processing unit 3031 is configured to combine the first index data and the index data in the hard disk area to obtain the second index data.

Again, in the embodiment of the present invention, the index data updating device updates the index data in the hard disk area, and then obtains the second index data. This process may be to add the first index data in the memory area to the hard disk area, and The index data originally stored in the area together constitute the second index data. Through the above method, the update of the index data in the hard disk area can be completed, thereby improving the feasibility and operability of the solution, and providing a specific basis for the realization of the solution.

Optionally, on the basis of the above embodiment corresponding to FIG. 9 , please refer to FIG. 10 , in another embodiment of the index data updating device 30 provided by the embodiment of the present invention,

The index data updating device 30 also includes:

The determining module 306A is configured to, after the second updating module 303 updates the index data in the hard disk area according to the first index data and obtains the second index data, according to the retrieval of the second index data Frequency determining third index data, wherein the third index data is a part of the second index data;

A replacing module 306B, configured to replace the first index data with the third index data determined by the determining module 306A, so as to store the third index data in the memory area.

Further, in the embodiment of the present invention, after the index data update device obtains the second index data, it can also determine the third index data according to the retrieval frequency of the second index data, and then synchronize the third index data to the memory area, so that the memory area The first index data originally stored in the middle is replaced with the third index data. Through the above method, when the user performs a search, it is more likely to directly extract the corresponding search results from the memory area. Usually, it is more efficient to find information from the memory than from the hard disk. Therefore, the search frequency will be Higher third index data is stored in memory, which is more conducive to the improvement of retrieval efficiency, thereby increasing the practicability of the scheme.

Optionally, on the basis of the above-mentioned embodiment corresponding to FIG. 6 , please refer to FIG. 11 , in another embodiment of the index data updating device 30 provided by the embodiment of the present invention,

The index data updating device 30 also includes:

The second receiving module 307A is configured to receive a data retrieval request;

The second obtaining module 307B is configured to obtain the first retrieval result corresponding to the first index data from the memory area according to the data retrieval request received by the second receiving module 307A, and obtain the first retrieval result corresponding to the first index data from the hard disk area Acquiring a second retrieval result corresponding to the second index data;

The first sending module 307C is configured to send the first search result or the second search result if the first search result obtained by the second obtaining module 307B is consistent with the second search result.

Secondly, in the embodiment of the present invention, a data retrieval process is introduced. The process is to first receive the data retrieval request sent by the user, and then obtain the first index corresponding to the first index data from the memory area according to the data retrieval request. As for the search result, the second search result corresponding to the second index data can be obtained from the hard disk area, and if it is detected that the first search result is consistent with the second search result, then one of the search results can be selected. Through the above method, the search results corresponding to the incremental index and the search results corresponding to the full index can be obtained, thereby improving the reliability and comprehensiveness of the search. In practical applications, you can choose between the first search result and the second search result One is to send, so as to ensure that the sent search results will not be repeated, thereby improving the rationality of the search.

Optionally, on the basis of the above embodiment corresponding to FIG. 11 , please refer to FIG. 12 , in another embodiment of the index data updating device 30 provided by the embodiment of the present invention,

The index data updating device 30 also includes:

The third obtaining module 308 is configured to obtain the first update duration of the first search result and the second search result before the first sending module 307C sends the first search result or the second search result. The second update duration of the result, wherein the first update duration is the time length between the moment when the first retrieval result is obtained and the current moment, and the second update duration is the moment when the second retrieval result is obtained the length of time to said current moment;

The first sending module 307C includes:

The first sending unit 307C1 is configured to send the first retrieval result if the first update duration is shorter than the second update duration;

The second sending unit 307C2 is configured to send the second retrieval result if the first update duration is longer than the second update duration.

Again, in the embodiment of the present invention, when the first search result is consistent with the second search result, the update duration of the two can be further used to select one of the search results, that is, the first update duration of the first search result is obtained first And the second update duration of the second search result, the first update duration is the time length between the time when the first search result is obtained and the current time, and the second update time is the time between the time when the second search result is obtained and the current time The length of time, and finally select the search result with a shorter update time as the sending object. Through the above method, the index data update device can provide users with more time-sensitive search results. At the same time, the first search result is usually stored in the memory. The first retrieval result, and the time to extract the required first retrieval result from the memory is shorter, thereby improving the operability and practicability of the solution.

Optionally, on the basis of the above-mentioned embodiment corresponding to FIG. 6 , please refer to FIG. 13 , in another embodiment of the index data updating device 30 provided by the embodiment of the present invention,

The index data updating device 30 also includes:

The third receiving module 309A is configured to receive a data retrieval request;

The fourth obtaining module 309B is configured to obtain the retrieval result corresponding to the second index data from the hard disk area according to the data retrieval request received by the third receiving module 309A;

The second sending module 309C is configured to send the retrieval result corresponding to the second index data obtained by the fourth obtaining module 309B.

Secondly, in the embodiment of the present invention, a data retrieval process is introduced. The process is to first receive the data retrieval request sent by the user, and then directly retrieve the retrieval result from the hard disk area according to the data retrieval request. The retrieval result is only It is stored in the hard disk area, not in the memory area, so there is no need to consume time and resources to find the retrieval results from the memory area. Through the above method, the consumption of memory resources can be reduced as much as possible, and the corresponding retrieval results can be directly extracted from the hard disk area, thereby improving the application efficiency and feasibility of the solution.

FIG. 14 is a schematic structural diagram of an index data updating device provided by an embodiment of the present invention. The index data updating device 400 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (central processing units). , CPU) 422 (eg, one or more processors) and memory 432, one or more storage media 430 (eg, one or more mass storage devices) for storing application programs 442 or data 444. Wherein, the memory 432 and the storage medium 430 may be temporary storage or persistent storage. The program stored in the storage medium 430 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Further, the central processing unit 422 may be configured to communicate with the storage medium 430 , and the index data updating apparatus 400 executes a series of instruction operations in the storage medium 430 .

The index data update device 400 may also include one or more power sources 426, one or more wired or wireless network interfaces 450, one or more input and output interfaces 458, and/or, one or more operating systems 441, such as Windows ServerTM , Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

The steps performed by the server in the foregoing embodiments may be based on the server structure shown in FIG. 14 .

Wherein, the central processing unit 422 can be used to perform the following steps:

Receive index data update request;

update the index data in the memory area according to the index data update request, and obtain first index data;

If the first index data satisfies a preset index data update condition, then update the index data in the hard disk area according to the first index data, and obtain second index data.

The central processing unit 422 can also be used to perform the following steps:

Acquiring a data amount corresponding to the first index data;

Judging whether the amount of data corresponding to the first index data reaches a storage capacity threshold, and if so, determining that the first index data satisfies the preset index data update condition.

The central processing unit 422 can also be used to perform the following steps:

Detecting the time value corresponding to the first index data;

It is judged whether the time value corresponding to the first index data reaches the target time, and if so, it is determined that the first index data satisfies the preset index data update condition.

The central processing unit 422 is specifically configured to perform the following steps:

Merge the first index data and the index data in the hard disk area to obtain the second index data.

The central processing unit 422 can also be used to perform the following steps:

determining third index data according to the retrieval frequency of the second index data, wherein the third index data is a part of the second index data;

replacing the first index data with the third index data, so as to store the third index data in the memory area.

The central processing unit 422 can also be used to perform the following steps:

receive data retrieval requests;

Obtaining a first retrieval result corresponding to the first index data from the memory area according to the data retrieval request, and obtaining a second retrieval result corresponding to the second index data from the hard disk area;

If the first search result is consistent with the second search result, sending the first search result or the second search result.

The central processing unit 422 can also be used to perform the following steps:

Obtain a first update duration of the first retrieval result and a second update duration of the second retrieval result, wherein the first update duration is the time between the moment when the first retrieval result is obtained and the current moment length, the second update duration is the time length between the moment when the second retrieval result is obtained and the current moment;

The central processing unit 422 is specifically configured to perform the following steps:

If the first update duration is less than the second update duration, then send the first retrieval result;

If the first update duration is longer than the second update duration, then send the second retrieval result.

The central processing unit 422 can also be used to perform the following steps:

receive data retrieval requests;

Acquiring a retrieval result corresponding to the second index data from the hard disk area according to the data retrieval request;

Sending the retrieval result corresponding to the second index data.

Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes such as U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk.

As mentioned above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still understand the foregoing The technical solutions recorded in each embodiment are modified, or some of the technical features are replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims (12)

1. A method for updating index data, the method being applied to an index data updating device, the index data updating device comprising a memory area and a hard disk area, the method comprising:

receiving an index data update request;

updating the index data in the memory area according to the index data updating request, and obtaining first index data;

if the first index data meets the preset index data updating condition, updating the index data in the hard disk area according to the first index data, and obtaining second index data;

receiving a data retrieval request;

acquiring a first search result corresponding to the first index data from the memory area according to the data search request, and acquiring a second search result corresponding to the second index data from the hard disk area;

If the first search result is consistent with the second search result, acquiring a first update time length of the first search result and a second update time length of the second search result, wherein the first update time length is the time length from the moment of acquiring the first search result to the current moment, and the second update time length is the time length from the moment of acquiring the second search result to the current moment;

if the first updating duration is smaller than the second updating duration, the first search result is sent;

and if the first updating time length is longer than the second updating time length, sending the second search result.

2. The method of claim 1, wherein after updating the index data in the memory area according to the index data update request and obtaining the first index data, the method further comprises:

acquiring the data quantity corresponding to the first index data;

judging whether the data quantity corresponding to the first index data reaches a storage capacity threshold value, if so, determining that the first index data meets the preset index data updating condition.

3. The method of claim 1, wherein after updating the index data in the memory area according to the index data update request and obtaining the first index data, the method further comprises:

detecting a time value corresponding to the first index data;

judging whether the time value corresponding to the first index data reaches a target time or not, if so, determining that the first index data meets the preset index data updating condition.

4. A method according to any one of claims 1 to 3, wherein updating the index data in the hard disk area according to the first index data and obtaining second index data comprises:

and merging the first index data with the index data in the hard disk area to obtain the second index data.

5. The method of claim 4, wherein after updating the index data in the hard disk area according to the first index data and obtaining the second index data, the method further comprises:

determining third index data according to the retrieval frequency of the second index data, wherein the third index data is part of index data in the second index data;

And replacing the first index data with the third index data to store the third index data into the memory area.

6. An index data updating device, wherein the index data updating device includes a memory area and a hard disk area, the index data updating device includes:

the first receiving module is used for receiving an index data updating request;

the first updating module is used for updating the index data in the memory area according to the index data updating request received by the first receiving module and obtaining first index data;

the second updating module is used for updating the index data in the hard disk area according to the first index data if the first index data updated by the first updating module meets the preset index data updating condition, and obtaining second index data;

the second receiving module is used for receiving a data retrieval request;

the second acquisition module is used for acquiring a first search result corresponding to the first index data from the memory area according to the data search request received by the second receiving module, and acquiring a second search result corresponding to the second index data from the hard disk area;

A third obtaining module, configured to obtain a first update duration of the first search result and a second update duration of the second search result if the first search result obtained by the second obtaining module is consistent with the second search result, where the first update duration is a time length from a time point when the first search result is obtained to a current time point, and the second update duration is a time length from a time point when the second search result is obtained to the current time point;

a first sending unit, configured to send the first search result if the first update duration is less than the second update duration;

and the second sending unit is used for sending the second search result if the first updating time length is longer than the second updating time length.

7. The index data updating device according to claim 6, characterized in that the index data updating device further comprises:

the first acquisition module is used for acquiring the data quantity corresponding to the first index data after the first update module updates the index data in the memory area according to the index data update request and obtains the first index data;

The first judging module is used for judging whether the data volume corresponding to the first index data acquired by the first acquiring module reaches a storage capacity threshold or not, and if yes, determining that the first index data meets the preset index data updating condition.

8. The index data updating device according to claim 6, characterized in that the index data updating device further comprises:

the detection module is used for detecting a time value corresponding to the first index data after the first update module updates the index data in the memory area according to the index data update request and obtains the first index data;

and the second judging module is used for judging whether the moment value corresponding to the first index data detected by the detecting module reaches the target moment or not, and if so, determining that the first index data meets the preset index data updating condition.

9. The index data updating apparatus according to any one of claims 6 to 8, wherein said second updating module includes:

and the processing unit is used for merging the first index data with the index data in the hard disk area and obtaining the second index data.

10. The index data updating device according to claim 9, characterized in that the index data updating device further comprises:

the determining module is used for determining third index data according to the retrieval frequency of the second index data after the second updating module updates the index data in the hard disk area according to the first index data and obtains the second index data, wherein the third index data is part of index data in the second index data;

and the replacing module is used for replacing the first index data by the third index data determined by the determining module so as to store the third index data into the memory area.

11. A server comprising a memory and a processor;

the memory stores a program;

the processor is configured to execute the program to implement the method for updating index data according to any one of claims 1 to 5.

12. A computer readable storage medium, wherein a program is stored in the computer readable storage medium, which when executed by a computer device, implements the method for updating index data according to any one of claims 1 to 5.

Images