Nothing Special   »   [go: up one dir, main page]

WO2021212757A1 - Upgrade maintenance method and apparatus for cluster, and electronic device and storage medium - Google Patents

Upgrade maintenance method and apparatus for cluster, and electronic device and storage medium Download PDF

Info

Publication number
WO2021212757A1
WO2021212757A1 PCT/CN2020/119110 CN2020119110W WO2021212757A1 WO 2021212757 A1 WO2021212757 A1 WO 2021212757A1 CN 2020119110 W CN2020119110 W CN 2020119110W WO 2021212757 A1 WO2021212757 A1 WO 2021212757A1
Authority
WO
WIPO (PCT)
Prior art keywords
version
node server
cluster node
software package
configuration file
Prior art date
Application number
PCT/CN2020/119110
Other languages
French (fr)
Chinese (zh)
Inventor
杨知
贺波
万书武
李均
Original Assignee
平安科技(深圳)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技(深圳)有限公司 filed Critical 平安科技(深圳)有限公司
Publication of WO2021212757A1 publication Critical patent/WO2021212757A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00Arrangements for software engineering
    • G06F8/60Software deployment
    • G06F8/65Updates
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00Arrangements for software engineering
    • G06F8/70Software maintenance or management
    • G06F8/71Version control; Configuration management

Definitions

  • This application relates to the technical field of software development, and in particular to a method, device, electronic equipment, and computer-readable storage medium for upgrading and maintaining a cluster.
  • This application provides a method, device, electronic device, and computer-readable storage medium for upgrading and maintaining a cluster.
  • a cluster upgrade and maintenance method provided by this application includes:
  • the version detection report determine whether the cluster node server needs to be differentially released
  • the version software package and configuration file in the cluster node server are updated according to the new version software package and the new configuration file, and then the version release is performed;
  • a preset restart program is used to restart the cluster node server after the version is released, and mark the cluster node server after the version is released.
  • the application also provides a cluster upgrade and maintenance device, the device includes:
  • the upload module is used to upload the new version of the software package and the new configuration file that need to be released on the cluster node server to the preset release machine;
  • the detection module is used to obtain the cluster node server that needs to be version released, perform corresponding version detection on the software package and configuration file in the cluster node server and the new version software package and the new configuration file, and generate a version detection report;
  • the judgment and execution module is used to judge whether the cluster node server needs to perform differential publishing according to the version detection report, if differential publishing is not required, perform version publishing on the cluster node server through the publishing machine, if If a differential release is required, the version software package and configuration file in the cluster node server are updated according to the new version software package and the new configuration file, and then the version release is performed;
  • the restart module is used to restart the cluster node server after the version is released by using a preset restart program, and mark the cluster node server after the version is released.
  • This application also provides an electronic device, which includes:
  • Memory storing at least one instruction
  • the processor executes the instructions stored in the memory to implement the following steps:
  • the version detection report determine whether the cluster node server needs to be differentially released
  • the version software package and configuration file in the cluster node server are updated according to the new version software package and the new configuration file, and then the version release is performed;
  • a preset restart program is used to restart the cluster node server after the version is released, and mark the cluster node server after the version is released.
  • the present application also provides a computer-readable storage medium in which at least one instruction is stored, and the at least one instruction is executed by a processor in an electronic device to implement the following steps:
  • the version detection report determine whether the cluster node server needs to be differentially released
  • the version software package and configuration file in the cluster node server are updated according to the new version software package and the new configuration file, and then the version release is performed;
  • a preset restart program is used to restart the cluster node server after the version is released, and mark the cluster node server after the version is released.
  • FIG. 1 is a schematic flowchart of a method for upgrading and maintaining a cluster provided by an embodiment of this application;
  • FIG. 2 is a schematic diagram of modules of a cluster upgrade and maintenance device provided by an embodiment of the application
  • FIG. 3 is a schematic diagram of the internal structure of an electronic device of a method for upgrading and maintaining a cluster provided by an embodiment of the application;
  • This application provides a method for upgrading and maintaining a cluster.
  • FIG. 1 it is a schematic flowchart of a cluster upgrade and maintenance method provided by an embodiment of this application.
  • the method can be executed by a device, and the device can be implemented by software and/or hardware.
  • the method for upgrading and maintaining the cluster includes:
  • the software package includes a jar package.
  • the jar package is also called JAR file (Java Archive, English: Java Archive) is a software package file format, usually used to aggregate a large number of Java class files, related metadata and resource (text, pictures, etc.) files into A file to develop application software or libraries for the Java platform.
  • the new configuration file includes: operating environment parameters, interface parameters, address parameters, and so on.
  • the new version of the jar package and the new configuration file that need to be released are determined by the version of the cluster node server. If the updated version of a certain cluster node server is 1.7, the updated version The content is the new version of the jar package and the new configuration file to be released.
  • the preset publishing machine performs functional operations by using a web interface, and is used to perform version publishing of the cluster node server.
  • the embodiment of the present application uploads the new version jar package and the new configuration file to a preset publishing machine through a web interface for storage, wherein the directory where the new version jar package and the new configuration file are stored is / appcom/deploy/release date/.
  • the information management of the cluster node server is performed through a web interface, so in the web interface, select the cluster node server that needs to be version released, and grab the jar package and the jar package in the cluster node server.
  • the configuration file performs corresponding version detection on the jar package and configuration file in the cluster node server and the new version jar package and the new configuration file, and generates a version detection report.
  • the capturing of the jar package and configuration file in the cluster node server can be implemented through Python, Shell, and Ansible scripts.
  • performing corresponding version detection of the jar package and configuration file in the cluster node server with the new version jar package and the new configuration file includes:
  • a version detection report is generated on the web page according to the version difference value.
  • the calculating the distance metric value between the jar package in the cluster node server and the new version jar package includes:
  • ⁇ X, Y represents the distance measurement value
  • X represents the md5 value of the jar package in the cluster node server
  • Y represents the md5 value of the new version of the jar package
  • ⁇ X represents the average md5 value of the jar package in the cluster node server
  • ⁇ Y represents the new The mean value of md5 of the version jar package
  • ⁇ X represents the standard deviation of the md5 value of the jar package in the cluster node server
  • ⁇ Y represents the standard deviation of the md5 value of the new version jar package
  • E represents the jar package in the cluster node server and the new version jar package
  • the calculating the similarity between the configuration file in the cluster node server and the new version configuration file includes:
  • sim(X,Y) represents the similarity
  • X represents the md5 value of the configuration file in the cluster node server
  • Y represents the md5 value of the new configuration file
  • X i and Y i respectively identify the cluster node server and the new configuration file.
  • the md5 value of the first bit is 3
  • the md5 value of the last bit is 5.
  • this application converts the characters in the md5 value into ASCII code Numerical value, for example, c is 99 after being converted into ASCII code value.
  • the similarity generated by it ranges from -1 to 1: -1 means that the configuration file in the cluster node server and the new version configuration file point to the exact opposite direction, 1 means that their points are exactly the same, 0 It usually means that they are independent, and the value in between indicates moderate similarity and dissimilarity.
  • the average value of the distance metric value and the similarity is calculated to obtain the version difference value, and the version difference value is compared with a preset threshold value, if the version difference value is less than the preset threshold value.
  • the threshold value of the cluster node server and the new version cluster node server have a difference result, if the difference result value is not less than the preset threshold value, it means that the cluster node server and the new version cluster node server are not in the difference result, Thus, the version detection report is generated according to the version difference value.
  • the preset threshold value is 0.95.
  • the difference result includes: the current version of the cluster node server is behind the newly released version of the cluster node server; the hardware and/or system of the cluster node server is different from the hardware and/or system of the new version of the cluster node server.
  • the embodiment of the present application ensures the accuracy of version detection by calculating the version difference value.
  • a preset restart program needs to be used to validate the release function, and the preset restart program is compiled through a shell script.
  • this application takes the Hadoop cluster as an example, and the preset restart service steps to restart the Hadoop cluster are: start Ldap and kerbores; start DNS service; execute servicentpd restart on all nodes in the cluster; start zookeeper, yarn; in the cluster Execute at any node: spark-sql can enter the spark command line, and execute a simple statement selectcount(*)from tablename to check whether spark is normal; hbase starts, rolling restart, the rolling restart steps are as follows:
  • this application marks the released version of the current cluster node service, so that subsequent releases of the cluster node server can clearly understand the current version and facilitate the upgrade and maintenance of multi-cluster node servers.
  • FIG. 2 it is a functional module diagram of the upgrade and maintenance device for the cluster of the present application.
  • the apparatus 100 for upgrading and maintaining the cluster described in this application can be installed in an electronic device.
  • the upgrade and maintenance device of the cluster may include an upload module 101, a detection module 102, a judgment set execution 103, and a restart module 103.
  • the module described in the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can complete fixed functions, and are stored in the memory of the electronic device.
  • each module/unit is as follows:
  • the upload module 101 is used to upload the new version software package and the new configuration file of the cluster node server to the preset publishing machine;
  • the detection module 102 is configured to obtain the cluster node server that needs to be version released, and perform corresponding version detection of the software package and configuration file in the cluster node server with the new version software package and the new configuration file to generate a version Test Report;
  • the judgment and execution module 103 is configured to judge whether the cluster node server needs to be differentially released according to the version detection report, if no differential release is required, execute the version on the cluster node server through the publishing machine Release, if differential release is required, the version software package and configuration file in the cluster node server are updated according to the new version software package and the new configuration file, and then the version release is performed;
  • the restart module 104 is configured to use a preset restart program to restart the cluster node server after the version is released, and mark the cluster node server after the version is released.
  • the uploading module 101 uploads the new version software package and the new configuration file of the cluster node server to the preset publishing machine.
  • the software package includes a jar package.
  • the jar package is also called JAR file (Java Archive, English: Java Archive) is a software package file format, usually used to aggregate a large number of Java class files, related metadata and resource (text, pictures, etc.) files into A file to develop application software or libraries for the Java platform.
  • the new configuration file includes: operating environment parameters, interface parameters, address parameters, and so on.
  • the new version of the jar package and the new configuration file that need to be released are determined by the version of the cluster node server. If the updated version of a certain cluster node server is 1.7, the updated version The content is the new version of the jar package and the new configuration file to be released.
  • the preset publishing machine performs functional operations by using a web interface, and is used to perform version publishing of the cluster node server.
  • the upload module 101 uploads the new version jar package and the new configuration file to a preset publishing machine for storage through a web interface, wherein the directory where the new version jar package and the new configuration file are stored is /appcom/deploy/release date/.
  • the detection module 102 obtains the cluster node server that needs to be version released, performs corresponding version detection on the software package and configuration file in the cluster node server and the new version software package and the new configuration file, and generates a version detection report.
  • the detection module 102 performs information management of the cluster node server through a web interface, so in the web interface, select the cluster node server that needs version release, and grab the cluster For the jar package and configuration file in the node server, perform corresponding version detection on the jar package and configuration file in the cluster node server and the new version jar package and the new configuration file, and generate a version detection report.
  • the capturing of the jar package and configuration file in the cluster node server can be implemented through Python, Shell, and Ansible scripts.
  • performing corresponding version detection of the jar package and configuration file in the cluster node server with the new version jar package and the new configuration file includes:
  • a version detection report is generated on the web page according to the version difference value.
  • the calculating the distance metric value between the jar package in the cluster node server and the new version jar package includes:
  • ⁇ X, Y represents the distance measurement value
  • X represents the md5 value of the jar package in the cluster node server
  • Y represents the md5 value of the new version of the jar package
  • ⁇ X represents the average md5 value of the jar package in the cluster node server
  • ⁇ Y represents the new The mean md5 value of the version jar package
  • ⁇ X represents the standard deviation of the md5 value of the jar package in the cluster node server
  • ⁇ Y represents the standard deviation of the md5 value of the new version jar package
  • E represents the jar package in the cluster node server and the new version jar package
  • the calculating the similarity between the configuration file in the cluster node server and the new version configuration file includes:
  • sim(X, Y) represents the similarity
  • X represents the md5 value of the configuration file in the cluster node server
  • Y represents the md5 value of the new configuration file
  • X i and Y i respectively identify the cluster node server and the new configuration file.
  • the md5 value of the first bit is 3
  • the md5 value of the last bit is 5.
  • this application converts the characters in the md5 value into ASCII code
  • Numerical value for example, c is 99 after being converted into ASCII code value.
  • the similarity generated by it ranges from -1 to 1: -1 means that the configuration file in the cluster node server and the new version configuration file point to the exact opposite direction, 1 means that their points are exactly the same, 0 It usually means that they are independent, and the value in between indicates moderate similarity and dissimilarity.
  • the detection module 102 in the embodiment of the present application calculates the average value of the distance metric value and the similarity to obtain the version difference value, and compares the version difference value with a preset threshold.
  • the difference value is less than the preset threshold value, it means that the cluster node server has a difference result from the new version cluster node server. If the difference result value is not less than the preset threshold value, it means the cluster node server and the new version cluster node The server is not in the difference result, and thus generates the version detection report according to the version difference value.
  • the preset threshold value is 0.95.
  • the difference result includes: the current version of the cluster node server is behind the newly released version of the cluster node server; the hardware and/or system of the cluster node server is different from the hardware and/or system of the new version of the cluster node server.
  • the detection module 102 described in the embodiment of the present application ensures the accuracy of version detection by calculating the version difference value.
  • the judgment and execution module 103 judges whether the cluster node server needs to be differentially released according to the version detection report. If no differential release is required, then the release machine performs version release on the cluster node server, if If a differential release is required, the version software package and configuration file in the cluster node server are updated according to the new version software package and the new configuration file, and then the version release is performed.
  • the restart module 104 uses a preset restart procedure to restart the cluster node server after the version is released, and marks the cluster node server after the version is released.
  • a preset restart program needs to be used to validate the release function, and the preset restart program is compiled through a shell script.
  • the embodiment of the application takes the Hadoop cluster as an example, and the preset restart service steps to restart the Hadoop cluster are: start Ldap and kerbores; start DNS service; execute servicentpd restart on all nodes in the cluster; start zookeeper, yarn; Execute on any node in the cluster: spark-sql can enter the spark command line and execute a simple statement selectcount(*)from tablename to check whether spark is normal; hbase starts, rolling restart, the rolling restart steps are as follows:
  • the restart module 104 marks the released version of the current cluster node service, so that subsequent releases of the cluster node server can clearly understand the current version, and facilitate the upgrade of multi-cluster node servers And maintenance.
  • FIG. 3 it is a schematic diagram of the structure of an electronic device implementing the method for upgrading and maintaining a cluster according to the present application.
  • the electronic device 1 may include a processor 10, a memory 11, and a bus, and may also include a computer program stored in the memory 11 and running on the processor 10, such as a cluster upgrade maintenance program 12.
  • the memory 11 includes at least one type of readable storage medium.
  • the readable storage medium may be nonvolatile or volatile.
  • the readable storage medium includes flash memory, mobile hard disk, and multimedia card.
  • Card-type memory for example: SD or DX memory, etc.
  • magnetic memory magnetic disk, optical disk, etc.
  • the memory 11 may be an internal storage unit of the electronic device 1 in some embodiments, for example, a mobile hard disk of the electronic device 1. In other embodiments, the memory 11 may also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart media card (SMC), and a secure digital (Secure Digital) equipped on the electronic device 1.
  • SD Secure Digital
  • flash card Flash Card
  • the memory 11 may also include both an internal storage unit of the electronic device 1 and an external storage device.
  • the memory 11 can be used not only to store application software and various types of data installed in the electronic device 1, such as the code of a cluster upgrade and maintenance program, etc., but also to temporarily store data that has been output or will be output.
  • the processor 10 may be composed of integrated circuits in some embodiments, for example, may be composed of a single packaged integrated circuit, or may be composed of multiple integrated circuits with the same function or different functions, including one or more Combinations of central processing unit (CPU), microprocessor, digital processing chip, graphics processor, and various control chips, etc.
  • the processor 10 is the control core (Control Unit) of the electronic device, which uses various interfaces and lines to connect the various components of the entire electronic device, and runs or executes programs or modules (such as clusters) stored in the memory 11 Upgrade and maintenance procedures, etc.), and call the data stored in the memory 11 to execute various functions of the electronic device 1 and process data.
  • Control Unit Control Unit
  • the bus may be a peripheral component interconnect standard (PCI) bus or an extended industry standard architecture (EISA) bus, etc.
  • PCI peripheral component interconnect standard
  • EISA extended industry standard architecture
  • the bus can be divided into address bus, data bus, control bus and so on.
  • the bus is configured to implement connection and communication between the memory 11 and at least one processor 10 and the like.
  • FIG. 3 only shows an electronic device with components. Those skilled in the art can understand that the structure shown in FIG. 3 does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown in the figure. Components, or a combination of certain components, or different component arrangements.
  • the electronic device 1 may also include a power source (such as a battery) for supplying power to various components.
  • the power source may be logically connected to the at least one processor 10 through a power management device, thereby controlling power
  • the device implements functions such as charge management, discharge management, and power consumption management.
  • the power supply may also include any components such as one or more DC or AC power supplies, recharging devices, power failure detection circuits, power converters or inverters, and power status indicators.
  • the electronic device 1 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.
  • the electronic device 1 may also include a network interface.
  • the network interface may include a wired interface and/or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is usually used in the electronic device 1 Establish a communication connection with other electronic devices.
  • the electronic device 1 may also include a user interface.
  • the user interface may be a display (Display) and an input unit (such as a keyboard (Keyboard)).
  • the user interface may also be a standard wired interface or a wireless interface.
  • the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc.
  • the display can also be appropriately called a display screen or a display unit, which is used to display the information processed in the electronic device 1 and to display a visualized user interface.
  • the cluster upgrade maintenance program 12 stored in the memory 11 in the electronic device 1 is a combination of multiple instructions, and when running in the processor 10, it can realize:
  • the version detection report determine whether the cluster node server needs to be differentially released
  • the version software package and configuration file in the cluster node server are updated according to the new version software package and the new configuration file, and then the version release is performed;
  • a preset restart program is used to restart the cluster node server after the version is released, and mark the cluster node server after the version is released.
  • the integrated module/unit of the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) .
  • modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional modules in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit may be implemented in the form of hardware, or may be implemented in the form of hardware plus software functional modules.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Stored Programmes (AREA)

Abstract

Disclosed are an upgrade maintenance method and apparatus for a cluster, and an electronic device and a computer-readable storage medium. The method comprises: uploading, to a preset releasing machine, a new version of a software package that needs to be released and a new configuration file; acquiring a cluster node server that needs to be subjected to version release, and performing version detection on a software package and a configuration file in the cluster node server and the new version of the software package and the new configuration file to generate a version detection report; executing version release on the cluster node server according to the version detection report and in a preset manner; and restarting and executing, by using a preset restart program, the cluster node server after same has been subjected to version release, and marking the cluster node server after same has been subjected to version release. The method can realize the automated upgrade maintenance of a cluster.

Description

集群的升级维护方法、装置、电子设备及存储介质Cluster upgrade and maintenance method, device, electronic equipment and storage medium
本申请要求于2020年4月23日提交中国专利局、申请号为CN202010329737.3,发明名称为“集群的升级维护方法、装置、电子设备及存储介质”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese patent application filed with the Chinese Patent Office on April 23, 2020, the application number is CN202010329737.3, and the invention title is "cluster upgrade and maintenance methods, devices, electronic equipment and storage media", all of which The content is incorporated in this application by reference.
技术领域Technical field
本申请涉及软件开发技术领域,尤其涉及一种集群的升级维护的方法、装置、电子设备及计算机可读存储介质。This application relates to the technical field of software development, and in particular to a method, device, electronic equipment, and computer-readable storage medium for upgrading and maintaining a cluster.
背景技术Background technique
随着大数据平台的迅速推广,各专业公司的大量使用,以及数据业务需求量增加,集群规模越来越来大,然而发明人意识到从大数据平台搭建到集群的版本升级维护,整个过程给运维人员带来极大的挑战,主要存在以下两个问题:With the rapid promotion of big data platforms, the large-scale use of professional companies, and the increase in demand for data services, the scale of clusters is getting larger and larger. However, the inventor realized that from the construction of the big data platform to the version upgrade and maintenance of the cluster, the whole process Bringing great challenges to operation and maintenance personnel, there are mainly the following two problems:
一、同一个集群存在操作系统版本不一致,硬件配置不一致;1. In the same cluster, the operating system version is inconsistent, and the hardware configuration is inconsistent;
二、集群环境信息只是通过一个Excel文档维护,存在更新不及时的情况,会导致升级维护风险增加。2. The cluster environment information is only maintained through an Excel file, and there are cases where the update is not timely, which will increase the risk of upgrade and maintenance.
发明内容Summary of the invention
本申请提供一种集群的升级维护的方法、装置、电子设备及计算机可读存储介质。This application provides a method, device, electronic device, and computer-readable storage medium for upgrading and maintaining a cluster.
本申请提供的一种集群的升级维护方法,包括:A cluster upgrade and maintenance method provided by this application includes:
将需要发布的新版本软件包和新配置文件上传到预设的发布机器中;Upload the new version of the software package and the new configuration file that need to be released to the preset release machine;
获取需要进行版本发布的集群节点服务器,将所述集群节点服务器中的软件包和配置文件与所述新版本软件包和新配置文件进行版本检测,生成版本检测报告;Obtain a cluster node server that needs to be version released, perform version detection on the software package and configuration file in the cluster node server and the new version software package and new configuration file, and generate a version detection report;
根据所述版本检测报告,判断所述集群节点服务器是否需要进行差异发布;According to the version detection report, determine whether the cluster node server needs to be differentially released;
若不需要进行差异发布,则通过所述发布机器对所述集群节点服务器执行版本发布;If there is no need to perform differential release, perform version release to the cluster node server through the release machine;
若需要进行差异发布,则根据所述新版本软件包和新配置文件对所述集群节点服务器中的版本软件包和配置文件进行更新后执行版本发布;If a differential release is required, the version software package and configuration file in the cluster node server are updated according to the new version software package and the new configuration file, and then the version release is performed;
利用预设的重启程序重启执行版本发布后的所述集群节点服务器,并对版本发布后的所述集群节点服务器进行标记。A preset restart program is used to restart the cluster node server after the version is released, and mark the cluster node server after the version is released.
本申请还提供一种集群的升级维护装置,所述装置包括:The application also provides a cluster upgrade and maintenance device, the device includes:
上传模块,用于将需要发布集群节点服务器的新版本软件包和新配置文件上传到预设的发布机器中;The upload module is used to upload the new version of the software package and the new configuration file that need to be released on the cluster node server to the preset release machine;
检测模块,用于获取需要进行版本发布的集群节点服务器,将所述集群节点服务器中的软件包和配置文件与所述新版本软件包和新配置文件进行对应的版本检测,生成版本检测报告;The detection module is used to obtain the cluster node server that needs to be version released, perform corresponding version detection on the software package and configuration file in the cluster node server and the new version software package and the new configuration file, and generate a version detection report;
判断及执行模块,用于根据所述版本检测报告,判断所述集群节点服务器是否需要进行差异发布,若不需要进行差异发布,则通过所述发布机器对所述集群节点服务器执行版本发布,若需要进行差异发布,则根据所述新版本软件包和新配置文件对所述集群节点服务器中的版本软件包和配置文件进行更新后执行版本发布;The judgment and execution module is used to judge whether the cluster node server needs to perform differential publishing according to the version detection report, if differential publishing is not required, perform version publishing on the cluster node server through the publishing machine, if If a differential release is required, the version software package and configuration file in the cluster node server are updated according to the new version software package and the new configuration file, and then the version release is performed;
重启模块,用于利用预设的重启程序重启执行版本发布后的所述集群节点服务器,并对版本发布后的所述集群节点服务器进行标记。The restart module is used to restart the cluster node server after the version is released by using a preset restart program, and mark the cluster node server after the version is released.
本申请还提供一种电子设备,所述电子设备包括:This application also provides an electronic device, which includes:
存储器,存储至少一个指令;及Memory, storing at least one instruction; and
处理器,执行所述存储器中存储的指令以实现如下步骤:The processor executes the instructions stored in the memory to implement the following steps:
将需要发布的新版本软件包和新配置文件上传到预设的发布机器中;Upload the new version of the software package and the new configuration file that need to be released to the preset release machine;
获取需要进行版本发布的集群节点服务器,将所述集群节点服务器中的软件包和配置文件与所述新版本软件包和新配置文件进行版本检测,生成版本检测报告;Obtain a cluster node server that needs to be version released, perform version detection on the software package and configuration file in the cluster node server and the new version software package and new configuration file, and generate a version detection report;
根据所述版本检测报告,判断所述集群节点服务器是否需要进行差异发布;According to the version detection report, determine whether the cluster node server needs to be differentially released;
若不需要进行差异发布,则通过所述发布机器对所述集群节点服务器执行版本发布;If there is no need to perform differential release, perform version release to the cluster node server through the release machine;
若需要进行差异发布,则根据所述新版本软件包和新配置文件对所述集群节点服务器中的版本软件包和配置文件进行更新后执行版本发布;If a differential release is required, the version software package and configuration file in the cluster node server are updated according to the new version software package and the new configuration file, and then the version release is performed;
利用预设的重启程序重启执行版本发布后的所述集群节点服务器,并对版本发布后的所述集群节点服务器进行标记。A preset restart program is used to restart the cluster node server after the version is released, and mark the cluster node server after the version is released.
本申请还提供一种计算机可读存储介质,所述计算机可读存储介质中存储有至少一个指令,所述至少一个指令被电子设备中的处理器执行以实现如下步骤:The present application also provides a computer-readable storage medium in which at least one instruction is stored, and the at least one instruction is executed by a processor in an electronic device to implement the following steps:
将需要发布的新版本软件包和新配置文件上传到预设的发布机器中;Upload the new version of the software package and the new configuration file that need to be released to the preset release machine;
获取需要进行版本发布的集群节点服务器,将所述集群节点服务器中的软件包和配置文件与所述新版本软件包和新配置文件进行版本检测,生成版本检测报告;Obtain a cluster node server that needs to be version released, perform version detection on the software package and configuration file in the cluster node server and the new version software package and new configuration file, and generate a version detection report;
根据所述版本检测报告,判断所述集群节点服务器是否需要进行差异发布;According to the version detection report, determine whether the cluster node server needs to be differentially released;
若不需要进行差异发布,则通过所述发布机器对所述集群节点服务器执行版本发布;If there is no need to perform differential release, perform version release to the cluster node server through the release machine;
若需要进行差异发布,则根据所述新版本软件包和新配置文件对所述集群节点服务器中的版本软件包和配置文件进行更新后执行版本发布;If a differential release is required, the version software package and configuration file in the cluster node server are updated according to the new version software package and the new configuration file, and then the version release is performed;
利用预设的重启程序重启执行版本发布后的所述集群节点服务器,并对版本发布后的所述集群节点服务器进行标记。A preset restart program is used to restart the cluster node server after the version is released, and mark the cluster node server after the version is released.
附图说明Description of the drawings
图1为本申请一实施例提供的集群的升级维护方法的流程示意图;FIG. 1 is a schematic flowchart of a method for upgrading and maintaining a cluster provided by an embodiment of this application;
图2为本申请一实施例提供的集群的升级维护装置的模块示意图;2 is a schematic diagram of modules of a cluster upgrade and maintenance device provided by an embodiment of the application;
图3为本申请一实施例提供的集群的升级维护方法的电子设备的内部结构示意图;FIG. 3 is a schematic diagram of the internal structure of an electronic device of a method for upgrading and maintaining a cluster provided by an embodiment of the application;
本申请目的的实现、功能特点及优点将结合实施例,参照附图做进一步说明。The realization, functional characteristics, and advantages of the purpose of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.
具体实施方式Detailed ways
应当理解,此处所描述的具体实施例仅仅用以解释本申请,并不用于限定本申请。It should be understood that the specific embodiments described here are only used to explain the present application, and are not used to limit the present application.
本申请提供一种集群的升级维护的方法。参照图1所示,为本申请一实施例提供的集群的升级维护方法的流程示意图。该方法可以由一个装置执行,该装置可以由软件和/或硬件实现。This application provides a method for upgrading and maintaining a cluster. Referring to FIG. 1, it is a schematic flowchart of a cluster upgrade and maintenance method provided by an embodiment of this application. The method can be executed by a device, and the device can be implemented by software and/or hardware.
在本实施例中,集群的升级维护的方法包括:In this embodiment, the method for upgrading and maintaining the cluster includes:
S1、将需要发布的新版本软件包和新配置文件上传到预设的发布机器中。S1. Upload the new version of the software package and the new configuration file that need to be released to the preset release machine.
本申请实施例中,所述软件包包括jar包。所述jar包又称为JAR文件(Java归档,英语:Java Archive)是一种软件包文件格式,通常用于聚合大量的Java类文件、相关的元数据和资源(文本、图片等)文件到一个文件,以便开发Java平台应用软件或库。所述新配置文件包括:运行环境参数、接口参数以及地址参数等等。In the embodiment of the present application, the software package includes a jar package. The jar package is also called JAR file (Java Archive, English: Java Archive) is a software package file format, usually used to aggregate a large number of Java class files, related metadata and resource (text, pictures, etc.) files into A file to develop application software or libraries for the Java platform. The new configuration file includes: operating environment parameters, interface parameters, address parameters, and so on.
在本申请的一个至少实施例中,所述需要发布的新版本jar包和新配置文件通过集群节点服务器的版本确定,如某个集群节点服务器更新后的版本为1.7,则其所更新的版本内容即为所述需要发布的新版本jar包和新配置文件。所述预设的发布机器通过使用Web界面进行功能操作,用于执行集群节点服务器的版本发布。In at least one embodiment of the present application, the new version of the jar package and the new configuration file that need to be released are determined by the version of the cluster node server. If the updated version of a certain cluster node server is 1.7, the updated version The content is the new version of the jar package and the new configuration file to be released. The preset publishing machine performs functional operations by using a web interface, and is used to perform version publishing of the cluster node server.
较佳地,本申请实施例通过web界面将所述新版本jar包和新配置文件上传到预设的发 布机器中进行存储,其中,所述新版本jar包和新配置文件存储的目录为/appcom/deploy/发布日期/。Preferably, the embodiment of the present application uploads the new version jar package and the new configuration file to a preset publishing machine through a web interface for storage, wherein the directory where the new version jar package and the new configuration file are stored is / appcom/deploy/release date/.
S2、获取需要进行版本发布的集群节点服务器,将所述集群节点服务器中的jar包和配置文件与所述新版本jar包和新配置文件进行对应的版本检测,生成版本检测报告。S2. Obtain a cluster node server that needs to be version released, perform corresponding version detection on the jar package and configuration file in the cluster node server and the new version jar package and new configuration file, and generate a version detection report.
在本申请的至少一个实施例通过web界面进行集群节点服务器的信息管理,于是在所述Web界面中,选择需要进行版本发布的集群节点服务器,并抓取所述集群节点服务器中的jar包和配置文件,将所述集群节点服务器中的jar包和配置文件与所述新版本jar包和新配置文件进行对应的版本检测,生成版本检测报告。所述抓取所述集群节点服务器中的jar包和配置文件可以通过Python、Shell以及Ansible脚本实现。In at least one embodiment of the present application, the information management of the cluster node server is performed through a web interface, so in the web interface, select the cluster node server that needs to be version released, and grab the jar package and the jar package in the cluster node server. The configuration file performs corresponding version detection on the jar package and configuration file in the cluster node server and the new version jar package and the new configuration file, and generates a version detection report. The capturing of the jar package and configuration file in the cluster node server can be implemented through Python, Shell, and Ansible scripts.
详细地,所述将所述集群节点服务器中的jar包和配置文件与所述新版本jar包和新配置文件进行对应的版本检测,包括:In detail, performing corresponding version detection of the jar package and configuration file in the cluster node server with the new version jar package and the new configuration file includes:
计算所述集群节点服务器中的jar包与所述新版本jar包中的距离度量值;Calculating the distance metric value between the jar package in the cluster node server and the jar package of the new version;
计算所述集群节点服务器中的配置文件与所述新版本配置文件的相似度;Calculating the similarity between the configuration file in the cluster node server and the new version configuration file;
将所述距离度量值与所述相似度进行平均值计算,得到版本差异值;Calculating the average value of the distance metric value and the similarity to obtain a version difference value;
根据所述版本差异值在所述web页面生成版本检测报告。A version detection report is generated on the web page according to the version difference value.
所述计算所述集群节点服务器中的jar包与所述新版本jar包中的距离度量值包括:The calculating the distance metric value between the jar package in the cluster node server and the new version jar package includes:
Figure PCTCN2020119110-appb-000001
Figure PCTCN2020119110-appb-000001
其中,ρ X,Y表示距离度量值,X表示集群节点服务器中jar包的md5值,Y表示新版本jar包的md5值,μ X表示集群节点服务器中jar包的md5均值,μ Y表示新版本jar包的md5均值,σ X表示集群节点服务器中jar包的md5值的标准差,σ Y表示新版本jar包的md5值的标准差,E表示集群节点服务器中jar包与新版本jar包的期望,若所述距离度量值为1即代表两个jar包完全一样,若所述距离度量值为-1即代表两个jar包完全不一样。 Among them, ρ X, Y represents the distance measurement value, X represents the md5 value of the jar package in the cluster node server, Y represents the md5 value of the new version of the jar package, μ X represents the average md5 value of the jar package in the cluster node server, and μ Y represents the new The mean value of md5 of the version jar package, σ X represents the standard deviation of the md5 value of the jar package in the cluster node server, σ Y represents the standard deviation of the md5 value of the new version jar package, and E represents the jar package in the cluster node server and the new version jar package It is expected that if the distance metric value is 1, it means that the two jar packages are completely the same, and if the distance metric value is -1, it means that the two jar packages are completely different.
所述计算所述集群节点服务器中的配置文件与所述新版本配置文件的相似度包括:The calculating the similarity between the configuration file in the cluster node server and the new version configuration file includes:
Figure PCTCN2020119110-appb-000002
Figure PCTCN2020119110-appb-000002
其中,sim(X,Y)表示相似度,X所述集群节点服务器中配置文件md5值,Y表示新配置文件的md5值,X i、Y i分别标识集群节点服务器与所述新配置文件第i位的md5值,如md5(jar)=38b8c2c1093dd0fec383a9d9ac940515,其第一位的md5值为3,最后一位的md5值为5,为简化计算,本申请将md5值中的字符用ASCII码转化为数值,例如,c经由ASCII码值转化后是99。其产生的相似性范围从-1到1:-1意味着所述集群节点服务器中的配置文件与所述新版本配置文件指向的方向正好截然相反,1表示它们的指向是完全相同的,0通常表示它们之间是独立的,而在这之间的值则表示中度的相似性和相异性。 Wherein, sim(X,Y) represents the similarity, X represents the md5 value of the configuration file in the cluster node server, Y represents the md5 value of the new configuration file, and X i and Y i respectively identify the cluster node server and the new configuration file. The md5 value of the i bit, such as md5(jar)=38b8c2c1093dd0fec383a9d9ac940515, the md5 value of the first bit is 3, and the md5 value of the last bit is 5. To simplify the calculation, this application converts the characters in the md5 value into ASCII code Numerical value, for example, c is 99 after being converted into ASCII code value. The similarity generated by it ranges from -1 to 1: -1 means that the configuration file in the cluster node server and the new version configuration file point to the exact opposite direction, 1 means that their points are exactly the same, 0 It usually means that they are independent, and the value in between indicates moderate similarity and dissimilarity.
进一步地,本申请实施例将所述距离度量值与所述相似度进行平均值计算,得到版本差异值,将所述版本差异值与预设的阈值对比,若所述版本差异值小于预设的阈值时,表示所述集群节点服务器与新版本集群节点服务器存在差异结果,若所述差异结果值不小于预设的阈值时,表示所述集群节点服务器与新版本集群节点服务器不在差异结果,从而根据所述版本差异值生成所述版本检测报告。优选地,所述预设的阈值为0.95。Further, in the embodiment of the present application, the average value of the distance metric value and the similarity is calculated to obtain the version difference value, and the version difference value is compared with a preset threshold value, if the version difference value is less than the preset threshold value. When the threshold value of the cluster node server and the new version cluster node server have a difference result, if the difference result value is not less than the preset threshold value, it means that the cluster node server and the new version cluster node server are not in the difference result, Thus, the version detection report is generated according to the version difference value. Preferably, the preset threshold value is 0.95.
所述差异结果包括:集群节点服务器的当前版本落后新发布集群节点服务器版本;集群节点服务器的硬件和/或系统与新版本集群节点服务器的硬件和/或系统存在差异。The difference result includes: the current version of the cluster node server is behind the newly released version of the cluster node server; the hardware and/or system of the cluster node server is different from the hardware and/or system of the new version of the cluster node server.
基于上述的实施方式,本申请实施例通过版本差异值的计算,确保了版本检测的精确性。Based on the foregoing implementation manner, the embodiment of the present application ensures the accuracy of version detection by calculating the version difference value.
S3、根据所述版本检测报告,判断所述集群节点服务器是否需要进行差异发布。S3. Determine whether the cluster node server needs to perform differential publishing according to the version detection report.
若不需要进行差异发布,执行S4、通过所述发布机器将所述集群节点服务器执行版本发布。If there is no need to perform differential publishing, execute S4 to publish the execution version of the cluster node server through the publishing machine.
若需要进行差异发布,执行S5、根据所述新版本软件包和新配置文件对所述集群节点服务器中的版本软件包和配置文件进行更新后执行版本发布。If differential release is required, execute S5, and perform version release after updating the version software package and configuration file in the cluster node server according to the new version software package and the new configuration file.
S6、利用预设的重启程序重启执行版本发布后的所述集群节点服务器,并对版本发布后的所述集群节点服务器进行标记,实现对版本发布后的集群节点服务器的升级维护。S6. Use a preset restart program to restart the cluster node server after the version is released, and mark the cluster node server after the version is released, so as to realize the upgrade and maintenance of the cluster node server after the version is released.
在本申请的至少一个实施例中,在完成所述集群节点服务器版本发布功能后,需要通过预设的重启程序使发版功能生效,所述预设的重启程序通过shell脚本进行编译。较佳地,本申请以hadoop集群为例,其预设的重启服务重启hadoop集群步骤为:启动Ldap、kerbores;启动DNS服务;在集群所有节点上,执行servicentpd restart;启动zookeeper,yarn;在集群中任意节点执行:spark-sql可进入spark命令行,并执行简单的语句selectcount(*)from tablename以检查spark是否正常;hbase启动,滚动重启,所述滚动重启步骤如下所示:In at least one embodiment of the present application, after the cluster node server version release function is completed, a preset restart program needs to be used to validate the release function, and the preset restart program is compiled through a shell script. Preferably, this application takes the Hadoop cluster as an example, and the preset restart service steps to restart the Hadoop cluster are: start Ldap and kerbores; start DNS service; execute servicentpd restart on all nodes in the cluster; start zookeeper, yarn; in the cluster Execute at any node: spark-sql can enter the spark command line, and execute a simple statement selectcount(*)from tablename to check whether spark is normal; hbase starts, rolling restart, the rolling restart steps are as follows:
1.停止索引新的数据;1. Stop indexing new data;
2.禁止分片分配;2. Prohibit fragment distribution;
3.关闭单个节点;3. Shut down a single node;
4.执行维护/升级;4. Perform maintenance/upgrade;
5.重启节点,然后确认它加入到集群中;5. Restart the node, and then confirm that it joins the cluster;
6.进行重启分片分配;6. Perform restart shard allocation;
7.重复第2到6步操作剩余节点;7. Repeat steps 2 to 6 to operate the remaining nodes;
8.恢复索引,批量重启(本申请使用shell脚本进行批量重启集群)。8. Restore the index and restart in batches (this application uses a shell script to restart the cluster in batches).
较佳地,在发版完成后,本申请对当前集群节点服务其发布的版本进行标记,以使后续集群节点服务器发布的版本能清晰了解当前版本情况,便于多集群节点服务器的升级与维护。Preferably, after the release is completed, this application marks the released version of the current cluster node service, so that subsequent releases of the cluster node server can clearly understand the current version and facilitate the upgrade and maintenance of multi-cluster node servers.
如图2所示,是本申请集群的升级维护装置的功能模块图。As shown in Figure 2, it is a functional module diagram of the upgrade and maintenance device for the cluster of the present application.
本申请所述集群的升级维护的装置100可以安装于电子设备中。根据实现的功能,所述集群的升级维护装置可以包括上传模块101、检测模块102、判断集执行103以及重启模块103。本发所述模块也可以称之为单元,是指一种能够被电子设备处理器所执行,并且能够完成固定功能的一系列计算机程序段,其存储在电子设备的存储器中。The apparatus 100 for upgrading and maintaining the cluster described in this application can be installed in an electronic device. According to the implemented functions, the upgrade and maintenance device of the cluster may include an upload module 101, a detection module 102, a judgment set execution 103, and a restart module 103. The module described in the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can complete fixed functions, and are stored in the memory of the electronic device.
在本实施例中,关于各模块/单元的功能如下:In this embodiment, the functions of each module/unit are as follows:
所述上传模块101,用于将需要发布集群节点服务器的新版本软件包和新配置文件上传到预设的发布机器中;The upload module 101 is used to upload the new version software package and the new configuration file of the cluster node server to the preset publishing machine;
所述检测模块102,用于获取需要进行版本发布的集群节点服务器,将所述集群节点服务器中的软件包和配置文件与所述新版本软件包和新配置文件进行对应的版本检测,生成版本检测报告;The detection module 102 is configured to obtain the cluster node server that needs to be version released, and perform corresponding version detection of the software package and configuration file in the cluster node server with the new version software package and the new configuration file to generate a version Test Report;
所述判断及执行模块103,用于根据所述版本检测报告,判断所述集群节点服务器是否需要进行差异发布,若不需要进行差异发布,则通过所述发布机器对所述集群节点服务器执行版本发布,若需要进行差异发布,则根据所述新版本软件包和新配置文件对所述集群节点服务器中的版本软件包和配置文件进行更新后执行版本发布;The judgment and execution module 103 is configured to judge whether the cluster node server needs to be differentially released according to the version detection report, if no differential release is required, execute the version on the cluster node server through the publishing machine Release, if differential release is required, the version software package and configuration file in the cluster node server are updated according to the new version software package and the new configuration file, and then the version release is performed;
所述重启模块104,用于利用预设的重启程序重启执行版本发布后的所述集群节点服务器,并对版本发布后的所述集群节点服务器进行标记。The restart module 104 is configured to use a preset restart program to restart the cluster node server after the version is released, and mark the cluster node server after the version is released.
详细地,所述集群的升级维护装置各模块在被所述电子设备的处理器所执行时,实施步骤如下:In detail, when each module of the cluster upgrade and maintenance apparatus is executed by the processor of the electronic device, the implementation steps are as follows:
所述上传模块101将需要发布集群节点服务器的新版本软件包和新配置文件上传到预设的发布机器中。The uploading module 101 uploads the new version software package and the new configuration file of the cluster node server to the preset publishing machine.
本申请实施例中,所述软件包包括jar包。所述jar包又称为JAR文件(Java归档,英语:Java Archive)是一种软件包文件格式,通常用于聚合大量的Java类文件、相关的元数据和资源(文本、图片等)文件到一个文件,以便开发Java平台应用软件或库。所述新配置文件包 括:运行环境参数、接口参数以及地址参数等等。In the embodiment of the present application, the software package includes a jar package. The jar package is also called JAR file (Java Archive, English: Java Archive) is a software package file format, usually used to aggregate a large number of Java class files, related metadata and resource (text, pictures, etc.) files into A file to develop application software or libraries for the Java platform. The new configuration file includes: operating environment parameters, interface parameters, address parameters, and so on.
在本申请的一个至少实施例中,所述需要发布的新版本jar包和新配置文件通过集群节点服务器的版本确定,如某个集群节点服务器更新后的版本为1.7,则其所更新的版本内容即为所述需要发布的新版本jar包和新配置文件。所述预设的发布机器通过使用Web界面进行功能操作,用于执行集群节点服务器的版本发布。In at least one embodiment of the present application, the new version of the jar package and the new configuration file that need to be released are determined by the version of the cluster node server. If the updated version of a certain cluster node server is 1.7, the updated version The content is the new version of the jar package and the new configuration file to be released. The preset publishing machine performs functional operations by using a web interface, and is used to perform version publishing of the cluster node server.
较佳地,所述上传模块101通过web界面将所述新版本jar包和新配置文件上传到预设的发布机器中进行存储,其中,所述新版本jar包和新配置文件存储的目录为/appcom/deploy/发布日期/。Preferably, the upload module 101 uploads the new version jar package and the new configuration file to a preset publishing machine for storage through a web interface, wherein the directory where the new version jar package and the new configuration file are stored is /appcom/deploy/release date/.
所述检测模块102获取需要进行版本发布的集群节点服务器,将所述集群节点服务器中的软件包和配置文件与所述新版本软件包和新配置文件进行对应的版本检测,生成版本检测报告。The detection module 102 obtains the cluster node server that needs to be version released, performs corresponding version detection on the software package and configuration file in the cluster node server and the new version software package and the new configuration file, and generates a version detection report.
在本申请的至少一个实施例中,所述检测模块102通过web界面进行集群节点服务器的信息管理,于是在所述Web界面中,选择需要进行版本发布的集群节点服务器,并抓取所述集群节点服务器中的jar包和配置文件,将所述集群节点服务器中的jar包和配置文件与所述新版本jar包和新配置文件进行对应的版本检测,生成版本检测报告。所述抓取所述集群节点服务器中的jar包和配置文件可以通过Python、Shell以及Ansible脚本实现。In at least one embodiment of the present application, the detection module 102 performs information management of the cluster node server through a web interface, so in the web interface, select the cluster node server that needs version release, and grab the cluster For the jar package and configuration file in the node server, perform corresponding version detection on the jar package and configuration file in the cluster node server and the new version jar package and the new configuration file, and generate a version detection report. The capturing of the jar package and configuration file in the cluster node server can be implemented through Python, Shell, and Ansible scripts.
详细地,所述将所述集群节点服务器中的jar包和配置文件与所述新版本jar包和新配置文件进行对应的版本检测,包括:In detail, performing corresponding version detection of the jar package and configuration file in the cluster node server with the new version jar package and the new configuration file includes:
计算所述集群节点服务器中的jar包与所述新版本jar包中的距离度量值;Calculating the distance metric value between the jar package in the cluster node server and the jar package of the new version;
计算所述集群节点服务器中的配置文件与所述新版本配置文件的相似度;Calculating the similarity between the configuration file in the cluster node server and the new version configuration file;
将所述距离度量值与所述相似度进行平均值计算,得到版本差异值;Calculating the average value of the distance metric value and the similarity to obtain a version difference value;
根据所述版本差异值在所述web页面生成版本检测报告。A version detection report is generated on the web page according to the version difference value.
所述计算所述集群节点服务器中的jar包与所述新版本jar包中的距离度量值包括:The calculating the distance metric value between the jar package in the cluster node server and the new version jar package includes:
Figure PCTCN2020119110-appb-000003
Figure PCTCN2020119110-appb-000003
其中,ρ X,Y表示距离度量值,X表示集群节点服务器中jar包的md5值,Y表示新版本jar包的md5值,μ X表示集群节点服务器中jar包的md5均值,μ Y表示新版本jar包的md5均值,σ X表示集群节点服务器中jar包的md5值的标准差,σ Y表示新版本jar包的md5值的标准差,E表示集群节点服务器中jar包与新版本jar包的期望,若所述距离度量值为1即代表两个jar包完全一样,若所述距离度量值为-1即代表两个jar包完全不一样。 Among them, ρ X, Y represents the distance measurement value, X represents the md5 value of the jar package in the cluster node server, Y represents the md5 value of the new version of the jar package, μ X represents the average md5 value of the jar package in the cluster node server, and μ Y represents the new The mean md5 value of the version jar package, σ X represents the standard deviation of the md5 value of the jar package in the cluster node server, σ Y represents the standard deviation of the md5 value of the new version jar package, and E represents the jar package in the cluster node server and the new version jar package It is expected that if the distance metric value is 1, it means that the two jar packages are completely the same, and if the distance metric value is -1, it means that the two jar packages are completely different.
所述计算所述集群节点服务器中的配置文件与所述新版本配置文件的相似度包括:The calculating the similarity between the configuration file in the cluster node server and the new version configuration file includes:
Figure PCTCN2020119110-appb-000004
Figure PCTCN2020119110-appb-000004
其中,sim(X,Y)表示相似度,X所述集群节点服务器中配置文件md5值,Y表示新配置文件的md5值,X i、Y i分别标识集群节点服务器与所述新配置文件第i位的md5值,如md5(jar)=38b8c2c1093dd0fec383a9d9ac940515,其第一位的md5值为3,最后一位的md5值为5,为简化计算,本申请将md5值中的字符用ASCII码转化为数值,例如,c经由ASCII码值转化后是99。其产生的相似性范围从-1到1:-1意味着所述集群节点服务器中的配置文件与所述新版本配置文件指向的方向正好截然相反,1表示它们的指向是完全相同的,0通常表示它们之间是独立的,而在这之间的值则表示中度的相似性和相异性。 Among them, sim(X, Y) represents the similarity, X represents the md5 value of the configuration file in the cluster node server, Y represents the md5 value of the new configuration file, and X i and Y i respectively identify the cluster node server and the new configuration file. The md5 value of the i bit, such as md5(jar)=38b8c2c1093dd0fec383a9d9ac940515, the md5 value of the first bit is 3, and the md5 value of the last bit is 5. To simplify the calculation, this application converts the characters in the md5 value into ASCII code Numerical value, for example, c is 99 after being converted into ASCII code value. The similarity generated by it ranges from -1 to 1: -1 means that the configuration file in the cluster node server and the new version configuration file point to the exact opposite direction, 1 means that their points are exactly the same, 0 It usually means that they are independent, and the value in between indicates moderate similarity and dissimilarity.
进一步地,本申请实施例所述检测模块102将所述距离度量值与所述相似度进行平均值计算,得到版本差异值,将所述版本差异值与预设的阈值对比,若所述版本差异值小于预设的阈值时,表示所述集群节点服务器与新版本集群节点服务器存在差异结果,若所述差异结果值不小于预设的阈值时,表示所述集群节点服务器与新版本集群节点服务器不在差异结果, 从而根据所述版本差异值生成所述版本检测报告。优选地,所述预设的阈值为0.95。Further, the detection module 102 in the embodiment of the present application calculates the average value of the distance metric value and the similarity to obtain the version difference value, and compares the version difference value with a preset threshold. When the difference value is less than the preset threshold value, it means that the cluster node server has a difference result from the new version cluster node server. If the difference result value is not less than the preset threshold value, it means the cluster node server and the new version cluster node The server is not in the difference result, and thus generates the version detection report according to the version difference value. Preferably, the preset threshold value is 0.95.
所述差异结果包括:集群节点服务器的当前版本落后新发布集群节点服务器版本;集群节点服务器的硬件和/或系统与新版本集群节点服务器的硬件和/或系统存在差异。The difference result includes: the current version of the cluster node server is behind the newly released version of the cluster node server; the hardware and/or system of the cluster node server is different from the hardware and/or system of the new version of the cluster node server.
基于上述的实施方式,本申请实施例所述检测模块102通过版本差异值的计算,确保了版本检测的精确性。Based on the foregoing implementation manner, the detection module 102 described in the embodiment of the present application ensures the accuracy of version detection by calculating the version difference value.
所述判断及执行模块103根据所述版本检测报告,判断所述集群节点服务器是否需要进行差异发布,若不需要进行差异发布,则通过所述发布机器对所述集群节点服务器执行版本发布,若需要进行差异发布,则根据所述新版本软件包和新配置文件对所述集群节点服务器中的版本软件包和配置文件进行更新后执行版本发布。The judgment and execution module 103 judges whether the cluster node server needs to be differentially released according to the version detection report. If no differential release is required, then the release machine performs version release on the cluster node server, if If a differential release is required, the version software package and configuration file in the cluster node server are updated according to the new version software package and the new configuration file, and then the version release is performed.
所述重启模块104利用预设的重启程序重启执行版本发布后的所述集群节点服务器,并对版本发布后的所述集群节点服务器进行标记。The restart module 104 uses a preset restart procedure to restart the cluster node server after the version is released, and marks the cluster node server after the version is released.
在本申请的至少一个实施例中,在完成所述集群节点服务器版本发布功能后,需要通过预设的重启程序使发版功能生效,所述预设的重启程序通过shell脚本进行编译。较佳地,本申请实施例以hadoop集群为例,其预设的重启服务重启hadoop集群步骤为:启动Ldap、kerbores;启动DNS服务;在集群所有节点上,执行servicentpd restart;启动zookeeper,yarn;在集群中任意节点执行:spark-sql可进入spark命令行,并执行简单的语句selectcount(*)from tablename以检查spark是否正常;hbase启动,滚动重启,所述滚动重启步骤如下所示:In at least one embodiment of the present application, after the cluster node server version release function is completed, a preset restart program needs to be used to validate the release function, and the preset restart program is compiled through a shell script. Preferably, the embodiment of the application takes the Hadoop cluster as an example, and the preset restart service steps to restart the Hadoop cluster are: start Ldap and kerbores; start DNS service; execute servicentpd restart on all nodes in the cluster; start zookeeper, yarn; Execute on any node in the cluster: spark-sql can enter the spark command line and execute a simple statement selectcount(*)from tablename to check whether spark is normal; hbase starts, rolling restart, the rolling restart steps are as follows:
1.停止索引新的数据;1. Stop indexing new data;
2.禁止分片分配;2. Prohibit fragment distribution;
3.关闭单个节点;3. Shut down a single node;
4.执行维护/升级;4. Perform maintenance/upgrade;
5.重启节点,然后确认它加入到集群中;5. Restart the node, and then confirm that it joins the cluster;
6.进行重启分片分配;6. Perform restart shard allocation;
7.重复第2到6步操作剩余节点;7. Repeat steps 2 to 6 to operate the remaining nodes;
8.恢复索引,批量重启(本申请使用shell脚本进行批量重启集群)。8. Restore the index and restart in batches (this application uses a shell script to restart the cluster in batches).
较佳地,在发版完成后,所述重启模块104对当前集群节点服务其发布的版本进行标记,以使后续集群节点服务器发布的版本能清晰了解当前版本情况,便于多集群节点服务器的升级与维护。Preferably, after the release is completed, the restart module 104 marks the released version of the current cluster node service, so that subsequent releases of the cluster node server can clearly understand the current version, and facilitate the upgrade of multi-cluster node servers And maintenance.
如图3所示,是本申请实现集群的升级维护的方法的电子设备的结构示意图。As shown in FIG. 3, it is a schematic diagram of the structure of an electronic device implementing the method for upgrading and maintaining a cluster according to the present application.
所述电子设备1可以包括处理器10、存储器11和总线,还可以包括存储在所述存储器11中并可在所述处理器10上运行的计算机程序,如集群的升级维护程序12。The electronic device 1 may include a processor 10, a memory 11, and a bus, and may also include a computer program stored in the memory 11 and running on the processor 10, such as a cluster upgrade maintenance program 12.
其中,所述存储器11至少包括一种类型的可读存储介质,所述可读存储介质可以是非易失性,也可以是易失性,所述可读存储介质包括闪存、移动硬盘、多媒体卡、卡型存储器(例如:SD或DX存储器等)、磁性存储器、磁盘、光盘等。所述存储器11在一些实施例中可以是电子设备1的内部存储单元,例如该电子设备1的移动硬盘。所述存储器11在另一些实施例中也可以是电子设备1的外部存储设备,例如电子设备1上配备的插接式移动硬盘、智能存储卡(Smart Media Card,SMC)、安全数字(Secure Digital,SD)卡、闪存卡(Flash Card)等。进一步地,所述存储器11还可以既包括电子设备1的内部存储单元也包括外部存储设备。所述存储器11不仅可以用于存储安装于电子设备1的应用软件及各类数据,例如集群的升级维护程序的代码等,还可以用于暂时地存储已经输出或者将要输出的数据。The memory 11 includes at least one type of readable storage medium. The readable storage medium may be nonvolatile or volatile. The readable storage medium includes flash memory, mobile hard disk, and multimedia card. , Card-type memory (for example: SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, etc. The memory 11 may be an internal storage unit of the electronic device 1 in some embodiments, for example, a mobile hard disk of the electronic device 1. In other embodiments, the memory 11 may also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart media card (SMC), and a secure digital (Secure Digital) equipped on the electronic device 1. , SD) card, flash card (Flash Card), etc. Further, the memory 11 may also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 11 can be used not only to store application software and various types of data installed in the electronic device 1, such as the code of a cluster upgrade and maintenance program, etc., but also to temporarily store data that has been output or will be output.
所述处理器10在一些实施例中可以由集成电路组成,例如可以由单个封装的集成电路所组成,也可以是由多个相同功能或不同功能封装的集成电路所组成,包括一个或者多个中央处理器(Central Processing unit,CPU)、微处理器、数字处理芯片、图形处理器及各种控制芯片的组合等。所述处理器10是所述电子设备的控制核心(Control Unit),利用各种接口和 线路连接整个电子设备的各个部件,通过运行或执行存储在所述存储器11内的程序或者模块(例如集群的升级维护程序等),以及调用存储在所述存储器11内的数据,以执行电子设备1的各种功能和处理数据。The processor 10 may be composed of integrated circuits in some embodiments, for example, may be composed of a single packaged integrated circuit, or may be composed of multiple integrated circuits with the same function or different functions, including one or more Combinations of central processing unit (CPU), microprocessor, digital processing chip, graphics processor, and various control chips, etc. The processor 10 is the control core (Control Unit) of the electronic device, which uses various interfaces and lines to connect the various components of the entire electronic device, and runs or executes programs or modules (such as clusters) stored in the memory 11 Upgrade and maintenance procedures, etc.), and call the data stored in the memory 11 to execute various functions of the electronic device 1 and process data.
所述总线可以是外设部件互连标准(peripheral component interconnect,简称PCI)总线或扩展工业标准结构(extended industry standard architecture,简称EISA)总线等。该总线可以分为地址总线、数据总线、控制总线等。所述总线被设置为实现所述存储器11以及至少一个处理器10等之间的连接通信。The bus may be a peripheral component interconnect standard (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus and so on. The bus is configured to implement connection and communication between the memory 11 and at least one processor 10 and the like.
图3仅示出了具有部件的电子设备,本领域技术人员可以理解的是,图3示出的结构并不构成对所述电子设备1的限定,可以包括比图示更少或者更多的部件,或者组合某些部件,或者不同的部件布置。FIG. 3 only shows an electronic device with components. Those skilled in the art can understand that the structure shown in FIG. 3 does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown in the figure. Components, or a combination of certain components, or different component arrangements.
例如,尽管未示出,所述电子设备1还可以包括给各个部件供电的电源(比如电池),优选地,电源可以通过电源管理装置与所述至少一个处理器10逻辑相连,从而通过电源管理装置实现充电管理、放电管理、以及功耗管理等功能。电源还可以包括一个或一个以上的直流或交流电源、再充电装置、电源故障检测电路、电源转换器或者逆变器、电源状态指示器等任意组件。所述电子设备1还可以包括多种传感器、蓝牙模块、Wi-Fi模块等,在此不再赘述。For example, although not shown, the electronic device 1 may also include a power source (such as a battery) for supplying power to various components. Preferably, the power source may be logically connected to the at least one processor 10 through a power management device, thereby controlling power The device implements functions such as charge management, discharge management, and power consumption management. The power supply may also include any components such as one or more DC or AC power supplies, recharging devices, power failure detection circuits, power converters or inverters, and power status indicators. The electronic device 1 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.
进一步地,所述电子设备1还可以包括网络接口,可选地,所述网络接口可以包括有线接口和/或无线接口(如WI-FI接口、蓝牙接口等),通常用于在该电子设备1与其他电子设备之间建立通信连接。Further, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and/or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is usually used in the electronic device 1 Establish a communication connection with other electronic devices.
可选地,该电子设备1还可以包括用户接口,用户接口可以是显示器(Display)、输入单元(比如键盘(Keyboard)),可选地,用户接口还可以是标准的有线接口、无线接口。可选地,在一些实施例中,显示器可以是LED显示器、液晶显示器、触控式液晶显示器以及OLED(Organic Light-Emitting Diode,有机发光二极管)触摸器等。其中,显示器也可以适当的称为显示屏或显示单元,用于显示在电子设备1中处理的信息以及用于显示可视化的用户界面。Optionally, the electronic device 1 may also include a user interface. The user interface may be a display (Display) and an input unit (such as a keyboard (Keyboard)). Optionally, the user interface may also be a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display can also be appropriately called a display screen or a display unit, which is used to display the information processed in the electronic device 1 and to display a visualized user interface.
应该了解,所述实施例仅为说明之用,在专利申请范围上并不受此结构的限制。It should be understood that the embodiments are only for illustrative purposes, and are not limited by this structure in the scope of the patent application.
所述电子设备1中的所述存储器11存储的集群的升级维护程序12是多个指令的组合,在所述处理器10中运行时,可以实现:The cluster upgrade maintenance program 12 stored in the memory 11 in the electronic device 1 is a combination of multiple instructions, and when running in the processor 10, it can realize:
将需要发布的新版本软件包和新配置文件上传到预设的发布机器中;Upload the new version of the software package and the new configuration file that need to be released to the preset release machine;
获取需要进行版本发布的集群节点服务器,将所述集群节点服务器中的软件包和配置文件与所述新版本软件包和新配置文件进行版本检测,生成版本检测报告;Obtain a cluster node server that needs to be version released, perform version detection on the software package and configuration file in the cluster node server and the new version software package and new configuration file, and generate a version detection report;
根据所述版本检测报告,判断所述集群节点服务器是否需要进行差异发布;According to the version detection report, determine whether the cluster node server needs to be differentially released;
若不需要进行差异发布,则通过所述发布机器对所述集群节点服务器执行版本发布;If there is no need to perform differential release, perform version release to the cluster node server through the release machine;
若需要进行差异发布,则根据所述新版本软件包和新配置文件对所述集群节点服务器中的版本软件包和配置文件进行更新后执行版本发布;If a differential release is required, the version software package and configuration file in the cluster node server are updated according to the new version software package and the new configuration file, and then the version release is performed;
利用预设的重启程序重启执行版本发布后的所述集群节点服务器,并对版本发布后的所述集群节点服务器进行标记。A preset restart program is used to restart the cluster node server after the version is released, and mark the cluster node server after the version is released.
具体地,所述处理器10对上述指令的具体实现方法可参考图1及图2对应实施例中相关步骤的描述,在此不赘述。Specifically, for the specific implementation method of the above-mentioned instructions by the processor 10, reference may be made to the description of the relevant steps in the corresponding embodiments of FIG. 1 and FIG. 2, which will not be repeated here.
进一步地,所述电子设备1集成的模块/单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个计算机可读取存储介质中。所述计算机可读介质可以包括:能够携带所述计算机程序代码的任何实体或装置、记录介质、U盘、移动硬盘、磁碟、光盘、计算机存储器、只读存储器(ROM,Read-Only Memory)。Further, if the integrated module/unit of the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) .
在本申请所提供的几个实施例中,应该理解到,所揭露的设备,装置和方法,可以通过其它的方式实现。例如,以上所描述的装置实施例仅仅是示意性的,例如,所述模块的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式。In the several embodiments provided in this application, it should be understood that the disclosed equipment, device, and method may be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.
所述作为分离部件说明的模块可以是或者也可以不是物理上分开的,作为模块显示的部 件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
另外,在本申请各个实施例中的各功能模块可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现,也可以采用硬件加软件功能模块的形式实现。In addition, the functional modules in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware, or may be implemented in the form of hardware plus software functional modules.
对于本领域技术人员而言,显然本申请不限于上述示范性实施例的细节,而且在不背离本申请的精神或基本特征的情况下,能够以其他的具体形式实现本申请。For those skilled in the art, it is obvious that the present application is not limited to the details of the foregoing exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or basic characteristics of the application.
因此,无论从哪一点来看,均应将实施例看作是示范性的,而且是非限制性的,本申请的范围由所附权利要求而不是上述说明限定,因此旨在将落在权利要求的等同要件的含义和范围内的所有变化涵括在本申请内。不应将权利要求中的任何附关联图标记视为限制所涉及的权利要求。Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of this application is defined by the appended claims rather than the above description, and therefore it is intended to fall into the claims. All changes in the meaning and scope of the equivalent elements of are included in this application. Any associated diagram marks in the claims should not be regarded as limiting the claims involved.
此外,显然“包括”一词不排除其他单元或步骤,单数不排除复数。系统权利要求中陈述的多个单元或装置也可以由一个单元或装置通过软件或者硬件来实现。第二等词语用来表示名称,而并不表示任何特定的顺序。In addition, it is obvious that the word "including" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claims can also be implemented by one unit or device through software or hardware. The second class words are used to indicate names, and do not indicate any specific order.
最后应说明的是,以上实施例仅用以说明本申请的技术方案而非限制,尽管参照较佳实施例对本申请进行了详细说明,本领域的普通技术人员应当理解,可以对本申请的技术方案进行修改或等同替换,而不脱离本申请技术方案的精神和范围。Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the application and not to limit them. Although the application has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the application can be Make modifications or equivalent replacements without departing from the spirit and scope of the technical solution of the present application.

Claims (20)

  1. 一种集群的升级维护方法,其中,所述方法包括:A cluster upgrade and maintenance method, wherein the method includes:
    将需要发布的新版本软件包和新配置文件上传到预设的发布机器中;Upload the new version of the software package and the new configuration file that need to be released to the preset release machine;
    获取需要进行版本发布的集群节点服务器,将所述集群节点服务器中的软件包和配置文件与所述新版本软件包和新配置文件进行版本检测,生成版本检测报告;Obtain a cluster node server that needs to be version released, perform version detection on the software package and configuration file in the cluster node server and the new version software package and new configuration file, and generate a version detection report;
    根据所述版本检测报告,判断所述集群节点服务器是否需要进行差异发布;According to the version detection report, determine whether the cluster node server needs to be differentially released;
    若不需要进行差异发布,则通过所述发布机器对所述集群节点服务器执行版本发布;If there is no need to perform differential release, perform version release to the cluster node server through the release machine;
    若需要进行差异发布,则根据所述新版本软件包和新配置文件对所述集群节点服务器中的版本软件包和配置文件进行更新后执行版本发布;If a differential release is required, the version software package and configuration file in the cluster node server are updated according to the new version software package and the new configuration file, and then the version release is performed;
    利用预设的重启程序重启执行版本发布后的所述集群节点服务器,并对版本发布后的所述集群节点服务器进行标记。A preset restart program is used to restart the cluster node server after the version is released, and mark the cluster node server after the version is released.
  2. 如权利要求1所述的集群的升级维护方法,其中,所述将所述集群节点服务器中的软件包和配置文件与所述新版本软件包和新配置文件进行版本检测,生成版本检测报告,包括:The method for upgrading and maintaining a cluster according to claim 1, wherein said performing version detection on the software package and configuration file in the cluster node server and the new version software package and new configuration file, and generating a version detection report, include:
    计算所述集群节点服务器中的软件包与所述新版本软件包中的距离度量值;Calculating a distance metric value between the software package in the cluster node server and the new version software package;
    计算所述集群节点服务器中的配置文件与所述新版本配置文件的相似度;Calculating the similarity between the configuration file in the cluster node server and the new version configuration file;
    将所述距离度量值与所述相似度进行平均值计算,得到版本差异值;Calculating the average value of the distance metric value and the similarity to obtain a version difference value;
    根据所述版本差异值,生成所述版本检测报告。According to the version difference value, the version detection report is generated.
  3. 如权利要求2所述的集群的升级维护方法,其中,所述计算所述集群节点服务器中的软件包与所述新版本软件包中的距离度量值,包括:The method for upgrading and maintaining a cluster according to claim 2, wherein the calculating the distance metric value between the software package in the cluster node server and the new version software package comprises:
    利用下述公式计算所述集群节点服务器中的软件包与所述新版本软件包中的距离度量值:Use the following formula to calculate the distance metric value between the software package in the cluster node server and the new version software package:
    Figure PCTCN2020119110-appb-100001
    Figure PCTCN2020119110-appb-100001
    其中,ρ X,Y表示距离度量值,X表示集群节点服务器中软件包的md5值,Y表示新版本软件包的md5值,μ X表示集群节点服务器中软件包的md5均值,μ Y表示新版本软件包的md5均值,σ X表示集群节点服务器中软件包的md5值的标准差,σ Y表示新版本软件包的md5值的标准差,E表示集群节点服务器中软件包与新版本软件包的期望。 Among them, ρ X, Y represents the distance measurement value, X represents the md5 value of the software package in the cluster node server, Y represents the md5 value of the new version of the software package, μ X represents the average md5 value of the software package in the cluster node server, and μ Y represents the new The mean value of md5 of the version software package, σ X represents the standard deviation of the md5 value of the software package in the cluster node server, σ Y represents the standard deviation of the md5 value of the new version software package, and E represents the software package in the cluster node server and the new version software package Expectations.
  4. 如权利要求2或3所述的集群的升级维护方法,其中,所述计算所述集群节点服务器中的配置文件与所述新版本配置文件的相似度,包括:The method for upgrading and maintaining a cluster according to claim 2 or 3, wherein the calculating the similarity between the configuration file in the cluster node server and the new version configuration file comprises:
    利用下述公式计算所述集群节点服务器中的配置文件与所述新版本配置文件的相似度:Use the following formula to calculate the similarity between the configuration file in the cluster node server and the new version configuration file:
    Figure PCTCN2020119110-appb-100002
    Figure PCTCN2020119110-appb-100002
    其中,sim(X,Y)表示相似度,X所述集群节点服务器中配置文件md5值,Y表示新配置文件的md5值,X i、Y i分别标识集群节点服务器与所述新配置文件第i位的md5值。 Wherein, sim(X,Y) represents the similarity, X represents the md5 value of the configuration file in the cluster node server, Y represents the md5 value of the new configuration file, and X i and Y i respectively identify the cluster node server and the new configuration file. The md5 value of the i bit.
  5. 如权利要求2或3所述的集群的升级维护方法,其中,所述根据所述版本差异值,生成所述版本检测报告,包括:The method for upgrading and maintaining a cluster according to claim 2 or 3, wherein said generating said version detection report according to said version difference value comprises:
    将所述版本差异值与预设的阈值对比;Comparing the version difference value with a preset threshold;
    若所述版本差异值小于预设的阈值时,则输出所述集群节点服务器与新版本集群节点服务器存在差异结果的结论;If the version difference value is less than the preset threshold value, output a conclusion that there is a difference result between the cluster node server and the new version cluster node server;
    若所述差异结果值不小于预设的阈值时,则输出所述集群节点服务器与新版本集群节点服务器不在差异结果的结论;If the difference result value is not less than the preset threshold value, output a conclusion that the cluster node server and the new version cluster node server are not in the difference result;
    根据是否存在差异结果的结论生成所述版本检测报告。The version test report is generated according to the conclusion of whether there is a difference result.
  6. 如权利要求5所述的集群的升级维护方法,其中,所述差异结果包括:集群节点服务器的当前版本落后新发布集群节点服务器版本;集群节点服务器的硬件和/或系统与新版本集群节点服务器的硬件和/或系统存在差异。The method for upgrading and maintaining a cluster according to claim 5, wherein the difference result includes: the current version of the cluster node server is behind the newly released version of the cluster node server; the hardware and/or system of the cluster node server and the new version of the cluster node server There are differences in hardware and/or systems.
  7. 如权利要求5所述的集群的升级维护方法,其中,所述预设的阈值为0.95。The method for upgrading and maintaining a cluster according to claim 5, wherein the preset threshold value is 0.95.
  8. 一种集群的升级维护装置,其中,所述装置包括:A cluster upgrade and maintenance device, wherein the device includes:
    上传模块,用于将需要发布集群节点服务器的新版本软件包和新配置文件上传到预设的发布机器中;The upload module is used to upload the new version of the software package and the new configuration file that need to be released on the cluster node server to the preset release machine;
    检测模块,用于获取需要进行版本发布的集群节点服务器,将所述集群节点服务器中的软件包和配置文件与所述新版本软件包和新配置文件进行对应的版本检测,生成版本检测报告;The detection module is used to obtain the cluster node server that needs to be version released, perform corresponding version detection on the software package and configuration file in the cluster node server and the new version software package and the new configuration file, and generate a version detection report;
    判断及执行模块,用于根据所述版本检测报告,判断所述集群节点服务器是否需要进行差异发布,若不需要进行差异发布,则通过所述发布机器对所述集群节点服务器执行版本发布,若需要进行差异发布,则根据所述新版本软件包和新配置文件对所述集群节点服务器中的版本软件包和配置文件进行更新后执行版本发布;The judgment and execution module is used to judge whether the cluster node server needs to perform differential publishing according to the version detection report, if differential publishing is not required, perform version publishing on the cluster node server through the publishing machine, if If a differential release is required, the version software package and configuration file in the cluster node server are updated according to the new version software package and the new configuration file, and then the version release is performed;
    重启模块,用于利用预设的重启程序重启执行版本发布后的所述集群节点服务器,并对版本发布后的所述集群节点服务器进行标记。The restart module is used to restart the cluster node server after the version is released by using a preset restart program, and mark the cluster node server after the version is released.
  9. 一种电子设备,其中,所述电子设备包括:An electronic device, wherein the electronic device includes:
    至少一个处理器;以及,At least one processor; and,
    与所述至少一个处理器通信连接的存储器;其中,A memory communicatively connected with the at least one processor; wherein,
    所述存储器存储有可被所述至少一个处理器执行的指令,所述指令被所述至少一个处理器执行,以使所述至少一个处理器能够执行如下步骤:The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the following steps:
    将需要发布的新版本软件包和新配置文件上传到预设的发布机器中;Upload the new version of the software package and the new configuration file that need to be released to the preset release machine;
    获取需要进行版本发布的集群节点服务器,将所述集群节点服务器中的软件包和配置文件与所述新版本软件包和新配置文件进行版本检测,生成版本检测报告;Obtain a cluster node server that needs to be version released, perform version detection on the software package and configuration file in the cluster node server and the new version software package and new configuration file, and generate a version detection report;
    根据所述版本检测报告,判断所述集群节点服务器是否需要进行差异发布;According to the version detection report, determine whether the cluster node server needs to be differentially released;
    若不需要进行差异发布,则通过所述发布机器对所述集群节点服务器执行版本发布;If there is no need to perform differential release, perform version release to the cluster node server through the release machine;
    若需要进行差异发布,则根据所述新版本软件包和新配置文件对所述集群节点服务器中的版本软件包和配置文件进行更新后执行版本发布;If a differential release is required, the version software package and configuration file in the cluster node server are updated according to the new version software package and the new configuration file, and then the version release is performed;
    利用预设的重启程序重启执行版本发布后的所述集群节点服务器,并对版本发布后的所述集群节点服务器进行标记。A preset restart program is used to restart the cluster node server after the version is released, and mark the cluster node server after the version is released.
  10. 如权利要求9所述的电子设备,其中,所述将所述集群节点服务器中的软件包和配置文件与所述新版本软件包和新配置文件进行版本检测,生成版本检测报告,包括:9. The electronic device according to claim 9, wherein said performing version detection of the software package and configuration file in the cluster node server with the new version software package and new configuration file, and generating a version detection report, comprises:
    计算所述集群节点服务器中的软件包与所述新版本软件包中的距离度量值;Calculating a distance metric value between the software package in the cluster node server and the new version software package;
    计算所述集群节点服务器中的配置文件与所述新版本配置文件的相似度;Calculating the similarity between the configuration file in the cluster node server and the new version configuration file;
    将所述距离度量值与所述相似度进行平均值计算,得到版本差异值;Calculating the average value of the distance metric value and the similarity to obtain a version difference value;
    根据所述版本差异值,生成所述版本检测报告。According to the version difference value, the version detection report is generated.
  11. 如权利要求10所述的电子设备,其中,所述计算所述集群节点服务器中的软件包与所述新版本软件包中的距离度量值,包括:11. The electronic device according to claim 10, wherein the calculating the distance metric value between the software package in the cluster node server and the new version software package comprises:
    利用下述公式计算所述集群节点服务器中的软件包与所述新版本软件包中的距离度量值:Use the following formula to calculate the distance metric value between the software package in the cluster node server and the new version software package:
    Figure PCTCN2020119110-appb-100003
    Figure PCTCN2020119110-appb-100003
    其中,ρ X,Y表示距离度量值,X表示集群节点服务器中软件包的md5值,Y表示新版本软件包的md5值,μ X表示集群节点服务器中软件包的md5均值,μ Y表示新版本软件包的md5均值,σ X表示集群节点服务器中软件包的md5值的标准差,σ Y表示新版本软件包的md5值的标准差,E表示集群节点服务器中软件包与新版本软件包的期望。 Among them, ρ X, Y represents the distance measurement value, X represents the md5 value of the software package in the cluster node server, Y represents the md5 value of the new version of the software package, μ X represents the average md5 value of the software package in the cluster node server, and μ Y represents the new The mean value of md5 of the version software package, σ X represents the standard deviation of the md5 value of the software package in the cluster node server, σ Y represents the standard deviation of the md5 value of the new version software package, and E represents the software package in the cluster node server and the new version software package Expectations.
  12. 如权利要求10或11所述的电子设备,其中,所述计算所述集群节点服务器中的配置文件与所述新版本配置文件的相似度,包括:The electronic device according to claim 10 or 11, wherein the calculating the similarity between the configuration file in the cluster node server and the new version configuration file comprises:
    利用下述公式计算所述集群节点服务器中的配置文件与所述新版本配置文件的相似度:Use the following formula to calculate the similarity between the configuration file in the cluster node server and the new version configuration file:
    Figure PCTCN2020119110-appb-100004
    Figure PCTCN2020119110-appb-100004
    其中,sim(X,Y)表示相似度,X所述集群节点服务器中配置文件md5值,Y表示新配置文件的md5值,X i、Y i分别标识集群节点服务器与所述新配置文件第i位的md5值。 Wherein, sim(X,Y) represents the similarity, X represents the md5 value of the configuration file in the cluster node server, Y represents the md5 value of the new configuration file, and X i and Y i respectively identify the cluster node server and the new configuration file. The md5 value of the i bit.
  13. 如权利要求10或11所述的电子设备,其中,所述根据所述版本差异值,生成所述版本检测报告,包括:The electronic device according to claim 10 or 11, wherein said generating said version detection report according to said version difference value comprises:
    将所述版本差异值与预设的阈值对比;Comparing the version difference value with a preset threshold;
    若所述版本差异值小于预设的阈值时,则输出所述集群节点服务器与新版本集群节点服务器存在差异结果的结论;If the version difference value is less than the preset threshold value, output a conclusion that there is a difference result between the cluster node server and the new version cluster node server;
    若所述差异结果值不小于预设的阈值时,则输出所述集群节点服务器与新版本集群节点服务器不在差异结果的结论;If the difference result value is not less than the preset threshold value, output a conclusion that the cluster node server and the new version cluster node server are not in the difference result;
    根据是否存在差异结果的结论生成所述版本检测报告。The version test report is generated according to the conclusion of whether there is a difference result.
  14. 如权利要求13所述的电子设备,其中,所述差异结果包括:集群节点服务器的当前版本落后新发布集群节点服务器版本;集群节点服务器的硬件和/或系统与新版本集群节点服务器的硬件和/或系统存在差异。The electronic device according to claim 13, wherein the difference result comprises: the current version of the cluster node server is behind the newly released version of the cluster node server; the hardware and/or system of the cluster node server and the hardware of the new version cluster node server / Or there is a difference in the system.
  15. 一种计算机可读存储介质,存储有计算机程序,其中,所述计算机程序被处理器执行时实现如下步骤:A computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the following steps:
    将需要发布的新版本软件包和新配置文件上传到预设的发布机器中;Upload the new version of the software package and the new configuration file that need to be released to the preset release machine;
    获取需要进行版本发布的集群节点服务器,将所述集群节点服务器中的软件包和配置文件与所述新版本软件包和新配置文件进行版本检测,生成版本检测报告;Obtain a cluster node server that needs to be version released, perform version detection on the software package and configuration file in the cluster node server and the new version software package and new configuration file, and generate a version detection report;
    根据所述版本检测报告,判断所述集群节点服务器是否需要进行差异发布;According to the version detection report, determine whether the cluster node server needs to be differentially released;
    若不需要进行差异发布,则通过所述发布机器对所述集群节点服务器执行版本发布;If there is no need to perform differential release, perform version release to the cluster node server through the release machine;
    若需要进行差异发布,则根据所述新版本软件包和新配置文件对所述集群节点服务器中的版本软件包和配置文件进行更新后执行版本发布;If a differential release is required, the version software package and configuration file in the cluster node server are updated according to the new version software package and the new configuration file, and then the version release is performed;
    利用预设的重启程序重启执行版本发布后的所述集群节点服务器,并对版本发布后的所述集群节点服务器进行标记。A preset restart program is used to restart the cluster node server after the version is released, and mark the cluster node server after the version is released.
  16. 如权利要求15所述的计算机可读存储介质,其中,所述将所述集群节点服务器中的软件包和配置文件与所述新版本软件包和新配置文件进行版本检测,生成版本检测报告,包括:15. The computer-readable storage medium according to claim 15, wherein said performing version detection on the software package and configuration file in the cluster node server and the new version software package and new configuration file to generate a version detection report, include:
    计算所述集群节点服务器中的软件包与所述新版本软件包中的距离度量值;Calculating a distance metric value between the software package in the cluster node server and the new version software package;
    计算所述集群节点服务器中的配置文件与所述新版本配置文件的相似度;Calculating the similarity between the configuration file in the cluster node server and the new version configuration file;
    将所述距离度量值与所述相似度进行平均值计算,得到版本差异值;Calculating the average value of the distance metric value and the similarity to obtain a version difference value;
    根据所述版本差异值,生成所述版本检测报告。According to the version difference value, the version detection report is generated.
  17. 如权利要求16所述的计算机可读存储介质,其中,所述计算所述集群节点服务器中的软件包与所述新版本软件包中的距离度量值,包括:15. The computer-readable storage medium of claim 16, wherein the calculating the distance metric value between the software package in the cluster node server and the new version software package comprises:
    利用下述公式计算所述集群节点服务器中的软件包与所述新版本软件包中的距离度量值:Use the following formula to calculate the distance metric value between the software package in the cluster node server and the new version software package:
    Figure PCTCN2020119110-appb-100005
    Figure PCTCN2020119110-appb-100005
    其中,ρ X,Y表示距离度量值,X表示集群节点服务器中软件包的md5值,Y表示新版本软件包的md5值,μ X表示集群节点服务器中软件包的md5均值,μ Y表示新版本软件包的md5均值,σ X表示集群节点服务器中软件包的md5值的标准差,σ Y表示新版本软件包的md5值的标准差,E表示集群节点服务器中软件包与新版本软件包的期望。 Among them, ρ X, Y represents the distance measurement value, X represents the md5 value of the software package in the cluster node server, Y represents the md5 value of the new version of the software package, μ X represents the average md5 value of the software package in the cluster node server, and μ Y represents the new The mean value of md5 of the version software package, σ X represents the standard deviation of the md5 value of the software package in the cluster node server, σ Y represents the standard deviation of the md5 value of the new version software package, and E represents the software package in the cluster node server and the new version software package Expectations.
  18. 如权利要求16或17所述的计算机可读存储介质,其中,所述计算所述集群节点服务器中的配置文件与所述新版本配置文件的相似度,包括:The computer-readable storage medium according to claim 16 or 17, wherein the calculating the similarity between the configuration file in the cluster node server and the new version configuration file comprises:
    利用下述公式计算所述集群节点服务器中的配置文件与所述新版本配置文件的相似度:Use the following formula to calculate the similarity between the configuration file in the cluster node server and the new version configuration file:
    Figure PCTCN2020119110-appb-100006
    Figure PCTCN2020119110-appb-100006
    其中,sim(X,Y)表示相似度,X所述集群节点服务器中配置文件md5值,Y表示新配置文件的md5值,X i、Y i分别标识集群节点服务器与所述新配置文件第i位的md5值。 Wherein, sim(X,Y) represents the similarity, X represents the md5 value of the configuration file in the cluster node server, Y represents the md5 value of the new configuration file, and X i and Y i respectively identify the cluster node server and the new configuration file. The md5 value of the i bit.
  19. 如权利要求16或17所述的计算机可读存储介质,其中,所述根据所述版本差异值,生成所述版本检测报告,包括:17. The computer-readable storage medium according to claim 16 or 17, wherein said generating said version detection report according to said version difference value comprises:
    将所述版本差异值与预设的阈值对比;Comparing the version difference value with a preset threshold;
    若所述版本差异值小于预设的阈值时,则输出所述集群节点服务器与新版本集群节点服务器存在差异结果的结论;If the version difference value is less than the preset threshold value, output a conclusion that there is a difference result between the cluster node server and the new version cluster node server;
    若所述差异结果值不小于预设的阈值时,则输出所述集群节点服务器与新版本集群节点服务器不在差异结果的结论;If the difference result value is not less than the preset threshold value, output a conclusion that the cluster node server and the new version cluster node server are not in the difference result;
    根据是否存在差异结果的结论生成所述版本检测报告。The version test report is generated according to the conclusion of whether there is a difference result.
  20. 如权利要求19所述的计算机可读存储介质,其中,所述差异结果包括:集群节点服务器的当前版本落后新发布集群节点服务器版本;集群节点服务器的硬件和/或系统与新版本集群节点服务器的硬件和/或系统存在差异。18. The computer-readable storage medium according to claim 19, wherein the difference result comprises: the current version of the cluster node server is behind the newly released version of the cluster node server; the hardware and/or system of the cluster node server and the new version of the cluster node server There are differences in hardware and/or systems.
PCT/CN2020/119110 2020-04-23 2020-09-29 Upgrade maintenance method and apparatus for cluster, and electronic device and storage medium WO2021212757A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010329737.3 2020-04-23
CN202010329737.3A CN111625252B (en) 2020-04-23 2020-04-23 Cluster upgrading maintenance method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
WO2021212757A1 true WO2021212757A1 (en) 2021-10-28

Family

ID=72271731

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/119110 WO2021212757A1 (en) 2020-04-23 2020-09-29 Upgrade maintenance method and apparatus for cluster, and electronic device and storage medium

Country Status (2)

Country Link
CN (1) CN111625252B (en)
WO (1) WO2021212757A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114466022A (en) * 2021-12-31 2022-05-10 苏州浪潮智能科技有限公司 Method, device and medium for acquiring server seed nodes in cluster
CN114785767A (en) * 2022-03-17 2022-07-22 中国电信集团工会上海市委员会 Method and system for connecting TDengine without installing client

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111625252B (en) * 2020-04-23 2024-05-28 平安科技(深圳)有限公司 Cluster upgrading maintenance method and device, electronic equipment and storage medium
CN112100070A (en) * 2020-09-15 2020-12-18 北京三快在线科技有限公司 Version defect detection method and device, server and storage medium
CN112631687B (en) * 2020-12-31 2024-08-02 中国农业银行股份有限公司 Service cluster configuration method, device and equipment
CN115001974A (en) * 2022-05-20 2022-09-02 中电信数智科技有限公司 Cluster system consistency detection method, device, system, equipment and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103605505A (en) * 2013-10-11 2014-02-26 索尔思光电(成都)有限公司 Sharing method of software configuration files
US20160110183A1 (en) * 2014-04-30 2016-04-21 Pivotal Software, Inc. Fast deployment across cloud platforms
CN108920167A (en) * 2018-08-03 2018-11-30 腾讯科技(深圳)有限公司 A kind of application program update method, apparatus and intelligent sound box
CN110955450A (en) * 2019-12-16 2020-04-03 北京智游网安科技有限公司 Application package file attribution statistical method, system and storage medium
CN111625252A (en) * 2020-04-23 2020-09-04 平安科技(深圳)有限公司 Cluster upgrading maintenance method and device, electronic equipment and storage medium

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102799445A (en) * 2012-05-03 2012-11-28 陈昊 Application upgrading method based on Android platform and system
CN102724308A (en) * 2012-06-13 2012-10-10 腾讯科技(深圳)有限公司 Software update method and software update system
CN105373410B (en) * 2015-12-22 2019-04-30 京信通信系统(中国)有限公司 Base station software difference upgrade method and its device
CN109298871B (en) * 2018-09-25 2021-09-28 东方电子股份有限公司 Software version monitoring and deploying method for intelligent scheduling system of power grid

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103605505A (en) * 2013-10-11 2014-02-26 索尔思光电(成都)有限公司 Sharing method of software configuration files
US20160110183A1 (en) * 2014-04-30 2016-04-21 Pivotal Software, Inc. Fast deployment across cloud platforms
CN108920167A (en) * 2018-08-03 2018-11-30 腾讯科技(深圳)有限公司 A kind of application program update method, apparatus and intelligent sound box
CN110955450A (en) * 2019-12-16 2020-04-03 北京智游网安科技有限公司 Application package file attribution statistical method, system and storage medium
CN111625252A (en) * 2020-04-23 2020-09-04 平安科技(深圳)有限公司 Cluster upgrading maintenance method and device, electronic equipment and storage medium

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114466022A (en) * 2021-12-31 2022-05-10 苏州浪潮智能科技有限公司 Method, device and medium for acquiring server seed nodes in cluster
CN114466022B (en) * 2021-12-31 2023-07-21 苏州浪潮智能科技有限公司 Method, device and medium for obtaining server seed nodes in cluster
CN114785767A (en) * 2022-03-17 2022-07-22 中国电信集团工会上海市委员会 Method and system for connecting TDengine without installing client

Also Published As

Publication number Publication date
CN111625252B (en) 2024-05-28
CN111625252A (en) 2020-09-04

Similar Documents

Publication Publication Date Title
WO2021212757A1 (en) Upgrade maintenance method and apparatus for cluster, and electronic device and storage medium
WO2021189899A1 (en) Link state tracking method and apparatus, and electronic device and computer storage medium
WO2021189906A1 (en) Target detection method and apparatus based on federated learning, and device and storage medium
WO2021189826A1 (en) Message generation method and apparatus, electronic device, and computer-readable storage medium
US11016836B2 (en) Graphical user interface for visualizing a plurality of issues with an infrastructure
WO2019090993A1 (en) Deployment method and apparatus for monitoring system, computer device and storage medium
WO2021184995A1 (en) Data processing method and data standard management system
WO2022142035A1 (en) System language internationalization maintenance method, apparatus, and computer-readable storage medium
CN112486957B (en) Database migration detection method, device, equipment and storage medium
CN112416957B (en) Data increment updating method and device based on data model layer and computer equipment
WO2022179121A1 (en) Automatic script generation method and apparatus, electronic device and storage medium
WO2022121172A1 (en) Text error correction method and apparatus, electronic device, and computer readable storage medium
CN112637341B (en) File uploading method and device, electronic equipment and storage medium
CN113051171A (en) Interface test method, device, equipment and storage medium
CN111694843A (en) Missing number detection method and device, electronic equipment and storage medium
CN113254446B (en) Data fusion method, device, electronic equipment and medium
CN113542387B (en) System release method and device, electronic equipment and storage medium
CN115002062A (en) Message processing method, device, equipment and readable storage medium
CN112541688B (en) Service data verification method and device, electronic equipment and computer storage medium
CN113434542A (en) Data relation identification method and device, electronic equipment and storage medium
CN112486532A (en) Method and device for managing configuration file, electronic equipment and storage medium
CN112685384A (en) Data migration method and device, electronic equipment and storage medium
CN115086047B (en) Interface authentication method and device, electronic equipment and storage medium
CN114727100B (en) Joint debugging method and device for monitoring equipment
CN115658968A (en) Service data number making method and device, electronic equipment and readable storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20932414

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20932414

Country of ref document: EP

Kind code of ref document: A1