Nothing Special   »   [go: up one dir, main page]

CN115860642B - A method and system for inbound and outbound management based on visual recognition - Google Patents

A method and system for inbound and outbound management based on visual recognition Download PDF

Info

Publication number
CN115860642B
CN115860642B CN202310051384.9A CN202310051384A CN115860642B CN 115860642 B CN115860642 B CN 115860642B CN 202310051384 A CN202310051384 A CN 202310051384A CN 115860642 B CN115860642 B CN 115860642B
Authority
CN
China
Prior art keywords
storage location
area
value
goods
warehouse
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310051384.9A
Other languages
Chinese (zh)
Other versions
CN115860642A (en
Inventor
石岩
李华伟
陈忠伟
王益亮
邓辉
沈锴
陆蕴凡
陈丁
李虎
赵越
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Xiangong Intelligent Technology Co ltd
Original Assignee
Shanghai Xiangong Intelligent Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Xiangong Intelligent Technology Co ltd filed Critical Shanghai Xiangong Intelligent Technology Co ltd
Priority to CN202310051384.9A priority Critical patent/CN115860642B/en
Publication of CN115860642A publication Critical patent/CN115860642A/en
Application granted granted Critical
Publication of CN115860642B publication Critical patent/CN115860642B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Abstract

本发明提供了一种基于视觉识别的出入库管理方法及系统，其中方法步骤包括：在图像帧中的仓库出入口位置标记出缓存区，标记出仓库内各库位并编号及获取位置数据，获取各缓存区所处图像帧中的位置数据；根据库位占用情况赋予各库位初始的第一状态值；记录进入入口缓存区的货物，并获取其所处图像帧中的位置数据

Figure ZY_1
；统计各库位的第一状态值，筛选出可接纳货物的库位集合E，以计算
Figure ZY_2
与E中每个库位的KIou值K1，并计算E中每个库位与出口缓存区位置数据的KIou值K2，后将K1与K2集合中的最小值的编号所对应的库位分配给货物，并更新该库位的第一状态值。籍此以基于计算机视觉技术实现仓储场景下的出入库的自动化管理。

Figure 202310051384

The present invention provides a visual recognition-based warehouse entry and exit management method and system, wherein the method steps include: marking the buffer area at the entrance and exit position of the warehouse in the image frame, marking and numbering each warehouse location in the warehouse and obtaining position data, obtaining The location data in the image frame where each buffer area is located; assign the initial first state value to each location according to the location occupancy; record the goods entering the entrance buffer area, and obtain the location data in the image frame where it is located

Figure ZY_1
;Statistics of the first status value of each storage location, filter out the collection of storage locations E that can accept goods, and calculate
Figure ZY_2
and the KIou value K1 of each storage location in E, and calculate the KIou value K2 of each storage location in E and the location data of the export buffer area, and then assign the storage location corresponding to the minimum number in the K1 and K2 set to Goods, and update the first status value of this location. In this way, the automated management of inbound and outbound warehouses in warehouse scenarios is realized based on computer vision technology.

Figure 202310051384

Description

一种基于视觉识别的出入库管理方法及系统A method and system for warehouse entry and exit management based on visual recognition

技术领域Technical Field

本发明涉及计算机视觉技术在自动化仓储领域的应用，尤其涉及一种基于视觉识别的出入库管理方法及系统。The present invention relates to the application of computer vision technology in the field of automated warehousing, and in particular to a warehouse entry and exit management method and system based on visual recognition.

背景技术Background Art

随着互联网经济的发展，货运快递量激增引起了仓储自动化改革，但复杂多样的仓储环境要实现自动化管理，目前还存在一定的技术门槛。特别是在仓库中的中转区，货物频繁的进出，库位状态改变频繁，非单一的进出口位置和极高的效率要求，给自动化管理带来了很大的挑战。在哪个库位、库位的哪个位置、进行出库还是入库则对自动化管理来说是很基础且重要的信息，同时也是对后续AGV小车的路径规划起到至关重要的指引作用。With the development of the Internet economy, the surge in freight and express delivery has led to the reform of warehouse automation. However, there are still certain technical barriers to realize automated management in complex and diverse warehouse environments. Especially in the transit area of the warehouse, the frequent entry and exit of goods, frequent changes in the status of storage locations, non-single import and export locations and extremely high efficiency requirements have brought great challenges to automated management. Which storage location, which position of the storage location, whether to ship out or enter the warehouse are very basic and important information for automated management, and also play a vital guiding role in the subsequent path planning of the AGV car.

目前现有技术通常采用光电传感器来反应库位内是否存在货物，但这种方法用在货运繁忙的仓储场景下，对光电传感器损耗较大，需要频繁更换，因此不利于仓库的高效运作管理，另一方面，设置光电传感器还需要大量的用电线路维护成本。At present, the existing technology usually adopts photoelectric sensors to reflect whether there are goods in the storage space. However, when this method is used in warehousing scenarios with busy freight traffic, the photoelectric sensors are subject to large wear and tear and need to be replaced frequently, which is not conducive to the efficient operation and management of the warehouse. On the other hand, the installation of photoelectric sensors also requires a lot of power line maintenance costs.

在另一种现有技术中，则提出采用新增二维码的方式进行管理，如通过扫描二维码来获取当前库位的信息，但这种做法首先需要人工操作，而且如果装在地上极易磨损，另外，如果库位需要放的东西发生改变，还要频繁的粘贴新的二维码，人力成本较高。In another prior art, it is proposed to use a new QR code for management, such as obtaining the information of the current storage location by scanning the QR code. However, this method first requires manual operation, and is very easy to wear out if installed on the ground. In addition, if the items to be placed in the storage location change, new QR codes must be frequently pasted, which has high labor costs.

可见上述此类现有技术，还无法高效准确的获取货物的出入库状态，以进行仓储自动化管理。为此发明人试图通过计算机视觉的目标检测技术和图像处理技术，来解决仓储自动化领域中出入库的管理难题。It can be seen that the above-mentioned prior art cannot efficiently and accurately obtain the in-and-out status of goods for warehouse automation management. Therefore, the inventor attempts to solve the in-and-out management problem in the field of warehouse automation by using computer vision target detection technology and image processing technology.

发明内容Summary of the invention

本发明的主要目的在于提供一种基于视觉识别的出入库管理方法及系统，以基于计算机视觉技术实现仓储场景下的出入库的自动化管理。The main purpose of the present invention is to provide a method and system for warehouse in and out management based on visual recognition, so as to realize the automated management of warehouse in and out in warehousing scenarios based on computer vision technology.

为了实现上述目的，根据本发明的第一个方面，提供了一种基于视觉识别的出入库管理方法，步骤包括：In order to achieve the above object, according to a first aspect of the present invention, a method for warehouse entry and exit management based on visual recognition is provided, the steps comprising:

步骤S100 在图像帧中的仓库出、入口位置标记出缓存区，并标记出仓库内各库位；为各个库位编号，获取各库位所处图像帧中的位置数据，获取各缓存区所处图像帧中的位置数据；Step S100: Mark the buffer area at the warehouse entrance and exit positions in the image frame, and mark each storage location in the warehouse; number each storage location, obtain the position data of each storage location in the image frame, and obtain the position data of each buffer area in the image frame;

步骤S200根据库位占用情况赋予各库位初始的第一状态值；Step S200 assigns an initial first state value to each storage location according to the storage location occupancy status;

步骤S300记录进入入口缓存区的货物，并获取其所处图像帧中的位置数据

Figure SMS_1
；Step S300 records the goods entering the entrance buffer area and obtains the position data of the goods in the image frame.
Figure SMS_1
;

步骤S400统计各库位的第一状态值，筛选出可接纳货物的库位集合E，以计算

Figure SMS_2
与E中每个库位的KIou值K1，并计算E中每个库位与出口缓存区位置数据的KIou值K2，后将K1与K2集合中的最小值的编号所对应的库位分配给货物，并更新该库位的第一状态值。Step S400 counts the first status value of each storage location, and selects the storage location set E that can accept the goods to calculate
Figure SMS_2
And the KIou value K1 of each storage location in E, and calculate the KIou value K2 of each storage location in E and the export buffer location data, then assign the storage location corresponding to the minimum value in the K1 and K2 sets to the goods, and update the first status value of the storage location.

在可能的优选实施方式中，所述KIou值K1的计算步骤包括：In a possible preferred implementation, the step of calculating the KIou value K1 includes:

步骤S410 将货物的位置数据所表示的区域设为A，将各个可接纳货物的库位的位置数据所表示的区域设为B；Step S410: The area represented by the location data of the goods is set as A, and the area represented by the location data of each storage location that can receive the goods is set as B;

步骤S420计算

Figure SMS_3
，Step S420: Calculate
Figure SMS_3
,

其中记A的左上角坐标为

Figure SMS_4
，右下角坐标为
Figure SMS_5
；记B左上角坐标为
Figure SMS_6
，右下角坐标为
Figure SMS_7
；The coordinates of the upper left corner of A are
Figure SMS_4
, the coordinates of the lower right corner are
Figure SMS_5
;Remember the coordinates of the upper left corner of B is
Figure SMS_6
, the coordinates of the lower right corner are
Figure SMS_7
;

参数

Figure SMS_8
; parameter
Figure SMS_8
;

参数

Figure SMS_10
= min(|min(
Figure SMS_13
,
Figure SMS_16
) - max(
Figure SMS_11
,
Figure SMS_12
)| , |min(
Figure SMS_15
,
Figure SMS_17
) - max(
Figure SMS_9
,
Figure SMS_14
) |) ; parameter
Figure SMS_10
= min(|min(
Figure SMS_13
,
Figure SMS_16
) - max(
Figure SMS_11
,
Figure SMS_12
)| , |min(
Figure SMS_15
,
Figure SMS_17
) - max(
Figure SMS_9
,
Figure SMS_14
) |) ;

参数

Figure SMS_18
。parameter
Figure SMS_18
.

在可能的优选实施方式中，所述KIou值K2的计算步骤包括：In a possible preferred implementation, the step of calculating the KIou value K2 includes:

步骤S430将各个可接纳货物的库位的位置数据所表示的区域设为B；将出口缓存区位置数据所表示的区域设为B'；Step S430: Set the area represented by the location data of each storage location that can receive goods as B; set the area represented by the location data of the export buffer area as B';

步骤S440计算

Figure SMS_19
- 10*α*
Figure SMS_20
， Step S440: Calculate
Figure SMS_19
- 10*α*
Figure SMS_20
,

其中记B的左上角坐标为(

Figure SMS_21
)，右下角坐标为(
Figure SMS_22
)；记B'左上角坐标为(
Figure SMS_23
)，右下角坐标为(
Figure SMS_24
)；The coordinates of the upper left corner of B are (
Figure SMS_21
), the coordinates of the lower right corner are (
Figure SMS_22
); let the coordinate of the upper left corner of B' be (
Figure SMS_23
), the coordinates of the lower right corner are (
Figure SMS_24
);

参数

Figure SMS_25
; parameter
Figure SMS_25
;

参数

Figure SMS_28
= min(|min(
Figure SMS_30
,
Figure SMS_33
) - max(
Figure SMS_27
,
Figure SMS_31
)| , |min(
Figure SMS_32
,
Figure SMS_34
)- max(
Figure SMS_26
,
Figure SMS_29
)|) ; parameter
Figure SMS_28
= min(|min(
Figure SMS_30
,
Figure SMS_33
) - max(
Figure SMS_27
,
Figure SMS_31
)| , |min(
Figure SMS_32
,
Figure SMS_34
)- max(
Figure SMS_26
,
Figure SMS_29
)|) ;

参数

Figure SMS_35
= max(0 , -(min(
Figure SMS_36
,
Figure SMS_37
) - max(
Figure SMS_38
,
Figure SMS_39
)))。 parameter
Figure SMS_35
= max(0 , -(min(
Figure SMS_36
,
Figure SMS_37
) - max(
Figure SMS_38
,
Figure SMS_39
))).

为了在本发明第一方面示例的基础上，进一步合理调度入库的位置，本发明的第二个方面，还提供了一种基于视觉识别的出入库管理方法，步骤包括：In order to further reasonably schedule the storage location based on the example of the first aspect of the present invention, the second aspect of the present invention further provides a storage management method based on visual recognition, the steps comprising:

步骤S100 在图像帧中的仓库出、入口位置标记出缓存区，并标记出仓库内各库位；为各个库位编号，获取各库位所处图像帧中的位置数据；在各库位周围设置附属区域，获取各附属区域所处图像帧中的位置数据；获取各缓存区所处图像帧中的位置数据；Step S100: Mark the buffer area at the warehouse entrance and exit positions in the image frame, and mark each storage location in the warehouse; number each storage location, and obtain the position data of each storage location in the image frame; set an auxiliary area around each storage location, and obtain the position data of each auxiliary area in the image frame; obtain the position data of each buffer area in the image frame;

步骤S200根据库位占用情况赋予各库位初始的第一状态值；对各库位的附属区域赋予第二状态值；Step S200 assigns an initial first status value to each storage location according to the storage location occupancy; assigns a second status value to the attached area of each storage location;

步骤S300记录进入入口缓存区的货物，并获取其所处图像帧中的位置数据

Figure SMS_40
；Step S300 records the goods entering the entrance buffer area and obtains the position data of the goods in the image frame.
Figure SMS_40
;

步骤S400统计各库位的第一状态值，筛选出可接纳货物的库位集合E，以计算

Figure SMS_41
与E中每个库位的KIou值K1，并计算E中每个库位与出口缓存区位置数据的KIou值K2，后将K1与K2集合中的最小值的编号所对应的库位分配给货物，并更新该库位的第一状态值；Step S400 counts the first status value of each storage location, and selects the storage location set E that can accept the goods to calculate
Figure SMS_41
and the KIou value K1 of each storage location in E, and calculate the KIou value K2 of each storage location in E and the location data of the export buffer area, then allocate the storage location corresponding to the number of the minimum value in the set of K1 and K2 to the goods, and update the first state value of the storage location;

步骤S500 根据已分配库位的附属区域的第二状态值，根据第一规则选择对应附属区域进入库位卸货。Step S500: According to the second status value of the subordinate area of the allocated storage location, the corresponding subordinate area is selected according to the first rule to enter the storage location for unloading.

在可能的优选实施方式中，所述KIou值K1的计算步骤包括：In a possible preferred implementation, the step of calculating the KIou value K1 includes:

步骤S410 将货物的位置数据所表示的区域设为A，将各个可接纳货物的库位的位置数据所表示的区域设为B；Step S410: The area represented by the location data of the goods is set as A, and the area represented by the location data of each storage location that can receive the goods is set as B;

步骤S420计算

Figure SMS_42
，Step S420: Calculate
Figure SMS_42
,

其中记A的左上角坐标为

Figure SMS_43
，右下角坐标为
Figure SMS_44
；记B左上角坐标为
Figure SMS_45
，右下角坐标为
Figure SMS_46
；The coordinates of the upper left corner of A are
Figure SMS_43
, the coordinates of the lower right corner are
Figure SMS_44
;Remember the coordinates of the upper left corner of B is
Figure SMS_45
, the coordinates of the lower right corner are
Figure SMS_46
;

参数

Figure SMS_47
;parameter
Figure SMS_47
;

参数

Figure SMS_48
= min(|min(
Figure SMS_51
,
Figure SMS_54
) - max(
Figure SMS_50
,
Figure SMS_52
)| , |min(
Figure SMS_55
,
Figure SMS_56
) - max(
Figure SMS_49
,
Figure SMS_53
) |) ; parameter
Figure SMS_48
= min(|min(
Figure SMS_51
,
Figure SMS_54
) - max(
Figure SMS_50
,
Figure SMS_52
)| , |min(
Figure SMS_55
,
Figure SMS_56
) - max(
Figure SMS_49
,
Figure SMS_53
) |) ;

参数

Figure SMS_57
。parameter
Figure SMS_57
.

在可能的优选实施方式中，所述KIou值K2的计算步骤包括：In a possible preferred implementation, the step of calculating the KIou value K2 includes:

步骤S430将各个可接纳货物的库位的位置数据所表示的区域设为B；将出口缓存区位置数据所表示的区域设为B'；Step S430: Set the area represented by the location data of each storage location that can receive goods as B; set the area represented by the location data of the export buffer area as B';

步骤S440计算

Figure SMS_58
- 10*α*
Figure SMS_59
， Step S440: Calculate
Figure SMS_58
- 10*α*
Figure SMS_59
,

其中记B的左上角坐标为(

Figure SMS_60
)，右下角坐标为(
Figure SMS_61
)；记B'左上角坐标为(
Figure SMS_62
)，右下角坐标为(
Figure SMS_63
)；The coordinates of the upper left corner of B are (
Figure SMS_60
), the coordinates of the lower right corner are (
Figure SMS_61
); let the coordinate of the upper left corner of B' be (
Figure SMS_62
), the coordinates of the lower right corner are (
Figure SMS_63
);

参数

Figure SMS_64
; parameter
Figure SMS_64
;

参数

Figure SMS_67
= min(|min(
Figure SMS_68
,
Figure SMS_72
) - max(
Figure SMS_66
,
Figure SMS_69
)| , |min(
Figure SMS_71
,
Figure SMS_73
)- max(
Figure SMS_65
,
Figure SMS_70
)|); parameter
Figure SMS_67
= min(|min(
Figure SMS_68
,
Figure SMS_72
) - max(
Figure SMS_66
,
Figure SMS_69
)| , |min(
Figure SMS_71
,
Figure SMS_73
)- max(
Figure SMS_65
,
Figure SMS_70
)|);

参数

Figure SMS_74
= max(0 , -(min(
Figure SMS_75
,
Figure SMS_76
) - max(
Figure SMS_77
,
Figure SMS_78
)))。 parameter
Figure SMS_74
= max(0 , -(min(
Figure SMS_75
,
Figure SMS_76
) - max(
Figure SMS_77
,
Figure SMS_78
))).

在可能的优选实施方式中，所述第一规则包括：In a possible preferred implementation, the first rule includes:

库位上下左右设有4个附属区域，所述第二状态值包括：开状态、允许开状态、关状态；There are 4 subsidiary areas on the upper, lower, left and right sides of the storage location, and the second state value includes: open state, allowed open state, and closed state;

S1若库位的4个附属区域中有一个为开状态，则其余3个区域为允许开状态；S1 If one of the four attached areas of the storage location is in the open state, the remaining three areas are in the allowed open state;

S2若某一附属区域相对的附属区域为开状态，则该附属区域本身必须为关状态；S2 If the subordinate area relative to a certain subordinate area is in the open state, then the subordinate area itself must be in the closed state;

S3只给允许开状态的附属区域分配入库任务，分配后，允许开状态变更为开状态，入库完成后变为允许开状态；S3 only assigns storage tasks to the subordinate areas in the allowed open state. After the assignment, the allowed open state changes to the open state, and after the storage is completed, it changes to the allowed open state.

S4若同一行/列中某个相同方位的附属区域的状态为开，则不允许同一行/列的相同附属区域为允许开状态，同时不允许相对的行/列中的相对的附属区域为允许开状态。S4: If the state of a subsidiary area in the same direction in the same row/column is open, the same subsidiary area in the same row/column is not allowed to be in the open state, and the relative subsidiary area in the relative row/column is not allowed to be in the open state.

在对应上述方法的情况下，本发明第三个方面，还提供了一种基于视觉识别的出入库管理系统，其包括：In the case of the above method, the third aspect of the present invention further provides a warehouse entry and exit management system based on visual recognition, which includes:

存储单元，用于存储包括上述中任一所述基于视觉识别的出入库管理方法步骤的程序，以供摄像机、视觉识别单元、处理单元、调度单元适时调取执行；A storage unit, used to store a program including any of the steps of the above-mentioned method for managing inbound and outbound storage based on visual recognition, so that the camera, the visual recognition unit, the processing unit, and the scheduling unit can call and execute the program in a timely manner;

摄像机，用于获取仓库区域图像帧，以向视觉识别单元发送；A camera, used to obtain image frames of the warehouse area to send to the visual recognition unit;

视觉识别单元，用于在图像帧中的仓库出、入口位置标记出缓存区，并标记出仓库内各库位；为各个库位编号，获取各库位所处图像帧中的位置数据，获取各缓存区所处图像帧中的位置数据；The visual recognition unit is used to mark the buffer area at the warehouse entrance and exit positions in the image frame, and mark each storage location in the warehouse; number each storage location, obtain the position data of each storage location in the image frame, and obtain the position data of each buffer area in the image frame;

视觉识别单元，进一步用于根据库位占用情况赋予各库位初始的第一状态值；记录进入入口缓存区的货物，并获取其所处图像帧中的位置数据

Figure SMS_79
；The visual recognition unit is further used to assign an initial first state value to each storage location according to the storage location occupancy; record the goods entering the entrance buffer area and obtain the position data of the goods in the image frame.
Figure SMS_79
;

处理单元，用于统计各库位的第一状态值，筛选出可接纳货物的库位集合E，以计算

Figure SMS_80
与E中每个库位的KIou值K1，并计算E中每个库位与出口缓存区位置数据的KIou值K2，后再计算出K1与K2集合中的最小值的编号所对应的库位，以向调度单元发送；The processing unit is used to count the first status value of each storage location, filter out the storage location set E that can accept the goods, and calculate
Figure SMS_80
and the KIou value K1 of each storage location in E, and calculate the KIou value K2 of each storage location in E and the location data of the export buffer area, and then calculate the storage location corresponding to the number of the minimum value in the set of K1 and K2 to send to the scheduling unit;

调度单元，用于将处理单元计算得出的库位分配给货物，并更新该库位的第一状态值。The scheduling unit is used to allocate the storage location calculated by the processing unit to the goods and update the first status value of the storage location.

在对应上述方法的情况下，本发明第四个方面，还提供了一种基于视觉识别的出入库管理系统，其包括：In the case of the above method, the fourth aspect of the present invention further provides a warehouse entry and exit management system based on visual recognition, which includes:

存储单元，用于存储包括如上所述基于视觉识别的出入库管理方法步骤的程序，以供摄像机、视觉识别单元、处理单元、调度单元适时调取执行；A storage unit for storing a program including the steps of the in-and-out warehouse management method based on visual recognition as described above, so that the camera, the visual recognition unit, the processing unit, and the scheduling unit can call and execute the program in a timely manner;

摄像机，用于获取仓库区域图像帧，以向视觉识别单元发送；A camera, used to obtain image frames of the warehouse area to send to the visual recognition unit;

视觉识别单元，用于在图像帧中的仓库出、入口位置标记出缓存区，并标记出仓库内各库位；为各个库位编号，获取各库位所处图像帧中的位置数据；在各库位周围设置附属区域，获取各附属区域所处图像帧中的位置数据；获取各缓存区所处图像帧中的位置数据；The visual recognition unit is used to mark the buffer area at the warehouse entrance and exit positions in the image frame, and mark each storage location in the warehouse; number each storage location, and obtain the position data of each storage location in the image frame; set an auxiliary area around each storage location, and obtain the position data of each auxiliary area in the image frame; obtain the position data of each buffer area in the image frame;

视觉识别单元，进一步用于根据库位占用情况赋予各库位初始的第一状态值；对各库位的附属区域赋予第二状态值；记录进入入口缓存区的货物，并获取其所处图像帧中的位置数据

Figure SMS_81
；The visual recognition unit is further used to assign an initial first state value to each storage location according to the storage location occupancy; assign a second state value to the attached area of each storage location; record the goods entering the entrance buffer area and obtain the position data of the goods in the image frame.
Figure SMS_81
;

处理单元，用于统计各库位的第一状态值，筛选出可接纳货物的库位集合E，以计算

Figure SMS_82
与E中每个库位的KIou值K1，并计算E中每个库位与出口缓存区位置数据的KIou值K2，后再计算出K1与K2集合中的最小值的编号所对应的库位，以向调度单元发送；The processing unit is used to count the first status value of each storage location, filter out the storage location set E that can accept the goods, and calculate
Figure SMS_82
and the KIou value K1 of each storage location in E, and calculate the KIou value K2 of each storage location in E and the location data of the export buffer area, and then calculate the storage location corresponding to the number of the minimum value in the set of K1 and K2 to send to the scheduling unit;

调度单元，用于将处理单元计算得出的库位分配给货物，并更新该库位的第一状态值，同时根据已分配库位的附属区域的第二状态值，根据第一规则指示由对应附属区域进入库位卸货。The scheduling unit is used to allocate the storage location calculated by the processing unit to the goods, and update the first status value of the storage location. At the same time, according to the second status value of the subsidiary area of the allocated storage location, it instructs the corresponding subsidiary area to enter the storage location for unloading according to the first rule.

在可能的优选实施方式中，所述视觉识别单元，将库位上下左右设有4个附属区域，并将第二状态值设置包括：开状态、允许开状态、关状态；In a possible preferred embodiment, the visual recognition unit sets four subsidiary areas on the upper, lower, left and right sides of the storage location, and sets the second state value to include: open state, allowed open state, and closed state;

所述调度单元依据的第一规则包括：S1若库位的4个附属区域中有一个为开状态，则其余3个区域为允许开状态；S2若某一附属区域相对的附属区域为开状态，则该附属区域本身必须为关状态；S3只给允许开状态的附属区域分配入库任务，分配后，允许开状态变更为开状态，入库完成后变为允许开状态；S4若同一行/列中某个相同方位的附属区域的状态为开，则不允许同一行/列的相同附属区域为允许开状态，同时不允许相对的行/列中的相对的附属区域为允许开状态。The first rule based on which the scheduling unit is based includes: S1 if one of the four subsidiary areas of the storage location is in the open state, the other three areas are in the allowed open state; S2 if the subsidiary area relative to a subsidiary area is in the open state, the subsidiary area itself must be in the closed state; S3 only assign warehousing tasks to subsidiary areas that are allowed to be in the open state, and after the assignment, the allowed open state is changed to the open state, and after the warehousing is completed, it becomes the allowed open state; S4 if the state of a subsidiary area in the same direction in the same row/column is open, the same subsidiary area in the same row/column is not allowed to be in the allowed open state, and the relative subsidiary areas in the relative rows/columns are not allowed to be in the allowed open state.

在对应上述方法的情况下，本发明第五个方面，还提供了一种基于视觉识别的出入库管理系统一种计算机设备，其包括存储器和处理器，所述存储器存储有计算机程序，其中所述处理器执行所述计算机程序时实现上述所述方法的步骤。In the case corresponding to the above method, the fifth aspect of the present invention also provides a computer device for a warehouse in and out management system based on visual recognition, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

在对应上述方法的情况下，本发明第六个方面，还提供了一种计算机可读存储介质，其上存储有计算机程序，其中所述计算机程序被处理器执行时实现上述所述的方法的步骤。In the case of corresponding to the above method, the sixth aspect of the present invention further provides a computer-readable storage medium, on which a computer program is stored, wherein the computer program implements the steps of the above method when executed by a processor.

.通过本发明提供的该基于视觉识别的出入库管理方法及系统，可基于视觉识别技术来准确高效的实时获取仓库中货物的出入库状态，便于管理货物入库，以及货区规划和装卸规划调配，可以更清晰的了解库位的动态使用情况，如库位已被占用，库位未被占用，库位计划被占用等状态，以计算机视觉技术来实现对仓储自动化管理提供更细粒度的信息。.Through the visual recognition-based warehousing management method and system provided by the present invention, the warehousing and in-and-out status of goods in the warehouse can be accurately and efficiently obtained in real time based on visual recognition technology, which is convenient for managing the warehousing of goods, as well as cargo area planning and loading and unloading planning and deployment. The dynamic use of the storage space can be more clearly understood, such as the status of the storage space being occupied, the storage space being unoccupied, the storage space being planned to be occupied, etc., and computer vision technology can be used to provide more fine-grained information for warehouse automation management.

另一方面，在一些实施方式中，本案还对库位设计了附属区域的状态管理方案，为货运机器人的货运路径规划提供了更多样的信息，及更多的调度合理性，提高了货物流通的效率及整体货运调度管理的可靠性。On the other hand, in some implementations, this case also designs a status management plan for the affiliated areas of the storage location, providing more diverse information and more scheduling rationality for the freight robot's freight path planning, thereby improving the efficiency of cargo circulation and the reliability of overall freight scheduling management.

附图说明BRIEF DESCRIPTION OF THE DRAWINGS

构成本申请的一部分的附图用来提供对本发明的进一步理解，本发明的示意性实施例及其说明用于解释本发明，并不构成对本发明的不当限定。在附图中：The drawings constituting a part of this application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

图1为本发明的基于视觉识别的出入库管理方法的第一、二实施例中仓库、仓库出入口缓存区及库位布局示意图；FIG1 is a schematic diagram of the layout of warehouses, warehouse entrance and exit buffer areas, and storage locations in the first and second embodiments of the inbound and outbound management method based on visual recognition of the present invention;

图2为本发明的基于视觉识别的出入库管理方法的第一、二实施例中KIou测量的概念示意图；FIG2 is a conceptual schematic diagram of KIou measurement in the first and second embodiments of the in-and-out warehouse management method based on visual recognition of the present invention;

图3为本发明的基于视觉识别的出入库管理方法的第一实施例的方法步骤示意图；FIG3 is a schematic diagram of method steps of a first embodiment of a method for warehouse entry and exit management based on visual recognition according to the present invention;

图4为本发明的基于视觉识别的出入库管理方法的第一、二实施例中KIou与其他Iou的效果对比示意图；FIG4 is a schematic diagram showing the effect comparison of KIou and other Iou in the first and second embodiments of the inbound and outbound management method based on visual recognition of the present invention;

图5为本发明的基于视觉识别的出入库管理方法的第一、二实施例中KIou与其他Iou的效果对比示意图，其中主要示意放大后的另外四种Iou的表现；FIG5 is a schematic diagram showing the comparison of the effects of KIou and other Iou in the first and second embodiments of the inbound and outbound management method based on visual recognition of the present invention, wherein the performance of the other four types of Iou after magnification is mainly shown;

图6为本发明的基于视觉识别的出入库管理系统的结构示意图；FIG6 is a schematic diagram of the structure of the in-and-out warehouse management system based on visual recognition of the present invention;

图7为本发明的基于视觉识别的出入库管理方法的第二实施例的方法步骤示意图；FIG7 is a schematic diagram of method steps of a second embodiment of the inbound and outbound management method based on visual recognition of the present invention;

图8为本发明的基于视觉识别的出入库管理方法的第二实施例中各库位附属区域示意图。FIG8 is a schematic diagram of the attached areas of each storage location in the second embodiment of the storage management method based on visual recognition of the present invention.

实施方式Implementation

为了使本领域的技术人员能够更好的理解本发明的技术方案，下面将结合实施例来对本发明的具体技术方案进行清楚、完整地描述，以助于本领域的技术人员进一步理解本发明。显然，本案所描述的实施例仅仅是本发明一部分的实施例，而不是全部的实施例。应当指出的是，对本领域的普通技术人员来说，在不脱离本发明构思及相互不冲突的前提下，本申请中的实施例及实施例中的特征可以相互组合。基于本发明中的实施例，在本领域普通技术人员没有做出创造性劳动前提下所获得的所有其他实施例，都应当属于本发明的揭露及保护范围。In order to enable those skilled in the art to better understand the technical solution of the present invention, the specific technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments, so as to help those skilled in the art further understand the present invention. Obviously, the embodiments described in this case are only embodiments of a part of the present invention, not all of the embodiments. It should be pointed out that for those of ordinary skill in the art, the embodiments in this application and the features in the embodiments can be combined with each other without departing from the concept of the present invention and without conflicting with each other. Based on the embodiments in the present invention, all other embodiments obtained without creative work by those of ordinary skill in the art should belong to the disclosure and protection scope of the present invention.

此外本发明的说明书和权利要求书及附图中的术语“第一”、“第二”、“S1”、“S2”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本发明的实施例能够以除了在这里描述的那些以外的顺序实施。同时本发明中的术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含。除非另有明确的规定和限定，术语“设置”、“布设”、“安装”、“相连”、“连接”应做广义理解，例如，可以是固定连接，也可以是可拆卸连接，或一体地连接；可以是机械连接，也可以是电连接；可以是直接相连，也可以通过中间媒介间接相连，可以是两个元件内部的连通。对于本领域的普通技术人员而言，可以根据具体情况，结合现有技术来理解上述术语在本案中的具体含义。In addition, the terms "first", "second", "S1", "S2", etc. in the specification, claims and drawings of the present invention are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those described herein. At the same time, the terms "including" and "having" and any of their variations in the present invention are intended to cover non-exclusive inclusions. Unless otherwise clearly specified and limited, the terms "set", "layout", "install", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be a connection between the two elements. For ordinary technicians in this field, the specific meanings of the above terms in this case can be understood according to the specific circumstances and in combination with the prior art.

请参阅图1至图5所示，本发明提供的该基于视觉识别的出入库管理方法，其步骤包括：Referring to FIG. 1 to FIG. 5 , the inbound and outbound warehouse management method based on visual recognition provided by the present invention comprises the following steps:

步骤S100：在图像帧中的仓库出、入口位置标记出缓存区，并标记出仓库内各库位；为各个库位编号，获取各库位所处图像帧中的位置数据，获取各缓存区所处图像帧中的位置数据；Step S100: Mark the buffer area at the warehouse entrance and exit positions in the image frame, and mark each storage location in the warehouse; number each storage location, obtain the position data of each storage location in the image frame, and obtain the position data of each buffer area in the image frame;

具体来说，假定该仓库是一个矩形区域，在仓库上方一定高度处安装镜头向下的摄像机，使摄像机能够完整且清晰的拍摄到整个仓库，由此可获得持续输出的视频流，该视频流本质上是一帧帧图像，例如帧率为25的摄像机的视频流本质上就是一秒内会输出25帧图像，因为摄像头是固定的，仓库的场地大小也是固定的，所以可以获得一张张固定大小和场地的图像帧，而后续的作业会以这一张张图像帧为识别及处理基础。Specifically, assuming that the warehouse is a rectangular area, a camera with a downward lens is installed at a certain height above the warehouse so that the camera can capture the entire warehouse completely and clearly. This will result in a continuously output video stream, which is essentially a frame of images. For example, a video stream from a camera with a frame rate of 25 essentially outputs 25 frames of images in one second. Because the camera is fixed and the size of the warehouse is fixed, image frames of fixed size and size can be obtained, and subsequent operations will be based on these image frames for recognition and processing.

如图1所示，示例中假设该仓库的的西方和南方是入口，东方和北方是出口，则本方案将在出口和入口处设置一个区域称为缓存区（Buffer），结合方位定义为NorthBuffer，South Buffer，WestBuffer和East Buffer。结合上文可知，入口对应着NorthBuffer和South Buffer，出口对应着West Buffer和East Buffer。As shown in Figure 1, in this example, it is assumed that the west and south of the warehouse are entrances, and the east and north are exits. In this solution, an area called a buffer is set at the entrance and exit, and the directions are defined as NorthBuffer, SouthBuffer, WestBuffer, and EastBuffer. As can be seen from the above, the entrance corresponds to NorthBuffer and SouthBuffer, and the exit corresponds to WestBuffer and EastBuffer.

之后，根据仓库大小给该区域规划m行n列矩形库位，每个库位给予一个编号

Figure SMS_83
，如1≤i≤m，1≤j≤n。After that, a rectangular storage location with m rows and n columns is planned for the area according to the size of the warehouse, and each storage location is given a number.
Figure SMS_83
, such as 1≤i≤m, 1≤j≤n.

而后，在图像上的仓库内区域中画上矩形库位，用前述

Figure SMS_89
表示，对应着真实的仓库中的库位，之后便可获取每个
Figure SMS_95
的在图像中的位置用
Figure SMS_102
（（
Figure SMS_85
，
Figure SMS_91
），（
Figure SMS_98
，
Figure SMS_104
））表示，由此方式，对应的四个Buffer区的位置可表示为：North Buffer用
Figure SMS_87
（（
Figure SMS_94
，
Figure SMS_101
），（
Figure SMS_108
，
Figure SMS_90
））表示，South Buffer用
Figure SMS_93
（（
Figure SMS_100
，
Figure SMS_107
），（
Figure SMS_86
，
Figure SMS_96
））表示，West Buffer用
Figure SMS_103
（（
Figure SMS_109
，
Figure SMS_84
），（
Figure SMS_97
，
Figure SMS_105
））表示，East Buffer用
Figure SMS_110
（（
Figure SMS_88
，
Figure SMS_92
），（
Figure SMS_99
，
Figure SMS_106
））表示。Then, draw a rectangular storage location in the warehouse area on the image, using the above
Figure SMS_89
Indicates that it corresponds to the actual warehouse location, and then each
Figure SMS_95
The position of in the image is
Figure SMS_102
（（
Figure SMS_85
,
Figure SMS_91
),
Figure SMS_98
,
Figure SMS_104
)) indicates that, in this way, the positions of the corresponding four buffer areas can be expressed as follows: North Buffer is represented by
Figure SMS_87
（（
Figure SMS_94
,
Figure SMS_101
),
Figure SMS_108
,
Figure SMS_90
)) indicates that the South Buffer is
Figure SMS_93
（（
Figure SMS_100
,
Figure SMS_107
),
Figure SMS_86
,
Figure SMS_96
)) indicates that West Buffer uses
Figure SMS_103
（（
Figure SMS_109
,
Figure SMS_84
),
Figure SMS_97
,
Figure SMS_105
)) indicates that East Buffer is
Figure SMS_110
（（
Figure SMS_88
,
Figure SMS_92
),
Figure SMS_99
,
Figure SMS_106
))express.

步骤S200：根据库位占用情况赋予各库位初始的第一状态值。Step S200: assigning an initial first status value to each storage location according to the storage location occupancy status.

例如，为了表示

Figure SMS_111
的占用状态，以供后续调度分配，需要设置一个状态值，即第一状态值，其表示当前库位是否被占用，如：初始状态值为0，表示未被占用、用1表示被占用、1/2表示目前未被占用但计划被占用（即将入库）。此外在本示例的基础上，本领域技术人员可以理解，该些状态可以根据实际货运需求，进行调整或改变，本案并不进行限制。For example, to express
Figure SMS_111
To determine the occupation status of a storage location for subsequent dispatch and allocation, a status value, namely the first status value, needs to be set, which indicates whether the current storage location is occupied, such as: the initial status value is 0, indicating that it is not occupied, 1 indicates that it is occupied, and 1/2 indicates that it is not currently occupied but is planned to be occupied (about to be put into storage). In addition, based on this example, those skilled in the art can understand that these statuses can be adjusted or changed according to actual freight needs, and this case does not limit them.

步骤S300：记录进入入口缓存区的货物，并获取其所处图像帧中的位置数据

Figure SMS_112
。Step S300: Record the goods entering the entrance buffer area and obtain their position data in the image frame
Figure SMS_112
.

具体来说，本案示例用Q = {

Figure SMS_113
}来表示所有仓库中的货物，用
Figure SMS_114
表示其中的某个货物；如一辆AGV小车载着货物从入口处进入仓库，首先会出现在缓存区，此时可给
Figure SMS_115
一个初始状态，如初始值为0，表示该货物未在库位中且未被分配库位，用1/2表示该货物已被分配库位但还未进入库位中，1表示该货物已在库位中。假定West Buffer与SouthBuffer为中转区的入口，East Buffer与North Buffer为仓库的出口。Specifically, this case example uses Q = {
Figure SMS_113
} to represent the goods in all warehouses, and use
Figure SMS_114
Indicates a certain cargo; if an AGV carrying cargo enters the warehouse from the entrance, it will first appear in the buffer area.
Figure SMS_115
An initial state, such as the initial value of 0, indicates that the goods are not in the storage location and have not been assigned a storage location, 1/2 indicates that the goods have been assigned a storage location but have not yet entered the storage location, and 1 indicates that the goods are already in the storage location. Assume that West Buffer and South Buffer are the entrances to the transit area, and East Buffer and North Buffer are the exits of the warehouse.

此时仓库上方的摄像头一直将图像传给视觉识别单元，视觉识别单元会检测到该货物在图像中的位置，用

Figure SMS_116
（（
Figure SMS_117
，
Figure SMS_118
），（
Figure SMS_119
，
Figure SMS_120
））表示。At this time, the camera above the warehouse keeps transmitting images to the visual recognition unit, which detects the location of the goods in the image.
Figure SMS_116
（（
Figure SMS_117
,
Figure SMS_118
),
Figure SMS_119
,
Figure SMS_120
))express.

步骤S400：统计各库位的第一状态值，筛选出可接纳货物的库位集合E，以计算

Figure SMS_122
与E中每个库位的KIou值，记为
Figure SMS_125
，并计算E中每个库位与出口缓存区位置数据的KIou值记为
Figure SMS_127
，
Figure SMS_123
，后将
Figure SMS_124
+
Figure SMS_126
集合与
Figure SMS_128
+
Figure SMS_121
集合的合集中最小的值的编号所对应的库位分配给货物，并更新该库位的第一状态值。Step S400: Count the first status values of each storage location, and select the storage location set E that can accept the goods to calculate
Figure SMS_122
and the KIou value of each storage location in E, denoted as
Figure SMS_125
, and calculate the KIou value of each storage location and the export buffer location data in E and record it as
Figure SMS_127
,
Figure SMS_123
, and later
Figure SMS_124
+
Figure SMS_126
Collection and
Figure SMS_128
+
Figure SMS_121
The storage location corresponding to the number with the smallest value in the collection is allocated to the goods, and the first status value of the storage location is updated.

具体来说，其中值得一提的是，如图2所示，假设最大的矩形区域为E，矩形区域A与矩形区域B相交，交集为矩形区域C，剩余区域记为D；Specifically, it is worth mentioning that, as shown in FIG2 , assuming that the largest rectangular area is E, rectangular area A intersects with rectangular area B, the intersection is rectangular area C, and the remaining area is recorded as D;

可知：E = A + B - C + D，

Figure SMS_129
， It can be seen that: E = A + B - C + D,
Figure SMS_129
,

而传统的Iou判断方法在本发明中存在以下缺陷，如果A、B不相交，则

Figure SMS_130
的值一直为0，在本发明中，假设识别到货物的框为A，库位为B，则在A靠近B的过程中，只要两者不相交，则
Figure SMS_131
一直为0，反应不出两者的位置关系；为了弥补这个缺点，本案提出了KIou判断方法。However, the traditional Iou judgment method has the following defects in the present invention: if A and B do not intersect, then
Figure SMS_130
The value of is always 0. In the present invention, assuming that the box of the identified goods is A and the storage location is B, when A approaches B, as long as the two do not intersect, then
Figure SMS_131
It is always 0, which cannot reflect the positional relationship between the two. In order to make up for this shortcoming, this case proposes the KIou judgment method.

统计每个

Figure SMS_132
的状态，将状态为未被占用的放入集合中，记为E，如E =
Figure SMS_133
，计算
Figure SMS_134
与E中每个库位的KIou的值，记为
Figure SMS_135
，其计算步骤示例包括：Statistics for each
Figure SMS_132
The state of unoccupied space is put into the set, recorded as E, such as E =
Figure SMS_133
,calculate
Figure SMS_134
The value of KIou for each storage location in E is expressed as
Figure SMS_135
, an example of its calculation steps include:

步骤S410 将货物的位置数据所表示的区域设为A，将各个可接纳货物的库位的位置数据所表示的区域设为B；Step S410: The area represented by the location data of the goods is set as A, and the area represented by the location data of each storage location that can receive the goods is set as B;

步骤S420计算

Figure SMS_136
，Step S420: Calculate
Figure SMS_136
,

其中记A的左上角坐标为

Figure SMS_137
，右下角坐标为
Figure SMS_138
；记B左上角坐标为
Figure SMS_139
，右下角坐标为
Figure SMS_140
；The coordinates of the upper left corner of A are
Figure SMS_137
, the coordinates of the lower right corner are
Figure SMS_138
;Remember the coordinates of the upper left corner of B is
Figure SMS_139
, the coordinates of the lower right corner are
Figure SMS_140
;

参数

Figure SMS_141
;parameter
Figure SMS_141
;

参数

Figure SMS_143
= min(|min(
Figure SMS_146
,
Figure SMS_149
) - max(
Figure SMS_144
,
Figure SMS_147
)| , |min(
Figure SMS_148
,
Figure SMS_150
) - max(
Figure SMS_142
,
Figure SMS_145
) |) ；parameter
Figure SMS_143
= min(|min(
Figure SMS_146
,
Figure SMS_149
) - max(
Figure SMS_144
,
Figure SMS_147
)| , |min(
Figure SMS_148
,
Figure SMS_150
) - max(
Figure SMS_142
,
Figure SMS_145
) |) ;

参数

Figure SMS_151
。parameter
Figure SMS_151
.

之后计算E中每个库位与两个出口Buffer区East Buffer与North Buffer的KIou的值，记为

Figure SMS_152
，
Figure SMS_153
，其计算步骤包括：Then calculate the KIou value of each storage location in E and the two exit buffer areas East Buffer and North Buffer, which is recorded as
Figure SMS_152
,
Figure SMS_153
, the calculation steps include:

步骤S430将各个可接纳货物的库位的位置数据所表示的区域设为B；将出口缓存区位置数据所表示的区域设为B'；Step S430: Set the area represented by the location data of each storage location that can receive goods as B; set the area represented by the location data of the export buffer area as B';

步骤S440如步骤S420那样计算

Figure SMS_154
- 10*α*
Figure SMS_155
，从而分别获取
Figure SMS_156
，
Figure SMS_157
。 Step S440 is calculated as in step S420.
Figure SMS_154
- 10*α*
Figure SMS_155
, thereby obtaining
Figure SMS_156
,
Figure SMS_157
.

最后，计算

Figure SMS_158
+
Figure SMS_159
集合与
Figure SMS_160
+
Figure SMS_161
集合的合集中最小的值，将最小值对应的ij既是分配给该货物的库位
Figure SMS_162
，同时将该
Figure SMS_163
的状态值更新为目前未被占用但计划被占用， 这样就完成了入库货物的库位分配。Finally, calculate
Figure SMS_158
+
Figure SMS_159
Collection and
Figure SMS_160
+
Figure SMS_161
The smallest value in the set, the ij corresponding to the smallest value is the storage location assigned to the goods
Figure SMS_162
, and at the same time
Figure SMS_163
The status value is updated to currently unoccupied but planned to be occupied, thus completing the storage location allocation for the incoming goods.

可见与现有其他Iou判断方法相比，只有KIou符合本方案及场景的要求，KIou具有其他Iou优点的同时，能够反映出例如所有时刻货物与库位的关系，其他Iou在货物与库位未有重合时是失效的。It can be seen that compared with other existing Iou judgment methods, only KIou meets the requirements of this solution and scenario. While KIou has the advantages of other Iou, it can reflect, for example, the relationship between goods and storage locations at all times. Other Iou are invalid when the goods and storage locations do not overlap.

如图4所示，其中除KIou外的其他四个Iou方案视觉上都重合到一块了，是因为坐标系的原因，要在一张图上展示出2000*2000的像素值下的各Iou的表现，但区域四个Iou的值都落在了（-1~0）之间，所以出现了上面所述的情况；如图5所示，为这四个Iou的放大后的表现，由此可见，本案提出的KIou方案更符合本方案的场景要求，能够较好的反应出两者的位置关系。As shown in Figure 4, except for KIou, the other four Iou schemes overlap visually. This is because of the coordinate system. In order to show the performance of each Iou under the pixel value of 2000*2000 on one picture, the values of the four Iou in the area all fall between (-1~0), so the situation described above occurs; as shown in Figure 5, this is the enlarged performance of these four Iou. It can be seen that the KIou scheme proposed in this case is more in line with the scene requirements of this scheme and can better reflect the positional relationship between the two.

对应上述方法，如图6所示，本发明提出的该基于视觉识别的出入库管理系统，其包括：Corresponding to the above method, as shown in FIG6 , the inbound and outbound warehouse management system based on visual recognition proposed by the present invention includes:

存储单元，用于存储包括如上所述基于视觉识别的出入库管理方法步骤的程序，以供摄像机、视觉识别单元、处理单元、调度单元适时调取执行。The storage unit is used to store a program including the steps of the warehouse entry and exit management method based on visual recognition as described above, so that the camera, visual recognition unit, processing unit, and scheduling unit can call and execute it in a timely manner.

摄像机，用于获取仓库区域图像帧，以向视觉识别单元发送。The camera is used to obtain image frames of the warehouse area to send to the visual recognition unit.

视觉识别单元，用于在图像帧中的仓库出、入口位置标记出缓存区，并标记出仓库内各库位；为各个库位编号，获取各库位所处图像帧中的位置数据，获取各缓存区所处图像帧中的位置数据。The visual recognition unit is used to mark the cache area at the warehouse entrance and exit positions in the image frame, and mark the various storage locations in the warehouse; number each storage location, obtain the position data of each storage location in the image frame, and obtain the position data of each cache area in the image frame.

视觉识别单元，进一步用于根据库位占用情况赋予各库位初始的第一状态值；记录进入入口缓存区的货物，并获取其所处图像帧中的位置数据

Figure SMS_164
。The visual recognition unit is further used to assign an initial first state value to each storage location according to the storage location occupancy; record the goods entering the entrance buffer area and obtain the position data of the goods in the image frame.
Figure SMS_164
.

处理单元，用于统计各库位的第一状态值，筛选出可接纳货物的库位集合E，以计算

Figure SMS_167
与E中每个库位的KIou值，记为
Figure SMS_168
，并计算E中每个库位与出口缓存区位置数据的KIou值记为
Figure SMS_171
，
Figure SMS_165
，后再计算出
Figure SMS_169
+
Figure SMS_170
集合与
Figure SMS_172
+
Figure SMS_166
集合的合集中最小的值的编号所对应的库位，以向调度单元发送。The processing unit is used to count the first status value of each storage location, filter out the storage location set E that can accept the goods, and calculate
Figure SMS_167
and the KIou value of each storage location in E, denoted as
Figure SMS_168
, and calculate the KIou value of each storage location and the export buffer location data in E and record it as
Figure SMS_171
,
Figure SMS_165
, and then calculate
Figure SMS_169
+
Figure SMS_170
Collection and
Figure SMS_172
+
Figure SMS_166
The storage location corresponding to the number of the smallest value in the collection is sent to the scheduling unit.

调度单元，用于将处理单元计算得出的库位分配给货物，并更新该库位的第一状态值。The scheduling unit is used to allocate the storage location calculated by the processing unit to the goods and update the first status value of the storage location.

如图1、2、4、5、7至图8所示，为了在本发明实施例一的基础上，进一步合理调度入库的位置，本发明实施例二还提供了一种基于视觉识别的出入库管理方法，其步骤包括：As shown in FIGS. 1, 2, 4, 5, 7 to 8, in order to further reasonably schedule the storage location based on the first embodiment of the present invention, the second embodiment of the present invention further provides a storage management method based on visual recognition, and the steps include:

步骤S100 在图像帧中的仓库出、入口位置标记出缓存区，并标记出仓库内各库位；为各个库位编号，获取各库位所处图像帧中的位置数据；在各库位周围设置附属区域，获取各附属区域所处图像帧中的位置数据；获取各缓存区所处图像帧中的位置数据。Step S100: Mark the cache area at the warehouse entrance and exit positions in the image frame, and mark each storage location in the warehouse; number each storage location and obtain the position data of each storage location in the image frame; set up an auxiliary area around each storage location and obtain the position data of each auxiliary area in the image frame; obtain the position data of each cache area in the image frame.

步骤S200根据库位占用情况赋予各库位初始的第一状态值；对各库位的附属区域赋予第二状态值；Step S200 assigns an initial first status value to each storage location according to the storage location occupancy; assigns a second status value to the attached area of each storage location;

步骤S300记录进入入口缓存区的货物，并获取其所处图像帧中的位置数据

Figure SMS_173
；Step S300 records the goods entering the entrance buffer area and obtains the position data of the goods in the image frame.
Figure SMS_173
;

步骤S400统计各库位的第一状态值，筛选出可接纳货物的库位集合E，以计算

Figure SMS_174
与E中每个库位的KIou值K1，并计算E中每个库位与出口缓存区位置数据的KIou值K2，后将K1与K2集合中的最小值的编号所对应的库位分配给货物，并更新该库位的第一状态值；Step S400 counts the first status value of each storage location, and selects the storage location set E that can accept the goods to calculate
Figure SMS_174
and the KIou value K1 of each storage location in E, and calculate the KIou value K2 of each storage location in E and the location data of the export buffer area, then allocate the storage location corresponding to the number of the minimum value in the set of K1 and K2 to the goods, and update the first state value of the storage location;

步骤S500 根据已分配库位的附属区域的第二状态值，根据第一规则选择对应附属区域进入库位卸货。Step S500: According to the second status value of the subordinate area of the allocated storage location, the corresponding subordinate area is selected according to the first rule to enter the storage location for unloading.

具体来说，在实施例一的方法的基础上，步骤S100中还在各库位周围设置了附属区域，并获取各附属区域所处图像帧中的位置数据。从而为后续调度判断入库的货物从库位的哪个方位入库比较合适提供了判断依据，进而提升入库的可靠性和调度合理性。Specifically, based on the method of the first embodiment, in step S100, an auxiliary area is set around each storage location, and the position data of each auxiliary area in the image frame is obtained. This provides a basis for subsequent scheduling to determine which direction of the storage location is more suitable for the goods to be stored, thereby improving the reliability of storage and the rationality of scheduling.

例如，设库位的宽度为w，高度为h。库位上下两边向外竖直平移1/5h距离，左右两边向外水平平移1/5w距离，其中需要注意的是，该距离可根据实际情况设置，此处仅为示例可能性，而未进行限制。For example, let the width of the storage location be w and the height be h. The upper and lower sides of the storage location are vertically translated outward by a distance of 1/5h, and the left and right sides are horizontally translated outward by a distance of 1/5w. It should be noted that the distance can be set according to the actual situation. This is only an example possibility and is not a limitation.

在得到额外的四个矩形区域后，根据方向，分别定义为NorthZone，SouthZone，WestZone和EastZone，分别简称为NZ，SZ，WZ和EZ，所以库位

Figure SMS_175
的这四个区域可以分别称作：
Figure SMS_176
。如图8所示。After getting the additional four rectangular areas, they are defined as NorthZone, SouthZone, WestZone and EastZone according to the direction, abbreviated as NZ, SZ, WZ and EZ respectively, so the storage location
Figure SMS_175
These four areas can be called:
Figure SMS_176
. As shown in Figure 8.

进一步的，

Figure SMS_177
的四个附属区域
Figure SMS_178
各有一个状态值即第二状态值，表示该区域是开状态、允许开状态或是关状态，初始值为0，表示关状态，若状态值为1/2，表示允许开状态，若状态值为1，表示开状态。Further,
Figure SMS_177
Four affiliated areas
Figure SMS_178
Each has a state value, namely the second state value, indicating whether the area is in an open state, an open state is allowed, or a closed state. The initial value is 0, indicating an closed state. If the state value is 1/2, it indicates an open state is allowed. If the state value is 1, it indicates an open state.

Figure SMS_183
的附属区域
Figure SMS_190
在图像中的位置用
Figure SMS_199
（（
Figure SMS_182
，
Figure SMS_192
），（
Figure SMS_195
，
Figure SMS_201
））表示，
Figure SMS_181
的附属区域
Figure SMS_188
在图像中的位置用
Figure SMS_197
（（
Figure SMS_202
，
Figure SMS_184
），（
Figure SMS_187
，
Figure SMS_196
））表示，
Figure SMS_206
的附属区域
Figure SMS_180
在图像中的位置用
Figure SMS_191
（（
Figure SMS_194
，
Figure SMS_204
），（
Figure SMS_179
，
Figure SMS_193
））表示，
Figure SMS_200
的附属区域
Figure SMS_205
在图像中的位置用
Figure SMS_185
（（
Figure SMS_189
，
Figure SMS_198
），（
Figure SMS_203
，
Figure SMS_186
））。
Figure SMS_183
The affiliated areas
Figure SMS_190
The position in the image is
Figure SMS_199
（（
Figure SMS_182
,
Figure SMS_192
),
Figure SMS_195
,
Figure SMS_201
))express,
Figure SMS_181
The affiliated areas
Figure SMS_188
The position in the image is
Figure SMS_197
（（
Figure SMS_202
,
Figure SMS_184
),
Figure SMS_187
,
Figure SMS_196
))express,
Figure SMS_206
The affiliated areas
Figure SMS_180
The position in the image is
Figure SMS_191
（（
Figure SMS_194
,
Figure SMS_204
),
Figure SMS_179
,
Figure SMS_193
))express,
Figure SMS_200
The affiliated areas
Figure SMS_205
The position in the image is
Figure SMS_185
（（
Figure SMS_189
,
Figure SMS_198
),
Figure SMS_203
,
Figure SMS_186
)).

当接计算货物

Figure SMS_207
从
Figure SMS_208
的哪个方位入库比较合适时，即判断从
Figure SMS_209
的四个附属区
Figure SMS_210
中哪个附属区进入
Figure SMS_211
，因为很多时候货物间的距离不能容纳两辆AGV小车并排或相向运动，再加上转向作业或复合机器人的机械臂安全工作范围等，所以要尽量避免一节路径上出现两个AGV小车。Calculate the goods when receiving
Figure SMS_207
from
Figure SMS_208
When it is more appropriate to store in which direction, it is determined from
Figure SMS_209
The four affiliated areas
Figure SMS_210
Which subsidiary area to enter
Figure SMS_211
Because in many cases the distance between the goods cannot accommodate two AGVs moving side by side or towards each other, and in addition to the safe working range of the robotic arm of the steering operation or the compound robot, it is necessary to try to avoid two AGVs on one section of the path.

为此本案设定了第一规则，如：For this purpose, the first rule is set in this case, such as:

1.若

Figure SMS_212
的4个附属区域中有一个为开状态，则其余3个区域为允许开状态。 1. If
Figure SMS_212
If one of the four subordinate areas is in the open state, the remaining three areas are in the allowed open state.

2.若某一附属区域相对的附属区域为开状态，则该区域本身必须为关状态。2. If the relative subordinate area of a subordinate area is in the open state, then the area itself must be in the closed state.

3.只能给允许开状态的附属区域分配入库任务，分配后，允许开状态变更为开状态，入库完成后变为允许开状态。3. Warehousing tasks can only be assigned to subsidiary areas that are in the allowed open state. After assignment, the allowed open state is changed to the open state, and after warehousing is completed, it becomes the allowed open state.

4.若同一行/列中某个相同方位的附属区域的状态为开，则不允许同一行/列的相同附属区域为允许开状态，不允许相对的行/列中的 相对的附属区域为允许开状态。4. If the state of a subordinate area in the same direction in the same row/column is open, the same subordinate area in the same row/column is not allowed to be in the open state, and the relative subordinate areas in the relative rows/columns are not allowed to be in the open state.

依照该规则程序，就可获得了一个完整的仓库中所有库位的状态，以及每个库位的四个附属区域的状态。按这种状态分配给缓存区的货物根据库位的指定附属区域进行入库即可，这样就完成了一个完整的循环，并且能够确保调度的合理性及可靠性。According to this rule procedure, the status of all storage locations in a complete warehouse and the status of the four subsidiary areas of each storage location can be obtained. According to this status, the goods allocated to the buffer area can be put into the warehouse according to the designated subsidiary areas of the storage location, thus completing a complete cycle and ensuring the rationality and reliability of scheduling.

对应上述方法，如图6所示，本发明提供的该基于视觉识别的出入库管理系统，包括：Corresponding to the above method, as shown in FIG6 , the inbound and outbound warehouse management system based on visual recognition provided by the present invention includes:

存储单元，用于存储包括如上所述基于视觉识别的出入库管理方法步骤的程序，以供摄像机、视觉识别单元、处理单元、调度单元适时调取执行。The storage unit is used to store a program including the steps of the warehouse entry and exit management method based on visual recognition as described above, so that the camera, visual recognition unit, processing unit, and scheduling unit can call and execute it in a timely manner.

摄像机，用于获取仓库区域图像帧，以向视觉识别单元发送。The camera is used to obtain image frames of the warehouse area to send to the visual recognition unit.

视觉识别单元，用于在图像帧中的仓库出、入口位置标记出缓存区，并标记出仓库内各库位；为各个库位编号，获取各库位所处图像帧中的位置数据；在各库位周围设置附属区域，获取各附属区域所处图像帧中的位置数据；获取各缓存区所处图像帧中的位置数据。The visual recognition unit is used to mark the cache area at the warehouse entrance and exit positions in the image frame, and mark the various storage locations in the warehouse; number each storage location and obtain the position data of each storage location in the image frame; set up ancillary areas around each storage location and obtain the position data of each ancillary area in the image frame; obtain the position data of each cache area in the image frame.

视觉识别单元，进一步用于根据库位占用情况赋予各库位初始的第一状态值；对各库位的附属区域赋予第二状态值；记录进入入口缓存区的货物，并获取其所处图像帧中的位置数据

Figure SMS_213
。The visual recognition unit is further used to assign an initial first state value to each storage location according to the storage location occupancy; assign a second state value to the attached area of each storage location; record the goods entering the entrance buffer area and obtain the position data of the goods in the image frame.
Figure SMS_213
.

处理单元，用于统计各库位的第一状态值，筛选出可接纳货物的库位集合E，以计算

Figure SMS_216
与E中每个库位的KIou值，记为
Figure SMS_217
，并计算E中每个库位与出口缓存区位置数据的KIou值记为
Figure SMS_219
，
Figure SMS_215
，后再计算出
Figure SMS_218
+
Figure SMS_220
集合与
Figure SMS_221
+
Figure SMS_214
集合的合集中最小的值的编号所对应的库位，以向调度单元发送。The processing unit is used to count the first status value of each storage location, filter out the storage location set E that can accept the goods, and calculate
Figure SMS_216
and the KIou value of each storage location in E, denoted as
Figure SMS_217
, and calculate the KIou value of each storage location and the export buffer location data in E and record it as
Figure SMS_219
,
Figure SMS_215
, and then calculate
Figure SMS_218
+
Figure SMS_220
Collection and
Figure SMS_221
+
Figure SMS_214
The storage location corresponding to the number of the smallest value in the collection is sent to the scheduling unit.

调度单元，用于将处理单元计算得出的库位分配给货物，并更新该库位的第一状态值，同时根据已分配库位的附属区域的第二状态值，根据第一规则指示由对应附属区域进入库位卸货。The scheduling unit is used to allocate the storage location calculated by the processing unit to the goods, and update the first status value of the storage location. At the same time, according to the second status value of the subsidiary area of the allocated storage location, it instructs the corresponding subsidiary area to enter the storage location for unloading according to the first rule.

对应上述实施例一和二的方法，本发明还提供了一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程序，其中所述处理器执行所述计算机程序时实现上述方法的步骤。Corresponding to the methods of the above-mentioned embodiments 1 and 2, the present invention further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned method when executing the computer program.

对应上述实施例一和二的方法，本发明还提供了一种计算机可读存储介质，其上存储有计算机程序，其中所述计算机程序被处理器执行时实现上述的方法的步骤。Corresponding to the methods of the above-mentioned embodiments 1 and 2, the present invention further provides a computer-readable storage medium on which a computer program is stored, wherein the computer program implements the steps of the above-mentioned method when executed by a processor.

综上所述，通过本发明提供的该基于视觉识别的出入库管理方法及系统，可基于视觉识别技术来准确高效的实时获取仓库中货物的出入库状态，便于管理货物入库，以及货区规划和装卸规划调配，可以更清晰的了解库位的动态使用情况，如库位已被占用，库位未被占用，库位计划被占用等状态，以计算机视觉技术来实现对仓储自动化管理提供更细粒度的信息。In summary, the in-and-out warehousing management method and system based on visual recognition provided by the present invention can accurately and efficiently obtain the in-and-out status of goods in the warehouse in real time based on visual recognition technology, which is convenient for managing the entry of goods into the warehouse, as well as cargo area planning and loading and unloading planning and allocation. It can more clearly understand the dynamic use of the storage location, such as the status of the storage location being occupied, the storage location being unoccupied, the storage location planned to be occupied, etc., and computer vision technology is used to provide more fine-grained information for warehouse automation management.

另一方面，本案还对库位设计了附属区域的状态管理方案，为货运机器人的货运路径规划提供了更多样的信息，及更多的调度合理性，提高了货物流通的效率及整体货运调度管理的可靠性。On the other hand, this case also designed a status management plan for the auxiliary areas of the storage location, providing more diverse information and more scheduling rationality for the freight robot's freight path planning, improving the efficiency of cargo circulation and the reliability of overall freight scheduling management.

以上公开的本发明优选实施例只是用于帮助阐述本发明。优选实施例并没有详尽叙述所有的细节，也不限制该发明仅为所述的具体实施方式。显然，根据本说明书的内容，可作很多的修改和变化。本说明书选取并具体描述这些实施例，是为了更好地解释本发明的原理和实际应用，从而使所属技术领域技术人员能很好地理解和利用本发明。本发明仅受权利要求书及其全部范围和等效物的限制，凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is only limited by the claims and their full scope and equivalents. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

本领域技术人员可以理解，除了以纯计算机可读程序代码方式实现本发明提供的系统、装置及其各个模块以外，完全可以通过将方法步骤进行逻辑编程来使得本发明提供的系统、装置及其各个模块以逻辑门、开关、专用集成电路、可编程逻辑控制器以及嵌入式微控制器等的形式来实现相同程序。所以，本发明提供的系统、装置及其各个模块可以被认为是一种硬件部件，而对其内包括的用于实现各种程序的模块也可以视为硬件部件内的结构；也可以将用于实现各种功能的模块视为既可以是实现方法的软件程序又可以是硬件部件内的结构。Those skilled in the art can understand that, in addition to implementing the system, device and its various modules provided by the present invention in a purely computer-readable program code, it is entirely possible to implement the same program in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers and embedded microcontrollers by logically programming the method steps. Therefore, the system, device and its various modules provided by the present invention can be considered as a hardware component, and the modules included therein for implementing various programs can also be regarded as structures within the hardware component; the modules for implementing various functions can also be regarded as both software programs for implementing the method and structures within the hardware component.

此外实现上述实施例方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成，该程序存储在一个存储介质中，包括若干指令用以使得单片机、芯片或处理器（processor）执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器（ROM，Read-Only Memory）、随机存取存储器（RAM，RandomAccess Memory）、磁碟或者光盘等各种可以存储程序代码的介质。In addition, all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including several instructions to enable a single-chip microcomputer, a chip or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), disk or optical disk and other media that can store program codes.

此外，本发明实施例的各种不同的实施方式之间也可以进行任意组合，只要其不违背本发明实施例的思想，其同样应当视为本发明实施例所公开的内容。In addition, various implementations of the embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the embodiments of the present invention, they should also be regarded as the contents disclosed in the embodiments of the present invention.

Claims (8)

1.一种基于视觉识别的出入库管理方法，其特征在于步骤包括：1. A method for warehouse entry and exit management based on visual recognition, characterized in that the steps include: 步骤S100 在图像帧中的仓库出、入口位置标记出缓存区，并标记出仓库内各库位；为各个库位编号，获取各库位所处图像帧中的位置数据，获取各缓存区所处图像帧中的位置数据；Step S100: Mark the buffer area at the warehouse entrance and exit positions in the image frame, and mark each storage location in the warehouse; number each storage location, obtain the position data of each storage location in the image frame, and obtain the position data of each buffer area in the image frame; 步骤S200根据库位占用情况赋予各库位初始的第一状态值；Step S200 assigns an initial first state value to each storage location according to the storage location occupancy status; 步骤S300记录进入入口缓存区的货物，并获取其所处图像帧中的位置数据 Cqh；Step S300 records the goods entering the entrance buffer area and obtains the position data C qh of the goods in the image frame; 步骤S400统计各库位的第一状态值，筛选出可接纳货物的库位集合E，以计算Cqh与E中每个库位的KIou值K1，并计算E中每个库位与出口缓存区位置数据的KIou值K2，后将K1与K2集合中的最小值的编号所对应的库位分配给货物，并更新该库位的第一状态值；Step S400 counts the first status value of each storage location, screens out the storage location set E that can accept the goods, calculates C qh and the KIou value K1 of each storage location in E, and calculates the KIou value K2 of each storage location in E and the location data of the export buffer area, and then allocates the storage location corresponding to the number of the minimum value in the set of K1 and K2 to the goods, and updates the first status value of the storage location; 其中，所述KIou值K1的计算步骤包括：The calculation step of the KIou value K1 includes: 步骤S410 将货物的位置数据所表示的区域设为A，将各个可接纳货物的库位的位置数据所表示的区域设为B；Step S410: The area represented by the location data of the goods is set as A, and the area represented by the location data of each storage location that can receive the goods is set as B; 步骤S420计算
Figure QLYQS_1
，Step S420: Calculate
Figure QLYQS_1
, 其中记A的左上角坐标为
Figure QLYQS_2
，右下角坐标为
Figure QLYQS_3
；记B左上角坐标为
Figure QLYQS_4
，右下角坐标为
Figure QLYQS_5
；The coordinates of the upper left corner of A are
Figure QLYQS_2
, the coordinates of the lower right corner are
Figure QLYQS_3
;Remember the coordinates of the upper left corner of B is
Figure QLYQS_4
, the coordinates of the lower right corner are
Figure QLYQS_5
; 参数
Figure QLYQS_6
;parameter
Figure QLYQS_6
; 参数
Figure QLYQS_9
= min(|min(
Figure QLYQS_12
,
Figure QLYQS_13
) - max(
Figure QLYQS_8
,
Figure QLYQS_11
)| , |min(
Figure QLYQS_14
,
Figure QLYQS_15
) - max(
Figure QLYQS_7
,
Figure QLYQS_10
) |)；parameter
Figure QLYQS_9
= min(|min(
Figure QLYQS_12
,
Figure QLYQS_13
) - max(
Figure QLYQS_8
,
Figure QLYQS_11
)| , |min(
Figure QLYQS_14
,
Figure QLYQS_15
) - max(
Figure QLYQS_7
,
Figure QLYQS_10
) |); 参数
Figure QLYQS_16
；parameter
Figure QLYQS_16
; 其中，所述KIou值K2的计算步骤包括：The calculation step of the KIou value K2 includes: 步骤S430将各个可接纳货物的库位的位置数据所表示的区域设为B；将出口缓存区位置数据所表示的区域设为B'；Step S430: Set the area represented by the location data of each storage location that can receive goods as B; set the area represented by the location data of the export buffer area as B'; 步骤S440计算
Figure QLYQS_17
- 10*α*
Figure QLYQS_18
，Step S440: Calculate
Figure QLYQS_17
- 10*α*
Figure QLYQS_18
, 其中记B的左上角坐标为(
Figure QLYQS_19
)，右下角坐标为(
Figure QLYQS_20
)；记B'左上角坐标为(
Figure QLYQS_21
)，右下角坐标为(
Figure QLYQS_22
)；The coordinates of the upper left corner of B are (
Figure QLYQS_19
), the coordinates of the lower right corner are (
Figure QLYQS_20
); let the coordinate of the upper left corner of B' be (
Figure QLYQS_21
), the coordinates of the lower right corner are (
Figure QLYQS_22
); 参数
Figure QLYQS_23
;parameter
Figure QLYQS_23
; 参数
Figure QLYQS_25
= min(|min(
Figure QLYQS_28
,
Figure QLYQS_30
) - max(
Figure QLYQS_26
,
Figure QLYQS_27
)| , |min(
Figure QLYQS_31
,
Figure QLYQS_32
)- max(
Figure QLYQS_24
,
Figure QLYQS_29
)|);parameter
Figure QLYQS_25
= min(|min(
Figure QLYQS_28
,
Figure QLYQS_30
) - max(
Figure QLYQS_26
,
Figure QLYQS_27
)| , |min(
Figure QLYQS_31
,
Figure QLYQS_32
)- max(
Figure QLYQS_24
,
Figure QLYQS_29
)|); 参数
Figure QLYQS_33
= max(0 , -(min(
Figure QLYQS_34
,
Figure QLYQS_35
) - max(
Figure QLYQS_36
,
Figure QLYQS_37
)))。parameter
Figure QLYQS_33
= max(0 , -(min(
Figure QLYQS_34
,
Figure QLYQS_35
) - max(
Figure QLYQS_36
,
Figure QLYQS_37
))). 2.一种基于视觉识别的出入库管理方法，其特征在于步骤包括：2. A method for warehouse entry and exit management based on visual recognition, characterized in that the steps include: 步骤S100 在图像帧中的仓库出、入口位置标记出缓存区，并标记出仓库内各库位；为各个库位编号，获取各库位所处图像帧中的位置数据；在各库位周围设置附属区域，获取各附属区域所处图像帧中的位置数据；获取各缓存区所处图像帧中的位置数据；Step S100: Mark the buffer area at the warehouse entrance and exit positions in the image frame, and mark each storage location in the warehouse; number each storage location, and obtain the position data of each storage location in the image frame; set an auxiliary area around each storage location, and obtain the position data of each auxiliary area in the image frame; obtain the position data of each buffer area in the image frame; 步骤S200根据库位占用情况赋予各库位初始的第一状态值；对各库位的附属区域赋予第二状态值；Step S200 assigns an initial first status value to each storage location according to the storage location occupancy; assigns a second status value to the attached area of each storage location; 步骤S300记录进入入口缓存区的货物，并获取其所处图像帧中的位置数据Cqh；Step S300 records the goods entering the entrance buffer area and obtains the position data C qh of the goods in the image frame; 步骤S400统计各库位的第一状态值，筛选出可接纳货物的库位集合E，以计算Cqh与E中每个库位的KIou值K1，并计算E中每个库位与出口缓存区位置数据的KIou值K2，后将K1与K2集合中的最小值的编号所对应的库位分配给货物，并更新该库位的第一状态值；Step S400 counts the first status value of each storage location, screens out the storage location set E that can accept the goods, calculates C qh and the KIou value K1 of each storage location in E, and calculates the KIou value K2 of each storage location in E and the location data of the export buffer area, and then allocates the storage location corresponding to the number of the minimum value in the set of K1 and K2 to the goods, and updates the first status value of the storage location; 步骤S500 根据已分配库位的附属区域的第二状态值，根据第一规则选择对应附属区域进入库位卸货 ；Step S500: According to the second status value of the subordinate area of the allocated storage location, the corresponding subordinate area is selected according to the first rule to enter the storage location for unloading; 其中，所述KIou值K1的计算步骤包括：The calculation step of the KIou value K1 includes: 步骤S410 将货物的位置数据所表示的区域设为A，将各个可接纳货物的库位的位置数据所表示的区域设为B；Step S410: The area represented by the location data of the goods is set as A, and the area represented by the location data of each storage location that can receive the goods is set as B; 步骤S420计算
Figure QLYQS_38
，Step S420: Calculate
Figure QLYQS_38
, 其中记A的左上角坐标为
Figure QLYQS_39
，右下角坐标为
Figure QLYQS_40
；记B左上角坐标为
Figure QLYQS_41
，右下角坐标为
Figure QLYQS_42
；The coordinates of the upper left corner of A are
Figure QLYQS_39
, the coordinates of the lower right corner are
Figure QLYQS_40
;Remember the coordinates of the upper left corner of B is
Figure QLYQS_41
, the coordinates of the lower right corner are
Figure QLYQS_42
; 参数
Figure QLYQS_43
;parameter
Figure QLYQS_43
; 参数
Figure QLYQS_45
= min(|min(
Figure QLYQS_49
,
Figure QLYQS_50
) - max(
Figure QLYQS_46
,
Figure QLYQS_48
)| , |min(
Figure QLYQS_51
,
Figure QLYQS_52
) - max(
Figure QLYQS_44
,
Figure QLYQS_47
) |) ;parameter
Figure QLYQS_45
= min(|min(
Figure QLYQS_49
,
Figure QLYQS_50
) - max(
Figure QLYQS_46
,
Figure QLYQS_48
)| , |min(
Figure QLYQS_51
,
Figure QLYQS_52
) - max(
Figure QLYQS_44
,
Figure QLYQS_47
) |) ; 参数
Figure QLYQS_53
；parameter
Figure QLYQS_53
; 其中，所述KIou值K2的计算步骤包括：The calculation step of the KIou value K2 includes: 步骤S430将各个可接纳货物的库位的位置数据所表示的区域设为B；将出口缓存区位置数据所表示的区域设为B'；Step S430: Set the area represented by the location data of each storage location that can receive goods as B; set the area represented by the location data of the export buffer area as B'; 步骤S440计算
Figure QLYQS_54
- 10*α*
Figure QLYQS_55
，Step S440: Calculate
Figure QLYQS_54
- 10*α*
Figure QLYQS_55
, 其中记B的左上角坐标为(
Figure QLYQS_56
)，右下角坐标为(
Figure QLYQS_57
)；记B'左上角坐标为(
Figure QLYQS_58
)，右下角坐标为(
Figure QLYQS_59
)；The coordinates of the upper left corner of B are (
Figure QLYQS_56
), the coordinates of the lower right corner are (
Figure QLYQS_57
); let the coordinate of the upper left corner of B' be (
Figure QLYQS_58
), the coordinates of the lower right corner are (
Figure QLYQS_59
); 参数
Figure QLYQS_60
;parameter
Figure QLYQS_60
; 参数
Figure QLYQS_62
= min(|min(
Figure QLYQS_65
,
Figure QLYQS_68
) - max(
Figure QLYQS_63
,
Figure QLYQS_64
)| , |min(
Figure QLYQS_67
,
Figure QLYQS_69
)- max(
Figure QLYQS_61
,
Figure QLYQS_66
)|) ;parameter
Figure QLYQS_62
= min(|min(
Figure QLYQS_65
,
Figure QLYQS_68
) - max(
Figure QLYQS_63
,
Figure QLYQS_64
)| , |min(
Figure QLYQS_67
,
Figure QLYQS_69
)- max(
Figure QLYQS_61
,
Figure QLYQS_66
)|) ; 参数
Figure QLYQS_70
= max(0 , -(min(
Figure QLYQS_71
,
Figure QLYQS_72
) - max(
Figure QLYQS_73
,
Figure QLYQS_74
)))。parameter
Figure QLYQS_70
= max(0 , -(min(
Figure QLYQS_71
,
Figure QLYQS_72
) - max(
Figure QLYQS_73
,
Figure QLYQS_74
))). 3.根据权利要求2所述的基于视觉识别的出入库管理方法，其特征在于，所述第一规则包括：3. The method for warehouse in and out management based on visual recognition according to claim 2, characterized in that the first rule comprises: 库位上下左右设有4个附属区域，所述第二状态值包括：开状态、允许开状态、关状态；There are 4 subsidiary areas on the upper, lower, left and right sides of the storage location, and the second state value includes: open state, allowed open state, and closed state; S1若库位的4个附属区域中有一个为开状态，则其余3个区域为允许开状态；S1 If one of the four attached areas of the storage location is in the open state, the remaining three areas are in the allowed open state; S2若某一附属区域相对的附属区域为开状态，则该附属区域本身必须为关状态；S2 If the subordinate area relative to a certain subordinate area is in the open state, then the subordinate area itself must be in the closed state; S3只给允许开状态的附属区域分配入库任务，分配后，允许开状态变更为开状态，入库完成后变为允许开状态；S3 only assigns storage tasks to the subordinate areas in the allowed open state. After the assignment, the allowed open state changes to the open state, and after the storage is completed, it changes to the allowed open state. S4若同一行/列中某个相同方位的附属区域的状态为开，则不允许同一行/列的相同附属区域为允许开状态，同时不允许相对的行/列中的相对的附属区域为允许开状态。S4: If the state of a subsidiary area in the same direction in the same row/column is open, the same subsidiary area in the same row/column is not allowed to be in the open state, and the relative subsidiary area in the relative row/column is not allowed to be in the open state. 4.一种基于视觉识别的出入库管理系统，其特征在于包括：4. A warehouse in and out management system based on visual recognition, characterized by comprising: 存储单元，用于存储包括如权利要求1所述基于视觉识别的出入库管理方法步骤的程序，以供摄像机、视觉识别单元、处理单元、调度单元适时调取执行；A storage unit for storing a program including the steps of the in-and-out warehouse management method based on visual recognition as claimed in claim 1, so that the camera, the visual recognition unit, the processing unit, and the scheduling unit can call and execute the program in a timely manner; 摄像机，用于获取仓库区域图像帧，以向视觉识别单元发送；A camera, used to obtain image frames of the warehouse area to send to the visual recognition unit; 视觉识别单元，用于在图像帧中的仓库出、入口位置标记出缓存区，并标记出仓库内各库位；为各个库位编号，获取各库位所处图像帧中的位置数据，获取各缓存区所处图像帧中的位置数据；The visual recognition unit is used to mark the buffer area at the warehouse entrance and exit positions in the image frame, and mark each storage location in the warehouse; number each storage location, obtain the position data of each storage location in the image frame, and obtain the position data of each buffer area in the image frame; 视觉识别单元，进一步用于根据库位占用情况赋予各库位初始的第一状态值；记录进入入口缓存区的货物，并获取其所处图像帧中的位置数据Cqh；The visual recognition unit is further used to assign an initial first state value to each storage location according to the storage location occupancy; record the goods entering the entrance buffer area, and obtain the position data C qh of the goods in the image frame; 处理单元，用于统计各库位的第一状态值，筛选出可接纳货物的库位集合E，以计算Cqh与E中每个库位的KIou值K1，并计算E中每个库位与出口缓存区位置数据的KIou值K2，后再计算出K1与K2集合中的最小值的编号所对应的库位，以向调度单元发送；The processing unit is used to count the first state value of each storage location, screen out the storage location set E that can accept the goods, calculate C qh and the KIou value K1 of each storage location in E, and calculate the KIou value K2 of each storage location in E and the location data of the export buffer area, and then calculate the storage location corresponding to the number of the minimum value in the set of K1 and K2, and send it to the scheduling unit; 调度单元，用于将处理单元计算得出的库位分配给货物，并更新该库位的第一状态值。The scheduling unit is used to allocate the storage location calculated by the processing unit to the goods and update the first status value of the storage location. 5.一种基于视觉识别的出入库管理系统，其特征在于包括：5. A warehouse in and out management system based on visual recognition, characterized by comprising: 存储单元，用于存储包括如权利要求2所述基于视觉识别的出入库管理方法步骤的程序，以供摄像机、视觉识别单元、处理单元、调度单元适时调取执行；A storage unit for storing a program including the steps of the in-and-out warehouse management method based on visual recognition as claimed in claim 2, so that the camera, the visual recognition unit, the processing unit, and the scheduling unit can call and execute the program in a timely manner; 摄像机，用于获取仓库区域图像帧，以向视觉识别单元发送；A camera, used to obtain image frames of the warehouse area to send to the visual recognition unit; 视觉识别单元，用于在图像帧中的仓库出、入口位置标记出缓存区，并标记出仓库内各库位；为各个库位编号，获取各库位所处图像帧中的位置数据；在各库位周围设置附属区域，获取各附属区域所处图像帧中的位置数据；获取各缓存区所处图像帧中的位置数据；The visual recognition unit is used to mark the buffer area at the warehouse entrance and exit positions in the image frame, and mark each storage location in the warehouse; number each storage location, and obtain the position data of each storage location in the image frame; set an auxiliary area around each storage location, and obtain the position data of each auxiliary area in the image frame; obtain the position data of each buffer area in the image frame; 视觉识别单元，进一步用于根据库位占用情况赋予各库位初始的第一状态值；对各库位的附属区域赋予第二状态值；记录进入入口缓存区的货物，并获取其所处图像帧中的位置数据Cqh；The visual recognition unit is further used to assign an initial first state value to each storage location according to the storage location occupancy; assign a second state value to the attached area of each storage location; record the goods entering the entrance buffer area, and obtain the position data C qh of the goods in the image frame; 处理单元，用于统计各库位的第一状态值，筛选出可接纳货物的库位集合E，以计算Cqh与E中每个库位的KIou值K1，并计算E中每个库位与出口缓存区位置数据的KIou值K2，后再计算出K1与K2集合中的最小值的编号所对应的库位，以向调度单元发送；The processing unit is used to count the first state value of each storage location, screen out the storage location set E that can accept the goods, calculate C qh and the KIou value K1 of each storage location in E, and calculate the KIou value K2 of each storage location in E and the location data of the export buffer area, and then calculate the storage location corresponding to the number of the minimum value in the set of K1 and K2, and send it to the scheduling unit; 调度单元，用于将处理单元计算得出的库位分配给货物，并更新该库位的第一状态值，同时根据已分配库位的附属区域的第二状态值，根据第一规则指示由对应附属区域进入库位卸货。The scheduling unit is used to allocate the storage location calculated by the processing unit to the goods, and update the first status value of the storage location. At the same time, according to the second status value of the subsidiary area of the allocated storage location, it instructs the corresponding subsidiary area to enter the storage location for unloading according to the first rule. 6.根据权利要求5所述的一种基于视觉识别的出入库管理系统，其特征在于，所述视觉识别单元，将库位上下左右设有4个附属区域，并将第二状态值设置包括：开状态、允许开状态、关状态；6. A visual recognition-based inbound and outbound storage management system according to claim 5, characterized in that the visual recognition unit sets four subsidiary areas on the upper, lower, left and right sides of the storage location, and sets the second state value to include: open state, allowed open state, and closed state; 所述调度单元依据的第一规则包括：S1若库位的4个附属区域中有一个为开状态，则其余3个区域为允许开状态；S2若某一附属区域相对的附属区域为开状态，则该附属区域本身必须为关状态；S3只给允许开状态的附属区域分配入库任务，分配后，允许开状态变更为开状态，入库完成后变为允许开状态；S4若同一行/列中某个相同方位的附属区域的状态为开，则不允许同一行/列的相同附属区域为允许开状态，同时不允许相对的行/列中的相对的附属区域为允许开状态。The first rule based on which the scheduling unit is based includes: S1 if one of the four subsidiary areas of the storage location is in the open state, the other three areas are in the allowed open state; S2 if the subsidiary area relative to a subsidiary area is in the open state, the subsidiary area itself must be in the closed state; S3 only assign warehousing tasks to subsidiary areas that are allowed to be in the open state, and after the assignment, the allowed open state is changed to the open state, and after the warehousing is completed, it becomes the allowed open state; S4 if the state of a subsidiary area in the same direction in the same row/column is open, the same subsidiary area in the same row/column is not allowed to be in the allowed open state, and the relative subsidiary areas in the relative rows/columns are not allowed to be in the allowed open state. 7.一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程序，其特征在于，所述处理器执行所述计算机程序时实现权利要求1至3中任一项所述方法的步骤。7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein the processor implements the steps of the method according to any one of claims 1 to 3 when executing the computer program. 8.一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现权利要求1至3中任一项所述方法的步骤。8. A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the method according to any one of claims 1 to 3 when executed by a processor.
CN202310051384.9A 2023-02-02 2023-02-02 A method and system for inbound and outbound management based on visual recognition Active CN115860642B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310051384.9A CN115860642B (en) 2023-02-02 2023-02-02 A method and system for inbound and outbound management based on visual recognition

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310051384.9A CN115860642B (en) 2023-02-02 2023-02-02 A method and system for inbound and outbound management based on visual recognition

Publications (2)

Publication Number Publication Date
CN115860642A CN115860642A (en) 2023-03-28
CN115860642B true CN115860642B (en) 2023-05-05

Family

ID=85657460

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310051384.9A Active CN115860642B (en) 2023-02-02 2023-02-02 A method and system for inbound and outbound management based on visual recognition

Country Status (1)

Country Link
CN (1) CN115860642B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116880477A (en) * 2023-07-07 2023-10-13 盐城工业职业技术学院 AGV intelligent guidance system and method applied in warehousing and logistics

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108876241A (en) * 2018-06-08 2018-11-23 四川木牛流马智能科技有限公司 A kind of storage space identification and management system of view-based access control model
CN114367457A (en) * 2022-01-15 2022-04-19 郭平芳 E-commerce storage automatic sorting device based on visual identification technology

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8965561B2 (en) * 2013-03-15 2015-02-24 Cybernet Systems Corporation Automated warehousing using robotic forklifts
US10071856B2 (en) * 2016-07-28 2018-09-11 X Development Llc Inventory management
JP6724244B2 (en) * 2017-04-12 2020-07-15 株式会社日立物流 Article management support device, article management support system, and article management support method
JP7475663B2 (en) * 2020-07-16 2024-04-30 東京ロボティクス株式会社 Mobile manipulator and control method and program thereof
CN112242940B (en) * 2020-07-31 2023-06-06 广州微林软件有限公司 Intelligent management system and management method for food in bins
CN112149555B (en) * 2020-08-26 2023-06-20 华南理工大学 Global vision-based multi-warehouse AGV tracking method
JP7036329B1 (en) * 2021-05-17 2022-03-15 株式会社アラヤ Worksite management system, worksite management method, and worksite management program
CN115601271B (en) * 2022-11-29 2023-03-24 上海仙工智能科技有限公司 A visual information anti-shake method, storage location state management method and system

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108876241A (en) * 2018-06-08 2018-11-23 四川木牛流马智能科技有限公司 A kind of storage space identification and management system of view-based access control model
CN114367457A (en) * 2022-01-15 2022-04-19 郭平芳 E-commerce storage automatic sorting device based on visual identification technology

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
基于3D视觉的青饲机拖车车斗自动识别与定位方法;苗中华;陈苏跃;何创新;金称雄;马世伟;徐双喜;;农业机械学报(第05期) *
智能仓储物流机器人产业技术应用研究;曽骥,韩巍;;现代信息科技(第01期) *

Also Published As

Publication number Publication date
CN115860642A (en) 2023-03-28

Similar Documents

Publication Publication Date Title
JP2022037223A (en) Method for controlling route of robot, server and memory medium
CN108958241B (en) Robot path control method, device, server and storage medium
CN111103887B (en) Multi-sensor-based multi-mobile-robot scheduling system design method
CN115860642B (en) A method and system for inbound and outbound management based on visual recognition
WO2018187943A1 (en) Method for creating grid map of intelligent robot
CN113219992A (en) Path planning method and cleaning robot
KR20200109275A (en) Vehicle track planning method, device, computer device and computer-readable storage medium
US20070145141A1 (en) Two-dimensional bar code, information processing device, information processing method, and program
CN111872935A (en) Robot control system and control method thereof
CN106910339A (en) Road information provides method, device and processing terminal
CN113485328A (en) Full-coverage path planning method and device, electronic equipment and storage medium
CN117664128A (en) Scheduling path generation method, generation device, robot, and storage medium
CN111847142A (en) Robot navigation method and device, computer readable storage medium and robot
Baek et al. Deep learning-based automated productivity monitoring for on-site module installation in off-site construction
CN115220447A (en) Multi-robot motion scheduling method and device
CN118753699A (en) Intelligent warehousing multi-layer robot system based on cloud-edge collaboration
CN115601271B (en) A visual information anti-shake method, storage location state management method and system
CN106873601A (en) Map parallel movement control method in grating map structure
CN112700025B (en) Coverage area distribution method and device, storage medium and electronic equipment
CN106339840A (en) Warehouse management system and method capable of realizing intelligent material storage and fetching
CN115057242B (en) Container management method based on combination of virtual reality and augmented reality
CN118838356A (en) AGV trolley self-adaptive navigation method and system
Basile et al. A Colored Timed Petri Net model for a cyber-physical view of automated warehouse systems
CN116310390B (en) Visual detection method and system for hollow target and warehouse management system
TWI824676B (en) Path planning system and path planning method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
PE01 Entry into force of the registration of the contract for pledge of patent right
PE01 Entry into force of the registration of the contract for pledge of patent right

Denomination of invention: A Visual Recognition Based Method and System for Inventory Management

Effective date of registration: 20230828

Granted publication date: 20230505

Pledgee: Bank of Communications Ltd. Shanghai New District Branch

Pledgor: Shanghai Xiangong Intelligent Technology Co.,Ltd.

Registration number: Y2023310000491

PC01 Cancellation of the registration of the contract for pledge of patent right
PC01 Cancellation of the registration of the contract for pledge of patent right

Granted publication date: 20230505

Pledgee: Bank of Communications Ltd. Shanghai New District Branch

Pledgor: Shanghai Xiangong Intelligent Technology Co.,Ltd.

Registration number: Y2023310000491