JP6671002B2 - refrigerator - Google Patents

Description

  The present invention relates to a refrigerator capable of photographing food in a storage and confirming a storage state on a display terminal or the like and a refrigerator system thereof.

  Generally, many refrigerators and refrigeration systems of this kind have been proposed. For example, the refrigerator and refrigeration system described in Patent Literature 1 are provided with a small camera in a storage, and the food storage state in the storage taken by the small camera is displayed on a display terminal such as a mobile phone via an Internet line. It is configured so that it can be confirmed.

  Patent Literature 2 discloses a refrigerator and a refrigeration system similar to Patent Literature 1. The refrigerator described in Patent Literature 1 captures a food state in a storage with a small camera when a refrigerator door is closed. It is configured to do so.

  Further, the refrigerator and the refrigeration system described in Patent Document 3 have, in addition to the configuration described in Patent Document 1, a display terminal further provided on the surface of the refrigerator, and the display terminal includes a fixed terminal and a refrigerator incorporated on the surface of the refrigerator. It consists of a display terminal such as a mobile phone that can be removed from the surface and carried, and both terminals are capable of displaying the food storage state in the storage taken by a small camera via the Internet line on its display unit, and when going out Of course, even when at home, the food storage status in the storage can be known without opening the refrigerator door.

JP 2002-236798 A JP 2007-46833 A JP-A-2002-81818

  As described above, many types of such refrigerators and refrigerator systems, that is, refrigerators and refrigerator systems capable of photographing food in a storage and confirming the storage status on a display terminal or the like have been proposed. Nothing has been put to practical use yet.

  Therefore, the present inventors have been studying the practical use of the refrigerator and the refrigerator system as described above, and have verified the practical use. As a result, they found that this kind of refrigerator and its refrigerator system had many problems that had to be solved in order to be put to practical use.

  One of the problems is that it is difficult to determine the food type in the storage by the small camera, and it is difficult to determine the food type.

  Patent Document 2 described above discloses that when photographing with a small camera, an image is taken with an interior lighting lamp in the storage, but food illuminated by an interior lighting lamp located on the inner wall of the storage is disclosed. When photographing with a small camera installed on the inner surface of the door, the image is dark and it is difficult to distinguish the food type as it is, or it is necessary to perform advanced correction processing to make it easier to distinguish the food type, which greatly increases the cost It is a major obstacle to commercialization.

The invention was made in view of the above problems, in which as the food in the reservoir can be satisfactorily captured food determination is providing easy to that refrigerators.

In order to solve the above conventional problems, a refrigerator of the present invention includes at least one storage, and a door that opens and closes the storage, and an imaging device for capturing an image of the inside of the storage is provided on the door. A first control means for turning on a lighting device provided in the storage based on that the door is opened, and a predetermined time after the door is closed. Second control means for causing the imaging device to execute imaging based on the lapse of time, and the illuminance of the illumination device when the imaging device performs imaging is the illumination when the door is open. rather lower than the illuminance of the illumination device, wherein until the illumination device by the first control means imaging is completed by the imaging device from lit, the lighting device, characterized in that it is maintaining a turned-on state And

Refrigerator of the present invention, it is possible to satisfactorily capturing the food reservoir, it is easy to determine the food savings built - in cabinet, it is possible to provide a refrigerator easy to use.

1 is a perspective view of a refrigerator according to an embodiment of the present invention. Schematic sectional view of the refrigerator A perspective view showing the inside when the storage door of the refrigerator is opened. (A) A perspective view in which a display terminal is integrally provided in the refrigerator, and (b) A perspective view in which the display terminal is detachably mounted in the refrigerator. The example of the mounting structure of the display terminal used for the same refrigerator is shown, (a) is sectional drawing at the time of attaching a display terminal integrally, (b) is sectional drawing at the time of attaching a display terminal detachably. The example of the visual recognition structure of the display terminal of the display terminal used for the refrigerator is shown, (a), (b) is a schematic cross-sectional view in the case where the outer panel of the door is made of glass, and (c) is configured of the outer panel of the door with a magic mirror Schematic sectional view Vertical sectional view showing an example of mounting the camera on the refrigerator The example of camera attachment of the refrigerator is shown, (a) is a cross section of the refrigerator door part in the state where the door was closed, (b) is a front view of the door inside side when the door is opened. (A) (b) (c) A longitudinal sectional view showing another example of a camera mounting example of the refrigerator. Longitudinal sectional view showing an example of camera installation in the case of a single door of the refrigerator The example of camera attachment of the refrigerator is shown, (a) is a cross section of the refrigerator door part in the state where the door was closed, (b) is a front view of the door inside side when the door is opened. Longitudinal sectional view showing another example of mounting a camera in the case of a single door of the refrigerator. Explanatory diagram showing a shooting state in the example of mounting the camera in FIG. Explanatory drawing showing an example of a camera attached to the refrigerator System configuration diagram of the refrigerator system in the embodiment Block diagram showing a refrigerator configuration of the refrigerator system Block diagram showing a server configuration of the refrigerator system 4 is a flowchart showing the photographing to data storing operation when the door of the refrigerator system is closed. Flow chart showing photographing to data storage operations during the refrigerator compartment cooling operation of the refrigerator system. 5 is a flowchart showing another example of the photographing to data storing operation of the refrigerator system. 5 is a flowchart showing another example of the photographing to data storing operation of the refrigerator system. 5 is a flowchart showing another example of the photographing to data storing operation of the refrigerator system. 5 is a flowchart showing another example of the photographing to data storing operation of the refrigerator system. Flowchart showing photographing to data storage operation when the door of the refrigerator system is opened 9 is a flowchart showing another example of operation of photographing to data storage when the door of the refrigerator system is opened. 15 is a flowchart showing another operation example of the photographing operation in FIGS. 4 is a flowchart showing a case in which the server device performs data storage and combining operations in the refrigerator system. Flow chart showing a case where data storage and synthesizing operation in the refrigerator system are performed on the refrigerator side Flow chart showing a case where data storage and synthesizing operation are performed on the display terminal side in the refrigerator system. Flow chart showing normal image data transmission and reception in the refrigerator system Flow chart showing transmission and reception of image data when image data is stored in a display terminal in the refrigerator system Flow chart showing a case where individual images and composite images are switched and transmitted and received in the refrigerator system. Flow chart showing image data transmission and reception at the time of a pre-transmission request in the same refrigerator system Front view of a display terminal for explaining switching of transmission and reception paths in the refrigerator system Diagram for explaining image synthesis processing in the refrigerator system

  Hereinafter, Embodiment 1 of the present invention will be described with reference to the drawings. Note that the present invention is not limited to the following embodiments.

[Embodiment 1]
<1. Configuration>
1-1. Configuration of refrigerator 1-1-1. Configuration of French Door Type Refrigerator First, a configuration example in the case of a French door type refrigerator will be described with reference to FIGS.

  1 to 3, a refrigerator main body 1 includes an outer box 2 made of metal (for example, an iron plate) opening forward, an inner box 3 made of hard resin (for example, ABS), and these outer box 2 and inner box 3. It is made of a heat insulating material 4 such as hard foamed urethane filled with foam. The refrigerator main body 1 has a plurality of storage compartments formed therein. This storage is located from the upper part of the refrigerator main body 1 to the refrigerator compartment 5, the switching room 6 located below the refrigerator compartment 5, the ice making room 7 juxtaposed to the switching room 6, and the lower part of the switching room 6 and the ice making room 7. There is a freezer compartment 8 and a vegetable compartment 9 located below the freezer compartment 8. The front surface of the refrigerator compartment 5 is freely opened and closed by a double door 10, and the front surfaces of the switching compartment 6, the ice making compartment 7, the freezing compartment 8 and the vegetable compartment 9 are drawer doors 11, 12 and 13. , 14 (hereinafter referred to as a pull-out door).

  A cooling chamber 16 is provided on the rear surface of the refrigerator main body 1, a cooler 17 for generating cool air, a blower fan 18 for supplying cool air to each chamber, and a damper (air volume adjusting means) for adjusting the air volume to the refrigerator compartment. 22 are provided. In addition, a compressor 19 is provided at the back of the top surface of the refrigerator body 1, and a condenser (not shown), a heat radiation pipe 20 for heat radiation, a capillary tube 21, and the above-described cooler 17 are sequentially arranged. The refrigerant is sealed in a refrigeration cycle connected in a ring shape, and a cooling operation is performed.

  Here, each of the doors 10 to 14 is filled with hard urethane as in the case of the refrigerator main body 1 so as to have a heat insulating property. In this refrigerator, of the doors 10 to 14, the door of the storage 5 serving as a refrigerator compartment is provided. A display terminal 15 is provided on one of the front surfaces of the display 10. The display terminal 15 is composed of a display unit such as a liquid crystal panel and a control unit for controlling the display unit as shown in FIG. 4A, and is integrally attached to the door 10, or as shown in FIG. It is formed of a smartphone or a tablet PC, is detachably attached to the door 10, and may be equipped with any method.

  5 (a) and 5 (b) show an example of the mounting configuration of the display terminal 15 on the door 10, FIG. 5 (a) shows a case where the display terminal 15 is integrally mounted, and FIG. This shows a case where the camera is detachably attached.

  5 (a) and 5 (b), the door 10 mainly includes an outer panel 10a facing the front side of the refrigerator, an inner panel 10b facing the inside of the refrigerator, an outer panel 10a and the inner panel. 10b and the heat insulating material 4 filled in between them. The outer panel 10a is made of a metal member such as a color steel plate or a transparent member such as a glass plate, and the inner plate 10b is a synthetic resin member formed by vacuum molding. A raised wall portion 10b-1 that protrudes toward the inside of the box in a square shape when viewed from the inside of the box is provided around the inner plate 10b. As described above, for example, hard urethane foam is used for the heat insulating material 4.

  In the door 10 of FIG. 5A in which the display terminal 15 is integrated, a resin storage frame 23 that covers the heat insulating material 4 is provided on the front side of the door 10 so as to be continuous with the jacket panel 10a. (If the outer panel 10a is a glass plate, it is provided integrally with the glass plate.) The display terminal 15 is fixed to the storage frame 23 via the terminal holder 24. Further, on the front side of the display terminal 15, a touch panel 25 is provided. A gap 26 is provided between the display terminal 15 and the touch panel 25 so that noise from the display terminal 15 does not cause the touch panel 25 to malfunction. The display terminal 15 and the touch panel 25 have a frame 27 covering the outer peripheral edge thereof attached and fixed to the surface of the door 10. In addition, since the heat insulating material 4 does not directly touch the display terminal 15 by providing the storage frame 23, the display terminal 15 is exposed to high temperature or deformed when the heat insulating material 4 is foamed. And other problems can be prevented. On the other hand, in the door 10 of FIG. 5B in which the display terminal 15 is detachable, a recess 28 is formed by providing a storage frame 23 on the front surface of the door 10, and the display terminal 15 is detachably attached to the recess 28. It is configured to do so.

  6 (a), 6 (b) and 6 (c) show an example of a see-through configuration of the display of the door 10 and the display terminal 15, and FIGS. 6 (a) and 6 (b) show a case where the outer panel 10a of the door 10 is a glass plate. In the case of 10a-1, FIG. 3 (c) shows a case where the outer panel 10a of the door 10 is constituted by a magic mirror 10a-2.

  In FIG. 6A, between the display terminal 15 embedded in the door 10 and the glass plate 10 a-1, there is provided a smoke sheet 29 that is transparent only in a portion facing the display terminal 15. Even when turned off and looks dark, the glass sheet 10a-1 other than the display terminal 15 also looks dark due to the smoke sheet 29, so that the boundary with the display terminal 15 is not conspicuous, and the appearance can be favorably maintained. In FIG. 6B, a transparent liquid crystal display is used as the display of the display terminal 15 and a black smoke sheet 29a is entirely laid along the rear surface of the transparent liquid crystal display. Can be improved. Further, in FIG. 6C, the glass plate 10a-1 in FIGS. 6A and 6B is a magic mirror 10a-2, and the smoke sheet 29 is not required. be able to. Each shows an example in which the display terminal 15 is integrally attached to the door 10. Note that, instead of the smoke sheet 29, painting may be performed on the back side of the glass plate 10a-1 so as to appear dark.

  FIGS. 7, 8 (a) and 8 (b) show an example of installation of a camera and an interior lamp as main parts of the present invention, and FIG. 4 is a longitudinal sectional view of a refrigerator with a door closed. 8A is a cross-sectional view showing an inner surface of the door of the refrigerator in a state where the door is closed, and FIG. 8B is a front view of the inner surface of the door when the door is opened.

  In FIGS. 7, 8A and 8B, shelves 30-1 to 30-4 are provided in the storage 5 of the refrigerator in a plurality of upper and lower stages. The shelves 30-1 to 30-4 are formed of a transparent material such as a transparent synthetic resin material or glass except for the surrounding frame.

  Reference numeral 31 denotes a storage container provided so as to be able to be drawn out below the lowermost shelf plate 30-4. Although the lowermost shelf plate 30-4 is fixed because it constitutes the upper surface of the storage container 31, the shelf plates 30-1 to 30-3 are provided with side wall projections provided on both side walls of the storage 5, respectively. It is mounted on 32-1 to 32-4 and is detachable. In particular, among the shelves 30-1 to 30-3 excluding the lowermost shelf 30-4, a plurality of corresponding side wall protrusions 32-2 are provided at different heights on the central shelf 30-2. The height of the shelf can be adjusted.

  Reference numerals 33-1 to 33-3 denote door pockets provided on the inner surface of the door 10 for opening and closing the opening of the storage 5, and are provided in a plurality of upper and lower stages. The door pockets 33-1 to 33-3 are also formed of a transparent material.

  Reference numerals 34-1 and 34-2 denote interior lighting lamps, which are provided on both side walls on the front opening and on the upper wall surface on the front opening of the storage 5 in front of the front ends of the shelves 30-1 to 30-4. In addition, in order to save energy, the interior lighting lamps 34-1 and 34-2 are provided with a substrate on which a plurality of LEDs are arranged, for example, extending in the vertical direction, and covered with a light-transmitting cover.

  Reference numerals 35-1 and 35-2 denote cameras provided on the front side of the storage 5 in the same manner as the in-compartment illumination lamps 34-1 and 34-2, and include, for example, a CCD camera or a CMOS camera. The type is not particularly limited, but is preferably strong against dew condensation and the like.

  In this embodiment, two cameras are provided, and one of the cameras 35-1 is attached to the upper door pocket 33-1 in the vicinity of the lower surface substantially at the center in the front-rear direction, and another camera 35-2 is provided. Is attached near the upper part of the lowermost door pocket 33-3. Here, the front-rear direction of the door pocket 33 is a direction perpendicular to a surface formed by the inner plate 10b of the door 10, and the same direction as the front-rear direction of the storage 5 when the door 10 is closed. .

  The upper camera 35-1 is provided substantially in front of the uppermost shelf 30-1, and the lower camera 35-2 is provided in front of the third shelf 30-3 in the figure. That is, each camera is not a shelf board 30-2 mounted on the plurality of side wall projections 32-2 provided on the assumption of height adjustment, but a shelf which is detachable but does not assume height adjustment. It is provided substantially in front of the plate 30-1 and the shelf 30-3.

  Further, both the cameras 35-1 and 35-2 are located at positions forward of the front ends of the respective shelves 30-1 to 30-4, as shown in FIG. 7, and are shown in FIG. 8B. As described above, the door 10 is provided so as to be located near the open end side end Y opposite to the pivot support portion X, that is, substantially at the center of the refrigerator 51 in the left-right direction. Almost the entire area of the warehouse can be photographed. In the case of this French door, the cameras 35-1 and 35-2 are provided on the side of the door 10 having a large area, and are set so as to be located substantially at the center of the refrigerator 51. In this example, the cameras 35-1 and 35-2 are provided on the side of the door 10 where the center pillar 36 (see FIG. 3) is not provided, and are set so as to be located substantially in the center of the refrigerator 51. . In the case of a French door, as is well known, the center pillar 36 is provided on one of the side surfaces of the two doors 10, and cool air in the storage 5 leaks from a gap generated between the two doors 10. This is a member provided to prevent the above.

  The cameras 35-1 and 35-2 may be provided with a camera 35-3 for exclusive use in the storage container as shown in FIG. 9A, or as shown in FIG. -2 can be freely moved up and down, or as shown in FIG. 3 (c), the lower camera 35-2 can be freely rotated up and down, and can be set arbitrarily according to the type of refrigerator to be installed. Good.

  The interior lighting lamps 34-1 and 34-2 are provided at a position farther from the storage than the cameras 35-1 and 35-2, as is apparent from FIG. The two lights are provided so as not to directly enter the photographing sections of the cameras 35-1 and 35-2.

  More specifically, as shown in FIG. 8A, the photographing units of the cameras 35-1 and 35-2 provide the irradiation amounts (maximum irradiation amounts) of the interior illumination lamps 34-1 and 34-2 in the irradiation direction. Is provided outside the range where the irradiation amount is と or more, in this example, the irradiation angle is 60 degrees.

  Although not shown, a light-shielding portion is provided on the camera side of the interior illuminating lights 34-1 and 34-2, or a light source such as an LED constituting the interior illuminating lights 34-1 and 34-2 is provided. A configuration in which the light from the interior illuminating lights 34-1 and 34-2 may be prevented from directly entering the photographing units of the cameras 35-1 and 35-2 by being disposed obliquely rearward and obliquely. The selection may be made appropriately in accordance with the installation conditions of the camera and the like. This is the same in the case of a single-door refrigerator described below.

1-1-2. Configuration of Single Door Type Refrigerator FIGS. 10 to 13 show an example of mounting a camera and an interior lamp when the door 10 is a single door. In the refrigerator of the single type door, the same parts as those of the refrigerator of the French door type described above are denoted by the same reference numerals, and description thereof will be omitted, and only different parts will be described.

  First, an example will be described with reference to FIGS. 10 and 11A and 11B.

  10 and 11 (a) and 11 (b), in the case of the single-door refrigerator, the upper camera 35-1 has a left-right divided door pocket 33-33, which is a general arrangement example of a single door. 4, 33-5, and 33-6 are provided on the raised wall portion 10b-2, and the lower camera 35-2 is provided on the inner surface of the door between the door pockets 33-7 and 33-8. . Like the French door type, the cameras 35-1 and 35-2 are provided at positions substantially in front of the shelves 30-1 and 30-3 which are detachable but do not require height adjustment. And as shown in FIG. 10, it is the front position from the front end of each shelf board 30-1 to 30-4, and near the center of the door 10, as can be understood from FIGS. The refrigerator 51 is provided so as to be located substantially at the center in the left-right direction, and the two cameras 35-1 and 35-2 can photograph almost the entire area in the refrigerator.

  Further, the interior lighting lamps 34-1 and 34-2 are provided in the same manner as the French door type. That is, the in-compartment illuminating lights 34-1 and 34-2 are provided on the front opening side walls and the front opening upper wall surface of the storage in front of the front ends of the respective shelves 30-1 to 30-4. There is a position where the light does not directly enter the photographing units of the cameras 35-1 and 35-2, that is, the photographing units of the cameras 35-1 and 35-2 are illuminated by the interior lighting lamps 34-1 and 34-2. It is provided outside the range where the irradiation amount is 以上 or more of the irradiation amount (maximum irradiation amount) in the direction, in this example, the irradiation angle is 60 degrees.

  FIGS. 12A, 12B and 13 show other examples of the single door type.

  In the case of this example, the cameras 35-1 and 35-2 are provided near the door open end Y on the opposite side of the shaft support X of the door 10, as shown in FIGS. The height positions of the cameras 35-1 and 35-2 are the same as those in the case of the single door. When the cameras 35-1 and 35-2 are provided near the door open end Y, an image is taken when the door 10 is opened at a predetermined angle, for example, 30 degrees. Accordingly, the photographing can take a wide range up to the vicinity of the opening of the storage 5 as shown by the broken line in FIG. Is substantially the same as the state in which the image is viewed when checking the image, and when the image is visually recognized on the display terminal 15, the image can be viewed in a natural form without a sense of discomfort.

1-1-3. FIG. 14 shows an example of a camera used for photographing in a refrigerator. It is most preferable that the cameras are provided at vertically separated positions as described in each of the above examples when photographing in the refrigerator. However, even if the upper and lower cameras 35-1 and 35-2 are installed close to each other as shown in FIG. As a result of verification, it was confirmed that the entire area of the inside could be photographed.

  Therefore, in such a case, it is conceivable that the cameras 35-1 and 35-2 are integrated into the camera module 41, and the modularization allows easy installation and cost reduction. In addition, modularization can be achieved by using a wider-angle lens such as a fisheye lens.

1-2. Next, the entire refrigerator system using the refrigerator as described above will be described with reference to FIGS.

  FIG. 15 shows an overall configuration of a refrigerator system 50 using the above-described refrigerator. As shown in FIG. 15, the refrigerator system 50 is a system for confirming the food situation in the refrigerator 51 by the display terminal 15 or the like.

  The refrigerator system 50 includes a refrigerator 51 and a display terminal 15, and further includes a wireless adapter (communication device) 53, a gateway device (relay device) 54, a router device 55, the Internet 56, and a server device 57.

  The refrigerator 51 is arranged in the house A of the user, for example, in a kitchen room in the house A.

  The wireless adapter (communication device) 53 is configured to be electrically connected to a later-described control unit of the refrigerator 51 and communicate with a router device 55 via a gateway device (relay device) 54. Wireless adapter 53 receives a signal transmitted from router device 55 and outputs the received signal to a control unit of refrigerator 51. Based on this signal, refrigerator 51 is configured to execute a corresponding operation.

  The wireless adapter 53 also obtains identification information (for example, a serial number and a model number) of the refrigerator 51 and image data in a refrigerator, which will be described later, from a storage unit of the refrigerator 51, and obtains a router via a gateway device (relay device) 54. It is configured to transmit to the device 55. The wireless adapter 53 may be provided integrally with the refrigerator 51 or may be configured to be detachable.

  The wireless adapter 53 includes a “connect” button 53a. The “connect” button 53 a is used to newly connect to the gateway device (relay device) 54. When the “connect” button 53 a is operated by the user, the wireless adapter 53 is configured to acquire the identification information (for example, a manufacturing number or a model number) of the refrigerator 51 from the storage unit of the refrigerator 51.

  As described above, the display terminal 15 is a general-purpose portable terminal such as a smartphone and a tablet PC (personal computer). In the case of the present embodiment, the display terminal 15 is detachable from the door 10 of the refrigerator 51 described above. It has a function (means) for connecting to the Internet 56 and a function (means) for communicating with the gateway device 54 described later.

  The display terminal 15 (for example, a smartphone) is configured to connect to the Internet 56 via a telephone network (for example, a 3G network). Additionally or alternatively, for example, the display terminal 15 is configured to connect to the Internet 56 via a public wireless LAN by a Wi-Fi communication function.

  By connecting to the Internet 56, for example, the display terminal 15 acquires a program for acquiring image data of foods in the refrigerator and data for energy-saving operation (for example, data such as the number of times of opening and closing of doors). Can be obtained from the homepage of the manufacturer of the refrigerator 51 via the Internet 56. By installing the acquired program on the display terminal 15 and activating the installed program, the display terminal 15 can acquire image data and the like of food in the refrigerator. That is, by operating the operation screen of the display 15a, the display terminal 15 can generate and transmit an operation signal for acquiring image data of the refrigerator 51. Details will be described later.

  The display terminal 15 is also connected to the router device 55 by general-purpose communication that does not go through the Internet 56, such as Wi-Fi communication, Bluetooth (registered trademark) communication, infrared communication, and the like, and the gateway device 54 is connected via the router device 55. It is configured to be connected for communication. A device (for example, a Wi-Fi antenna) for that purpose is incorporated in the display terminal 15.

  The gateway device (relay device) 54 is a device that relays the communication between the wireless adapter 53 and the display terminal 15. For example, the gateway device (relay device) 54 is connected to the wireless adapter 53 and a special low power wireless special small frequency band (924.0 to 928.0 MHz). ) Is configured to perform communication using the signal. The frequency band of communication between the gateway device 54 and the wireless adapter 53 is preferably a low frequency band that reaches a long distance, and is installed in the user's house A where the refrigerator 51 is installed together with the router device 55.

  The gateway device 54 is also configured to communicate with the display terminal 15 via the router device 55. That is, the gateway device 54 is configured such that communication (first communication) can be performed only via the router device 55 without passing through the display terminal 15 and the Internet 56.

  The gateway device 54 is further configured to be connectable to the Internet 56 via a router device 55. Thus, the gateway device 54 can communicate (second communication) with the display terminal 15 via the Internet 56 and the router device 55, and can also communicate with the server device 57.

  Therefore, the gateway device 54 communicates with the display terminal 15 via communication not via the Internet 56 (ie, first communication via only the router device 55) and communication via the Internet 56 (ie, via the Internet 56 and the router device 55). (Second communication). The reason for this configuration will be described later.

  The server device 57 is prepared, for example, by a manufacturer of the refrigerator 51. The server device 57 manages, for the display terminal 15, the refrigerator 51, and the wireless adapter 53, information necessary for access and authentication. For example, the server device 57 determines whether or not the identification information of the display terminal 15 included in the communication request signal matches the identification information of the display terminal 15 stored (registered) in association with the wireless adapter 53. Perform authentication (determination). Then, when they match, the server device 57 authenticates the display terminal 15 to be accessed as a regular display terminal, and permits communication with the refrigerator 51 in the house A.

  Further, the server device 57 stores the image data of the food in the refrigerator transmitted from the refrigerator 51, the data for performing the energy saving operation, and the like. These data are transmitted to the display terminal 15 based on a data calling signal from the display terminal 15. That is, the user can operate the display terminal 15 to display the food image data in the refrigerator on the display and check it.

1-3. FIG. 16 shows a system configuration of a refrigerator used in the refrigeration system.

  16, the refrigerator 51 includes an interface (I / F) 58, a controller (control unit) 59, a storage unit 60, a door opening / closing detection unit 61, interior lighting lamps 34-1, 34-2, and a camera 35-. 1, 35-2, a fan drive circuit 62, a compressor drive circuit 63, a blower fan 18, and a compressor 19, and are supplied with power from an AC power supply 64 such as a commercial power supply in the house A, for example.

  The interface (I / F) 58 exchanges data and the like between the wireless adapter 53 and the controller 59 of the refrigerator 51.

  The controller (control unit) 59 controls the operation of the refrigerator. The controller 59 controls the cameras 35-1 and 35-2 and the in-compartment illumination lamps 34-1 and 34-2 to photograph the food in the refrigerator, and temporarily stores the image data in the storage unit 60. And transmits it to the server device 57 on the Internet 56. Further, if there is a request signal for pre-transmission of image data from the display terminal 15 via the server device 57, an instruction is sent to the display terminal 15 via the server device 57 to execute transmission immediately after photographing the food. The details will be described later.

  The storage unit 60 rewrites and stores, in addition to the identification code and the control program, image data of food in the refrigerator taken by the cameras 35-1 and 35-2 and data for energy saving operation.

  The interior lighting lamps 34-1 and 34-2 are controlled to be turned on and off by the controller 59 based on a signal from the door opening / closing detection unit 61, and are turned on when the door 10 is opened, and for several seconds when the door 10 is closed. Turns off after a delay time.

  The cameras 35-1 and 35-2 are controlled by the controller 59 based on a signal from the door opening / closing detection unit 61, and photograph the food in the refrigerator when the door 10 is closed. The details will be described later.

  The fan drive circuit 62 receives a control signal from the controller 59 and controls driving of the rotation of the blower fan 18.

  The compressor drive circuit 63 receives a control signal from the controller 59 and controls the drive of the compressor 19 such as the rotation speed.

  The blower fan 18 operates under the control of the fan drive circuit 62 to generate an airflow for circulating cool air.

  The compressor 19 compresses a refrigerant (not shown) circulating in the refrigerator 51 under the control of the compressor drive circuit 63.

1-4. Configuration of Server Device Next, a configuration example of the server device of the refrigerator system according to Embodiment 1 will be described with reference to FIG. Here, the configuration of the server device 57 in FIG. 15 will be described as an example.

  As shown in FIG. 17, the server device 57 includes a server interface 65, a server control unit 66, and a data storage unit 67.

  The server interface 65 is a communication unit that exchanges data and the like between the display terminal 15 and the router device 55 and the server device 57 via the Internet 56 under the control of the server control unit 66.

  The server control unit 66 controls the overall operation of the server device 57. The server control unit 66 also performs a joining process on the image data of the food transmitted from the refrigerator 51 and stores the image data in the data storage unit 67. When the display terminal 15 receives a signal of a request for pre-transmission of image data, the data storage unit The latest image data of the image data stored in 67 is transmitted to the display terminal 15 each time the image data is transmitted from the refrigerator 51, and the display terminal 15 is rewritten and updated. Of course, when a data calling signal is transmitted from the display terminal 15 in a state where there is no signal for a request for pre-transmission of image data, the latest image data stored in the data storage unit 67 is transmitted to the display terminal 15.

  The data storage unit 67 stores necessary data such as a management program and an application program for executing the refrigerator system. Further, the data storage unit 67 rewrites and stores the latest data of the food image data transmitted from the refrigerator 51 under the control of the server control unit 66. The data storage unit 67 includes, for example, an HDD (Hard Disc Drive), an SSD (Solid State Drive), and the like.

  It should be noted that the server device 57 can be constructed as either a server composed of a single large computer or a cloud server composed of a large number of computers.

<2. Operation>
Next, the operation of the refrigerator and the refrigeration system configured as described above will be described. In the following description, the same operations will be described with the same step numbers appended.

2-1. Photographing operation of food 2-1-1. FIG. 18 is a flowchart showing an operation in a case where the food in the storage is photographed when the door 10 of the refrigerator 51 is closed.

  When the door 10 is opened to put food in and out (Y in Step 1), the interior lighting lamps 34-1 and 34-2 are turned on (Step 2), and when the door 10 is closed (Y in Step 3), The illuminance of the interior lighting lamps 34-1 and 34-2 is lowered (step 4), and after a while, for example, several seconds, the cameras 35-1 and 35-2 are activated to photograph the food in the interior (step 5). . Thereafter, the photographed image data is stored in the storage unit 60 of the refrigerator 51 (step 6), and the interior lighting lamps 34-1 and 34-2 are turned off (step 7).

  Here, it is preferable that the illuminance of the interior illuminating lights 34-1 and 34-2 be reduced during the above-described photographing, because the food can be clearly photographed. That is, when the food is photographed by turning on the interior lightings 34-1 and 34-2, the reflected light is strongly reflected on the food from the entire peripheral surface excluding the opening of the storage because the interior of the storage is white. However, the food becomes whitish and the clear photographing becomes difficult. However, when the illuminance of the interior illuminating lights 34-1 and 34-2 is reduced as in the above-described example, the influence of the reflected light from the inner surface of the storage can be significantly reduced, and a clear image can be captured. It becomes.

2-1-2. FIG. 19 is a flowchart showing the operation in the case where the food in the storage room is photographed when the refrigerator is cooled after the food in the refrigerator is put in and taken out of the refrigerator.
When the door 10 is opened to put food in and out (Y in Step 1), the interior lights 34-1 and 34-2 are turned on (Step 2), the door 10 is closed (Y in Step 3), and the refrigerator compartment is opened. It is determined whether or not the cooling operation is performed (step 8). If the blower fan 18 is ON, the damper 22 is open, and the refrigerator is in the cooling operation, the illuminance of the interior lighting lamps 34-1 and 34-2 is reduced in the same manner as when the door is closed (step 4). ), Activate the cameras 35-1 and 35-2 to photograph the food in the refrigerator (step 5). Thereafter, the photographed image data is stored in the storage unit 60 of the refrigerator 51 (step 6), and the interior lighting lamps 34-1 and 34-2 are turned off (step 7).

  In this way, by taking a picture during the refrigerator compartment cooling operation, for example, even if dew condensation occurs due to a temperature difference between the inside and outside of the storage on the surface of the food newly placed in the storage, this condensation is eliminated by the cooling operation, Disappears. Therefore, photographing by the cameras 35-1 and 35-2 is performed so clearly that characters written on the food packaging container can be read, which is preferable.

  During the operation, when the door 10 is closed (Y in step 1), the interior illuminating lights 34-1 and 34-2 are turned off once and the interior illuminating lights 34-1 and 34-1 are again illuminated before photographing. If the illuminance is reduced by turning on -2, it is possible to suppress wasteful power consumption caused by keeping the interior lighting lamps 34-1 and 34-2 lit.

2-1-2-a. FIG. 20 is a flowchart showing an operation in a case where the food in the storage is photographed in conjunction with the door opening and closing operation after the food in the refrigerator is taken in and out.

  When the door 10 is opened to put food in and out (Y in Step 1), the interior lighting lamps 34-1 and 34-2 are turned on (Step 2), and when the door 10 is closed (Y in Step 3), A predetermined time is measured after the door is closed (step 101), and when it is determined that the predetermined time has elapsed, the cameras 35-1 and 35-2 are energized (step 102). Then, as in the case of the photographing when the door is closed, the illuminance of the interior lighting lamps 34-1 and 34-2 is reduced (step 4), and the cameras 35-1 and 35-2 are activated to photograph the food in the refrigerator. (Step 5). Thereafter, the energization of the cameras 35-1 and 35-2 is terminated (step 103), and the captured image data is stored in the storage unit 60 of the refrigerator 51 (step 6), and the interior lighting lamps 34-1 and 34-2 are stored. Is turned off (step 7).

  In this way, by measuring a predetermined time after detecting the closing of the door and judging that the predetermined time has elapsed, the photographing of the cameras 35-1 and 35-2 is started. Even in this case, time for removing the fogging can be secured, and clear image data can be taken. The predetermined time depends on the ambient temperature of the refrigerator, the humidity, the opening time of the door, and the like.For example, the higher the ambient temperature and humidity, the longer the predetermined time, or the longer the opening time of the door, the longer the predetermined time. The predetermined time may be changed according to conditions.

  The timing for energizing the cameras 35-1 and 35-2 is set to a predetermined time after the door is closed. However, any timing may be used as long as the time is from the detection of the opening of the door to the photographing. By energizing the camera after detecting the opening of the door, the effect of preventing the fogging of the lens can be enhanced, but it can be set as appropriate in balance with the power consumption.

  If the predetermined time is set to be long (for example, several minutes), the interior lighting in step 2 is temporarily turned off, and a predetermined time before the shooting, the illuminance suitable for shooting is again set. It may be turned on to shoot. The energization timing of the camera may be appropriately set before a predetermined time before shooting.

  The termination of energization of the camera (step 103) has been described before the storage of the image data (step 6), but may be performed after the storage of the image data (step 6).

2-1-2-b. Photographing linked to time FIG. 21 is a flowchart showing an operation in a case where food in the storage is photographed based on a predetermined time period or a predetermined time.

  It is determined whether a predetermined time has elapsed or the predetermined time has elapsed since the previous shooting (step 201), and in the case of Yes, it is determined whether the door 10 is closed (step 202). Then, power is supplied to the cameras 35-1 and 35-2 (step 102). Then, the illuminances of the interior lighting lamps 34-1 and 34-2 are turned on at a predetermined illuminance (step 203), and the cameras 35-1 and 35-2 are activated to photograph the food in the interior (step 5). ). Thereafter, the energization of the cameras 35-1 and 35-2 is terminated (step 103), and the captured image data is stored in the storage unit 60 of the refrigerator 51 (step 6), and the interior lighting lamps 34-1 and 34-2 are stored. Is turned off (step 7).

  As described above, it is determined whether a predetermined time has elapsed or a predetermined time has elapsed since the previous imaging, and the imaging by the cameras 35-1 and 35-2 is started, so that the periodic imaging of the stored food is performed. Data can be obtained. Thus, for example, it is possible to periodically acquire and monitor not only the presence or absence of food in the refrigerator, but also the state change of stored food as an image.

  In the present embodiment, the operation in the case where the food in the storage is photographed based on the predetermined time period or the predetermined time has been described, but the food in the storage is operated in conjunction with the above-described door opening / closing operation. (2-1-2-a. Imaging linked with door opening / closing operation) may be combined. For example, the food in the storage may be photographed in conjunction with the door opening and closing operation, and the food in the storage may be photographed based on a predetermined time period or a predetermined time. Further, based on photographing the food in the storage in conjunction with the door opening / closing operation, the food in the storage is photographed based on a predetermined time period or a predetermined time when there is no door opening / closing for a predetermined time. You may do so.

2-1-2-c. FIG. 22 is a flowchart showing an operation in a case where food in the storage is photographed based on a photographing instruction from the outside.

  When there is a photographing instruction from the outside (step 301), it is determined whether or not the door 10 is closed (step 202). When the door 10 is closed, power is supplied to the cameras 35-1 and 35-2 (step 301). 102). Then, the illuminances of the interior lighting lamps 34-1 and 34-2 are turned on at a predetermined illuminance (step 203), and the cameras 35-1 and 35-2 are activated to photograph the food in the interior (step 5). ). Thereafter, the energization of the cameras 35-1 and 35-2 is terminated (step 103), and the captured image data is stored in the storage unit 60 of the refrigerator 51 (step 6), and the interior lighting lamps 34-1 and 34-2 are stored. Is turned off (step 7). Thereafter, the image data is transmitted (step 302).

  As described above, the food in the storage is photographed based on the photographing instruction from the outside, and the image data is transmitted, so that the stored food in the storage can be confirmed in real time as needed by the image.

  In the present embodiment, the operation in the case where the food in the storage is photographed based on an external photographing instruction has been described, but the operation of the food in the storage in conjunction with the above-described door opening / closing operation is described. (2-1-2-a. Shooting linked to the door opening / closing operation), or when a predetermined time has elapsed since the previous shooting or at a predetermined time (2-1-2). -B. Shooting linked to time).

2-1-2-d. FIG. 23 is a flowchart showing an operation in a case where the food in the storage is photographed in conjunction with the opening / closing operation of the refrigerator compartment damper.

  When the door 10 is opened to put food in and out (Y in Step 1), the interior lighting lamps 34-1 and 34-2 are turned on (Step 2), and when the door 10 is closed (Y in Step 3), After detecting the closing of the door, it is determined whether the damper of the refrigerating compartment is opened or closed (step 401). After the determination and the predetermined time has elapsed (Y in step 101), the cameras 35-1 and 35-2 are energized (step 102). Then, the illuminance of the interior lighting lamps 34-1 and 34-2 is reduced (step 4), and the cameras 35-1 and 35-2 are activated to photograph the food in the interior (step 5). Thereafter, the energization of the cameras 35-1 and 35-2 is terminated (step 103), and the captured image data is stored in the storage unit 60 of the refrigerator 51 (step 6), and the interior lighting lamps 34-1 and 34-2 are stored. Is turned off (step 7).

  When the damper of the refrigerating room is closed in step 401 (N in step 401), the process stands by until the damper of the refrigerating room is opened (step 402), and when the damper of the refrigerating room is detected to be open (step 402). In step 402 (Y), it is determined whether a predetermined time has elapsed since the opening of the damper was detected. After the predetermined time has elapsed (Y in step 403), the cameras 35-1 and 35-2 are energized (step 102). Thereafter, the same operation as described above is performed.

  In this way, by photographing the inside of the refrigerator with the damper in the refrigerator compartment open, it is possible to remove the fogging and condensation of the cameras 35-1, 35-2 due to the introduction of the cool air dehumidified by the cooler into the refrigerator. And clear image data can be obtained. Further, since the predetermined time has elapsed while the damper of the refrigerator compartment is open, the fogging and condensation of the cameras 35-1 and 35-2 can be reliably removed.

  In the present embodiment, the operation in the case where the food in the storage is photographed in conjunction with the opening / closing operation of the refrigerator compartment damper has been described. However, from the viewpoint of introducing dehumidified cool air, the operation of the compressor 19 and the blower fan 18 is described. Time is the premise.

  Further, when the ambient temperature is low, the operation time of the compressor 19 or the open state of the damper of the refrigerator is shortened. Therefore, when the ambient temperature is low, the setting of the predetermined time is shortened. May be. In addition, a mode may be provided in which the operation of the compressor 19 is forcibly performed at the time of photographing or the open state of the damper of the refrigerator compartment is maintained.

2-1-3. Photographing when Door is Open FIG. 24 is a flowchart showing an operation in the case of a single door, in which food in the storage is photographed when the door 10 is opened at a predetermined angle.

  When the door 10 is opened to put food in and out (Y in step 1), the interior lighting lamps 34-1 and 34-2 are turned on (step 2), and the opening angle of the door becomes a predetermined angle, for example, as shown in FIG. When the opening angle reaches 30 degrees as shown (Y in step 10), the illuminance of the interior illuminating lights 34-1 and 34-2 is reduced (step 4), and the cameras 35-1 and 35-2 are started to activate the warehousing. The food inside is photographed (step 5). Thereafter, the photographed image data is stored in the storage unit 60 of the refrigerator 51 (Step 6), and when the door is closed (Y in Step 11), the interior lighting lamps 34-1 and 34-2 are turned off (Step 7). .

  If the photographing is performed when the door 10 is opened at a predetermined angle as in this example, it is possible to photograph the interior of the storage 5 over a wide area as described above, and the image becomes natural without any discomfort. It is.

  In addition, when photographing when the predetermined angle is opened, photographing is performed at least twice at the time of the door opening operation and at the time of the door closing operation. At this time, by storing the image data (at the time of closing the door) photographed last in the operation time from one opening to closing of the door in the storage unit 60, the latest storage of the food that has been put in and taken out is stored. Food condition can be photographed.

  Needless to say, in step 10 for detecting whether or not the opening angle of the door 10 reaches the predetermined angle, if the door 10 is closed before reaching the predetermined angle, the in-compartment illuminating light 34-1. , 34-2 are turned off, and it is waited for the door 10 to be opened again.

  As described above, the typical photographing operation has been described by way of example. In addition to the above-described photographing operation, for example, photographing is performed when a predetermined time has elapsed since the last photographing, or when a photographing instruction is given from the display terminal 15. It is also possible to perform a photographing operation such as performing.

2-1-3-a. Another Example of Photographing when Door is Open FIG. 25 shows another example of a case of a single door, in which the food in the storage is photographed when the door 10 is opened at a predetermined angle. It is a flowchart.

  When the door 10 is opened to take in and out food (Y in step 1), when the opening angle of the door becomes a predetermined angle, for example, 30 degrees or more (Y in step 10), the cameras 35-1, 35- 2 (step 102). Then, the door 10 shifts from the open state to the closing operation, and when the opening angle of the door 10 becomes a predetermined angle, for example, an opening angle of 30 degrees (Y in step 501), the cameras 35-1 and 35-2 are activated, The food in the refrigerator is photographed (step 5). When the door 10 is closed (Y in Step 503) without the opening angle of the door 10 again exceeding a predetermined angle, for example, an opening angle of 30 degrees (N in Step 502), the cameras 35-1, 35- The power supply to the refrigerator 2 is terminated (step 103), and the captured image data is stored in the storage unit 60 of the refrigerator 51 (step 6).

  If the opening angle of the door does not become a predetermined angle, for example, an opening angle of 30 degrees or more (N of step 10), the door 10 can be opened without exceeding an opening angle of a predetermined angle, for example, 30 degrees. Is closed (Y in step 504), the process returns to the standby state without photographing (step 505).

  After the cameras 35-1 and 35-2 are activated and the food in the refrigerator is photographed (step 5), when the opening angle of the door 10 exceeds a predetermined angle, for example, 30 degrees again (step 5). 502) It is determined that there is a re-input of food, it is determined that re-photographing is necessary (step 506), the door 10 shifts from the open state to the closing operation again, and the opening angle of the door 10 is a predetermined angle, for example, It is determined that the opening angle is 30 degrees (step 501). Thereafter, the same operation as described above is performed.

  As described above, if the image is taken when the door 10 is opened at a predetermined angle, the interior of the storage can be imaged over a wide area as described above. At this time, there is no need to adjust the illuminance of the interior illuminating lights 34-1 and 34-2 to a predetermined illuminance as described in the photographing when the door is closed, and the ordinary interior illumination of the interior of the door 10 and the external With the illumination from, image data captured in a natural state as seen by a user can be acquired.

  Further, in the present embodiment, the operation of opening and closing is repeated when the opening angle of the door 10 reaches a predetermined angle (for example, 30 degrees) without completely closing the door 10 due to re-feeding of food or the like. Even in this case, it is possible to acquire image data taken immediately before the door 10 is closed, that is, image data of the final storage state of the food.

2-1-4. Illumination at the time of photography Food illumination at the time of photography, as already described with reference to FIGS. 18, 19, and 24, turns on the interior illumination lamps 34-1 and 34-2 and then reduces the illuminance. Irradiate and shoot. In some cases, it is also conceivable to perform imaging by lowering the illuminance and illuminating the interior lighting lamp 34-1 on both sides of the storage room or only the interior lighting lamp 34-2 on the upper surface of the storage room. The reason for photographing with the reduced illuminance will be described later.

  As another example of food irradiation at the time of photographing, the following may be considered. That is, FIG. 26 is a flow chart showing a state of turning on and off the interior illuminating lights 34-1 and 34-2 at the time of photographing by the cameras 35-1 and 35-2. This corresponds to the operation indicated by the broken line.

  The above-described photographing in FIGS. 18, 19, and 24 is an example in which the inside illumination lamps 34-1 and 34-2 are turned on and then the illuminance is reduced to perform photographing. Each of the interior lighting lamps 34-1 on the side is turned off, in other words, only one of the interior lighting lamps 34-1 on both sides of the storage is turned on, and the food is irradiated and photographed. In this case, the interior illuminating lamp 34-2 on the upper surface of the storage may be turned on or off, but any one may be selected as appropriate so that the illuminance in the interior is optimal for photographing.

  In FIG. 26, before starting photographing with the cameras 35-1 and 35-2, one of the interior lighting lamps 34-1 on both sides of the storage already turned on, for example, the right interior lighting lamp is turned off ( Step 12), photograph the food that is being illuminated by the left interior lamp (step 13) and store it (step 14). Thereafter, the right interior lighting is turned on (step 15), and then the left interior lighting is turned off (step 16), and the food illuminated by the right interior lighting is photographed (step 17). ). The image data is stored (step 18), and the right interior lamp is turned off (step 19). Thus, the image data stored by irradiating the food from one side is combined into one image and stored.

  As described above, the reason why the interior lighting lamps 34-1 and 34-2 are turned on one by one to photograph food and combine them into one is as follows. That is, as described above, when taking a picture, the illuminance of the interior illuminating lamps 34-1 and 34-2 is reduced to take a picture. However, if the illuminance is too large, the image becomes dark and lacks sharpness. Therefore, there is a limit to the decrease in illuminance. For this reason, the illuminance is maintained to some extent, but even with the illuminance, the influence of the reflected light may remain when the internal volume of the storage is small. However, as described in this example, when the left and right ones of the interior illuminating lamp 34-1 are turned on and the food is photographed, the influence of the reflected light can be reduced, and the interior illuminating lamp 34-1 is turned on. The surface on the side where you are can shoot with moderate brightness. Therefore, if the combining process of combining the left and right images of the food photographed with the appropriate brightness is performed, a clear image is obtained as a whole, which is preferable.

  The image processing apparatus further includes a storage unit 60 that stores an image captured by the imaging device, and a correction unit that corrects the image. When a plurality of images are combined, the storage unit 60 stores the image corrected by the correction unit. In the case of a single image, the specification may be such that an image taken without using the correction means is stored. This eliminates the need to perform complicated arithmetic processing on all images, and allows the user to promptly check a single uncorrected image according to the user's request, thereby improving the usability according to the application. A high refrigerator can be provided.

2-2. Saving and Combining Images Next, saving and combining of image data captured by the above-described operations will be described.

2-2-1 Saving and Combining in Server FIG. 27 shows a case where image data is saved and combined in the server device 57. When the inside of the refrigerator is photographed by the operations shown in FIGS. The refrigerator 51 is connected to the Internet 56 from the wireless adapter 53 via the gateway device 54 and the router device 55 (step 20), and the individual image data photographed and stored by each camera is transmitted to the server device 57 connected to the Internet 56. Send it (step 21). The image data transmitted via the Internet 56 is subjected to a combining process (step 22) for combining individual image data by the server control unit 66 of the server device 57. The combined image data is stored in the data storage unit 67 of the server device 57 (step 23).

  If images are combined and stored in the server device 57 as in this example, the load on the controller 59 and the storage unit 60 on the refrigerator side can be reduced, and an Internet-compatible refrigerator can be provided at low cost. can do.

FIG. 28 shows a case where image data is stored and synthesized on the refrigerator side. When the inside of the refrigerator is photographed by the operations shown in FIGS. 18 to 24 including FIG. 26 described above. The controller (control unit) 59 of the refrigerator 51 extracts the individual image data from each camera stored in the storage unit 60, combines them, and combines them (step 24). The combined image data is stored (Step 25), and the refrigerator 51 is connected to the Internet 56 from the wireless adapter 53 via the gateway device 54 and the router device 55 (Step 20), and the stored image data is connected to the Internet 56. The information is transmitted to the server device 57 (step 21). The composite image data transmitted via the Internet 56 is stored in the data storage unit 67 of the server device 57 (Step 23).

  In this example, the load on the controller 59 and the storage unit 60 on the refrigerator side increases, but the image data can be transmitted and received with the display terminal 15 without passing through the data storage unit 67 of the server device 57, and the system operation cost can be reduced. can do.

2-2-3. FIG. 29 shows a case where image data is stored and synthesized on the display terminal 15 side. When the inside of the refrigerator is photographed by the operations shown in FIGS. 18 to 24 including FIG. Connects to the Internet 56 from the wireless adapter 53 via the gateway device 54 and the router device 55 (step 20), and transmits image data not to the server but to the display terminal 15 (step 26). The image data transmitted via the Internet 56 is subjected to a combining process (step 27) for combining the image data by the display terminal 15. The synthesized image data is stored in the data storage unit of the display terminal 15 (step 28).

  In this case as well, although the load on the display terminal 15 increases, the load on the controller 59 and the storage unit 60 on the refrigerator side can be reduced, and an Internet-compatible refrigerator can be provided at low cost.

  As described above, the transmission and the combining operation of the image data have been described by exemplifying typical ones. However, the storage and the combining portion need not always be performed at the same portion, for example, the server device 57. , And the image data is stored in the server device 57, and the image data can be arbitrarily set according to the system.

2-3. Normal transmission and reception of image data 2-3-1. Transmission and Reception of Image Data Stored in Server and Refrigerator FIG. 30 is a flowchart showing transmission and reception between the server device 57 or the refrigerator 51 and the display terminal 15, showing transmission and reception of image data stored in the server device 57 or the refrigerator 51. is there.

First, the user presses a server connection button (not shown) displayed on the display of the display terminal 15 at a shopping destination, for example, and makes a request to read the image data stored in the data storage unit 67 of the server device 57 ( Step 31). It is detected whether or not the image data has been taken in from the server device 57 (step 32). If the image data has not been transferred, the display terminal 15 connects to the Internet 56 (step 33). The image data stored in the data storage unit 67 of the server device 57 or the storage unit 60 of the refrigerator is taken out (step 34), and the image data is transferred to the display terminal 15 such as a smart phone or the like which is the data calling source via the Internet 56. Is transferred (step 35). It is detected whether or not image data has been captured from the server device 57 (step 32), and when the image data has been captured, the image data is output to the display of the display terminal 15 (step 36).
Thereafter, it is detected whether or not the enlargement button (not shown) displayed on the display of the display terminal 15 has been pressed (step 37). When it is detected that the enlargement button has been pressed, the storage in the storage output to the display is displayed. Image data is created by enlarging the image data to 1.1 times or more by digital conversion. The enlarged image data is output to the display of the display terminal 15 (step 36).

  Further, thereafter, it is detected whether or not a ten-key button (not shown) displayed on the display is pressed (step 38). When the ten-key button is pressed, the enlarged image data in the storage output to the display is detected. The image data is output to the display as the image data of the location specified by the ten-key button (step 36). Further, it is detected whether the ten-key button has been pressed (step 38). If not detected, end.

  The enlargement button and the ten-key button for displaying image data at a predetermined position may be a method in which the surface of the display is slid with a finger, and such a configuration improves usability.

  As described above, according to the refrigerator system, the contents of the refrigerator 51 can be checked at a glance at a shopping place, so that forgetting to purchase can be prevented, and wastefulness of purchasing extra ingredients can be eliminated. . In addition, in a home where two people work together, it is possible to check the inside of the storage 5 of the refrigerator 51 just before returning home and to purchase foods and the like on the way home, thereby reducing the time for the user.

  In addition, by displaying an enlarged display or displaying a predetermined location, even a display terminal 15 that has been downsized like the display terminal 15 displays an image in an enlarged manner. You can see.

2-3-2. Transmission and Reception of Image Data Stored in Display Terminal FIG. 31 shows a case where image data is already stored and stored in the display terminal 15. In this case, as described with reference to FIG. 29, the image data is transmitted to and stored in the display terminal 15, and when a request to read the image data is made (step 31), the image data is fetched from the storage unit of the display terminal 15. To output image data to the display of the display terminal 15 (step 36). The subsequent enlarged display operation and local display are as described with reference to FIG.

2-3-3. Image data switching reception The images photographed by the cameras 35-1 and 35-2 include the photographed images (the upper half or the lower half in the storage) of each of the cameras 35-1 and 35-2 and the respective photographed images. An example is shown in which a combined image combined (combined image from the top to the bottom in the storage) is stored and stored, and these images can be switched to be transmitted / received and displayed on the display terminal 15.

  FIG. 32 is a flowchart showing the operation in such a case.

  First, the user presses an individual image / synthesized image call button (not shown) displayed on the display of the display terminal 15 (step 41), and calls either the individual image or the synthesized image (step 42). ). Thereafter, either the individual image data or the composite image data is fetched by the flow shown in FIG. 30 or FIG. 31 and displayed on the display of the display terminal 15.

  When the image displayed on the display terminal 15 can be switched to either an individual image or a composite image as in this example, usually, the composite image is called up and displayed, so that the food situation in the storage can be checked at a glance. However, when it is difficult to determine the food due to being at the corner of the storage or the like, switching to an individual image and displaying the image enables reliable determination, thereby improving usability.

2-4. FIG. 33 shows a flowchart in the case where there is a pre-transmission request for an image from the display terminal 15.

  First, the user operates the display terminal 15 to perform a request operation for a pre-request signal (step 51). The server device 57 or the refrigerator 51 receives the request signal (step 52), reads and transmits image data (individual image data or composite image data, the same applies hereinafter) stored in the data storage unit 67 or the storage unit 60 (step 52). 53). The display terminal 15 detects whether image data from the server device 57 or the refrigerator 51 has been taken in (step 54), and connects to the Internet 56 from the display terminal 15 if image data has not been transmitted (step 55). The server devices 57 take out the image data stored in the refrigerator 51 (step 56), and transmit the image data to the display terminal 15 via the Internet 56 (step 57). It is detected whether or not image data has been taken in from the server device 57 (step 54). When the image data is detected, the process waits until there is a call for the image data (N in step 58). If the image data is called (Y in step 58), the image data is output to the display of the display terminal 15 (step 59).

  Thus, when the display terminal 15 requests image data, even if the access to the server device 57 is concentrated and the transmission of the image data from the server device 57 is delayed, the image data is requested in real time. Can be displayed on the display to check the image. Therefore, it is possible to avoid a situation where the user waits for the image data to be displayed at the shopping destination, and it is possible to shorten the shopping time.

  Further, if there is a pre-transmission request for the image from the display terminal 15, the server device 57 sets the image data transmitted every time the door 10 of the refrigerator 51 is closed to the display terminal 15 each time. If the user who has the image data is set so as to leave the image data transmitted, the user can know this every time the door is opened and closed. Therefore, for example, when the power consumption is likely to increase due to an increase in the number of times of opening and closing the door, the user can call home and talk to a child or the like so as to refrain from using the refrigerator 51. Convenience is also improved.

2-5. Switching of transmission / reception paths for image data and the like FIG. 34 shows a display on the display terminal 15 when a communication path is selected. The flow of transmission and reception of image data is as described above, but the purpose of this embodiment is to change the communication path to speed up the acquisition of image data.

  Specifically, the user can select whether to perform the display terminal 15 directly with the gateway device 54 via the router device 55 or with the server device 57 via the Internet 56.

  In FIG. 34, reference numerals 70 and 71 designate communication path selection buttons displayed on the display of the display terminal 15, and reference numeral 70 designates "direct" selection for directly connecting the display terminal 15 to the refrigerator 51 via the router device 55 and the gateway device 54. A button 71 is a “server” selection button for connecting to the server device 57 or the refrigerator 51 via the Internet 56. Numerals 72 and 73 are data selection buttons for calling data, 72 is image data of food, and 73 is a button for energy saving related data such as the number of times the door is opened and closed. 74 is a transmission button.

  When the “direct” selection button 70 is selected, the display terminal 15 is set to “direct” communication. When the user touches the send button 74 in a state where the “direct” communication is set, the image data request signal of the display terminal 15 is sent from the display terminal 15 directly to the gateway device (relay device) 54 via the router device 55. Sent. Gateway device 54 transmits the received data request signal to wireless adapter 53 of refrigerator 51. Upon receiving the image data request signal, the wireless adapter 53 transmits the image data request signal to the controller 59 of the refrigerator 51. Upon receiving the image data request signal, the controller 59 of the refrigerator 51 reads the image data stored in the storage unit 60 and transmits the image data to the display terminal 15 via the wireless adapter 53 and the router device 55.

  Therefore, as shown by the broken line in FIG. 15, when the user is located in the house A, in a broad sense, when the display terminal 15 is in a position where it can directly communicate with the router device 55, the display terminal 15 is connected to the router device 55. , And can directly acquire image data from the storage unit 60 of the refrigerator 51.

  As a result, if a transmission response of image data to a user operation on the display terminal 15 is delayed due to line congestion due to concentration of access to the Internet 56, the user sets the display terminal 15 to " By switching to “direct” communication, image data can be obtained without being congested by the line of the Internet 56. Therefore, the user can display the image on the display of the display terminal 15 in real time without waiting time, and can confirm the food situation in the storage 5.

  On the other hand, as shown by the solid line in FIG. 15, when the user is located outside the house A, in a broad sense, when the display terminal 15 is located at a position where it cannot directly communicate with the router device 55, the user selects the “server” selection button. By touching 71, the display terminal 15 communicates with the server device 57 via the Internet 56 and reads the image data of the refrigerator 51. That is, if the user is outside the house A and touches the “server” selection button 71 to set “server” communication, the display terminal 15 obtains the image data of the refrigerator even if the user is at a remote location, You can know the storage situation.

  As described above, the refrigerator system according to the present embodiment allows the user to select either “direct” communication or “server” communication. By selecting communication, it is possible to view images in a comfortable state without being affected by the congestion of the Internet line.

  As described above, the refrigerator and the refrigerator system according to the present invention have been described by exemplifying the specific configuration, but the present invention is not limited to the above configuration.

  For example, in the above-described embodiment, the gateway device 54 is interposed in the communication path between the display terminal 15 and the wireless adapter 53 of the refrigerator 51 and the Internet 56. However, the gateway device 54 is not necessarily required. May be configured to directly communicate with the display terminal 15, the wireless adapter 53 of the refrigerator 51, and the Internet 56 via this, so that the system configuration can be simplified.

  Further, in the above embodiment, when the user is outside the house A, the image data is obtained from the server device 57. The image data stored in the storage unit 60 of the refrigerator 51 may be directly read from the refrigerator 51 by communicating with the refrigerator 51. In this case and when performing “direct” communication, it is preferable to perform image combining processing in the storage unit 60 of the refrigerator 51 and transmit the image data after the combining processing.

  Further, in this embodiment, an example is shown in which a “direct” selection button 70 and a “server” selection button 71 are displayed on the display of the display terminal 15 and the user selects “direct” communication or “server” communication. However, this may be achieved by providing the display terminal 15 with a function of automatically switching to “direct” communication when the radio wave of the router device 55 is caught, so that “direct” communication is automatically selected. Can be improved.

<3. Effect>
As described above, the refrigerator and the refrigerator system according to the embodiment of the present invention perform various functions and effects as described with the specific configuration. According to this, it is possible to provide an inexpensive refrigerator and a refrigeration system that can take a good picture of the food in the storage and can easily distinguish the food.

  That is, in the refrigerator according to the present embodiment, the cameras 35-1 and 35-2 photograph the food in the storage 5 from the front side of the storage 5, and at this time, the illumination lights 13-0 and 34-2 in the storage are also stored in the storage. The food in the storage 5 is illuminated from the same front side as the photographing direction by the cameras 35-1 and 35-2. Therefore, the photographed image is a clear image without the darkness or inconsistency as in the case where the interior lighting lamps 34-1 and 34-2 are provided on the inner wall of the storage 5. Further, it is not necessary to ask the server device 57 for image processing for brightening and sharpening the captured image.

  Further, since the cameras 35-1 and 35-2 and the in-compartment illuminating lights 34-1 and 34-2 are provided so as to be located forward of the front ends of the shelves 30-1 to 30-4, The image in the storage 5 taken by the cameras 35-1 and 35-2 is the same as the state in which the user opens the door 10 of the refrigerator 51 and actually looks at it. Therefore, when the user displays the image data on the display terminal 15, the user can confirm the food situation in the storage 5 without any discomfort. Moreover, since the in-compartment lightings 34-1 and 34-2 illuminate the food on the shelves from the front front of the shelves 30-1 to 30-4, they are mounted on the front ends of the shelves 30-1 to 30-4. The placed food can be photographed brightly and clearly.

  In addition, since the cameras 35-1 and 35-2 are provided so as to be located on the inner surface of the door 10 of the refrigerator 51 and substantially in the center of the storage 5 on the left and right sides, the cameras 35-1 and 35-2 are used. The photographed image can be shaped so as to be closer to the state where the user opens the door 10 of the refrigerator 51 and actually looks at the image. In addition, it is possible to take a large distance between the cameras 35-1 and 35-2 and the food on the shelf board to photograph a wide area in the storage 5, and to display the food storage situation in a wider area. It becomes possible to confirm at 15. In addition, in this example, a compressor 19 is provided at the uppermost portion of the storage 5 where the camera 35-1 and 35-2 are relatively hard to capture when the cameras 35-1 and 35-2 are provided on the inner surface of the door so that the uppermost portion protrudes forward. This allows the photographing of this portion as well, and the compressor 19 is provided in the upper part of the main body to improve the storage capacity of the lower part of the main body, and also to make the photographing by the cameras 35-1 and 35-2 good. be able to.

  Further, since the cameras 35-1 and 35-2 are provided so as to be located substantially at the center in the front-rear direction of the door pockets 33-1 and 33-2 on the inner surface of the door, not only the food in the storage 5 but also the door pocket. A part of the food stored in 33-1 and 33-2, for example, a milk pack can also be photographed. Since the user has stored it himself / herself, he / she can recognize what it is even if only a part thereof is shown. Therefore, it is possible to photograph almost all foods in the storage.

  If the camera is provided with a camera 14-3 dedicated to the storage container as shown in FIG. 9A, the food in the storage container 31 can be clearly photographed. 9B, the food in the storage container 31 can also be made up of two cameras, that is, the camera 35-2 can be moved up and down as shown in FIG. It is possible to shoot efficiently with a small number of cameras.

  On the other hand, the interior lighting lamps 34-1 and 34-2 are provided at positions deeper than the cameras 35-1 and 35-2, as is clear from FIGS. Since the light of 34-2 is provided so as not to directly enter the photographing sections of the cameras 35-1 and 35-2, the light of the interior lighting lamps 34-1 and 34-2 is provided by the cameras 35-1 and 35-2. It is possible to prevent the camera 35-1 and 35-2 from getting worse when entering the camera, and it is possible to always obtain a good image. This is because a light-blocking portion is provided on the camera side of the interior illuminating lights 35-1 and 35-2, or a light source such as an LED constituting the interior illuminating lights 34-1 and 34-2 is inclined obliquely rearward. The same effect can be obtained by adopting a configuration such as arrangement.

  In addition, the illuminances in the storages of the interior illuminating lamps 34-1 and 34-2 are 150 lux to 200 lux, and when photographing is performed with the illuminance as it is, the food becomes whitish as described in the section of the photographing operation in FIG. The image is difficult to determine. For this reason, in this embodiment, the configuration is such that the illuminance is reduced and the image is taken. Thereby, even with a CCD camera or a CMOS camera, a clear image can be captured with appropriate brightness. The illuminance of the in-compartment illuminating lights 34-1 and 34-2 is controlled by the server control unit 66 so that the illuminance is lower than normal only when the door 10 is closed or the door 10 is opened and an image is taken. By performing such control, it is possible to maintain clear illuminance as usual and make it easier to see the inside of the refrigerator while performing normal shooting without shooting, thereby enabling clear shooting. The decrease in the illuminance may be caused by, for example, turning off one or both of the interior lighting lamps 34-1 and 34-2 or extinguishing both interior lighting lamps 34-1 and 34-2. Various considerations such as lowering the illuminance of the lamps 34-1 and 34-2 may be made, and may be appropriately selected.

  The images in the storage 5 taken by the cameras 35-1 and 35-2 are transmitted to the server device 57 by transmitting the images of the cameras 35-1 and 35-2 to the server device 57 or the refrigerator 51, or The image is combined and synthesized in the display terminal 15. In this embodiment, the image is easily synthesized.

  That is, the cameras 35-1 and 35-2 are provided on the door inner surface corresponding to the front ends of the shelves 30-1 to 30-4, in this example, the upper shelf 30-1 and the third shelf 30-. 3 is provided on the inner surface of the door corresponding to the front end, so that the images taken by the cameras 35-1 and 35-2 are centered on the shelves 30-1 and 30-3 where the line Z in FIG. It will be taken as.

  Therefore, the image data in the storage 5 photographed by the two cameras 35-1 and 35-2 can be combined and processed at the shelf 30-2 which is the boundary line Z. That is, as shown in the “photographed raw image” in FIG. 35, the upper and lower images commonly show the shelf board 30-2, and the curvature of the “photographed raw image” is corrected and “after the curvature correction processing”. After the “image”, if the common shelf 30-2 transferred to the upper and lower images is matched, an image of the entire storage is obtained. As a result, it is not necessary to perform an extremely difficult correction process of aligning and combining each food placed on the shelf plate at the time of the image combining process for each food, and the image combining process becomes easier. Become. Therefore, the processing program for image combination can be simplified, and the capacity of the control unit of the server device 57, the refrigerator 51, or the display terminal 15 for image combination processing needs to be large. And can contribute to speeding up the processing speed.

  Further, since the two cameras 35-1 and 35-2 are provided substantially in front of the shelves 30-1 and 30-3 whose height cannot be adjusted, the height of the shelves 30-35 can be adjusted. Even when the height of the image No. 2 is changed, the image combining process can be easily performed by adjusting the range of the combining process according to the height of the shelf 30-2 during the image combining process.

  The present invention can provide an inexpensive refrigerator and a refrigeration system that can easily photograph food in a storage and easily determine the food, and can be used not only for household refrigerators but also for refrigerators in chain stores such as convenience stores. The present invention also includes a freezer), the convenience store headquarters checks the food situation in the refrigerator of each convenience store, and automatically delivers the food, and refrigerators installed in facilities such as factories and laboratories. The present invention is also applicable to business use in which foods, goods, and the like in the inside are checked by the material department and automatically delivered.

DESCRIPTION OF SYMBOLS 1 Refrigerator main body 2 Outer box 3 Inner box 4 Insulation material 5 Storage (refrigeration room)
Reference Signs List 6 switching room 7 ice making room 8 freezing room 9 vegetable room 10 door 10a jacket panel 10a-1 glass plate 10a-2 magic mirror 10b inner plate 10b-1, 10b-2 raised wall portion 11-14 pull-out door 15 display terminal 15a display 16 cooling room 17 cooler 18 blower fan 19 compressor 20 radiating pipe 21 capillary tube 22 damper 23 storage frame 24 terminal holder 25 touch panel 26 gap 27 frame 28 recess 29, 29a smoke sheet 30-1 to 30-4 Shelf board 31 Storage container 32-1 to 32-4 Side wall convex portion 33-1 to 33-8 Door pocket 34-1, 34-2 Interior lighting 35-1, 35-2, 35-3 Camera 36 Center pillar 41 Camera module 50 Refrigerator system 51 Refrigerator 53 Wireless adapter (communication) Equipment)
53a "Connect" button 54 Gateway device (relay device)
55 router device 56 Internet 57 server device 58 interface (I / F)
59 Controller (control unit)
Reference Signs List 60 storage unit 61 door open / close detection unit 62 fan drive circuit 63 compressor drive circuit 64 AC power supply 65 server interface 66 server control unit 67 data storage unit 70 "direct" selection button 71 "server" selection button 72 image data selection button 73 energy saving Related data selection button 74 Send button A House

Claims (11)

A refrigerator provided with at least one storage and a door that opens and closes the storage, and an imaging device for imaging the inside of the storage is provided on the door, based on that the door is opened. A first control unit that turns on a lighting device provided in the storage, and a second control unit that causes the imaging device to execute imaging based on a predetermined time that has elapsed since the door was closed. and a second control means, the illuminance of the illumination device when the imaging apparatus executes the imaging by the rather low than the illuminance of the illumination device, said first control means when said door is open The refrigerator is characterized in that the lighting device is maintained in a lit state from when the lighting device is turned on to when imaging by the imaging device is completed . The refrigerator according to claim 1, further comprising a third control unit that turns off the lighting device based on completion of imaging by the imaging device. The lighting device, a refrigerator according to claim 1 or 2, characterized in that provided on the top surface of the reservoir. The lighting device, a refrigerator according to claim 1 or 2, characterized in that provided on the side wall portion of the reservoir. The lighting device, the top surface of the reservoir, and refrigerator according to claim 1 or 2, characterized in that provided on the side wall portion of the reservoir. The refrigerator according to any one of claims 1 to 5 , further comprising a wireless communication device configured to transmit an image captured by the imaging device to the outside. The wireless communications apparatus, after the front Symbol illumination apparatus is turned off, the refrigerator according to claim 6, characterized in that to transmit the image the imaging device imaged the outside. The refrigerator according to any one of claims 1 to 7 , wherein the storage is a refrigerator. The refrigerator according to any one of claims 1 to 8 , wherein the door is a double door that includes a first door and a second door. The refrigerator according to claim 9 , wherein the imaging device is provided on the first door, and a center pillar is provided on the second door. The predetermined time, the temperature around the refrigerator, moisture around the refrigerator The refrigerator according to claim 1, 10, characterized in that varies with conditions that include any of the opening time of the door.

Images