EP3892182A1

EP3892182A1 - Method and system for determining filling information of a dishwasher and for preparing a dishwasher washing cycle

Info

Publication number: EP3892182A1
Application number: EP21167272.0A
Authority: EP
Inventors: Leonardo Sergi
Original assignee: Candy SpA
Current assignee: Candy SpA
Priority date: 2020-04-09
Filing date: 2021-04-07
Publication date: 2021-10-13
Also published as: IT202000007582A1

Abstract

The method comprises the steps of acquiring a digital color image of at least one rack (21, 22) of the dishwasher (2) containing items to be washed in a given use condition; therefore, determining at least one rack filling information, according to the features of the digital color image acquired; and then providing a user, by means of a graphical interface (32), with at least one determined rack filling information.

The at least one rack filling information comprises a filling level, expressed in terms of volume, or percentage of volume, occupied in the rack, and/or one or more of the following load condition information: volume, or percentage of volume, occupied in the rack, partitioned by item types belonging to a predefined set of item types; and/or volume, or percentage of volume, occupied in the rack, partitioned by material types belonging to a predefined set of material types which the items to be washed are made of; and/or area or percentage of area occupied in the rack; and/or number of items to be washed, present in the rack; and/or number of items to be washed, present in the rack, partitioned by type of items belonging to a predefined set of item types; and/or; and/or position occupied in the rack by each item present in the rack, according to a granularity defined by a predefined set of rack sub-areas; and/or position of the cutlery rack, according to a granularity defined by a predefined set of rack sub-areas. Methods and systems for preparing a dishwasher washing cycle, based on the results of the method for determining information relating to the filling are also described.

Description

TECHNOLOGICAL BACKGROUND OF THE INVENTION

Field of application

In general, the present invention relates to the technical field of the preparation of a dishwasher washing cycle, assisted by electronic processing, to support a user of the dishwasher.
More in particular, the invention relates to a method, based on electronic processing, for determining information relating to the filling of a dishwasher.

Description of the prior art

A dishwashing machine (or more briefly dishwasher), in particular a domestic one, conventionally comprises several racks, i.e., baskets (usually, an upper rack or basket, a lower rack or basket and a cutlery rack or basket) in which to arrange the items to be washed.
Once the dishwasher is filled and closed, the user sets a washing program or cycle, from among a certain number of stored programs, or sets some variables of the washing cycle, and therefore controls the execution of the washing cycle.
In known solutions, the user acts through a dishwasher control interface, provided, for example, with knobs or buttons, or by means of remote control with electronic devices.
In both cases, in the known solutions, the possibilities offered by the dishwasher control interface, to adapt the washing program to the actual dishwasher load situation, are limited.
The user usually chooses the cycle to be carried out on the basis of generic considerations, considering the generic information available thereto with regard to the load and the type of items to be washed.
This often leads to unsatisfactory results, in terms of non-optimal washing quality, risk of damaging the most fragile items, and with an unsatisfactory compromise between the different needs of thoroughly washing some heavy items, and simultaneously washing in a more delicate but no less accurate manner other lighter or more fragile or transparent items.
The need is therefore particularly felt to assist the user in selecting a washing program adapted to the specific dishwasher load situation.
This need, felt for many household appliances, is particularly critical for a dishwasher, in the light of the very wide range of possible dishwasher load conditions, which depend on numerous variables: the mix of item types which may be present, even very different from one another (for example, glass or plastic glasses, ceramic plates, metal pans or pots with metal or wooden handles, porcelain cups, metal cutlery, and so on); the different types of materials of the items present (glass, plastic, metal, wood, ceramic, and so on); the distribution of the items between the baskets and the arrangement within each basket; the load or filling conditions (for example, percentage of volume occupied).
Furthermore, even if software programs were available, capable of identifying an optimal cycle given a certain load condition, the issue would remain of how to input an accurate "description" of the load situation to this program, precisely due to the numerous variables which characterize such situation.
In fact, at the moment, no known solutions exist such as to be capable of obviating these issues and satisfying the aforesaid needs.
Consequently, the needs of the users, mentioned above, relating to washing quality, minimization of the risks of damaging the most fragile items, a satisfactory compromise between different washing requirements for different types of loaded items, and, last but not least, efficiency and energy saving of the washing cycle, are also unsatisfied.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide a method for determining the filling information of a dishwasher, in particular, a domestic one, which allows to at least partially obviate the drawbacks mentioned above with reference to the background art, and to respond to the aforementioned needs particularly felt in the technical field considered. Such an object is achieved by means of a method according to claim 1.
Further embodiments of such a method are defined in claims 2-12.
It is also the object of the present invention to provide a method for preparing and performing a dishwashing cycle, by means of a dishwasher, which is based on the results of the aforesaid method for determining filling information.
Such an object is achieved by means of a method according to claim 13.
A further embodiment of such method is defined in claim 14.
It is also the object of the present invention a system for preparing and performing a dishwashing cycle, capable of performing the aforesaid method. Such a system is defined in claim 15.
A further embodiment of the system is defined in claim 16.
Furthermore, it is a particular object of the present invention to assist a user in the selection of a washing program for the dishwasher, in such a way to consider the load situation, and/or to assist a user in optimizing the partition of the items between the dishwasher racks/baskets, and/or in optimizing the arrangement of the items inside each rack/basket.

BRIEF DESCRIPTION OF THE DRAWINGS

Further features and advantages of the methods and of the systems according to the invention will become apparent in the following description of preferred embodiments, given by way of indicative, non-limiting examples, with reference to the accompanying Figures, in which:

Figure 1 diagrammatically shows an embodiment of a system for preparing and performing a dishwashing cycle, according to the invention, capable of implementing a corresponding embodiment of the method for preparing and performing a dishwashing cycle;
Figures 2 and 3 show examples of partition into sub-areas of the upper and lower racks, respectively, according to an embodiment of the method according to the invention;
Figures 4-5 show two examples of tables for presenting the results obtained with the implementation of an embodiment of the method for determining the filling information of a dishwasher;
Figure 6 shows an example of an arrangement of kitchenware, inside a dishwasher rack, which the method according to the present invention is capable of identifying and describing;
Figure 7 shows a further example of a table for presenting the results obtained with the implementation of an embodiment of the method for determining the filling information of a dishwasher;
Figure 8 diagrammatically shows a graphical interface made available by a software application comprised in the present invention, at different steps of the method for preparing and performing a dishwashing cycle according to the invention.

DETAILED DESCRIPTION

With reference to Figures 1-8, a method for determining filling information of a dishwasher 2, in particular, a domestic one, is described.
The method comprises the steps of acquiring a digital color image of at least one rack (i.e., basket) 21, 22 of the dishwasher 2 containing items to be washed in a given use condition; then, determining at least one rack filling information, according to the features of the digital color image acquired; and then providing a user, by means of a graphical interface 32, with at least one determined rack filling information.
The aforesaid at least one rack filling information comprises a filling level, expressed in terms of volume, or percentage of volume, occupied in the rack, and/or one or more of the following load condition information:

volume, or percentage of volume, occupied in the rack, partitioned by type of item belonging to a predefined set of item types; and/or
volume, or percentage of volume, occupied in the rack, partitioned by type of material belonging to a predefined set of material types which the items to be washed are made of; and/or
area or percentage of area occupied in the rack; and/or
number of items to be washed, present in the rack; and/or
number of items to be washed, present in the rack, partitioned by type of items belonging to a predefined set of item types; and/or
position occupied in the rack by each item present in the rack, according to a granularity defined by a predefined set of rack sub-areas; and/or
position of the cutlery rack, according to a granularity defined by a predefined set of rack sub-areas.

According to an embodiment of the method, the aforesaid acquiring step comprises acquiring a respective digital color image of each of at least two of the racks of the dishwasher 2, an upper rack 21 and a lower rack 22.
The determining step comprises determining, for each of the at least two racks 21, 22 the aforesaid at least one rack filling information, according to the features of the respective digital color image acquired.
In accordance with an embodiment of the method, the aforesaid determining and providing steps comprise determining and providing at least the rack filling level.
According to an implementation option, the aforesaid determining step comprises determining the rack filling level, and furthermore determining the following load condition information: the volume, or the percentage of volume, occupied in the rack, partitioned according to the aforesaid item types, and partitioned according to the aforesaid material types; and the number of items present in the rack, for each of the aforesaid item types, and for each of the aforesaid material types.
According to an embodiment of the method, the determining step comprises determining the rack filling level and all the load condition information, mentioned above.
In accordance with an embodiment, the method further comprises determining the following further load condition information:

types of items present in the rack, among a plurality of types belonging to a predefined set of item types; and/or
type of material which the items to be washed, present in the rack, are made of, among a plurality of material types belonging to a predefined set of material types.

In accordance with different possible implementation options of the method, the determining step comprises determining the rack filling level and any combination of one or more of the load condition information, mentioned above.
According to an implementation option, the providing step comprises providing the user with all or with a desired subset of the filling level information and of the determined load conditions.
In accordance with an embodiment of the method, the aforesaid step of determining at least one rack filling information comprises processing such digital image by means of a processing employing artificial intelligence and/or machine learning techniques and/or at least one trained algorithm.
On the basis of such processing, the method provides for recognizing and/or determining, for each of the one or more racks of the dishwasher of which the image was acquired, the aforesaid rack filling level information and/or the aforesaid load condition information.
According to an embodiment of the method, the aforesaid processing employing artificial intelligence techniques and/or at least one trained algorithm is based on one or more preliminary training steps, carried out before the operating steps previously described.
Each preliminary training step comprises training an artificial intelligence and/or machine learning algorithm to determine and/or recognize a predefined set of filling information, comprising one or more of the following filling level information and/or load conditions:

volume or percentage of volume occupied by the items to be washed; and/or
area or percentage of area occupied by the items to be washed; and/or
number of present items to be washed; and/or
types of items present in the rack, among a plurality of types belonging to a predefined set of item types; and/or
volume or percentage of volume occupied in the rack partitioned by each of the item types identified; and/or
number of present items to be washed, partitioned by each of the identified item types; and/or
type of material which the items to be washed are made of, among a plurality of material types belonging to a predefined set of material types; and/or
volume or percentage of volume occupied, partitioned by each type of material identified; and/or
number of items present, partitioned by each type of material identified; and/or
position occupied by each of the items identified, according to a granularity defined by a predefined set of rack sub-areas; and/or
position of the cutlery rack, according to a granularity defined by a predefined set of basket sub-areas.

The aforesaid training step comprises acquiring a plurality of digital color images of a rack, each corresponding to a known basket load condition, in which the value and/or content of each of the information belonging to the aforesaid predefined set of filling information to be determined is known.
The artificial intelligence and/or machine learning algorithm is then trained on the basis of the plurality of digital color images acquired and on the basis of the respective known values and/or contents of each of the filling information to be determined, corresponding to the respective known rack load conditions represented by the respective digital color images acquired.
According to different embodiments, the step of acquiring a plurality of images in the training step comprises acquiring different images in different lighting and/or shooting or framing conditions (with or without flash), and/or in the presence of different disturbances which may occur in actual situations; and/or with different resolutions compatible with different possible image acquisition means of the user, so as to train the artificial intelligence and/or machine learning algorithm to recognize the desired information in a plurality of corresponding actual situations of image acquisition by the user.
According to another embodiment, which may be combined with the preceding one, the step of acquiring a plurality of images in the training step comprises acquiring images in which portions or fractions of one or more items placed in a dishwasher rack, and not the entire items, are captured, so as to train the artificial intelligence and/or machine learning algorithm to recognize an item even when it is even only partially framed and/or when even only a portion of the item is framed.
It should be noted that the method described herein therefore includes performing a recognition, even in the aforesaid non-optimal conditions, only on the basis of the training of the algorithm (and without requiring the user to take multiple images and compare them with one another).
It should also be noted that the method may employ different algorithms or artificial intelligence and/or machine learning techniques, known per se, for recognizing items, which, however, are trained in the specific manners described above.
According to some implementation examples of the method, the training step is carried out by dividing the set of digital images acquired in known filling conditions into two subsets, a training subset and a testing subset. The results gradually obtained by the trained algorithm on the basis of the training subset are verified by means of the testing subset, so as to validate the results obtained, obtain and evaluate the performance metrics, evaluate the percentage of error obtainable from the estimates, and possibly refine the training with additional training images.
Conventionally, this involves the acquisition of several hundreds, or even thousands of digital images, acquired in respective known conditions, during the algorithm training step.
In an actually implemented training example, 338 images were employed to form a testing subset.
For the training subset, 1869 color photographs were taken (digital images acquired for training purposes), of which 754 in ambient light conditions with flash, 753 in ambient light conditions without flash, 362 without ambient light with flash.
The known load conditions, for each acquired image, have been prepared by arranging in the most different manners, in the upper and lower basket of the dishwasher, a plurality of commercial items with features fully known, in particular the volume and material.
The structural features, in particular the size, of the upper and lower racks of the dishwasher considered, were provided to the algorithm as a training input.
The actions described above may of course be repeated on a plurality of different dishwashers.
The photographs were taken from a reference position considered common for the user (for example, at the sternum of a 170 cm tall person and at a distance of 20 or 30 cm from the fully open dishwasher door).
According to an implementation option of the method, the results of the trained algorithm are combined with deterministic formulae or algorithms.
For example, the trained algorithm may be used to identify the types of items and the respective types of material present in the rack, the number of items and the position thereof.
In this case, the method further comprises the step of estimating the volume and/or the percentage of volume occupied in the rack, and/or the volume and/or the percentage of volume occupied in the rack partitioned by material type and item type, and/or the area of occupation of the rack on the basis of the results provided by the trained algorithm, processed by means of formulae and/or analytical algorithms, on the basis of mathematical relationships and/or predefined criteria.
According to an implementation option, the preliminary training step comprises defining a set of categories of recognizable items, each category being defined by the type of item to be washed and the type of material of the item to be washed; then, associating a respective estimated volume to each category; then, training the recognition algorithm to recognize the category of an item comprised in an acquired image, by means of a training carried out on the basis of a plurality of acquired test color images, corresponding to respective known rack filling situations, in which items of a known category and of a known volume and shape are placed in the rack.
In this case, the determining step comprises identifying the number of items present in the rack and recognizing the category of each item, by processing the acquired color image carried out by the trained algorithm; and estimating the total volume occupied by the present items on the basis of the number of items identified and the respective types recognized.
The volume for each category is estimated by means of an approximate predefined value dictated by experience, or it is calculated on the basis of approximate rules or formulas established a priori based on experience.
In accordance with an implementation option, the aforesaid determining step further comprises estimating the volume occupied by each item category identified as the product of the number of items identified belonging to the category and the predefined volume associated with the category; and estimating the total volume occupied in the basket as the sum of the estimated volumes for each item category identified.
According to an implementation option, the aforesaid determining step further comprises estimating a percentage of volume occupied as the ratio between the estimated total volume occupied in the rack and a known volume value of the rack.
In various possible implementation variants of the method, algorithmic techniques are used aimed at reaching a reasonable and effective compromise between the accuracy of the estimates and the complexity of the algorithms and the related training.
In fact, it is considered that, for the purposes of the present invention, a reasonable approximation is acceptable in estimating, for example, the volumes of the items to be washed, so as to simplify the algorithms, given the many different situations which may characterize a load condition (numerous product categories, numerous sub-categories for each category, numerous possible arrangements of the items to be washed, and so on).
For example, once an item has been recognized, the volume estimate is obtained by applying different weights to standard volume values of the item, depending on whether the item has been placed by the user in the lower or upper rack.
The weights are predefined in the light of experience, considering the usual arrangements chosen by the users, for example, the fact that larger items are usually arranged in the lower rack.
Therefore, for example, a metal pot in the upper rack is associated with a lower weight, and therefore estimated with a lower volume occupied, with respect to a metal pot placed in the lower rack.
Similar considerations apply to plates or other kitchenware.
According to different possible implementation options of the method, the aforesaid predefined set of recognizable item types comprises one or more of the following types: glasses and/or glass crystals with stems and/or plates and/or pots and/or trays and/or cups and/o cutlery and/or utensils and/or other kitchenware.
In accordance with different possible embodiments of the method, the aforesaid predefined set of recognizable material types comprises one or more of the following types: glass and/or ceramic/porcelain and/or metals and/or plastic/silicone and/or wood and/or other material.
According to an implementation option of the method, for each of said recognizable item types, a further recognition is made for identifying the belonging of the item to a respective coded sub-type.
In accordance with an embodiment, the method further comprises the steps of identifying the number of items present in the basket and recognizing, for each item, the respective type and the material type, by processing the acquired color image.
In this case, the determining step comprises determining, for each "item type - material type" pair, one or more of the following information: volume or percentage of volume occupied in the rack and/or area or percentage of area occupied in the rack; and/or number of items to be washed, present in the basket; and/or position occupied in the rack defined by a predefined set of rack sub-areas.
According to an implementation option, the providing step comprises providing information organized in a two-dimensional matrix, in which each column comprises a respective item type, from a predefined set of item types, and each row comprises a material type, from a predefined set of material types, and in which a cell of the matrix contains as result, for the respective "item type - material type" pair, one or more of the following determined information: volume or percentage of volume occupied in the rack; and/or area or percentage of area occupied in the rack; and/or number of items to be washed, present in the rack; and/or position occupied in the rack defined by a predefined set of rack sub-areas.
According to a particular implementation option, the aforesaid matrix is a simplified matrix, adapted to provide results only on the cells corresponding to "item type - material type" combinations, predefined as significant.
According to an embodiment of the method, the recognition of the position of each item present in a dishwasher rack is provided.
Furthermore, the aforesaid predefined set of rack sub-areas which defines an identifiable position granularity comprises: for the upper dishwasher rack 21 a first number of upper rack sub-areas; and/or; for the lower dishwasher rack 22 a second number of lower rack sub-areas.
In such case, the position of each item is expressed by indicating the lower or upper rack sub-area in which the item is identified.
According to an implementation option (shown in Figure 2), for the upper dishwasher rack 21, three upper rack sub-areas (SA1, SA2, SA3) are defined, arranged side by side with respect to one another in the sense of a length of the upper rack, each corresponding to about one third the width of the rack.
According to an implementation option (shown in Figure 3), three lower basket sub-areas are defined for the lower dishwasher rack 22, comprising: a first lower rack sub-area SA6 which extends along the entire length of the lower rack and occupies a first half of the width of the lower rack; a second lower rack sub-area SA4 which occupies a part of the second half of the width of the lower rack and extends for about two thirds of the length of the lower rack; a third lower rack sub-area SA5 which occupies the remaining part of the second half of the width of the lower rack and extends for about a third of the length of the lower rack.
According to an implementation option, sub-area matrices are provided as a result, containing the considered filling level and load condition information, expressed for each sub-area.
Examples of such sub-area matrices are shown in Figures 4 and 5, in which the determined and presented results are, respectively, the occupied volume and the number of items, for each sensible "item type - material type" combination, within a sub-area.
It should be noted that, in Figure 5, even the number of items may take on a decimal value, therefore not an integer. This is linked to another interesting aspect of the method, i.e., the ability to also identify the position of an item which is partly in one sub-area and partly in another sub-area, also assigning the item fractions to the relevant sub-area. Such situation is illustrated, for example, in Figure 6.
Some details, given by way of explanation and not by way of limitation, will be reported below on the possible volumetric approximations adopted (in a particular implementation example) during the training step. In such step, volumetric or dimensional or functional information for each of the items arranged in the dishwasher are known. After the algorithm has recognized an item, an approximate volume is assigned to such item according to some criteria, exemplified below.
In such examples, three general criteria are adopted:

a) items with a shape similar to a cylinder, of which diameter and height are known, are assigned the volume of the cylinder having such diameter and height;
b) items with a shape similar to a parallelepiped, of which the three dimensions are known, or of which two dimensions are known and the third one may be reasonably estimated, are assigned the volume of the parallelepiped having such dimensions;
c) for items whose capacity is known, the volume is estimated on the basis of the capacity (converted, for example, from cl or l to cm³).

For items which may be separated into several parts (for example, pan and handle, or pot and handle, or cup and saucer), the total volume is estimated as the sum of the volumes estimated for the individual parts.
For a lid, the volume is estimated on the basis of the diameter of the relative pot and an estimated thickness (for example, 0.5 cm).
For glasses, cups and trays, a volume estimate based on the capacity is adopted.
For dinner plates and soup bowls, an estimate based on the cylindrical approximation may be adopted, i.e., on the basis of the known diameter and a height estimated on the basis of average height values for dinner plates and soup bowls.
For containers, an estimate based on parallelepiped approximation may be adopted, i.e., as a product of the three dimensions.
For cutlery, an estimate based on the parallelepiped approximation may be adopted, i.e., a product of the three dimensions, if known, or a product of two known dimensions for an estimate of the third dimension; and so on.
An example of the results of the volumetric approximation described above (and the standard deviations of the error committed, where available, enclosed in brackets) is given in the matrix of Figure 7, which shows the estimated volume, for a given dishwasher basket filling condition, for each significant "item type - material type" pair.
A matrix of a similar type may also be presented in an operational phase, after the estimate has been made, based on the processing of an image acquired in a given situation. The volume may be estimated on the basis of approximations such as those reported above, or it may be calculated by assigning each item to a category and adding known and predefined standard values for each category; in such case, the greater the number of categories considered, and, therefore, the finer the classification of the items, the better the approximation obtained.
In accordance with an embodiment of the method, the determining step further comprises determining indications and/or recommendations regarding the arrangement of the items in the dishwasher, on the basis of the filling and load level information detected by means of the processing the digital image acquired.
In such case, the step of providing information to the user comprises: presenting in a numerical or graphical form the information detected, among said obtainable rack filling level information and basket load condition information; and/or also providing the user with written notifications containing said indications and/or recommendations regarding the arrangement of items in the dishwasher.
According to various embodiments, the processing which allows the user to reach such indications/suggestions comprises predefined conversion tables based on the percentage of types of items among the known cycle programs.
The suggestions for arranging items provide indications for moving items identified in an inappropriate rack, or in an inappropriate position inside a rack; and/or suggestions to arrange some items differently to improve load balancing, if imbalances have been detected, on the basis of predefined evaluation criteria (and incorporated into the algorithm); or even efficiency tips, in terms of optimum load with respect to the filling percentage, on the basis of known dishwasher operation efficiency criteria.
According to a more specific implementation example, the aforesaid indications and/or suggestions provided to the user comprise a load optimization indication between upper and lower dishwasher rack and suggestions for moving items from one rack to the other one; and/or an indication of item position optimization within a rack and recommendations for moving items between different rack sub-areas.
In accordance with an embodiment of the method, the steps of acquiring a digital color image and providing information to the user are carried out by means of a portable electronic device 3 equipped with digital image acquisition means 31 and a graphical interface 32, on the basis of a software application and/or program stored and executable in electronic processing means 30 of the portable electronic device 3.
According to an embodiment of the method, the step of determining at least one rack filling level and/or basket load condition information is carried out by means of a processing performed by one or more remote software programs stored and executed in remote electronic processing means 4.
The remote electronic processing means 4 are operatively connected to the electronic processing means 30 of the portable electronic device 3, and the aforesaid one or more remote software programs are interoperable with the software application and/or program stored and executable in the portable electronic device.
A method for preparing and performing a dishwashing cycle by means of a dishwasher 2, in particular a domestic one, is now described.
This method comprises the steps of providing a dishwasher 2; then, arranging the kitchenware in the racks 21, 22 of the dishwasher; then, performing a method for determining filling information of a dishwasher according to any one of the embodiments described above; furthermore, defining a recommended washing program, on the basis of the results of the method for determining the filling information.
The method further includes generating a washing preparation notification, which may be consulted by the user, in which such washing preparation notification contains at least one washing instruction, comprising the washing parameters of the recommended washing program and also one or more confirmation requests and/or auxiliary washing preparation tips in addition to the aforesaid washing instruction.
The step of providing information to the user comprises providing the user with the aforesaid washing preparation notification, in addition to the rack filling level information and the rack load condition.
According to an embodiment, such method further comprises the steps of requesting the user for a validation of the generated washing instruction, in which the validation comprises an intentional confirmation or modification of the generated washing instruction and an intentional confirmation to start the dishwashing cycle according to the generated or modified washing instruction.
Finally, the steps are provided of transmitting (S10) the aforesaid generated or modified washing instruction to an electronic control unit 20 of the dishwasher 2, and of performing the washing cycle by means of the dishwasher 2, controlling the dishwasher operation by means of the electronic control unit 20 according to the generated or modified washing instruction transmitted.
According to two different implementation variants, the step of transmitting the generated or modified washing instruction to an electronic control unit of the dishwasher is carried out by means of a transmission of electronic signals and bypassing a dishwasher user interface 23 for the manual selection of washing parameters, or by means of a manual command by means of the dishwasher user interface 23.
According to an implementation example, the transmission of the washing instruction to the dishwasher electronic control means 20 occurs by means of a wireless signal connection between the portable electronic device 3 and the dishwasher electronic control means 20.
A system for preparing and performing a dishwashing cycle 1 is now described, with reference to Figure 1.
This system 1 comprises a dishwasher 2, a portable electronic device 3, and remote electronic processing means 4.
The dishwasher 2 is configured to perform a dishwashing cycle, and comprises a dishwasher electronic control unit 20, configured to control the performance of a washing program associated with the aforesaid washing cycle, on the basis of at least one washing instruction received.
The portable electronic device 3 comprises digital image acquisition means 31, configured to acquire a digital color image of at least one basket 21, 22 of the dishwasher containing items to be washed in a condition of use, and also portable electronic device electronic processing means 30 and a portable electronic device graphical interface 32.
Such graphical interface 32 is configured to provide a user with at least one rack filling information and/or a washing preparation notification containing at least one washing instruction relating to a recommended washing program; and/or at least one confirmation request and/or auxiliary washing preparation tips in addition to the washing instruction.
The remote electronic processing means 4 are operatively connected to the portable electronic device 3, and are configured to receive (S7) from the portable electronic device digital data corresponding to the acquired digital image, to determine at least one rack filling information and/or a recommended washing program and/or auxiliary washing preparation tips, on the basis of a digital color image processing, performed by one or more remote software programs stored and executed in the remote electronic processing means 4; and are further configured to transmit (S8) to the portable electronic device 3 the at least one rack filling information and/or the recommended washing program and/or the auxiliary washing preparation tips determined.
The aforesaid electronic processing means of the portable electronic device 30 are configured to perform, by means of a device software application and/or program, the following steps:

preparing a presentation for the user of the at least one rack filling information and/or recommended washing program and/or auxiliary washing preparation tips determined;
receiving instructions (S9) from the user to validate the at least one determined washing instruction, corresponding to the recommended washing cycle which has been determined, or a washing instruction modified by the user.

The aforesaid at least one basket filling information comprises a filling level, expressed in terms of volume, or percentage of volume, occupied in the rack, and/or one or more of the following load condition information:
volume, or percentage of volume, occupied in the rack, partitioned by item types belonging to a predefined set of item types; and/or volume, or percentage of volume, occupied in the rack, partitioned by material types belonging to a predefined set of material types which the items to be washed are made of; and/or area or percentage of area occupied in the rack; and/or number of items to be washed, present in the rack; and/or number of items to be washed, present in the rack, partitioned by type of items belonging to a predefined set of item types; and/or; and/or position occupied in the rack by each item present in the basket, according to a granularity defined by a predefined set of basket sub-areas; and/or position of the cutlery rack, according to a granularity defined by a predefined set of rack sub-areas.
According to an embodiment of the system, the portable electronic device electronic processing means 30 are further configured to transmit (S10) the aforesaid washing instruction determined, or modified by the user, to the electronic control unit of the dishwasher 20.
According to an implementation option, the transmitting step (S10) is performed by transmitting electronic signals and bypassing a dishwasher user interface 23 for the manual selection of washing parameters.
According to another implementation option, the command to the dishwasher is performed by means of a manual command by means of the dishwasher user interface 23.
According to an embodiment of the system, the aforesaid portable electronic device 3 is a smartphone or tablet, equipped with digital camera 31, and in which the aforesaid software application or program of the portable electronic device is stored, executable by the processing means 30 of the smartphone or tablet.
Figure 8 shows an example of use of a smartphone 3 as a portable electronic device in the context of the method described herein, showing the screens of a graphical interface 32 controlled by the software application which supports the method, installed and executable in the smartphone 3, in relation to different steps of the method:

S1 acquisition of a color photograph of the upper dishwasher rack;
S2 acquisition of a color photograph of the lower dishwasher rack;
S3 by means of the graphical interface 32, the filling level, or the load percentage, of each of the racks is indicated;
S4 by means of the graphical interface 32, a suggestion is provided on how to move some items from the upper rack to the lower one, or vice versa, to optimize the load;
S5 by means of the graphical interface, suggestions are provided on how to move items within each of the racks, to optimize the position of the items, and to obtain a better washing performance;
S6 by means of the graphical interface 32, an indication is provided on the washing program to be set, in the light of the particular load condition identified.

In accordance with an implementation option, the aforesaid device software application is configured to present the user with a guide masking, superimposed on the frame of the dishwasher basket, before acquiring the image, to assist the user, aiming at optimizing the framing and acquisition of the image.
According to various possible implementation options, the system 1 is configured to perform a method for preparing and performing a dishwashing cycle by means of a dishwasher 2, according to any one of the embodiments of such method, previously described.
It is worth noting that the object of the present invention is fully achieved by the methods and the system described above by virtue of the functional and structural features thereof.
In fact, the method according to the invention allows to assist a user in the selection of a washing program for the dishwasher, with the presentation of information, tips and suggestions which consider the load situation, and/or the optimization of the partition of the items between the dishwasher baskets, and/or the optimization of the arrangement of the items inside each basket.
The information, washing cycle tips and suggestions provided to the user are determined by a processing which receives as input exhaustive information about the filling level and the load condition of the real situation prior to a specific washing cycle.
Such input information, determined on the basis of an artificial intelligence processing of one or more acquired digital images, comprise information relating to a vast plurality of variables, such as the mix of item types which may be present, even very different from one another (for example, glass or plastic glasses, ceramic plates, metal pans or pots with metal or wooden handles, porcelain cups, metal cutlery, and so on); the different types of materials of the present items (glass, plastic, metal, wood, ceramic, and so on); the distribution of the items between the racks and the arrangement within each rack; the load or filling conditions (for example, percentage of volume occupied).
The availability of such information regarding numerous relevant variables allows to effectively describe (albeit with reasonable approximations, adapted to the type of application considered) the very wide range of possible dishwasher load conditions.
Ultimately, the method and the system of the invention allow to satisfy the needs of the users, previously mentioned, relating to washing quality, minimization of the risk of damaging the most fragile items, a satisfactory compromise between different washing requirements for different types of loaded items, and, last but not least, efficiency and energy saving of the washing cycle.
A person skilled in the art may make changes and adaptations to the embodiments of the methods and systems described above or may replace elements with others which are functionally equivalent to satisfy contingent needs without departing from the scope of protection of the following claims. Each of the features described as belonging to a possible embodiment may be achieved irrespective of the other embodiments described.

Claims

A method for determining filling information of a dishwasher (2), in particular, a domestic one, comprising:
- acquiring a digital color image of at least one rack (21, 22) of the dishwasher (2) containing items to be washed in a given use condition;

- determining at least one rack filling information, according to features of the digital color image acquired;

- providing a user, by means of a graphical interface (32), said at least one rack filling information,
wherein said at least one rack filling information comprises:
- a filling level, expressed in terms of volume, or percentage of volume, occupied in the rack,
and/or one or more of the following load condition information:

- volume, or percentage of volume, occupied in the basket, partitioned by type of item belonging to a predefined set of item types; and/or

- volume, or percentage of volume, occupied in the rack, partitioned by type of material belonging to a predefined set of material types which the items to be washed are made of; and/or

- area or percentage of area occupied in the rack; and/or

- number of items to be washed, present in the rack; and/or

- number of items to be washed, present in the rack, partitioned by type of items belonging to a predefined set of item types; and/or

- position occupied in the rack by each item present in the rack, according to a granularity defined by a predefined set of rack sub-areas; and/or

- position of the cutlery rack, according to a granularity defined by a predefined set of rack sub-areas.
A method according to any one of the preceding claims, wherein said determining and providing steps comprise determining and providing at least said rack filling level, and wherein said determining step comprises determining the rack filling level, and also the following load condition information:
- the volume, or the percentage of volume, occupied in the rack, partitioned according to said item types, and partitioned according to said material types; and

- the number of items present in the rack, for each of said item types, and for each of said material types.
A method according to any one of the preceding claims, wherein said determining step comprises determining said rack filling level and all the aforesaid load condition information,
and/or wherein the providing step comprises providing the user with all or with a desired subset of the filling level information and of the determined load conditions.
A method according to any one of the preceding claims, further comprising determining the following additional load condition information:
- types of items present in the rack, among a plurality of types belonging to a predefined set of item types; and/or

- type of material which the items to be washed, present in the rack, are made of, among a plurality of material types belonging to a predefined set of material types.
A method according to any one of the preceding claims, wherein said determining step comprises:
- processing said digital image by means of a processing employing artificial intelligence and/or machine learning techniques and/or at least one trained algorithm;

- recognizing and/or determining, on the basis of said processing, for each of the one or more racks of the dishwasher the image of which was acquired, said rack filling level information and/or said load condition information,
wherein said processing employing artificial intelligence techniques and/or at least one trained algorithm is based on one or more preliminary training steps, carried out before the steps comprised in claim 1, wherein each preliminary training step comprises:
- training an artificial intelligence and/or machine learning algorithm to determine and/or recognize a predefined set of filling information, comprising one or more of the following filling level information and/or load conditions:
- volume or percentage of volume occupied by the items to be washed; and/or

- area or percentage of area occupied by the items to be washed; and/or

- number of present items to be washed; and/or

- types of items present in the rack, among a plurality of types belonging to a predefined set of item types; and/or

- volume or percentage of volume occupied in the rack partitioned by each of the item types identified; and/or

- number of present items to be washed, partitioned by each of the identified item types; and/or

- type of material which the items to be washed are made of, among a plurality of material types belonging to a predefined set of material types; and/or

- volume or percentage of volume occupied, partitioned by each type of material identified; and/or

- number of items present, partitioned by each type of material identified; and/or

- position occupied by each of the items identified, according to a granularity defined by a predefined set of rack sub-areas, and/or

- position of the cutlery rack, according to a granularity defined by a predefined set of rack sub-areas;
and wherein the method provides that said training step comprises:
- acquiring a plurality of digital color images of a rack, each corresponding to a known rack load condition, wherein the value and/or content of each of the information belonging to said predefined set of filling information to be determined is known;

- training the artificial intelligence and/or machine learning algorithm on the basis of said plurality of digital color images acquired and on the basis of the respective known values and/or contents of each of the filling information to be determined, corresponding to the respective known rack load conditions represented by the respective digital color images acquired.
A method according to claim 5, wherein:
the step of acquiring a plurality of images in the training step comprises acquiring different images in different lighting and/or shooting conditions, and/or in the presence of disturbances which may occur in actual situations; and/or acquiring images with different resolutions compatible with different possible image acquisition means of the user, so as to train the artificial intelligence and/or machine learning algorithm to recognize the desired information in a plurality of corresponding actual situations,

and/or wherein the step of acquiring a plurality of images in the training step comprises acquiring images in which portions or fractions of one or more items placed in a dishwasher rack, and not the entire items, are captured, so as to train the artificial intelligence and/or machine learning algorithm to recognize an item even when it is even only partially framed and/or when even only a portion of the item is framed,

and/or wherein the trained algorithm is used to identify the item types and the respective types of material present in the rack, and also the number of items and the position of each item, wherein the method further comprises the step of estimating the volume and/or the percentage of volume occupied in the basket, and/or the volume and/or the percentage of volume occupied in the rack partitioned by material type and item type, and/or the area of occupation of the rack on the basis of the results provided by the trained algorithm, processed by means of formulae and/or analytical algorithms, on the basis of mathematical relationships and/or predefined criteria.
A method according to any one of claims 5 to 6, wherein the preliminary training step comprises:
- defining a set of categories of recognizable items, each category being defined by type of item to be washed and type of material of the items to be washed;

- associating to each category a respective estimated volume;

- training the recognition algorithm to recognize the category of an item comprised in an acquired image, by means of a training carried out on the basis of a plurality of acquired test color images, corresponding to respective known basket filling situations, wherein the items of a known category and of a known volume and shape are placed in the rack;
and wherein the determining step comprises:
- identifying the number of items present in the basket and recognizing the category of each item, by processing the acquired color image by means of the trained algorithm;

- estimating the overall volume occupied by the present items on the basis of the number of items identified and the respective types recognized.
A method according to claim 7, wherein said determining step further comprises:
- estimating the volume occupied by each item category identified as the product of the number of items identified belonging to said category and the predefined volume associated with said category;

- estimating the overall volume occupied in the basket as the sum of the estimated volumes for each item category identified.
A method according to any one of the preceding claims, wherein said predefined set of recognizable item types comprises one or more of the following types:
glasses and/or crystal glasses with stem and/or plates and/or pots and/or trays and/or cups and/or cutlery and/or utensils and/or kitchenware; and wherein said predefined set of recognizable material types comprises one or more of the following types: glass and/or ceramic/porcelain and/or metals and/or plastic/silicone and/or wood,
and/or wherein, for each of said recognizable item types, a further recognition is made to identify the belonging of the item to a respective coded sub-type,

and/or wherein the method further comprises the step of identifying the number of items present in the rack and recognizing, for each item, the respective type and the type of material, by processing the acquired color image, and wherein the step of determining comprises determining, for each "item type - material type" pair, one or more of the following information:
- volume or percentage of volume occupied in the rack; and/or

- area or percentage of area occupied in the rack; and/or

- number of items to be washed, present in the rack; and/or

- position occupied in the rack defined by a predefined set of rack sub-areas;

and/or wherein the providing step comprises providing information organized in a two-dimensional matrix, wherein each column comprises a respective item type, from a predefined set of item types, and each row comprises a material type, from a predefined set of material types, and wherein a cell of the matrix contains as result, for the respective "item type - material type" pair, one or more of the following determined information:
- volume or percentage of volume occupied in the rack; and/or

- area or percentage of area occupied in the rack; and/or

- number of items to be washed, present in the rack; and/or

- position occupied in the basket defined by a predefined set of rack sub-areas,

wherein the matrix is adapted to provide results only on the cells corresponding to "item type - material type" combinations, predefined as significant.
A method according to any one of the preceding claims, wherein said acquiring step comprises acquiring a respective digital color image of each of at least two of said dishwasher racks (21, 22), an upper rack (21) and a lower rack (22), and wherein said determining step comprises determining, for each of the at least two racks, said at least one rack filling information, according to the features of the respective digital color image acquired, and wherein the recognition of the position of each item present in each of the racks of the dishwasher is provided,
wherein said predefined set of rack sub-areas which defines an identifiable position granularity comprises:
- for the upper dishwasher rack (21) a first number of upper rack sub-areas; and/or

- for the lower dishwasher rack (22) a second number of lower rack sub-areas;
and wherein the position of each item is expressed by indicating the lower or upper rack sub-area in which the item is identified,
and/or wherein the following is defined:
- for the upper dishwasher rack (21), three upper rack sub-areas (SA1, SA2, SA3), arranged side by side with respect to one another in the sense of a length of the upper rack, each corresponding to about one third the width of the rack;

- for the lower dishwasher rack (22), three lower rack sub-areas (SA4, SA5, SA6), comprising:
a first lower rack sub-area (SA6) which extends on the entire length of the lower basket and occupies a first half of the width of the lower rack;

a second lower rack sub-area (SA4) which occupies a part of the second half of the width of the lower rack and extends for about two thirds of the length of the lower rack;

a third lower rack sub-area (SA5) which occupies the remaining part of the second half of the width of the lower rack and extends for about a third of the length of the lower rack,

and/or wherein sub-area matrices are provided as a result, containing the considered filling level and load condition information, expressed for each sub-area.
A method according to any one of the preceding claims, wherein the determining step further comprises determining indications and/or recommendations regarding the arrangement of the items in the dishwasher, on the basis of the filling and load level information detected on the basis of the processing the digital image acquired,
and wherein the step of providing information to the user comprises:
- presenting in a numerical or graphical form the information detected, among said obtainable rack filling level information and rack load condition information; and/or

- also providing the user with written notifications containing said indications and/or recommendations regarding the arrangement of items in the dishwasher,
wherein said indications and/or recommendations provided to the user comprise:
- indication of load optimization between upper and lower dishwasher rack and recommendations for moving items from one rack to the other one; and/or

- indication of item position optimization within a rack and recommendations for moving items between different rack sub-areas.
A method according to any one of the preceding claims, wherein the steps of acquiring a digital color image and providing information to the user are carried out by means of a portable electronic device (3) equipped with digital image acquisition means and graphical interface, on the basis of a software application and/or program stored and executable in electronic processing means of the portable electronic device,
wherein the step of determining at least one rack filling level and/or rack load condition information is carried out by means of a processing performed by one or more remote software programs stored and executed in remote electronic processing means (4),
said remote electronic processing means (4) being operatively connected with the electronic processing means of the portable electronic device (30),
said one or more remote software programs being interoperable with the software application and/or program stored and executable in the portable electronic device (3).
A method for preparing and performing a dishwashing cycle by means of a dishwasher (2), in particular a domestic one, comprising the steps of:
- providing a dishwasher (2);

- arranging the kitchenware in the racks (21, 22) of the dishwasher (2);

- performing a method for determining filling information of a dishwasher according to any one of claims 1-12;

- defining a recommended washing program, on the basis of the results of the method for determining the dishwasher filling information;

- generating a washing preparation notification, which may be consulted by the user, wherein said washing preparation notification contains at least one washing instruction, comprising the washing parameters of the recommended washing program and also one or more confirmation requests and/or auxiliary washing preparation tips in addition to said washing instruction;
and wherein the step of providing information to the user comprises providing the user with said washing preparation notification, in addition to said rack filling level information and the rack load condition.
A method according to claim 13, comprising the steps of:
- requesting the user for a validation of the generated washing instruction, said validation comprising an intentional confirmation or modification of the generated washing instruction and an intentional confirmation to start the dishwashing cycle according to the generated or modified washing instruction,

- transmitting said generated or modified washing instruction to an electronic control unit of the dishwasher, by means of a transmission of electronic signals and bypassing a dishwasher user interface (23) for the manual selection of washing parameters, or by means of a manual command by means of the dishwasher user interface (23);

- performing the washing cycle by means of the dishwasher (2), commanding the dishwasher operation by means of the electronic control unit (20) according to the generated, or modified, washing instruction transmitted.
A system (1) for preparing and performing a dish washing cycle comprising:
- a dishwasher (2), configured to perform a dishwashing cycle, and comprising a dishwasher electronic control unit (20), configured to control the performance of a washing program associated with said washing cycle, on the basis of at least one washing instruction received;

- a portable electronic device (3), comprising:
- digital image acquisition means (31), configured to acquire a digital color image of at least one rack (21, 22) of the dishwasher containing items to be washed in a use condition;

- electronic processing means of the portable electronic device (30);

- a portable mobile device graphical interface (32), configured to provide a user with at least one rack filling information and/or a washing preparation notification containing at least one washing instruction relating to a recommended washing program; and/or at least one confirmation request and/or auxiliary washing preparation tips in addition to the washing instruction;

- remote electronic processing means (4), operatively connected to the portable electronic device (3), and configured to:
- receive (S7) from the portable electronic device (3) digital data corresponding to the acquired digital image;

- determine at least one rack filling information and/or a recommended washing program and/or auxiliary washing preparation tips, on the basis of a digital color image processing, carried out by one or more remote software programs stored and executed in remote electronic processing means;

- transmit (S8) to the portable electronic device (3) the at least one rack filling information and/or the recommended washing program and/or the auxiliary washing preparation tips determined;
wherein the electronic processing means of the portable electronic device (3) are configured to perform, by means of a device software application and/or program, the following steps:
- preparing a presentation for the user of said at least one rack filling information and/or recommended washing program and/or auxiliary washing preparation tips determined;

- receiving instructions from the user to validate the at least one determined washing instruction, corresponding to the recommended washing cycle which has been determined, or a washing instruction modified by the user;
wherein said at least one rack filling information comprises:
- a filling level, expressed in terms of volume, or percentage of volume, occupied in the rack,
and/or one or more of the following load condition information:
- volume, or percentage of volume, occupied in the rack, partitioned by type of item belonging to a predefined set of item types; and/or

- volume, or percentage of volume, occupied in the rack, partitioned by type of material belonging to a predefined set of material types which the items to be washed are made of; and/or

- area or percentage of area occupied in the rack; and/or

- number of items to be washed, present in the rack,
and/or

- number of items to be washed, present in the rack, partitioned by type of items belonging to a predefined set of item types; and/or

- position occupied in the rack by each item present in the rack, according to a granularity defined by a predefined set of rack sub-areas; and/or

- position of the cutlery rack, according to a granularity defined by a predefined set of rack sub-areas.
A system (1) according to claim 15, wherein the electronic processing means of the portable electronic device (3) are further configured to transmit (S10) said washing instruction determined, or modified by the user, to the electronic control unit (20) of the dishwasher (2), by means of a transmission of electronic signals and bypassing a dishwasher user interface (23) for the manual selection of washing parameters, or by means of a manual command by means of the dishwasher user interface (23),
and/or wherein said portable electronic device (3) is a smartphone or tablet, equipped with digital camera (31), and in which said software application or program of the portable electronic device (3) is stored, executable by the processing means of the smartphone or tablet.