KR20220156116A - Method and apparatus for monitoring alcohol intake using smart ring - Google Patents

Abstract

The present specification relates to a method for monitoring a drinking status of a user using a smart ring, as a method for monitoring a drinking status of a user using a smart ring, performed by a medical information management server, the method comprising the steps of: obtaining user motion information through a smart ring mounted on a user's finger; estimating a drinking status of the user by analyzing the user motion information; and transmitting the user motion information or estimated drinking status information to a medical institution server.

Description

Method and device for monitoring drinking state using smart ring {METHOD AND APPARATUS FOR MONITORING ALCOHOL INTAKE USING SMART RING}

Embodiments relate to a device and method for monitoring a drinking state of a user using a smart ring. More specifically, it relates to a method and apparatus for estimating a user's drinking state, such as drinking amount, based on user's motion information obtained using a smart ring, and monitoring the user's drinking state based on the estimated information.

The World Health Organization (WHO) has designated alcohol as a first-class carcinogen, and the International Agency for Research on Cancer (IRAC) under the World Health Organization has also designated alcohol, the main component of alcohol, and acetaldehyde, a by-product of alcohol, as a first-class carcinogen. warning of adverse effects.

The liver is the largest organ in our body, where many cells gather and supply the energy needed by our brain from the metabolism of various nutrients the body needs, and play a role in combining or detoxifying toxic substances. If there is an abnormality in the liver, various side effects such as jaundice or anemia appear, and it is an essential organ to maintain a healthy life.

The liver is known to be an organ that is particularly affected by alcohol consumption due to its ability to detoxify toxic substances. Alcoholic liver disease caused by alcohol can be classified into three types: alcoholic fatty liver, alcoholic hepatitis, and alcoholic cirrhosis. It is known that the risk of alcoholic liver disease is closely related to the amount of alcohol consumed.

Alcoholic fatty liver occurs mostly in heavy drinkers, and when alcohol is drunk frequently and excessively, alcoholic fatty liver occurs, and if it is severe, it causes alcoholic hepatitis or fibrosis, and if it gets worse, it causes alcoholic cirrhosis.

In view of the above, in order to reduce the negative effects of alcohol on the body, such as preventing alcoholic liver disease, even if drinking information is not input from the drinker, the drinking amount is estimated from the drinker's behavior and A method for estimating and monitoring drinking status is needed.

In the following, a method and device for monitoring a user's drinking state using a smart ring will be described.

Korean Patent Registration No. 10-1621774

An object of the present specification is to provide a drinking state monitoring method and device using a smart ring.

This specification monitors the user's drinking state by acquiring sensor information using a smart ring, estimating the user's drinking status, type of drinking, frequency of drinking, and amount of drinking, and providing an appropriate alarm when it is estimated that drinking is over a certain amount so that the user's It is an object of the present invention to provide a method and apparatus capable of controlling the amount of alcohol consumed.

An object of the present specification is to provide a method and apparatus for continuously monitoring and managing a user's health status by storing user's drinking information.

The problem to be solved in the present specification is not limited to the above, and can be extended to various matters that can be derived by the embodiments of the invention described below.

A method for monitoring a user's drinking state using a smart ring, performed by a medical information management server according to an embodiment of the present specification, includes the steps of acquiring user motion information through a smart ring mounted on a user's finger, the Analyzing user motion information to estimate a drinking state of the user, and transmitting the user motion information or the estimated drinking state information to a medical institution server.

In an embodiment, the estimating of the drinking state of the user may include estimating a shape of a cup held by the user, a movement path of the cup, a bending angle and an angular velocity of the cup based on the user motion information; and and estimating at least one of drinking status, type of drinking, frequency of drinking, and amount of drinking of the user based on the estimated shape of the cup, movement path of the cup, bending angle and angular velocity of the cup.

In one embodiment, the method may further include transmitting a request for generating an alarm of one or more of vibration, sound, and light to the smart ring when the drinking amount of the user is equal to or greater than a threshold value.

In an embodiment, the method may further include transmitting drinking amount information of the user to the medical institution when the drinking amount of the user is equal to or greater than a threshold value, and receiving health state information of the user from the medical institution. can

In one embodiment, the step of estimating the user's drinking state comprises measuring the EMG signal of the user's finger on which the smart ring is worn, and estimating the drinking amount of the user according to the degree of muscle contraction from the EMG signal. steps may be included.

In one embodiment, the smart ring includes one or more of a camera, an acceleration sensor, a temperature sensor, a bending sensor, a gyro sensor, a gravity sensor, and a speed sensor, and two or more are worn on one or both hands of the user. It can be.

In one embodiment, the user motion information is 3-dimensional motion information of the user's hand wearing the smart ring, which is calculated based on information about the location, tilt change, pressure, and distance of the smart ring in a 3-dimensional space. can include

As a program stored in a computer-readable recording medium according to an embodiment of the present specification, it may be composed of instructions for performing the above-described method.

The present specification may provide a method and apparatus for monitoring a drinking state of a user using a smart ring.

In the present specification, three-dimensional motion information of a user's hand wearing a smart ring is obtained using sensor information included in a smart ring, and from information collected through machine learning, the user's drinking status, drinking type, number of drinks, and amount of alcohol are determined. An estimation method can be provided.

The present specification may provide a method of estimating a user's drinking state, continuously monitoring the estimated information, and providing appropriate feedback through a smart ring for health management of the user.

The present specification can provide a method of estimating and monitoring the user's health condition due to drinking in real time by managing the collected drinking state information of the user and providing it to a medical institution.

It should be understood that the effects of the present specification are not limited to those described above, and can be extended to various contents that can be derived from the detailed description of the embodiments of the present invention below.

1 is a diagram illustrating an example of an operating environment of a system according to an embodiment of the present specification.
2 is a block diagram illustrating an internal configuration of a computing device 200 according to an embodiment of the present specification.
3 is a system diagram illustrating a method of collecting user motion information and monitoring a user's drinking state using a smart ring according to an embodiment of the present specification.
4 is a diagram illustrating a function of a smart ring for obtaining user motion information according to an embodiment of the present specification.
5A to 5D are diagrams illustrating an example of wearing a smart ring and a drinking operation according to various embodiments of the present specification.
6A to 6D are diagrams showing examples of shapes of hands holding various types of cups according to various embodiments of the present specification.
7 is a diagram illustrating a method of analyzing artificial intelligence-based smart ring acquisition information and estimating a user state according to an embodiment of the present specification.
8 is a flowchart illustrating a method of monitoring a drinking state of a user collected through a smart ring according to an embodiment of the present specification.
9 is a flowchart illustrating a specific method of estimating and managing a user's drinking amount using a smart ring according to an embodiment of the present specification.

In describing the embodiments of the present specification, if it is determined that a detailed description of a known configuration or function may obscure the gist of the embodiment of the present specification, the detailed description thereof will be omitted. And, in the drawings, parts not related to the description of the embodiments of the present specification are omitted, and similar reference numerals are attached to similar parts.

In the embodiments of the present specification, when a component is said to be "connected", "coupled" or "connected" with another component, this is not only a direct connection relationship, but also an indirect connection between which another component exists. It may also include a causal connection. In addition, when a component "includes" or "has" another component, this means that it may further include another component without excluding other components unless otherwise stated. .

In the embodiments of the present specification, terms such as first and second are used only for the purpose of distinguishing one component from another, and do not limit the order or importance of components unless otherwise specified. don't Therefore, within the scope of the embodiments herein, a first component in an embodiment may be referred to as a second component in another embodiment, and similarly, a second component in an embodiment may be referred to as a first component in another embodiment. can also be called

In the embodiments of the present specification, components that are distinguished from each other are intended to clearly describe each characteristic, and do not necessarily mean that the components are separated. That is, a plurality of components may be integrated to form a single hardware or software unit, or a single component may be distributed to form a plurality of hardware or software units. Therefore, even if not mentioned separately, such an integrated or distributed embodiment is also included in the scope of the embodiments of the present specification.

In this specification, a network may be a concept including both wired and wireless networks. In this case, the network may refer to a communication network through which data exchange between devices, systems, and devices may be performed, and is not limited to a specific network.

The embodiments described herein may have aspects that are entirely hardware, part hardware and part software, or entirely software. In this specification, “unit”, “apparatus” or “system” refers to computer-related entities such as hardware, a combination of hardware and software, or software. For example, in this specification, a unit, module, device, or system, etc., refers to a running process, processor, object, executable file, thread of execution, program, and/or computer. (computer), but is not limited thereto. For example, both an application running on a computer and a computer may correspond to parts, modules, devices, or systems of the present specification.

In addition, in the present specification, a device may be a fixed device such as a PC or a home appliance having a display function, as well as a mobile device such as a smart phone, a tablet PC, a wearable device, and a head mounted display (HMD). Also, as an example, the device may be an in-vehicle cluster or an Internet of Things (IoT) device. That is, in this specification, a device may refer to devices capable of operating an application, and is not limited to a specific type. In the following, for convenience of description, a device in which an application operates is referred to as a device.

In this specification, the communication method of the network is not limited, and connections between components may not be connected in the same network method. The network may include not only a communication method using a communication network (eg, a mobile communication network, wired Internet, wireless Internet, broadcasting network, satellite network, etc.), but also short-distance wireless communication between devices. For example, a network may include objects and all communication methods that objects may network, and is not limited to wired communication, wireless communication, 3G, 4G, 5G, or other methods. For example, a wired and/or network may include Local Area Network (LAN), Metropolitan Area Network (MAN), Global System for Mobile Network (GSM), Enhanced Data GSM Environment (EDGE), High Speed Downlink Packet Access (HSDPA), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Zigbee, Wi-Fi, Voice over VoIP Internet Protocol), LTE Advanced, IEEE802.16m, WirelessMAN-Advanced, HSPA+, 3GPP Long Term Evolution (LTE), Mobile WiMAX (IEEE 802.16e), UMB (formerly EV-DO Rev. C), Flash-OFDM, iBurst and It may refer to a communication network using one or more communication methods selected from the group consisting of MBWA (IEEE 802.20) systems, HIPERMAN, Beam-Division Multiple Access (BDMA), Wi-MAX (World Interoperability for Microwave Access), and ultrasonic communication. However, it is not limited thereto.

Components described in various embodiments do not necessarily mean essential components, and some may be optional components. Accordingly, embodiments composed of a subset of components described in the embodiments are also included in the scope of the embodiments of the present specification. In addition, embodiments including other components in addition to the components described in various embodiments are also included in the scope of the embodiments of the present specification.

Hereinafter, with reference to the drawings, look at the embodiments of the present specification in detail.

1 is a diagram illustrating an example of an operating environment of a system according to an embodiment of the present specification. Referring to FIG. 1 , a user device 110 and one or more servers 120 , 130 , and 140 are connected through a network 1 . 1 is an example for explanation of the invention, and the number of user devices or servers is not limited as shown in FIG. 1 .

The user device 110 may be a fixed terminal implemented as a computer system or a mobile terminal. The user device 110 includes, for example, a smart phone, a mobile phone, a navigation device, a computer, a laptop computer, a digital broadcast terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a tablet PC, and a game console. console), wearable device, smart ring, IoT (internet of things) device, VR (virtual reality) device, AR (augmented reality) device, and the like. As an example, in embodiments user device 110 is one of a variety of physical computer systems capable of communicating with other servers 120 - 140 over network 1 using substantially wireless or wired communication schemes. can mean

Each server may be implemented as a computer device or a plurality of computer devices that communicate with the user device 110 through the network 1 to provide commands, codes, files, contents, services, and the like. For example, the server may be a system that provides each service to the user device 110 connected through the network 1 . As a more specific example, the server may provide the user device 110 with a desired service (eg, information provision, etc.) through an application as a computer program installed and driven in the user device 110 . As another example, the server may distribute a file for installing and running the above-described application to the user device 110, receive user input information, and provide a corresponding service.

2 is a block diagram illustrating an internal configuration of a computing device 200 according to an embodiment of the present specification. This computing device 200 may be applied to one or more user devices 110-1 and 110-2 or servers 120-140 described above with reference to FIG. 1, and each device and server may add some components or By being configured except for, it may have the same or similar internal configuration.

Referring to FIG. 2 , a computing device 200 may include a memory 210 , a processor 220 , a communication module 230 and a transceiver 240 . The memory 210 is a non-temporary computer-readable recording medium, and is a non-perishable large-capacity memory such as RAM (random access memory), ROM (read only memory), disk drive, SSD (solid state drive), flash memory, and the like. A permanent mass storage device may be included. Here, a non-perishable mass storage device such as a ROM, SSD, flash memory, disk drive, etc. may be included in the above-described device or server as a separate permanent storage device distinct from the memory 210 . In addition, the memory 210 stores an operating system and at least one program code (for example, a browser installed and driven on the user device 110 or a code for an application installed on the user device 110 to provide a specific service). It can be. These software components may be loaded from a computer-readable recording medium separate from the memory 210 . The separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD/CD-ROM drive, and a memory card.

In another embodiment, software components may be loaded into the memory 210 through the communication module 230 rather than a computer-readable recording medium. For example, at least one program is a computer program installed by files provided by developers or a file distribution system (eg, the above-described server) that distributes installation files of applications through the network 1 (eg, a server). It may be loaded into the memory 210 based on the above-described application).

The processor 220 may be configured to process commands of a computer program by performing basic arithmetic, logic, and input/output operations. Instructions may be provided to processor 220 by memory 210 or communication module 230 . For example, processor 220 may be configured to execute received instructions according to program codes stored in a recording device such as memory 210 .

The communication module 230 may provide a function for the user device 110 and the servers 120 to 140 to communicate with each other through the network 1, and the device 110 and/or the servers 120 to 140, respectively. A function for communicating with other electronic devices may be provided.

The transceiver 240 may be a means for interface with an external input/output device (not shown). For example, the external input device may include devices such as a keyboard, mouse, microphone, and camera, and the external output device may include devices such as a display, a speaker, and a haptic feedback device.

Also, in other embodiments, the computing device 200 may include more components than those of FIG. 2 depending on the nature of the device to which it is applied. For example, when the computing device 200 is applied to the user device 110, it is implemented to include at least some of the above-described input/output devices, or a transceiver, a Global Positioning System (GPS) module, a camera, various sensors, It may further include other components such as databases and the like. As a more specific example, when the user device is a smartphone, various devices such as an acceleration sensor, a gyro sensor, a camera module, various physical buttons, buttons using a touch panel, input/output ports, and a vibrator for vibration are generally included in the smartphone. It may be implemented to include more components.

For example, a smart ring and a server (medical information management server, medical institution server) described below may operate based on the computing device 200 of FIG. 2 described above. That is, components of the computing device 200 may be implemented in the smart ring, and through this, the user's finger motion information, posture information, gaze information, muscle contraction information, and various other information may be obtained. Also, as an example, the medical information management server or medical institution server in the following may be a server or system that operates based on FIG. 1 described above. That is, the medical information management server or medical institution server may communicate with other devices through a network as shown in FIG. 1 . In addition, as an example, the smart ring may perform communication directly or through a user device with a medical information management server or a medical institution server through communication based on the above-described FIG. 1 . In the following, a method of estimating whether or not to drink, the type of drinking, the frequency of drinking, and the amount of drinking using the smart ring based on the above bar, and providing appropriate feedback for health management through the smart ring based on this information is described.

3 is a diagram illustrating a system in which a method of collecting user motion information and monitoring a user's drinking state by using a smart ring according to an embodiment of the present specification is performed.

Referring to FIG. 3 , a smart ring may be a device 310-1 or 310-2 including various sensors mounted on a user's hand. Here, as an example, the smart ring may be each individual device. That is, each smart ring may acquire user motion information and transmit it to the medical information management server 10 directly or through an intermediate router. Here, the medical information management server 10 may be a system or server capable of communicating with a smart ring as a computing device based on FIG. 2 described above. In addition, as another example, the smart ring can transmit motion information collected through various sensors to a user device such as a smart phone or tablet PC, and the user device collects the information to the medical information management server 10 through the network 1. information can be transmitted. The user device may also be a computing device based on FIG. 2 and may control the operation of the smart ring through an application installed on the user device. For example, the operation mode of the smart ring may be changed through an application, or power on/off control, change of collected information, and the like may be controlled.

In addition, the medical information management server 10 may obtain 3D motion information of the user's hand by analyzing the sensor information received through the smart ring, and use the 3D motion information of the user's hand to determine whether or not to drink, the type of alcohol, Information such as the amount or frequency of drinking can be estimated. The medical information management server 10 may transmit the estimated drinking information of the user to the medical institution server 320 . In another embodiment, together with or separately from the estimated drinking information of the user, sensor information of the smart ring or 3D motion information of the hand (eg, data before user drinking information estimation) may be transmitted to the medical institution server 320 .

In one embodiment, the smart ring may be a set of two devices 310-1 and 310-2. That is, the smart ring may be a device composed of two devices 310-1 and 310-2 and implemented through interworking. For example, the two devices 310-1 and 310-2 of the smart ring may be the first smart ring 310-1 and the second smart ring 310-2, respectively, but are limited to the above-mentioned names or numbers. It doesn't work.

For example, each of the first smart ring 310-1 and the second smart ring 310-2 may include a plurality of input units and output units, which will be described later.

At this time, each of the first smart ring 310-1 and the second smart ring 310-2 may obtain various information related to the user's hand motion through a sensor. For example, each of the first smart ring 310-1 and the second smart ring 310-2 may obtain motion information about the user's hand. The motion information of the user's hand can be calculated from the position and tilt change of the smart ring in the 3D space, pressure, and relative distance information.

As a specific example, when the user's hand motion information is acquired through only one smart ring, there may be a limit to the accuracy of the information. In consideration of the above points, each of the first smart ring 310-1 and the second smart ring 310-2 may be mounted on a user's finger to obtain motion information. The server 10 may calculate the distance between the smart rings 310-1 and 310-2 by comparing the acquired motion information and estimate a specific motion type of the finger. Here, the number of smart rings is not limited to two and may be three or more.

For example, the first smart ring 310-1 may be fitted to the index finger of the user's right hand, and the second smart ring 310-2 may be fitted to the middle finger of the user's right hand. Alternatively, the second smart ring 310-2 is inserted into the index finger of the user's right hand, and the first smart ring 310-1 and the second smart ring 310-2 may be fitted to different joints, but are limited thereto It is not.

In this way, by wearing a plurality of smart rings, it is possible to obtain more accurate information about finger movements. For example, a hand gesture may be identified by using not only acceleration information of each of two or more smart rings fitted in each hand, but also relative distance information between each smart ring. This identification may be implemented by server 10 .

As a specific example, when a user holds a cup in his hand and performs a drinking action, the user moves the cup up and down or tilts it while holding the cup with one hand (with his or her fingers or with the entirety of the fingers and palm). judgment can be made. At this time, the first smart ring 310-1 and the second smart ring 310-1 worn on different fingers (eg, thumb and index finger) or one finger (eg, only the index finger) on one hand. 2) The size of the cup held can be estimated using acceleration information and relative distance through mutual interworking. When the size of the estimated cup is less than or equal to a predetermined value, it may be determined that the cup is soju or liquor, and it may be estimated that soju or liquor is being drunk. The size of the estimated cup may be subdivided and determined to be a liquor cup when the size is equal to or less than the first threshold, and determined to be a soju cup when the size is greater than the first threshold but equal to or less than the second threshold.

For example, from a user wearing the first smart ring 310-1 and the second smart ring 310-2 on one or both hands, respectively, when the user drinks water, beer, or soju Depending on the type of drinking, such as cases, three-dimensional motion information such as an acceleration sensor is acquired to collect training data, and a machine learning model for measuring drinking can be trained. Machine learning can be used in combination with supervised learning, unsupervised learning, and reinforcement learning. Based on the learning, the three-dimensional motion information obtained from the first smart ring 310-1 and the second smart ring 310-2 mounted on the user's finger is input into the machine learning model to estimate whether or not drinking is done, and drinking number of times can be counted.

When learning the above-described machine learning model, various factors may be considered. For example, the angle and angular velocity at which the first smart ring 310-1 and/or the second smart ring 310-2 are bent around the mouth can be measured through an acceleration sensor, etc., and each value is weighted can be granted For example, since the bending angle and angular velocity will be different when drinking water (negative number) and when drinking alcohol (drinking), it can be estimated that the greater the angle and the faster the angular velocity, the more drinking. accuracy can be further improved. In addition, for example, if it is moved to or bent around the mouth after a specific shock (vibration), it can be evaluated as having a higher probability of drinking than a negative number.

As another example, drinking can be estimated through the time holding the glass. For example, if the time holding the glass is measured to be more than a predetermined time, the probability of being estimated as drinking alcohol rather than water can be increased. In addition, when the user's gaze directs the glass, the probability that it is estimated as alcohol rather than water may be given a high weight. According to the experimental results, in the case of alcohol, there is a tendency to stare at the alcohol in the glass before drinking, so based on the learning result, the probability that this case is judged as alcohol may increase. Whether or not the eyes are directed toward the cup can be estimated by measuring a change in the angle of the neck by linking a smart ring with a sensor attached to an accessory (eg, earring, glasses, hat, etc.) mounted on the face/head. Alternatively, when a camera sensor is included in the smart ring and the user's eyes are measured to be directed toward the smart ring through image analysis, it may be determined that the user's gaze is directed toward the cup.

For example, if two or more smart rings are worn on two or more fingers of one hand or one finger of one hand, the size of the cup held can be estimated. In this case, it is judged as soju or liquor, and the amount and frequency of drinking can be estimated by estimating the soju or liquor drinking situation. At this time, the shape of the gripped hand can be estimated based on the relative distance between the two smart rings, and for example, the size of the gripped cup, such as the diameter, can be estimated by measuring the degree of bending and bending of the fingers. Based on the estimated size of the cup, it may be determined as a liquor cup if it is less than or equal to the first threshold, and if it is greater than the first threshold but less than or equal to the second threshold, it may be determined as a soju cup.

As another example, the smart rings 310-1 and 310-2 may include EMG signal measurement sensors (electrodes) on their surfaces, and measure electrical signals (EMG signals) generated from the fingers before and after the user drinks alcohol. It can measure the degree of contraction (tension) of the muscle. For example, the lower the degree of muscle contraction is measured, the higher the amount of drinking may be measured, and the accuracy of estimating the amount of drinking according to the degree of muscle contraction can be further increased through machine learning.

Also, as an example, the operation information of the user obtained through the smart rings 310-1 and 310-2 as described above may be transmitted to the server 10. At this time, the server 10 monitors the user's motion information received from the smart rings 310-1 and 310-2 in real time, analyzes the monitored user's motion information, and determines one of drinking status, drinking type, drinking amount, and number of times of drinking. more can be inferred. Analysis of the server 10 may be performed based on machine learning, and the server 10 may transmit the analyzed drinking information of the user to the medical institution server 320 .

The medical institution server 320 checks the user's drinking information and outputs the user's health information (eg, liver disease information, various cancer occurrence probability information, etc.) according to mechanical analysis or clinical analysis by medical personnel. can do. In other words, based on the information obtained from the smart ring (inclination of each smart ring, temperature, pressure, moving distance, distance between each other, etc.), the user's drinking status (drinking status, type of drinking, amount of drinking, frequency of drinking and estimated alcohol intake, etc. ) can be determined. The medical institution server 320 or the medical information management server 10 may set the user's drinking threshold based on the user's health information stored, and monitor the user's drinking amount and number of times of drinking in real time so that the user drinks more than the threshold amount. You can provide feedback so that you don't.

For example, the medical institution server 320 or the medical information management server 10 analyzes the user's operation information obtained from the smart rings 310-1 and 310-2, and the drinking information is stored in the user device (user's smartphone). Alternatively, it may be transmitted to a health care service account to which the user has subscribed) or feedback corresponding to drinking information may be provided. For example, the medical information management server 10 (or received from the medical institution server 320) is a smart ring when the user's drinking frequency is greater than or equal to a predetermined number within a predetermined time and/or the drinking amount is estimated to be greater than or equal to a threshold amount. may provide one or more alarms of vibration, sound, and light. For example, the smart ring can generate blue light emission or vibration stimulation, which is a signal to drink water. In addition, the smart ring can provide feedback to refrain from drinking, such as generating both sound and vibration stimulation to stop drinking. The medical information management server 10 transmits a request to call a medical person (specialist, etc.) to the medical institution server 320 or send an ambulance to the medical institution server 320 when it is determined that the user's health condition requires treatment due to the user's drinking state. can The medical information management server 10 may receive information on clinical judgment from the medical institution server 320 .

In another example, the smart rings 310-1 and 310-2 communicate with the user device and communicate with the medical information management server 10 through the user device to transmit user operation information and receive related feedback. have. While the application is running, the user device can receive the user's drinking state information and related feedback information from the medical information management server 10, and the user's health condition is determined to be a state in need of treatment from the medical information management server 10 In this case, user location information may be transmitted to the medical information management server 10 or the medical institution server 320 or a phone call may be received.

4 illustrates a function of a smart ring for acquiring motion information of a user according to an embodiment of the present specification.

For example, the smart ring may perform various functions in order to perform the above-described operation. Specifically, the smart ring may have a wireless communication function in order to interwork with the aforementioned medical institution server or user device. That is, the smart ring may transmit information acquired from the user to a system or other device through wireless communication. Also, the smart ring may store information acquired from the user. For example, the smart ring may store biometric information (or sensor information or user motion information) obtained from a user for a certain period of time and transmit the stored information to a system or other device based on a certain period, and is limited to the above-described embodiment. It doesn't work. In addition, the smart ring may include a camera, an acceleration sensor, a temperature sensor, a bending sensor, a gyro sensor, a gravity sensor, a speed sensor, an EMG signal measurement sensor, and other various sensors for sensing the user's movement. Operation information of can be obtained. Also, as an example, the smart ring may obtain the user's heart rate or other information using a pressure sensor, and may have a waterproof function. In addition, the smart ring may operate based on a charging method, and through this, the above-described operations may be performed. Also, as an example, the smart ring itself may include a display unit and display information being measured based on the display unit, and is not limited to the above-described embodiment.

For example, the medical information management server 10 may estimate the user's finger posture in a 3D space by analyzing the value of the acceleration sensor of the smart ring. In this case, the estimation accuracy of the finger posture may be improved by further using distance information between the smart rings. For example, the relative distance information between smart rings fitted to the user's finger and the acceleration value of each smart ring can be used to estimate the finger posture, and through this, the size of the cup held can be estimated. Based on this, the medical information management server 10 may determine the user's drinking type.

In addition, the medical information management server 10 may store the user's personal information in a database (not shown). Here, 'personal information' may include, but is not limited to, one or more of age, gender, place of residence, family history of a specific disease, chronic disease, family composition, prescription data, medical records, and liver health information. Meanwhile, the medical information management server 10 may acquire information such as a user's medical record through the medical institution server 320 with the consent of the user.

The medical institution server 320 may obtain user drinking information from the medical information management server 10 and generate clinical determination information based on the obtained user drinking information. The clinical determination information may be information input by a medical person based on user drinking information or a learning result output using user drinking information as an input by a specific learning module. The above learning module may be configured in the medical institution server 320 or the medical information management server 10 .

5A to 5D show an example of wearing a smart ring and a drinking operation according to various embodiments of the present specification.

For example, referring to FIG. 5A , the user may wear the first smart ring 510-1 on the index finger and the second smart ring 510-2 on the middle finger. The size of the cup held by the user depends on the relative positions of the first smart ring 510-1 and the second smart ring 510-2 and the first smart ring 510-1 and the second smart ring 510-2. It can be estimated using the degree of bending of the finger through the pressure sensor information for the finger. Since it is common to drink alcohol from different glasses depending on the type, the type of alcohol can be estimated from the measured size of the glass.

In addition, as an example, as shown in FIG. 5A, when a user holds a glass and drinks alcohol, unlike the case of drinking water, the glass tends to be greatly bent around the mouth, and drinking with a small glass such as a soju glass or a liquor glass In this case, this tendency is more observed. Therefore, each value is weighted for the bending angle and angular velocity around the mouth measured through the first smart ring 510-1 and/or the second smart ring 510-2, and the angle is greatly bent and the angular velocity It can be assumed that the faster you go, the more you drink.

As another example, referring to FIG. 5B , the user may wear the first smart ring 510-1 on his ring finger and the second smart ring 510-2 on his little finger. The size of the cup held by the user may be estimated using the degree of bending of the finger through pressure sensor information on the first smart ring 510-1 and the second smart ring 510-2. Since it is common to drink alcohol from different glasses depending on the type, the type of alcohol can be estimated from the measured size of the glass.

For example, as shown in FIG. 5B , when a user drinks alcohol after lifting a glass and colliding with another glass, the shock sensor of the first smart ring 510-1 and/or the second smart ring 510-2 After a specific shock (vibration) is sensed through, a motion of moving or bending around the mouth can be measured. In this case, it may be evaluated that the probability of drinking is higher than that of a negative number.

As another example, the user may wear the first smart ring 510-1 and the second smart ring 510-2 on the same finger. For example, referring to FIG. 5C , the user may wear both the first smart ring 510-1 and the second smart ring 510-2 on the index finger. The size of the cup held by the user may be estimated using the degree of bending of the finger through pressure sensor information on the first smart ring 510-1 and the second smart ring 510-2. Since it is common to drink alcohol from different glasses depending on the type, the type of alcohol can be estimated from the measured size of the glass.

For example, drinking can be estimated through the time holding a glass. For example, if the time of holding the glass is measured to be more than a predetermined time, the probability of being estimated as drinking alcohol rather than water can be increased. As shown in FIG. 5C , when the user's gaze is directed toward the glass, a probability that it is estimated as alcohol rather than water may be given a high weight. According to the experimental results, in the case of alcohol, there is a tendency to stare at the alcohol in the glass before drinking, so based on the learning result, the probability that this case is judged as alcohol may increase. Whether or not the gaze is directed toward the cup can be estimated by measuring the change in angle of the neck by linking the smart ring with a sensor attached to an accessory (eg, earring, glasses, hat, etc.) mounted on the face and head. When the user's gaze stays toward the glass for a certain period of time and then moves to or bends around the mouth, a weight can be given so that the probability of drinking is higher than that of a negative number.

Also, for example, referring to FIG. 5D , the user may wear the first smart ring 510-1 on his right finger and the second smart ring 510-2 on his left finger. Both the first smart ring 510-1 and the second smart ring 510-2 can be worn on the index fingers of both hands. The size of the cup held by the user is the distance between the first smart ring 510-1 and the second smart ring 510-2 and the first smart ring 510-1 and the second smart ring 510-2. It can be estimated using the degree of bending of the finger through pressure sensor information and the like for . Since it is common to drink alcohol from different glasses depending on the type, the type of alcohol can be estimated from the measured size of the glass.

As shown in FIG. 5D , when a user holds a glass with both hands and drinks, the glass tends to bend greatly around the mouth, unlike when drinking water, and when drinking with a small glass such as a soju glass or a liquor glass A trend is more observed. Therefore, each value is weighted for the bending angle and angular velocity around the mouth measured through the first smart ring 510-1 and/or the second smart ring 510-2, and the angle is greatly bent and the angular velocity It can be assumed that the faster you go, the more you drink.

In the above example, the number of smart rings is not limited to two, but may be three or more, and the worn position is not limited.

6A to 6D are diagrams showing examples of shapes of hands holding various types of cups according to various embodiments of the present specification. Depending on the type of alcohol, the shape of the glass changes, and as the shape of the glass changes, the shape of the hand holding the daughter also changes.

For example, referring to FIG. 6A , in a state where the first smart ring 610-1 is worn on the user's thumb and the second smart ring 610-2 is worn on the middle finger, the user holds a beer mug with a handle. It could be holding and drinking. Sensors such as pressure sensors and gravity sensors for the relative positions of the first smart ring 610-1 and the second smart ring 610-2 and the first smart ring 610-1 and the second smart ring 610-2 The user's holding state, the shape and weight of the cup, etc. can be estimated using the degree of bending of the finger and the pressing of the finger by the cup using information obtained from the cup. Since it is common to drink alcohol from different glasses according to the type, the type of alcohol can be estimated from the measured user's holding state, size and weight of the glass, and in the illustrated example, it is judged as a beer glass and estimated as a beer drinking situation. can

As another example, referring to FIG. 6B , the first smart ring 610-1 is worn on the thumb of the first user, and the second smart ring 610-2 is worn on the index finger of the first user. 1 The user may drink while holding a wine glass. In addition, while the third smart ring 610-3 is worn on the second user's thumb and the fourth smart ring 610-4 is worn on the second user's index finger, the second user also holds a wine glass and drinks. may be the case. Sensors such as pressure sensors and gravity sensors for the relative positions of the first smart ring 610-1 and the second smart ring 610-2 and the first smart ring 610-1 and the second smart ring 610-2 The holding state of the first user, the shape and weight of the cup, etc. may be estimated using the degree of bending of the finger and the pressing of the finger by the cup using information obtained from the cup. The relative positions of the third and fourth smart rings 610-3 and 610-4 and sensors such as pressure sensors and gravity sensors for the third and fourth smart rings 610-3 and 610-4 The holding state of the second user, the shape of the cup, the weight of the cup, etc. may be estimated using the degree of bending of the finger and the pressing of the finger by the cup using information obtained from the cup. The first smart ring 610-1, the second smart ring 610-2, the third smart ring 610-3, and the fourth smart ring 610-4 are within a certain distance (eg, For example, when located at a distance where Bluetooth communication is possible, etc.), the motion measurement information of the first user and the second user can be shared with each other. type can be inferred. In the illustrated example, it may be determined as a wine glass and may be estimated as a wine drinking situation.

As another example, referring to FIG. 6C , in a state where the first smart ring 610-1 is worn on the user's thumb and the second smart ring 610-2 is worn on the index finger, the user holds a glass of makgeolli and drinks it. may be the case. Sensors such as pressure sensors and gravity sensors for the relative positions of the first smart ring 610-1 and the second smart ring 610-2 and the first smart ring 610-1 and the second smart ring 610-2 The user's holding state, the size and weight of the cup, and the like can be estimated using the degree of bending of the finger and the pressing of the finger by the cup using information obtained from the cup. Since it is common for alcohol to be drunk from different glasses depending on the type, the type of alcohol can be estimated from the measured user's holding state and the size and weight of the glass. can

As another example, referring to FIG. 6D , while the first smart ring 610-1 is worn on the user's index finger and the second smart ring 610-2 is worn on the ring finger, the user holds a wine glass and drinks. may be the case. Sensors such as pressure sensors and gravity sensors for the relative positions of the first smart ring 610-1 and the second smart ring 610-2 and the first smart ring 610-1 and the second smart ring 610-2 The user's holding state, the size and weight of the cup, and the like can be estimated using the degree of bending of the finger and the pressing of the finger by the cup using information obtained from the cup. Since it is common for alcohol to be drunk from different glasses according to the type, the type of alcohol can be estimated from the measured user's holding state and the size and weight of the glass. can In particular, in the case of drinking soju or liquor with a small size of the glass, if the angle and angular velocity of the glass bending around the mouth are above a certain level, it can be estimated that the drinking situation is soju or liquor, and a weight can be assigned to this.

7 is a diagram illustrating a method of analyzing artificial intelligence-based smart ring acquisition information and estimating a user state according to an embodiment of the present specification. In one embodiment of the present specification, it is a diagram showing a method of utilizing artificial intelligence-based smart ring acquisition information. Such artificial intelligence-based utilization may be executed by the medical information management server 10.

Referring to FIG. 7 , the medical information management server 10 may output user state information based on artificial intelligence by using user motion information acquired through a smart ring as an input. A learning model for user motion information based on artificial intelligence may be implemented internally or externally in the medical information management server 10 . As an example, the motion information may be information about a user obtained through a sensor attached to a smart ring as described above. In this case, the user motion information may be analyzed through the learning model as input information of the learning model. As a specific example, the learning model may check whether the user has gripped the cup, the user's finger gesture, finger bending, hand bending and acceleration based on the input information.

In one embodiment, the server 10 inputs the user's motion information as input information to a motion information learning model, and from the motion information learning model, the user's state, that is, at least one of drinking status, drinking type, drinking amount, and number of times of drinking. can be output as output information.

In one example, the activity information learning model may output information on at least one of whether a cup is held, the size and weight of the cup held, and a hand gesture type as output information based on the user motion information. In addition, in order to improve learning accuracy of the learning model, user activity information input to the activity information learning model may be pre-processed before being input. For example, user motion information acquired from the smart ring while the user is moving to a location for a predetermined period of time may be excluded. This is because, for example, focusing on data of a time interval in which the user's hand moves to hold the cup and a time interval in which the user moves after holding the cup can improve the accuracy of hand gesture identification.

However, this is an example, and data during this period may be useful in some cases.

For example, the smart ring may obtain user motion information about a finger motion based on at least one of an ultrasonic sensor, an acceleration sensor, a temperature sensor, and a bending sensor. At this time, the learning model of the system analyzes the user's motion information for the finger motion based on deep learning to determine whether the cup is gripped, whether the cup is gripped with the size and weight of the cup, or the type of cup held. . For example, information about finger pressure, angle, relative position between a plurality of smart rings, etc. when holding a soju glass can be recognized through a learning model analyzed through deep learning. Then, based on the motion information from the smart ring, the learning model determines the movement of the cup after holding the cup, whether or not an impact is applied to the cup, whether the cup stays in a certain position for a considerable time, and the angle at the height around the mouth. Based on whether or not is bent rapidly and at this time, whether the user is drinking based on the rotational angular velocity of the hand, and in case of drinking, information related to the type of alcohol, the amount of alcohol consumed, the number of times of drinking, etc. can

Then, based on the learning model, the drinking state of the user may be confirmed and analyzed. For example, the type of alcohol being drunk and the amount of alcohol consumed at one time may be estimated based on the shape and weight of the glass through the learning model. The number of drinks can be determined based on the angle and angular velocity of the mouth when the hand moves up and down, and the amount of alcohol can be analyzed in real time based on the amount and number of drinks drunk at one time, and this can be provided as an output value.

An EMG signal collected through an EMG signal measurement sensor (electrode) included on the surface of the smart ring can be input as an input value of the learning model. Experimentally, the degree of muscle contraction (tension) tends to be lower after the user drinks than before drinking. It is possible to estimate the amount of drinking according to the degree of muscle contraction through machine learning by measuring the electrical signal (electromagnetic signal) generated from the finger wearing the smart ring.

Meanwhile, as an example, an output value obtained through the learning model as described above may be fed back to the learning model. That is, the learning model continuously updates based on input information acquired through the smart ring and output information output based thereon, and based on this, the accuracy of state analysis can be increased.

The medical information management server 10 monitors the user's drinking state in real time based on the user's drinking state analysis result value that is output, and if it is determined that the user's drinking amount (or number of drinking times) exceeds the threshold value, the smart ring A notification may be sent and an alarm (eg, light, sound, vibration, etc.) may be generated to the user through the smart ring. In another example, the medical information management server 10 may transmit the user's drinking state analysis result output to the medical institution server 320 . The above-described condition analysis and transmission of results may be performed on a real-time basis, and the medical institution server 320 provides a clinical judgment based on the user's drinking condition analysis result value, allows a medical person to directly contact the user, or tracks the user's location. You can send an ambulance.

As another example, information obtained based on the above-described learning model may be transmitted to the medical information management server 10 or the medical institution server 320 . Here, the medical information management server 10 or the medical institution server 320 may generate management information by analyzing and evaluating information thereon. At this time, the generated management information may be provided to the user or fed back to the above-described learning model. For example, management information that is fed back may include, but is not limited to, alcoholic liver disease risk factor discovery or findings and various cancer risk factors determined based on user condition analysis information.

Through the feedback described above, the above-described learning model can be continuously updated, and based on this, the accuracy and utilization of pattern analysis can be increased.

8 is a flowchart illustrating a method of monitoring a drinking state of a user through a smart ring according to an embodiment of the present specification. This method may be implemented by the medical information management server 10 described above.

Referring to FIG. 8 , the method for monitoring the user's drinking state through the smart ring, implemented by the medical information management server 10, includes obtaining user motion information through the smart ring (S810), It may include estimating the user's drinking state by analyzing the user's motion information (S820) and transmitting the user's motion information or the user's drinking state estimation information to the medical institution server (S830).

In one embodiment, the step of estimating the drinking state (S820) is the shape of the cup held by the user and the angle at which the cup is bent using user motion information obtained through a plurality of smart rings worn on one or both hands, respectively. and estimating the angular velocity, presence or absence of shock in the cup, and the moving path of the cup, and estimating whether or not to drink, the type of drink, the number of drinks and the amount of drink based on the estimated information. The shape of the cup may include the size, type, and weight of the cup, and may include an operation of estimating the user's drinking amount through user motion information through various sensor information collected from the above-described smart ring.

9 is a flowchart illustrating a specific method of estimating and managing a drinking amount of a user using a smart ring according to an embodiment of the present specification.

9 is a flowchart illustrating a specific method of estimating, monitoring, and feeding back a user's drinking amount using a smart ring according to an embodiment of the present specification. This method may be implemented by the medical information management server 10 described above. Referring to FIG. 9 , a method for estimating and managing a user's drinking amount through a smart ring, implemented by the medical information management server 10, includes acquiring user motion information through the smart ring (S910), Analyzing the user motion information to estimate the user's drinking amount (S920), determining whether the drinking amount exceeds the threshold (S930), and sending an alarm request to the smart ring if the drinking amount exceeds the threshold (S940) can include

In one embodiment, the step of estimating the user's drinking amount (S920) is the shape of the cup held by the user and the angle at which the cup is bent using user motion information obtained through a plurality of smart rings worn on one or both hands, respectively. And angular velocity, presence or absence of shock in the cup, and the moving path of the cup can be estimated, and based on the estimated information, the amount of drinking (or the number of drinking) can be estimated based on determining whether or not to drink and the type of drinking. The shape of the cup may include the size, type, and weight of the cup, and may include an operation of estimating the user's drinking amount through user motion information through various sensor information collected from the above-described smart ring. The drinking amount may include the amount of alcohol calculated based on the type of drinking, the size of the cup and the number of cups drunk, the amount displayed in ml or l, or the number of times the user drinks. The amount of drinking may be estimated cumulatively as a result of analyzing the user's motion information obtained through the smart ring.

In the step of determining whether the drinking amount exceeds the threshold value (S930) according to an embodiment, if the drinking amount does not exceed the threshold value, return to the step of obtaining user motion information through the smart ring (S910) and again the drinking amount and drinking amount according to an embodiment. Counting and monitoring will continue. When the drinking amount exceeds the threshold value, an alarm generation request is transmitted to the smart ring of the medical information management server 10 (S940). The smart ring may generate a warning alarm to the user in a manner such as light, sound, or vibration, and is not necessarily limited thereto, and various modifications are possible. Alternatively, in another example, the medical information management server 10 transmits related information (the user motion information or the user drinking state estimation information) to the medical institution server 320 when the drinking amount exceeds the threshold, so as to make a clinical judgment of the medical staff. You can request actions such as receiving a call from a medical staff or sending an ambulance. The drinking amount information transmitted to the medical institution server 320 is managed as health information of the user and can be used to evaluate and manage the risk of related diseases such as alcoholic liver disease and various cancers. The threshold may be set to, for example, 6 glasses of soju or 500ml of beer through clinical data or experience, and may be individually set differently in consideration of individual health conditions.

The embodiments described above may be at least partially implemented as a computer program and recorded on a computer-readable recording medium. Computer-readable recording media on which programs for implementing the embodiments are recorded include all types of recording devices in which computer-readable data is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, and optical data storage devices. In addition, computer-readable recording media may be distributed in computer systems connected through a network, and computer-readable codes may be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing this embodiment can be easily understood by those skilled in the art to which this embodiment belongs.

The present specification reviewed above has been described with reference to the embodiments shown in the drawings, but this is only exemplary, and those skilled in the art will understand that various modifications and variations of the embodiments are possible therefrom. However, such modifications should be considered within the technical protection scope of the present specification. Therefore, the true technical protection scope of the present specification should be determined to include other implementations, other embodiments, and equivalents to the claims by the technical spirit of the appended claims.

Claims (8)

As a method of monitoring a user's drinking state using a smart ring, which is performed by a medical information management server,
Acquiring user motion information through a smart ring mounted on a user's finger;
estimating a drinking state of the user by analyzing the user motion information; and
A method for monitoring a user's drinking state using a smart ring, comprising transmitting the user's motion information or the estimated drinking state information to a medical institution server.
According to claim 1,
Estimating the drinking state of the user,
estimating a shape of the cup held by the user, a moving path of the cup, a bending angle, and an angular velocity of the cup based on the user motion information; and
and estimating at least one of drinking status, type of drinking, frequency of drinking, and amount of drinking of the user based on the estimated shape of the cup, movement path of the cup, bending angle and angular velocity of the cup. A method of monitoring a user's drinking status using the ring.
According to claim 2,
Monitoring the user's drinking state using the smart ring, further comprising the step of transmitting a request for generating an alarm of one or more of vibration, sound, and light to the smart ring when the drinking amount of the user is greater than or equal to the threshold value. Way.
According to claim 2,
transmitting information on the drinking amount of the user to the medical institution when the drinking amount of the user is equal to or greater than the threshold value; and
The method of monitoring the user's drinking state using a smart ring, characterized in that it further comprises the step of receiving the user's health state information from the medical institution.
According to claim 1,
Estimating the drinking state of the user,
Measuring the EMG signal of the user's finger on which the smart ring is worn, and estimating the drinking amount of the user according to the degree of muscle contraction from the EMG signal, characterized in that it comprises the step of estimating the drinking state of the user using the smart ring. How to monitor.
According to claim 1,
The smart ring,
A user using a smart ring that includes one or more of a camera, an acceleration sensor, a temperature sensor, a bending sensor, a gyro sensor, a gravity sensor, and a speed sensor and is worn on one or both hands of the user. How to monitor your drinking status.
According to claim 1,
The user action information,
Using a smart ring, characterized in that it includes 3-dimensional motion information of the user's hand wearing the smart ring, which is calculated based on the position, tilt change, pressure, and distance information of the smart ring in a 3-dimensional space. How to monitor user's drinking status.
A computer program stored in a computer-readable recording medium combined with hardware to perform the method according to any one of claims 1 to 7.

Images