KR101778548B1

KR101778548B1 - Conference management method and system of voice understanding and hearing aid supporting for hearing-impaired person

Info

Publication number: KR101778548B1
Application number: KR1020150160435A
Authority: KR
Inventors: 윤재민
Original assignee: 얄리주식회사
Priority date: 2015-11-16
Filing date: 2015-11-16
Publication date: 2017-09-18
Also published as: KR20170056942A

Abstract

The present invention relates to a conference management method and system, and more particularly, to a conference management method capable of supporting speech recognition and hearing support for the hearing-impaired person to facilitate the hearing- And a system.
According to another aspect of the present invention, there is provided a conference management method capable of supporting speech recognition and hearing support for a hearing impaired person using a conference management server in a conference including a hearing impaired person, comprising: (a) Receiving a conference attendance request data signal from a user terminal using the NFC tag attached in the conference hall and transmitting a conference attendance response data signal to the terminal; (b) after the step (a), the conference management server receives the speech data from the user terminal, recognizes the received speech data, and stores the speech-recognized text data; (c) transmitting the stored speech-recognized text data to the user terminal by the conference management server; And (d) creating a meeting record in the conference management server with the voice recognized text data stored in step (b), and transmitting the meeting minutes response data corresponding to the meeting minutes request data signal from the user terminal, And transmitting the created minutes to the user terminal so that the requested minutes are displayed.

Description

TECHNICAL FIELD The present invention relates to a conference management method and system for voice recognition and hearing aid for the hearing impaired,

The present invention relates to a conference management method and system, and more particularly, to a conference management method capable of supporting speech recognition and hearing support for the hearing-impaired person to facilitate the hearing- And a system.

People often think they can imitate hearing impairment by clasping their ears with hands, but this means that the hearing impairment is reduced in size and that they can not hear a specific frequency of sound. These frequencies can not be regenerated because they disappear regardless of amplification.

In general, hearing aids used by people with hearing impairments are simply amplifiers that make them louder, making them the perfect solution to hearing impairment. The hearing aids can not distinguish between sounds that should be heard louder and those that should be reduced, but they also amplify the sounds of the surroundings as well as the words to be heard.

People with perceptual problems like most overlapping hearing impaired people do not transmit sound impulses to the brain because of damage to the auditory nerves. In the case of mild hearing loss or some severe hearing loss, it is effective when using a hearing aid. However, in the case of moderate hearing loss, consonants and vowels are hardly distinguished in the high frequency range, so it is possible to hear the sound, but it can not be understood. To solve this problem, it is found that the speech recognizer is in need of help.

On the other hand, the meetings in the organizational society are good communication tools that collect various opinions of the members, get new ideas and make decisions in order to solve the problems of the organizations, companies and organizations. However, Time and cost have been put in.

In order to establish a desirable meeting culture, 'unnecessary meeting time reduction' and 'efficient meeting proceeding' have already been discussed in many companies. In the organizational culture, the awareness that meetings are important enough to influence the competitiveness of companies , Companies and organizations are strongly pushing 'conference innovation'. This is not only for general companies but also for public corporations and public organizations as a part of management innovation, and it is a reality that successful companies and organizations basically make efforts to reduce waste of time by strictly observing meeting time and regulations. To improve business performance. Companies such as LG Electronics' '111 Culture' campaign, Samsung's' 337 Campaign ', Hewlett-Packard's' Coffee Break', Canon's' Standing Meeting 'and GE's' Workout Town Meeting' And a variety of improvement efforts are underway.

The conference, which is a necessary device for communication and decision making within the organization, is being promoted by the recent advances in IT technology, but it can not completely replace the face to face, Are mostly held at face-to-face meetings.

Hearing impaired people seem to have difficulties in distinguishing them from normal people and have no major inconvenience in daily life, but they have serious difficulties in obtaining information and forming a social network because they have language and hearing impairments which are very important in social life. In addition to those who are inherently hearing impaired, hearing loss patients due to the surrounding environment (illnesses, injuries, noise, aging, etc.) are steadily increasing, and social problems of hearing impaired people are no longer a problem.

Hearing impaired people who use sign language, spoken language, and spoken language as a means of communication have to be restricted from communication with other people, such as needing a sign language interpreter or using a variety of auxiliary devices. It is true that the level of educational achievement is low, and there are many cases where they face realistic difficulties in getting a job or getting into a society. Therefore, there is a growing need and interest in reducing the constraints on social and economic activities of disabled people and increasing their participation in the situation where the disabled population reaches 5% of the total population.

Technology and software development and dissemination for disabled people are being actively carried out in advanced countries around the world. Various types of technologies for communication for the hearing impaired are also increasingly spreading. However, in Korea, Interactive Voice & Video Response (IVVR) and telecommunication relay services are mainly performed in communication service companies.

Moreover, in spite of various efforts to improve the efficiency of the conference mentioned above, ineffectual situations in which time and various costs are put in such a way that it is actually a task to organize or share the contents of meetings at various meetings, seminars, Many of the participants in the conference have difficulties in communicating with the deaf or those who are hearing impaired. However, while the videoconferencing market used for teleconferencing has been growing steadily over the last few years, the conference management system, which recognizes the contents of the meeting and summarizes and shares minutes, has not been introduced in the domestic and overseas markets .

KR 10-2013-0107385 KR 10-2012-0100598 KR 10-2010-0131331

Disclosure of Invention Technical Problem [8] Accordingly, the present invention has been made in an effort to solve the above problems, and it is an object of the present invention to provide a conference And to provide a method and system for meeting management that provides voice recognition and hearing aid for the hearing impaired which can help communication and decision making.

In order to achieve the above object, there is provided a conference management method capable of supporting speech recognition and hearing assistance for a hearing impaired person using a conference management server in a meeting including a hearing impaired person according to the present invention, Receiving a conference attendance request data signal from a user terminal using the NFC tag attached in the conference hall and transmitting a conference attendance response data signal to the terminal; (b) after the step (a), the conference management server receives the speech data from the user terminal, recognizes the received speech data, and stores the speech-recognized text data; (c) transmitting the stored speech-recognized text data to the user terminal by the conference management server; And (d) creating a meeting record in the conference management server with the voice recognized text data stored in step (b), and transmitting the meeting minutes response data corresponding to the meeting minutes request data signal from the user terminal, And transmitting the created minutes to the user terminal so that the requested minutes are displayed.

Preferably, in the step of recognizing and storing the speech data signal received from the conference management server, the storage is stored in the conference management server in real time and classified into meeting rooms and written in minutes.

According to another aspect of the present invention, there is provided a conference management method capable of supporting speech recognition and hearing assistance for a hearing impaired person using a terminal connected to a conference management server in a conference including a hearing impaired person, A user terminal joining a conference management server transmits a conference attendance request data signal to a conference management server using an NFC tag attached to the conference hall and receives a conference attendance response data signal from the conference management server; (b) a user terminal connected to the conference management server through step (a) transmits the speech data to the conference management server, and the transmitted speech data is speech-recognized, ; And (c) after the step (b), the user terminal transmits a meeting request data signal to the conference management server, and receives the meeting response data corresponding thereto and displays the meeting record.

Preferably, the step (a) includes: (i) installing a hearing / conference program at the attendance terminal; And (ii) inviting and inviting participation in the conference or meeting using the installed hearing / conference program.

Preferably, in the above step (ii), the organizer prepares the meeting room, invites the member, and approves the participation request of the participant.

Preferably, after the hearing / conference program is installed in the terminal in step (i), the hearing ability of the terminal user is measured through the installed hearing / conference program, and the frequency is shifted corresponding to the measured hearing, .

According to the present invention, there is an effect to be applied to a wide range of courses, vocational education, etc. for the purpose of enhancing the participation of the hearing-impaired people in social activities and improving the quality of life, and it is possible to use not only hearing-impaired persons who are difficult to speak but also blind people.

It also has the effect of improving work efficiency and reducing various costs by shortening the meeting time.

FIG. 1 is a sequence diagram illustrating a conference management method capable of speech recognition and hearing aid support for a hearing impaired person according to the present invention.
FIG. 2 is a flowchart illustrating a conference management method using a hearing / conference algorithm in a terminal in a conference management method capable of supporting speech recognition and hearing for a hearing impaired person according to the present invention.
3 is a flowchart illustrating a method of speech recognition in a conference management method capable of speech recognition and hearing aid support for a hearing impaired person according to the present invention.
4 is a block diagram of a conference management system capable of supporting speech recognition and hearing assistance for the hearing impaired according to the present invention.
5 is a diagram illustrating an example of speech recognition and displayed on a terminal in a conference management method capable of supporting speech recognition and hearing aid for a hearing impaired person according to the present invention

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Prior to this, terms and words used in the present specification and claims should not be construed as limited to ordinary or dictionary terms, and the inventor should appropriately interpret the concepts of the terms appropriately It should be interpreted in accordance with the meaning and concept consistent with the technical idea of the present invention based on the principle that it can be defined. Therefore, the embodiments described in this specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention and do not represent all the technical ideas of the present invention. Therefore, It is to be understood that equivalents and modifications are possible.

1 is a sequence diagram illustrating a conference management method capable of supporting speech recognition and hearing for the hearing impaired according to the present invention.

Attendees, including presenters and listeners attending conferences held in any offline conference room, connect to the conference management server 100 using the speaker terminal 200 and the listener terminal 300, The listener transmits the conference attendance request data signal to the conference management server 100 using the speaker terminal 200 and the listener terminal 300 in step S110 and transmits the conference attendance request data signal to the conference management server 100 100 transmits a conference attendance response (acceptance) data signal to the speaker terminal 200 and the listener terminal 300 (S111). At this time, the presenter terminal 200 and the user of the listener terminal 300 are allowed to attend the conference, and the speaker 200 and the listener terminal 300 are connected to the conference management server 100 (S120). For the sake of convenience, in the sequence diagram of FIG. 1, a presenter's conference attendance request data signal is first transmitted through the presenter terminal 200. However, the order of connection of the presenter terminal 200 and the listener terminal 300 is not fixed, In which the presenter and the listener enter and are connected regardless of the order. Here, the NFC tag attached within the conference hall may be used for transmitting the conference attendance request data signal for accessing the conference management server 100. In the NFC tag, the conference hall related to the conference and the information of the attendance terminal capable of attendance are stored. It is possible that the terminals 200 and 300 recognize tags. QR codes can also be used in this way.

After that, the user of the presenter terminal 200 and the user of the listener terminal 300 are started. In the present invention, the conference starts from the announcement of the presenter terminal 200 for convenience. The speaker terminal 200 transmits the speaker's utterance data signal to the conference management server 100 (S130). At this time, the speaker 200 is connected to a microphone and an earphone for inputting the contents of the speech and outputting the contents of the speech, which will be described later. Thereafter, the conference management server 100 receiving the speaker's speech data signal recognizes the speech data signal and stores the speech-recognized text data (S131). Here, speech recognition is performed by detecting speech in a section where speech is read to disperse the speech signal to a virtual server, and synthesizing the dispersed speech signal results. The conference management server 100 transmits the speech recognition speech data signal to the speaker terminal 200 and the speaker terminal 300 in step S132 and the speaker terminal 200 and the speaker terminal 300 transmit the speech recognition speech data (S133). &Lt; / RTI >

Thereafter, the listener terminal 300 starts utterance related to the speaker's uttered contents displayed in step S133, and the listener terminal 300 transmits the speaker's utterance contents data signal to the conference management server 100 (S140) .

The conference management server 100 receiving the speech data signal of the listener performs speech recognition of the speech data of the listener and stores it (S141). Then, the conference management server 100 transmits a speech recognition text data signal of the listener to the speaker terminal 200 and the listener terminal 300 (S142). At this time, the speaker terminal 200 and the listener terminal 300 display the voice-recognized text data signal of the listener sent from the conference management server 100 (S143). Here, the text data display in steps S133 and S143 may be displayed as a chat window, and may be displayed as an example as shown in FIG.

Meanwhile, the speaker terminal 200 and the listener terminal 300 display the voice-recognized data signal, and move to a frequency corresponding to the hearing of the terminal users to output the voice-recognized contents to the users. The hearing ability is measured through the hearing aid / support program, and the frequency is shifted in accordance with the audible hearing characteristic, and the contents of the speech are outputted. That is, if the low-frequency region can be heard based on the list of hearing loss, the frequency of the high-frequency band is moved and compressed in the low-frequency band so that the frequency band originally distributed widely in the high- To be heard.

After the conference is continued, the conference management server 100 creates a conference record based on the speech data of the presenter and the listener, which are recognized and stored as speech (S200).

The presenter terminal 200 transmits a meeting attendance request data signal to the conference management server 100 in step S210 and receives the meeting attendance response data signal from the meeting management server 100. The speaker 200 requesting the meeting attendant displays the meeting invitation . In the same way, the listener terminal 300 also transmits a meeting attendant request data signal to the conference management server 100 using the listener terminal 300. The meeting management server 100 receives the meeting attendant answer data signal from the listener terminal 300 300, and the listener terminal displays the minutes (S223).

2 is a flowchart showing a sequence of a hearing / conference algorithm of a terminal in a conference management method capable of speech recognition and hearing processing for a hearing impaired person according to the present invention.

The presenter and the listener for attending the conference download and install the hearing / conference program on the web using the presenter terminal 200 and the listener terminal 300 (S101). At this time, the hearing ability of the terminal user is measured through the installed hearing / conference program, and the speech data is outputted at the meeting corresponding to the measured hearing ability.

Thereafter, when a hearing / conference program is installed (S101), a step S102 of inviting and inviting participation in the meeting and meeting is performed. At this time, there is a chairman who organizes the meeting regardless of the speaker and the listener. The chairman can create a meeting room and invite the members through the hearing / conference program. Members who attend the meeting room as a member, Quot; request "message, which is then included in the conference room list.

Then, the conference attendees in the conference room list attend the offline conference room and participate in the conference (S103).

3 is a flowchart illustrating a method of speech recognition in a conference management method supporting speech recognition and hearing processing for a hearing impaired person according to the present invention.

The speaker starts speaking using the microphone, removes the noise from the input speech (S411), and detects the voice (S412). At this time, if the terminal can not detect the voice, the user inputs the voice using the microphone again.

Thereafter, when the terminal detects the voice, the voice data sensed by the terminal is transmitted to the conference management server, and the conference management server performs voice recognition (S413) on the transmitted voice data.

When voice recognition is performed (S413), the conference management server stores voice (S414), and stores it in the voice database and voice information database (S416). At this time, the conference management server transmits the voice-recognized text data to the terminal, and the terminal displays the transmitted voice data in text (S415).

The voice stored in the step S414 is moved to a voice of a frequency band compensating for the user's hearing loss stored in the hearing characteristic database (S421) (if the user has a hearing loss in the low frequency band, the voice of the low frequency band is shifted to the high frequency band , And if the user has a hearing loss in the high frequency band, the sound of the high frequency band is shifted to the low frequency band), and a voice is output to the listener earphone, and the hearing characteristic database of the step S421, (S419), and the audiovisual characteristic is checked (S420) to store the hearing characteristic of the user.

If there is a mistaken part of speech in the voice text displayed on the terminal in step S415, the user selects a mistaken part (S417). Thereafter, the voice database of step S416 and the voice stored in the voice information database are reproduced (S418), and the voice is amplified and outputted based on the listener's hearing characteristic database.

4 is a block diagram illustrating a conference management system capable of supporting speech recognition and hearing for the hearing impaired.

A conference management system capable of supporting speech recognition and hearing assistance for a hearing impaired person includes a microphone and a hearing aid earphone (210, 310) for inputting a speech content and outputting a speaking content, a microphone and a hearing aid earphone (210, 310) A speaker 200 and a speaker 300 connected to the speakers 200 and 300 and receiving a speech voice signal input through the microphones 211 and 311 and displaying the transmitted speech voice signal, And a conference management server 100 for performing speech recognition of the received speech voice signal and storing and managing the received speech voice signal.

The hearing aid earphones 210 and 310 are connected to the terminals 200 and 300 by wired or wireless connection. The hearing aid earphones 210 and 310 are connected to the earphone terminals of the terminals 200 and 300 by wired lines and include microphones 211 and 311 and speakers 212 and 312. The microphones 211 and 311 receive a sound in a low frequency range and a medium high frequency range, and receive sound even in a social conversation distance of 150 cm to 200 cm and a public conversation distance of 200 cm or more. Speakers 212 and 312 limit the amplification to prevent hearing damage.

The presenter and listener terminals 200 and 300 are project terminals including a portable electronic device such as a smart phone, a tablet PC, and a dedicated device or a project screen. Portable electronic devices measure and store the hearing ability of the user, Receiving modules 250 and 350 for transmitting and receiving voice data signals input from the microphones 210 and 310 to and from the conference management server 100. The voice transmission / And a meeting record display module (260, 360) for requesting a meeting record created by the conference management server (100), and displaying the received meeting record by request.

The conference management server 100 includes a terminal data transmission / reception module 110 for managing a data signal transmitted from the terminals 200 and 300 and a response signal corresponding to the data signal and the terminal data transmission / reception module 110 A voice data management module 120 for receiving and storing voice data signals of conference participants transmitted from the terminals 200 and 300 and a voice recognition module 120 for recognizing voice data signals received by the voice data management module 120, Module 130 and a translation module 140 for translating the speech recognition data signal that has been voice-recognized through the continuous speech recognition module 130 into a language required by the attendee or an arbitrary portion of the attendee wondering.

The terminal data transmission / reception module 110 manages transmission / reception of a data signal received from the terminals 200 and 300 and a data signal responding thereto, for example, a transmission / reception management of a conference attendance request data signal and a response data signal responsive thereto And transmits and receives voice response data to the terminals 200 and 300. The terminal 200 and the terminal 300 transmit voice data of the terminal user and voice recognition data of the received voice data to the terminals 200 and 300 send.

The voice data management module 120 manages and stores the voice data received through the terminal data transmission / reception module 110 and stores the voice data and the voice data interlocked with the continuous voice recognition module 130 .

The continuous speech recognition module 130 notifies the voice data transmitted through the voice data management module 120 to a middleware (not shown) by detecting a breathing interval instead of a single sentence unit, Lt; / RTI > That is, instead of waiting for the completion of several sentences when speaking about their opinions in a plurality of sentences, such as a speech or an announcement, the sentence is sent to the respective speech recognition server (middleware) , Sequentially receiving the results of speech recognition from the respective speech recognition servers, and outputting the combined results in real time, the speech recognition waiting time can be drastically reduced.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. It will be understood that various modifications and changes may be made without departing from the scope of the appended claims.

100: conference management server
110: terminal data transmission / reception module
120: voice data management server
130: Continuous speech recognition server
140: translation server
200, 300: terminal
210, 310: Microphone-Assisted Helping Earphone

Claims

A conference management method capable of supporting speech recognition and hearing assistance for a hearing impaired person using a conference management server in a meeting including a hearing impaired person,
(a) receiving a conference attendance request data signal from a user terminal to which the conference management server intends to attend a conference using an NFC tag attached to the conference hall, and transmitting a conference attendance response data signal to the terminal;
(b) after the step (a), the conference management server receives the speech data from the user terminal, recognizes the received speech data, and stores the speech-recognized text data;
(c) transmitting the stored speech-recognized text data to the user terminal by the conference management server; And
(d) creating a meeting record in the conference management server from the voice recognized text data stored in step (b), transmitting the meeting attendance response data corresponding to the meeting attendee request data signal from the user terminal, Transmitting the created minutes to the user terminal so that the requested minutes are displayed;
And a speech recognition and hearing aid support for the hearing impaired.

The method according to claim 1,
In the step of recognizing and storing the speech data signal received from the conference management server, the storage is stored in the conference management server in real time,
A conference management method capable of voice recognition and hearing assistance for the hearing impaired.

A conference management method capable of supporting speech recognition and hearing support for a hearing impaired person using a terminal connected to a conference management server in a meeting including a hearing impaired person,
(a) A user terminal who wants to attend a conference transmits a conference attendance request data signal to the conference management server using an NFC tag attached to the conference hall and receives a conference attendance response data signal from the conference management server, Connected;
(b) a user terminal connected to the conference management server through step (a) transmits the speech data to the conference management server, and the transmitted speech data is speech-recognized, ; And
(c) after the step (b), the user terminal transmits a meeting request data signal to the conference management server and receives a meeting attendance response data corresponding thereto and displays a meeting record
And a speech recognition and hearing process for the hearing impaired.

The method of claim 3,
Before the step (a)
(I) installing a hearing / conference program at a conference attendance terminal; And
(Ii) hoping and inviting participation in the conference or meeting using the installed hearing / conference program
And a speech recognition and hearing aid support for the hearing impaired person.

The method of claim 4,
In the above step (ii), the organizer is responsible for creating the meeting room, inviting the members, and approving the participation request of the participant
A conference management method capable of voice recognition and hearing assistance for the hearing impaired.

The method of claim 4,
After the hearing / conference program is installed in the terminal in the step (i), the hearing ability of the terminal user is measured through the installed hearing / conference program, and the frequency is shifted corresponding to the measured hearing,
And a speech recognition and hearing aid support for the hearing impaired person.

delete