WO2023189422A1

WO2023189422A1 - Control device, control method, haptic presentation system, and program product

Info

Publication number: WO2023189422A1
Application number: PCT/JP2023/009309
Authority: WO
Inventors: 修一赤岩; 慶介木野井
Original assignee: 株式会社栗本鐵工所
Priority date: 2022-03-30
Filing date: 2023-03-10
Publication date: 2023-10-05
Also published as: JP2023148851A

Abstract

The present invention provides a control device, a control method, a haptic presentation system, and a program product for improving the perceptibility of haptic sensations presented by a haptic presentation device. The control device is a control device of a haptic presentation device having a displacement unit provided as to be displaceable in response to an operation by an operator, the haptic presentation device generating a kinesthetic sensation in response to an operation on the displacement unit in accordance with the magnitude of displacement of the displacement unit to present a haptic sensation of a displayed object, wherein control data for generating the kinesthetic sensation is stored on a per-object basis in correspondence with the magnitude of displacement of the displacement unit, and the control device corrects the correspondence relationship between the control data and the magnitude of displacement for the control data on the basis of a setting and outputs corrected control data associated with the magnitude of displacement of the displacement unit to the haptic presentation device.

Description

Control devices, control methods, tactile presentation systems and program products

The present invention relates to a control device, a control method, a tactile presentation system, and a program product that improve the perceptibility of tactile sensations presented by a tactile presentation device.

Of the five human senses, image and audio transmission technologies related to visual and auditory senses are becoming more precise, and recently, various technologies for presenting tactile sensations (haptics) have been proposed.

As a tactile presentation technology, it is possible to reproduce the vibrations that occur when an image of an object displayed on a display with a built-in touch panel is touched, mainly using three types of vibrations (ERM: Eccentric Rotating Mass, LRA: Linear resonant actuator, piezo element). Proposed. However, when presenting tactile sensations on a touch panel, it is difficult to reproduce deeper tactile sensations, that is, not only the skin of fingers but also the force sensations generated in muscles and tendons.

Based on knowledge of magnetorheological fluids, the inventors have proposed a tactile presentation device that can be operated by an operator and reproduces different sensations depending on the object (Patent Document 1, etc.).

Patent No. 6906275

The tactile presentation device includes a displacement section that is a curved plate or the like that can be displaced relative to the base by an operator's operation. The displacement part can be displaced by the operator without resistance when the tactile presentation device is not controlled, but the tactile presentation device under control causes the displacement portion to be displaced according to the amount of displacement (position) of the displacement portion. The tactile sensation is presented by changing the magnitude of the force sense, the output method of the force sense, etc.

The manner in which the displacement portion of this tactile presentation device is displaced differs depending on the size of the operator's hands, differences in muscles, etc. Further, depending on the operator, when an operation to displace the displacement portion is performed, the ease of perceiving the sensation as a touch differs depending on the timing at which the force sensation is output from the displacement portion. Therefore, it is desirable to be able to set the tactile presentation device so that each operator can easily perceive the tactile sensation.

The present invention has been made in view of the above circumstances, and aims to provide a control device, a control method, a tactile presentation system, and a program product that improve the perceptibility of tactile sensations presented by a tactile presentation device. do.

A control device according to an embodiment of the present disclosure includes a displacement section that is displaceable in response to an operation by an operator, and generates a force sensation in response to an operation on the displacement section according to the amount of displacement of the displacement section. A control device for a tactile presentation device that displays a tactile sensation of a displayed object, the control device storing control data for generating the tactile sensation for each of the objects in association with a displacement amount of the displacement portion. and corrects the correspondence between the displacement amount of the control data and the control data based on the settings, and outputs the corrected control data associated with the displacement amount of the displacement portion to the tactile presentation device.

A control method according to an embodiment of the present disclosure includes a displacement section that is displaceable in response to an operation by an operator, and generates a force sensation in response to an operation on the displacement section according to a displacement amount of the displacement section. A control method for a tactile presentation device that presents a tactile sensation of a displayed object by causing a computer connected to the tactile presentation device to generate the haptic sensation in association with a displacement amount of the displacement portion. Control data is stored for each object, and based on settings, the correspondence between the displacement amount of the control data and the control data is corrected, and the corrected control is made in correspondence with the displacement amount of the displacement part. It includes a processing unit that outputs data to the tactile presentation device.

A program product according to an embodiment of the present disclosure includes a displacement section that is displaceable in response to an operation by an operator, and generates a force sensation in response to an operation on the displacement section according to the amount of displacement of the displacement section. control data for causing the force sensation to be stored in a computer connected to a tactile presentation device that presents a tactile sensation of the displayed object, in association with a displacement amount of the displacement portion, for each of the objects; a process of correcting the correspondence between the displacement amount of the control data and the control data based on the settings, and outputting the corrected control data associated with the displacement amount of the displacement part to the tactile presentation device. Execute.

In the control device, control method, and program product of the present disclosure, when the displacement portion of the tactile presentation device generates a force sensation according to the amount of displacement, the tactile sensation is presented after the displacement portion starts displacement. are corrected individually.

A control device according to an embodiment of the present disclosure receives, as the setting, a shift in the displacement amount for starting to output the control data that continuously changes in ascending order of the displacement amount.

In the control device of the present disclosure, the control data is set so that the tactile sensation of the object can be continuously reproduced as the displacement amount changes in ascending order. The discrepancies up to that point are corrected individually.

A control device according to an embodiment of the present disclosure receives the settings on a settings screen that includes a slide bar that indicates a position of deviation with respect to the range of the displacement amount.

In the control device of the present disclosure, it is possible to set the deviation as a continuous value using a slide bar.

A control device according to an embodiment of the present disclosure receives the settings on a settings screen that selectably displays a plurality of candidates having different amounts of deviation with respect to the range of displacement amounts.

The control device of the present disclosure allows selection from multiple candidates.

A tactile presentation system according to an embodiment of the present disclosure includes a displacement section that is displaceable in response to an operation by an operator, and generates a force sensation in response to an operation on the displacement section according to the amount of displacement of the displacement section. a tactile presentation device that generates and presents a tactile sensation of a displayed object; and an information processing device that is communicatively connected to the tactile presentation device and includes a display section, and the information processing device is configured to detect a displacement amount of the displacement section. Control data that causes the force sensation and visual data to be displayed on the display unit are stored for each object in association with the control data and the control data and Correct the correspondence with the visual data, output the corrected control data associated with the displacement amount of the displacement part to the tactile presentation device, and calculate the displacement of the displacement part based on the corrected visual data. An image is output to the display section according to the amount.

In the tactile presentation system of the present disclosure, when the displacement part of the tactile presentation device generates a sense of force according to the amount of displacement, the time from when the displacement part starts to be displaced until the tactile sense is presented is individual to the operator. It is corrected to

According to the present disclosure, it is possible to adjust the tactile presentation by the tactile presentation device according to the operator, and it is possible to improve the degree of tactile perception.

FIG. 1 is a schematic diagram showing a tactile presentation system. FIG. 1 is a block diagram showing the configuration of an information processing device. It is an explanatory view showing an example of contents of a sensory DB. FIG. 2 is a block diagram showing the configuration of a tactile presentation device. 2 is a flowchart illustrating an example of a basic processing procedure for tactile presentation in the tactile presentation system. FIG. 2 is a schematic diagram showing an example of tactile presentation in the tactile presentation system. 3 is a flowchart illustrating an example of a setting method. An example of a setting screen displayed on the display unit is shown. 7 is a flowchart illustrating an example of a correction procedure based on a presentation start position. It is an explanatory view showing the result of correction. Another example of the setting screen displayed on the display unit is shown. FIG. 2 is a schematic diagram showing an example of tactile presentation in the tactile presentation system. FIG. 2 is a schematic diagram of a tactile presentation system in a second embodiment. It is a block diagram showing the composition of HMD of a 2nd embodiment.

The present disclosure will be specifically described with reference to drawings showing embodiments thereof. In the following embodiments, implementation of a setting method in a tactile presentation system will be described.

(First embodiment)
FIG. 1 is a schematic diagram showing a tactile presentation system 100. The tactile presentation system 100 includes an information processing device 1 and a tactile presentation device 2. The information processing device 1 and the tactile presentation device 2 are communicably connected by short-range wireless communication and exchange data with each other.

The information processing device 1 uses a smartphone as shown in FIG. The information processing device 1 may be a tablet terminal or a laptop-type PC (Personal Computer) instead of a smartphone.

The tactile presentation device 2 is a device that an operator can operate by moving his or her finger while holding it along the displacement portion 202. The tactile presentation device 2 reads the position of the displacement part 202 that is displaced by the operator's finger movement, controls the built-in MRF (Magneto-Rheological Fluid) device 24 according to the position, and moves the displacement part 202 of the operator. This is a device that produces a sense of force using a reaction force (rotational resistance) to the operation of the machine, and presents a tactile sensation. The form of the displacement part 202 of the tactile presentation device 2 is not limited to that shown in FIG. 1, but may be stick-like or cushion-like covered with a cover. The tactile presentation device 2 may employ a motor, a piezo element, or the like instead of the MRF device 24, and may generate a force sensation by rotational force or vibration in response to an operation by the operator. It may be combined with something that provides a warm or cold sensation. The structure may be such that it is installed on the ground or a wall and is operated by the operator's palm or foot.

In the tactile presentation system 100, the tactile presentation device 2 cooperates with the information processing device 1, and causes the information processing device 1 to output the image and audio of the object, while causing the tactile presentation device 2 to output the tactile sensation of the object. In the example shown in FIG. 1, when a soft object is displayed on the display unit 13 of the information processing device 1 and the operator pushes the displacement part 202 of the tactile presentation device 2 with a finger, the tactile presentation device 2 becomes squishy or fluffy. While outputting a tactile sensation, the display unit 13 displays an image that changes as if the object is being pushed in, and the audio output unit 14 outputs an audible “squeak” sound that corresponds to the object being pushed in. Output audio.

FIG. 2 is a block diagram showing the configuration of the information processing device 1. The information processing device 1 includes a processing section 10, a storage section 11, a communication section 12, a display section 13, an audio output section 14, and an operation section 15. The processing unit 10 is a processor using a CPU (Central Processing Unit) and/or a GPU (Graphics Processing Unit). The processing unit 10 executes a process described below based on a control program P1 for tactile presentation stored in the storage unit 11.

The storage unit 11 uses nonvolatile memory such as flash memory and SSD (Solid State Drive). The storage unit 11 stores data referenced by the processing unit 10. The control program P1 (program product) is downloaded from the information processing device 1 or another program server device via the communication unit 12 and stored in an executable manner. The control program P1 stored in the storage unit 11 may be the control program P8 stored in the computer-readable storage medium 8, which is read out and stored by the processing unit 10.

The storage unit 11 stores a sensory database (DB) 110 that includes tactile data, visual data, and auditory data of the object to be output. The sensory DB 110 stores tactile data, visual data (images, videos), and auditory data to be output at each displacement for each displacement data of a displacement part in the tactile presentation device 2, in association with an object ID that identifies the object. (audio) (see Figure 3). The storage unit 11 further stores setting data for each operator.

The communication unit 12 is a communication module for short-range wireless communication, for example, Bluetooth (registered trademark). The processing unit 10 can transmit and receive data to and from the tactile presentation device 2 through the communication unit 12 .

The display unit 13 is a display such as a liquid crystal display or an organic EL (Electro Luminescence) display. The display unit 13 is, for example, a display with a built-in touch panel. The processing unit 10 displays, on the display unit 13, an operation screen for causing the tactile sensation presentation device 2 to output a tactile sensation and an image of the target object, based on the control program P1.

The audio output unit 14 includes a speaker and the like. The processing unit 10 causes the audio output unit 14 to output the sound of the object, music, etc. based on the control program P1.

The operation unit 15 is a user interface that allows input and output to and from the processing unit 10, and is a touch panel built into the display unit 13. The operation unit 15 may be a physical button. The operation unit 15 may be a voice input unit.

FIG. 3 is an explanatory diagram showing an example of the contents of the sensory DB 110. As shown in FIG. 3, the sensory DB 110 stores the current value of the current to the MRF device of the tactile presentation device 2 for each displacement amount (angle) of the displacement part 202 as tactile data in association with the object ID. There is. The objects are plural, and may include real objects that present a tactile sensation, such as still life objects such as balls or balloons, foods such as gummies, vegetables, or fruits, and animals such as dogs, cats, or fish. Characters such as slimes and monsters may also be included.

The relationship between the amount of displacement of the displacement section 202 of the sensory DB 110 and the current value is set according to the actual dimensions of the movable range of the displacement section 202 and the dimensions of the object. For example, if the movable range of the displacement unit 202 is 50 mm, and the object is a sphere that will not burst, such as a small gummy bear, and the size is set to 15 mm, the tactile sensation will be reproduced within the first 15 mm, In the remaining 35 mm range, a current value is set that produces a rotational resistance that makes it difficult to push in any further. Alternatively, a large current value that makes it impossible to push in the range of 12 mm to 15 mm may be set, and the current values thereafter may be set to zero. In addition, if the target object is a large object such as the seat surface of a sofa, and the size greatly exceeds the movable range of the displacement unit 202, the tactile data may be such as to present the sensation up to 50 mm from the surface of the sofa seat surface. The current value is set. If the target object is a virtual character such as a monster, the current value is set based on the size according to the setting of the character.

Similarly, the sensory DB 110 stores images (frame images) for each position (angle) of the displacement part 202 as visual data in association with the object ID. Here, the frame image is one still image that is recognized as an animation image when displayed continuously. Similarly, audio data for each position (angle) of the displacement section 202 is stored as auditory data in association with the object ID. The auditory data may be waveform data that differs depending on each angle. It may also be a timestamp of the audio corresponding to each angle.

FIG. 4 is a block diagram showing the configuration of the tactile presentation device 2. As shown in FIG. 1, the tactile presentation device 2 is configured by providing a flat bottomed cylindrical grip 200 with a band-shaped flat plate displacement portion 202 having a curved portion partially along the circumferential direction. Although the displacement part 202 is made of a flexible material, it may also be made of a highly rigid material and rotatably supported via the grip body 200 and a support shaft. A binding tool 203 in the form of a cloth tape is provided on the outer surface of the distal end of the displacement portion 202 . A link mechanism 204 is provided on the inner surface of the distal end of the displacement portion 202 to connect to the rotating shaft of the rotor of the MRF device 24 housed inside the gripper 200 .

As shown in FIG. 1, the operator uses the gripping body 200 by, for example, holding it with the thumb and middle finger and inserting the index finger into the binding tool 203 with the finger such as the index finger along the displacement part 202. The operator can move the displacement part 202 by pushing in the index finger, or can move the displacement part 202 away from the grip body 200 by extending the index finger.

The tactile presentation device 2 includes a gripping body 200 as shown in FIG. The grip body 200 has an MRF device 24 built therein. The control section 20, the storage section 21, the communication section 22, and the power supply section 23 may be provided integrally with the grip body 200, or may be provided as separate bodies connected to the grip body 200 wirelessly or by wire.

The control unit 20 includes a processor such as a CPU and an MPU (Micro-Processing Unit), and a memory such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The control unit 20 is, for example, a microcontroller. The control unit 20 controls each component based on a control program P2 stored in the built-in ROM to realize tactile presentation.

The storage unit 21 is an auxiliary storage memory for the control unit 20, and stores control data (tactile data) of the MRF device 24 in a rewritable manner.

The communication unit 22 is a communication module for short-range wireless communication, for example, Bluetooth (registered trademark). The control unit 20 can transmit and receive data to and from the information processing device 1 through the communication unit 22 .

The control unit 20 is connected to the power supply unit 23, the MRF device 24, and the sensor 25 via I/O, and exchanges signals with each other.

The power supply unit 23 includes a rechargeable battery. The power supply unit 23 supplies power to each component and the MRF device 24 when turned on.

The MRF device 24 has a yoke that sandwiches a disk-shaped rotor with a gap between them, and generates a magnetic field by passing a control current through a coil provided in the yoke, thereby generating a magnetic field sealed in the gap. The viscosity (shear stress) of the viscous fluid is controlled to provide rotational resistance to the rotor. When the control unit 20 controls the magnitude of the control current to the MRF device 24, the rotational resistance is immediately changed.

The sensor 25 measures the position (angle) of the displacement section 202 and outputs it to the control section 20. The sensor 25 measures and outputs the displacement of the displacement portion 202 as an angle. The sensor 25 may include a plurality of sensors such as a gyro sensor and an acceleration sensor.

In the tactile presentation device 2 configured as described above, when the displacement section 202 is operated by the operator, the displacement of the displacement section 202 is caused to move in the direction of rotation of the MRF device 24 toward the rotation axis of the rotor via the link mechanism 204. communicated. Since the rotating shaft rotates freely when the MRF device 24 is not operating, that is, while the control current is zero, the displacement portion 202 fluctuates without resistance. On the other hand, when the MRF device 24 operates and the control current is not zero, the viscosity (shear stress) of the magnetorheological fluid inside the MRF device 24 is changed depending on the magnitude of the current flowing to the MRF device 24. By continuously changing the magnitude of the current to the MRF device 24 or by vibrating the current value at a predetermined frequency, the control unit 20 can change the resistance force applied to the displacement unit 202 and the method of its appearance. .

In this way, the tactile sensation presentation device 2 varies the resistance (current value) according to the amount of pushing of the displacement section 202 to present a smooth tactile sensation, or increases the resistance as the pushing amount increases to create a tighter feel. It is possible to present a tactile sensation with a certain hardness, or to present a crunchy tactile sensation by repeatedly increasing and decreasing the resistance.

FIG. 5 is a flowchart illustrating an example of a basic processing procedure for tactile presentation in the tactile presentation system 100. When the operator starts the control program P1 and turns on the power of the tactile presentation device 2, the processing unit 10 of the information processing device 1 starts the following process in cooperation with the tactile presentation device 2.

The processing unit 10 displays on the display unit an operation screen that includes a list of object candidates for which tactile sensation is to be displayed (step S101), and accepts the selection of the object (step S102). The processing unit 10 reads tactile data, image data, and audio data corresponding to the object ID of the selected object from the sensory DB 110 of the storage unit 11 (step S103).

The processing unit 10 corrects the read tactile data, image data, and audio data based on the presentation start position set in a process described later, and temporarily stores the corrected data (step S104). The correction method in step S104 will be described in detail later.

The processing unit 10 establishes communication between the tactile presentation device 2 and the communication unit 12 (step S105), and transmits the temporarily stored corrected tactile data to the tactile presentation device 2 (step S106). The processing unit 10 causes the display unit 13 to display that tactile presentation is to be started (step S107).

Upon receiving the tactile data (step S201), the control unit 20 of the tactile presentation device 2 stores it in the storage unit 21 (step S202).

The control unit 20 samples a signal corresponding to the displacement amount (angle) of the displacement unit 202 output from the sensor 25 (step S203). The control unit 20 transmits the displacement amount obtained by sampling to the information processing device 1 (step S204), and refers to the current value corresponding to the obtained displacement amount from the tactile data stored in the storage unit 21 (step S204). S205), outputs the referenced current to the MRF device 24 (step S206), and transmits the process to step S203. The processing in steps S203 to S206 continues until an operation to end the processing is performed on the information processing device 1 side.

The information processing device 1 receives the displacement amount from the tactile presentation device 2 (step S108), and the processing section 10 refers to the image and audio corresponding to the received displacement amount from the visual data and auditory data in the storage section 11 ( Step S109) and output from the display section 13 and audio output section 14 (Step S110), and the process returns to Step S108. Each time the amount of displacement is transmitted from the tactile presentation device 2, the information processing device 1 outputs an image and audio corresponding to the amount of displacement.

The processing unit 10 determines whether a termination operation has been performed (step S111). If it is determined that the end operation has not been performed (S111: NO), the process returns to step S108, and the processes of steps S108-S110 are repeated.

If it is determined that the end operation has been performed (S111: YES), the processing unit 10 ends the display, disconnects communication with the tactile presentation device 2 (step S112), and ends the process.

FIG. 6 is a schematic diagram showing an example of tactile presentation in the tactile presentation system 100. FIG. 6 shows changes in images and sounds displayed on the display unit 13 in response to the amount of displacement in the tactile data transmitted to the tactile presentation device 2. As shown in FIG. 6, the image changes depending on the amount of depression of the displacement part 202, and the way the depression corresponds to the tactile sensation changes. In addition, as shown in FIG. 6, when displaying an image of the operator's finger in the image, contact, that is, from the initial position (displacement amount is zero) output to the MRF device 24, to the target object in the image. An image of the touching finger may be displayed.

In the tactile presentation system 100 that presents the tactile sense in this way, when the information processing device 1 corrects the tactile data in step S104 of the processing procedure shown in the flowchart of FIG. The output of the device 2 can be tailored to each operator.

FIG. 7 is a flowchart illustrating an example of a setting method. The processing procedure shown below is started when the setting menu is selected within the operation screen displayed based on the control program P1. The processing procedure shown in FIG. 7 may be automatically started when the processing based on the control program P1 is executed for the first time. Initially, the amount of displacement corresponding to the presentation start position is zero.

The processing unit 10 displays a setting screen including a bar corresponding to the angular range of the displacement amount (step S301). The processing unit 10 receives the selection of the presentation start position using the bar (step S302), determines and stores the amount of displacement (angle) corresponding to the selected presentation start position (step S303). In step S303, the processing unit 10 determines and stores the position of the displacement amount in the displacement amount range (0 to 90°) from the ratio of the length of the selected position to the length of the bar. That is, when the length of the bar is 100, in the case where the length to the selected position is 5, it is about 5%, so 5° is determined as the displacement amount corresponding to the presentation start position.

The processing unit 10 creates tactile data in which the current value up to the amount of displacement (angle) corresponding to the presentation start position is set to zero, and a predetermined current value is set to the amount of displacement greater than or equal to the displacement amount corresponding to the presentation start position. (Step S304). The processing unit 10 transmits the created tactile data to the tactile presentation device 2 (step S305), and causes the display unit 13 to display a message urging you to try it on the setting screen (step S306).

Based on the received tactile data, the control unit 20 of the tactile presentation device 2 causes a current to flow through the MRF device 24 according to the amount of displacement of the displacement portion, so that the amount of displacement of the displacement portion 202 is smaller than the amount of displacement corresponding to the set presentation start position. It is possible to perform a test in which a predetermined reaction force (rotational resistance) is generated only when the holder is pushed in. This allows the operator to test the amount of time (play/displacement) before starting to feel a reaction force.

The processing unit 10 displays a determination button on the setting screen (step S307), and accepts whether or not the presentation start position selected in step S302 is acceptable as a result of the trial. The processing unit 10 determines whether the OK button has been selected (step S308). If it is determined that the enter button has not been selected (S308: NO), the processing unit 10 returns the process to step S302 and repeats the process until the enter button is selected.

If it is determined that the enter button has been selected (S308: YES), the processing unit 10 stores the displacement amount (angle) determined in step S303 in the storage unit 11 as a correction amount (step S309), and ends the process. do.

FIG. 8 shows an example of the setting screen 130 displayed on the display unit 13. When the settings menu is selected by the operation unit 15, the processing unit 10 displays the settings screen 130 on the display unit 13. The settings screen 130 includes a bar interface 131 that includes a control 132 that is slidable over a length that corresponds to a range of displacement amounts. The operator can move the control 132 up and down on the touch panel built into the display section 13.

The setting screen 130 includes a test button 133 and a decision button 134. When the test button 133 is selected by the operator, the processing unit 10 detects this using the operation unit 15, and determines the displacement amount in step S303 based on the position of the moved control 132. If the test button 133 is selected, the processing unit 10 executes the processes from step S304 to step S308.

If the decision button 134 is selected, the processing unit 10 executes steps S308-S309.

As a result, the amount of correction of the presentation start position when operating the displacement unit 202 is set for each operator, and is reflected in step S104 in the process shown in the flowchart of FIG. FIG. 9 is a flowchart showing an example of a correction procedure based on the presentation start position, and FIG. 10 is an explanatory diagram showing the results of the correction.

The processing unit 10 creates corrected tactile data by adding the displacement amount stored as the correction amount to each displacement amount of the tactile data read in step S104 (step S401). Thereby, it is possible to shift the amount of displacement at which the continuously changing current value in the tactile data starts to be output.

The processing unit 10 creates corrected auditory data by adding the displacement amount stored as the correction amount to each displacement amount of the read auditory data (step S402).

The processing unit 10 creates corrected visual data by adding the displacement amount stored as the correction amount to each displacement amount of the read visual data (step S403), and ends the correction.

In step S403, when displaying the image of the operator's finger virtually superimposed on the image that is the visual data, the processing unit 10 determines that when the image of the operator's finger is virtually superimposed and displayed, the same amount of displacement as the amount of displacement stored as the correction amount is obtained from the tactile presentation device 2. At the transmitted timing, coordinates for displaying the finger image so that the finger image appears to be in contact with the target object may be calculated. In this case, the processing unit 10 determines the timing at which the tactile sensation occurs and the image of the finger appearing in contact with the object based on the presentation start position and the amount of displacement (angle) at which the current value in the tactile data becomes non-zero. The image may be adjusted so that the timing matches.

Through the correction, the tactile data, auditory data, and visual data become data in which current values, audio data, and image data are associated with each displacement amount after the correction amount is added, as shown in FIG. . That is, the range of the amount of displacement (movable range) of the displacement portion 202 is corrected so as to deviate from the range of the presented sensation.

FIG. 11 shows another example of the setting screen 130 displayed on the display unit 13. As shown in FIG. 8, when the setting menu is selected by the operation unit 15, the processing unit 10 displays the setting screen 130 on the display unit 13. On the setting screen 130, three patterns are displayed as candidates, with respect to the range of displacement amount, with different patterns of how far from the top the tactile presentation should start (tactile presentation position).

In the example of FIG. 11 as well, the setting screen 130 includes a test button 133 and a decision button 134. It is possible to select one by pressing the decision button 134. In the example shown in FIG. 11, of the three candidates, the center candidate is highlighted by a thick line frame (cursor), indicating that this candidate has been selected by tapping on the operation unit 15. There is.

When the test button 133 is selected by the operator, the processing unit 10 detects this using the operation unit 15, and determines the displacement amount in step S303 based on the position of the moved control 132. If the test button 133 is selected, the processing unit 10 executes the processes from step S304 to step S308.

Even in the example of the setting screen 130 shown in FIG. 11, if the enter button 134 is selected, the processing unit 10 executes steps S308-S309.

FIG. 12 is a schematic diagram showing an example of tactile presentation in the tactile presentation system 100. In the example shown in FIG. 12, compared with the example shown in FIG. 6, changes in the image and sound displayed on the display unit 13 are shown in response to the amount of displacement in the corrected tactile data. In FIG. 12, the state of the tactile data before correction is shown by a broken line. As shown in FIG. 12, after the correction, compared to FIG. 6, the tactile sensation starts to be presented from the displacement section 202 from a position where the displacement section 202 is pushed in a little (equivalent to 5 degrees), and similarly, images and sounds is output.

In this way, it is possible to adjust the presentation start position of the tactile sense when using the tactile sense presentation device 2 according to the operator's preference and sense. By adapting the method of presenting the tactile sensation in the tactile sensation presentation device 2 to the operator, the degree of perception by the operator is improved.

Although the information processing device 1 has been described as storing one correction amount, the present invention is not limited to this. The storage unit 11 of one information processing device 1 stores correction amounts for different operators, and after receiving the selection of the operator who operates the tactile presentation device 2, the processing unit 10 , the process shown in the flowchart of FIG. 5 may be executed using the set correction amount.

(Second embodiment)
In the second embodiment, an HMD (Head Mounted Display) is used to present visual information regarding a target object as a three-dimensional image. FIG. 13 is a schematic diagram of the tactile presentation system 100 in the second embodiment, and FIG. 14 is a block diagram showing the configuration of the HMD 3. The configuration of the tactile presentation system 100 in the second embodiment is the same as that of the tactile presentation system 100 in the first embodiment, except that the HMD 3 is used and the details of the processing by using the HMD 3 are different. . Therefore, among the configurations of the tactile presentation system 100 of the second embodiment, the same components as those of the tactile presentation system 100 of the first embodiment are given the same reference numerals, and detailed description thereof will be omitted.

The HMD 3 includes a display section 31, a motion detection section 32, a space detection section 33, and a connection section 34. The HMD 3 may be provided with a display section 31, a motion detection section 32, a space detection section 33, and a connection section 34 in the main body, or some may be provided separately and mutually exchange control signals via a communication medium. It may be possible to do so. The HMD 3 may have the functions of the information processing device 1 and be configured integrally.

The display unit 31 includes, for example, a small liquid crystal display, an optical lens, and an optical system mechanism, and is capable of displaying a three-dimensional image at a viewing angle of 110° or more. The display unit 31 is a transparent or semi-transparent glass-like display, and receives an image signal (including a video signal) output from the information processing device 1 and displays it in a superimposed manner in the operator's actual field of view. The display section 31 is not limited to a glass-like one, and may display an image based on an image signal output from the information processing device 1 superimposed on an image taken of real space by a camera provided facing forward. It's okay.

The motion detection unit 32 includes a plurality of triaxial acceleration sensors and gyro sensors installed in various directions on the main body (the cover of the display unit 31 and the attachment belt), and aggregates signals from a group of these sensors. and a control circuit that outputs the output. The movement detection unit 32 detects the movement of the wearer's head.

The space detection unit 33 includes two or more infrared cameras arranged outwardly in parallel on the outer surface of the main body, and an infrared LED disposed at an intermediate position between the infrared cameras so as to similarly emit infrared rays outward. use The space detection unit 33 functions as a depth sensor that measures the distance to an object that exists outside the main body of the HMD 3. The space detection unit 33 can measure and output the distance from the HMD 3 main body to a still object such as a wall or floor in the location of the operator wearing the HMD 3. Similarly, the space detection unit 33 can measure and output the distance from the HMD 3 main body to the operator's arm, hand, and finger.

The connection unit 34 is an interface for connecting to the information processing device 1. The HMD 3 outputs signals corresponding to the results measured by the motion detection unit 32 and the space detection unit 33 to the information processing device 1, and also acquires the video signal output from the information processing device 1 and displays it on the display unit. 31.

A speaker may be provided in the HMD 3, and the audio output from the audio output unit 14 of the information processing device 1 may be output from the speaker.

In the tactile presentation system 100 of the second embodiment configured as described above, the information processing device 1 uses the HMD 3 and the distance to the still object in the real space measured by the space detection unit 33 to determine the three-dimensional shape of the virtual object. The original image is superimposed and displayed in accordance with the operator's movement detected by the movement detection unit 32 to realize AR display. To select an object, the space detection section 33 uses the tactile presentation device 2 to select the coordinates in the three-dimensional image of the virtual object displayed on the display section 31 of the HMD 3 (coordinates that match the real space). When it is detected that the gripped fingers are overlapped, the processing unit 10 of the information processing device 1 detects this as being selected.

By combining the AR display using the HMD 3 and the tactile presentation device 2, the operator can visually observe an image in which the object is virtually placed in real space, and feel the touch of the object using the tactile presentation device. It is possible to memorize it in 2. At this time, similar to what was explained in the first embodiment, the tactile presentation start position in the tactile presentation device 2 can be corrected according to each operator, thereby further improving the degree of perception. .

The embodiments disclosed above are illustrative in all respects and are not restrictive. The scope of the present invention is indicated by the claims, and includes all changes within the meaning and range equivalent to the claims.

1 Information processing device 10 Processing unit 11 Storage unit 13 Display unit 130 Setting screen P1 Control program 2 Tactile presentation device 20 Control unit 21 Storage unit 202 Displacement unit 24 MRF device

Claims

It has a displacement part that is displaceable in response to an operation by an operator, and generates a force sensation in response to the operation on the displacement part according to the amount of displacement of the displacement part, and presents a tactile sensation of the displayed object. A control device for a tactile presentation device, comprising:
Control data for generating the force sensation is stored for each object in association with a displacement amount of the displacement portion,
Correcting the correspondence between the displacement amount of the control data and the control data based on the settings,
A control device that outputs corrected control data associated with a displacement amount of the displacement portion to the tactile presentation device.
The control device according to claim 1, wherein a shift in the amount of displacement for starting to output the control data that changes continuously in ascending order of the amount of displacement is accepted as the setting.
The control device according to claim 2, wherein the setting is received on a setting screen including a slide bar indicating a position of deviation with respect to the range of the displacement amount.
The control device according to claim 2, wherein the settings are accepted on a settings screen that selectably displays a plurality of candidates having different amounts of deviation from the displacement amount range.
It has a displacement part that is displaceable in response to an operation by an operator, and generates a force sensation in response to the operation on the displacement part according to the amount of displacement of the displacement part, and presents a tactile sensation of the displayed object. A method for controlling a tactile presentation device, comprising:
A computer connected to the tactile presentation device,
Control data for generating the force sensation is stored for each object in association with a displacement amount of the displacement portion,
Correcting the correspondence between the displacement amount of the control data and the control data based on the settings,
A control method including a processing section that outputs corrected control data associated with a displacement amount of the displacement section to the tactile presentation device.
It has a displacement part that is displaceable in response to an operation by an operator, and generates a force sensation in response to the operation on the displacement part according to the amount of displacement of the displacement part, and presents a tactile sensation of the displayed object. a tactile presentation device that
an information processing device that is communicatively connected to the tactile presentation device and includes a display unit;
The information processing device includes:
Control data for causing the force sensation and visual data to be displayed on the display unit are stored for each object in association with the displacement amount of the displacement unit,
Correcting the correspondence between the displacement amount of the control data and the control data and visual data based on the settings,
outputting corrected control data associated with the displacement amount of the displacement portion to the tactile presentation device;
A tactile presentation system that outputs an image to the display unit according to the amount of displacement of the displacement unit based on the corrected visual data.
It has a displacement part that is displaceable in response to an operation by an operator, and generates a force sensation in response to the operation on the displacement part according to the amount of displacement of the displacement part, and presents a tactile sensation of the displayed object. a computer connected to a tactile presentation device that
Control data for generating the force sensation is stored for each object in association with a displacement amount of the displacement portion,
Correcting the correspondence between the displacement amount of the control data and the control data based on the settings,
A program product that executes a process of outputting corrected control data associated with a displacement amount of the displacement portion to the tactile presentation device.