JP5945966B2 - Portable terminal device, portable terminal program, server, and image acquisition system - Google Patents

Description

  The present invention relates to a portable terminal device, a portable terminal program, a server, and an image acquisition system for publicly providing an image privately photographed at a public venue.

  In recent years, there are many cases where a photographer is allowed to freely take a picture in a public venue such as a concert venue. In this case, the organizer may collect and manage the captured images via the Internet or the like, and may later use them as content materials.

  Patent Document 1 discloses a system that allows only a visitor to share an image that is privately photographed in a public venue. In this prior art, the wireless portable terminal is equipped with GPS and photographing means, and the server side permits registration of the photographed image only to the wireless portable terminal located in the predetermined zone within the predetermined period, and the registered wireless portable terminal Delivers the selected content at the destination address and transmission time specified by.

JP 2006-101157 A

  However, in the above-described prior art, it is difficult to efficiently collect and manage images privately taken by each visitor in a public venue via wireless communication. The reason is that each user himself / herself sends (uploads) an image arbitrarily selected at an arbitrary time to the server, so that it takes a long time, and similar images overlap. This is because the collected contents are biased, such as those that are not transmitted.

  On the other hand, there can be considered a method of automatically transmitting all shot images simultaneously to a large number of mobile terminal devices existing in the venue in the same time zone. In this case, a wireless communication device installed in the venue On the other hand, since radio communication is concentrated geographically and temporally, the traffic load becomes excessive and communication becomes difficult. In particular, it is impossible to individually transmit and receive a large-capacity photographed image such as a moving image by wireless communication.

  An object of the present invention is to provide a mobile terminal device, a mobile terminal program, a server, and an image acquisition system capable of automatically and smoothly transmitting and receiving only appropriate captured images from a large number of mobile terminal devices in the same venue. It is in.

  In order to achieve the above object, the present invention provides a predetermined method through wireless communication with a wireless communication device that is possessed by a user at a predetermined position in a predetermined area and in which the area in the area is a wireless communicable range. A portable terminal device capable of transmitting and receiving information to and from the server, wherein the position input means for inputting the position information of the predetermined position by the operation of the user, the image photographing means for photographing an image, and the photographing photographed by the image photographing means Based on the captured image storage means for storing the posture information and time information of the mobile terminal device detected when the image is captured in association with the captured image together with the image, and the position information and the posture information. Image determining means for determining whether the captured image is a photograph of a predetermined gaze position in the region, and before the image determining means determines that the gaze position has been captured. When the posture information, the position information, and the time information corresponding to the photographed image are transmitted to the server via the wireless communication device and the transmission is permitted by the transmission permission information received as a reply, the photographing is performed. Image transmitting means for transmitting an image to the server via the wireless communication device.

In order to achieve the above object, the present invention includes a portable terminal device capable of transmitting and receiving information to and from a predetermined server via wireless communication with a wireless communication device in which a region within a predetermined region is a wireless communication possible range. A computer-executable program for a portable terminal, a position input step for inputting position information of a predetermined position in the region of a user who owns the portable terminal device, an image photographing step for photographing an image, and the image A captured image storage step for storing, in addition to the captured image captured in the capturing step , the orientation information and time information of the mobile terminal device detected when the image is captured in association with the captured image, the position information, and the orientation An image determination step for determining whether the captured image is a photograph of a predetermined gaze position in the region based on the information; and The posture information, the position information, and the time information corresponding to the captured image determined to have captured the gaze position in the determination step are transmitted to the server via the wireless communication device and received as a reply An image transmission step of transmitting the captured image to the server via the wireless communication device when transmission is permitted by the transmission permission information.

  In order to achieve the above-described object, the present invention provides a mobile phone possessed by a user at a predetermined position in an area through wireless communication with a wireless communication apparatus having a predetermined area as a wireless communicable range. A server capable of transmitting / receiving information to / from a terminal device, wherein the posture information and the time based on position information, posture information, and time information of the predetermined position received from the portable terminal device via the wireless communication device Usefulness determination means for determining the usefulness of the captured image corresponding to the information, and transmission permission information for the captured image whose usefulness is determined to be higher than a certain level by the usefulness determination means via the wireless communication device. An image receiving unit that transmits to the terminal device and receives the captured image from the portable terminal device as a reply, and a received image that stores the captured image received by the image receiving unit Characterized in that it comprises a 憶 means.

  In order to achieve the above object, the present invention is capable of wireless communication with a mobile terminal device held by a user at a predetermined position within a predetermined area, and with the mobile terminal apparatus using the area within the area as a wireless communicable range. An image acquisition system including a wireless communication device and a server connected to the wireless communication device so that information can be transmitted and received, wherein the portable terminal device inputs position information of the predetermined position by an operation of the user. A position input unit, an image capturing unit that captures an image, a captured image captured by the image capturing unit, and attitude information and time information of the mobile terminal device detected when the image is captured, and the captured image The photographed image is obtained by photographing a predetermined gaze position in the region based on the photographed image storage means that stores the information in association with the position information and the posture information. Image determination means for determining whether or not the posture information, the position information, and the time information corresponding to the captured image determined to have captured the gaze position by the image determination means via the wireless communication device. Image transmission means for transmitting the captured image to the server via the wireless communication device when transmission is permitted by the transmission permission information received as a reply to the server. Is based on the position information, the posture information, and the time information received from the portable terminal device via the wireless communication device, and determines the usefulness of the captured image corresponding to the posture information and the time information. Usefulness determination means for determining, and transmission permission information for a photographed image whose usefulness is determined to be higher than a certain level by the usefulness determination means via the wireless communication device. An image receiving unit that transmits to the terminal device and receives the captured image from the portable terminal device as a reply; and a received image storage unit that stores the captured image received by the image receiving unit. Features.

  According to the present invention, only appropriate photographed images can be automatically and smoothly transmitted and received from a large number of portable terminal devices in the same venue.

It is explanatory drawing which shows the main structures of an image acquisition system. It is a functional block diagram which shows the main structures of a portable terminal device. It is a figure explaining the determination method of gaze position imaging in a portable terminal device. It is a sequence chart which shows the detail about information transmission / reception between a portable terminal device and a server. It is a figure which represents typically the picked-up image database memorize | stored in the portable terminal device side. It is a figure which represents typically the meeting place database memorize | stored on the server side. It is a figure which represents typically the collection image database memorize | stored on the server side. It is a flowchart which shows the processing content which CPU of the portable terminal device which started the exclusive application performs. It is a flowchart which shows the detailed processing content of the image imaging | photography recording process shown in FIG. It is a flowchart which shows the detailed process content of the image transmission process shown in FIG. It is a flowchart which shows the processing content which CPU of a server performs for every communicable time slot | zone. It is a figure showing the other structure which reduces the traffic load of the radio | wireless communication in a hall.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the present embodiment, an image of a music concert held at a public concert hall venue (predetermined area; hereinafter referred to as the venue) is privately photographed with the camera of the individual mobile terminal device of the visitor. Explain the case of publicly providing

  FIG. 1 is an explanatory diagram showing the main configuration of an image acquisition system 1 according to this embodiment. As shown in FIG. 1, the public part system installed in the venue 100 includes an in-venue access point 101 (wireless communication device) in which the audience seats (regions in the venue) in the venue 100 are wirelessly communicable. And an in-venue access point 101 that communicates with the inside of the venue and a server 103 connected so as to be able to send and receive information via a wired LAN 102 or the like.

  The in-venue access point 101 can send and receive information to and from the mobile terminal device 10 such as a smartphone possessed by the user Y (visitor, spectator) located in the spectator seat within the wireless communicable range 104 via a wireless LAN. ing. The server 103 is composed of a computer including a large-capacity storage device 103a (received image storage means) such as a hard disk.

  Then, a user Y who desires a service by the image acquisition system 1 of the present embodiment, that is, a music concert audience, has previously installed a dedicated application for the music concert (hereinafter abbreviated as a dedicated application as appropriate) on the Internet or the like. Come to the concert hall venue 100 with the device 10.

  The image acquisition system 1 of the present embodiment transmits and receives captured images via wireless communication between the mobile terminal device 10 and the server 103 possessed by a large number of users Y gathered at the concert hall venue 100.

  FIG. 2 is a functional block diagram showing the main configuration of the mobile terminal device 10. In FIG. 2, the mobile terminal device 10 includes a CPU 11, a ROM 12, a RAM 13, a nonvolatile storage device 14, a clock 15, a touch panel display 16, a camera 17 (image photographing means), an orientation sensor 18, A three-dimensional acceleration sensor 19 and a wireless LAN communication unit 20 are provided.

  The CPU 11 has a function of controlling the mobile terminal device 10 as a whole by performing various calculations according to the operation of a predetermined program and exchanging information with other units and outputting various control instructions.

  The ROM 12 is an information storage medium in which various processing programs and other necessary information are written in advance. The RAM 13 is an information storage medium on which information necessary for executing the various programs is written and read. The non-volatile storage device 14 is a non-volatile information storage medium such as a flash memory, for example, and stores a program for the dedicated application, and various information such as seat information, captured images, and captured image databases, which will be described later. The clock 15 has a function of detecting the current time.

  The touch panel display 16 integrates a liquid crystal display as a display unit and a touch panel as an input unit, and functions as a user interface of the mobile terminal device 10.

  The camera 17 includes an imaging pixel such as a CCD and captures an image in the imaging direction. As the types of images to be photographed, both still images and moving images are possible, but in the example of this embodiment, still images are handled as photographed images.

  The direction sensor 18 has a function of detecting the direction in which the imaging direction of the camera 17 is directed by detecting geomagnetism.

  The three-dimensional acceleration sensor 19 has a function of detecting the inclination (elevation angle) of the imaging direction of the camera 17 with respect to the horizontal direction by detecting each gravitational acceleration in the orthogonal three-axis directions.

  The wireless LAN communication unit 20 has a function of controlling information communication with the in-venue access point 101 in accordance with a wireless LAN standard such as so-called Wi-Fi (registered trademark).

  The image collection function by the image acquisition system 1 having the above configuration will be described. The image acquisition system 1 of this embodiment transmits and receives captured images via wireless communication between the mobile terminal device 10 and the server 103 possessed by a large number of spectators (users) gathered at the concert hall venue 100 in this example. I do. First, each user inputs position information of his / her seat (predetermined position) in the venue 100 to the mobile terminal device 10 by an operation input to the mobile terminal device 10 before the performance. Specific examples of the operation input include manual input of a seat number written on a ticket or a seat, input of information from an electronic ticket, or reading of a barcode printed on a ticket or a seat. About this, a well-known method should just be used and detailed description is abbreviate | omitted. In the hall 100, the user freely photographs in any direction and at any timing using the camera 17 built in the mobile terminal device 10.

  The organizer of the concert or the like collects and manages the images privately taken by each user in the public venue in this way on the server 103 via wireless communication. However, it is difficult to efficiently do so. It was. The reason for this is that only the images arbitrarily selected by the users themselves at the arbitrary time after the concert performance are sent (uploaded) to the server 103, so that it takes a long time to collect and the contents are similar. This is because the collected contents are biased, such as duplicated images and those that are not transmitted.

  On the other hand, a method of automatically transmitting all captured images simultaneously to a large number of mobile terminal devices 10 existing in the venue 100 during the concert performance is conceivable. Wireless communication concentrates spatially and temporally on the in-venue access point 101 installed in the venue, resulting in excessive traffic load and difficulty in communication. In particular, it is impossible to individually transmit and receive a large-capacity photographed image such as a moving image by wireless communication.

  Therefore, in the image acquisition system 1 of the present embodiment, among the images that the user has freely photographed, only appropriate ones are narrowed down and transmitted by both the mobile terminal device 10 and the server 103. Specifically, the mobile terminal device 10, together with the captured image captured by the camera 17, posture information (azimuth angle, elevation angle) and time information (capturing time) of the mobile terminal device 10 detected when the image is captured. Is stored in association with the photographed image. Furthermore, the mobile terminal device 10 is an image acquisition target such as a performer on a predetermined gaze position in the venue 100, that is, a performer on the stage, based on the position information (seat position) and posture information. It is determined whether the subject position is taken. As a result, it is possible to automatically narrow down only images that are suitable for transmission to the server 103 by the mobile terminal device 10 itself, among images that are freely photographed by the user. Note that the gaze position information may be received separately from the server 103 in advance and periodically updated in correspondence with the time information (detailed explanation is omitted for this point).

  A gaze position shooting determination method in the mobile terminal device 10 will be described with reference to FIG. FIG. 3A is a plan view of an arrangement relationship between a specific user Y in the venue 100 and an image acquisition target performer existing on the stage, and FIG. 3B is the same arrangement. It is the side view which looked at the relationship from the side.

  First, in FIG. 3A, in the illustrated example, it is assumed that the stage in the hall 100 is located in the direction of the true north direction (N0 °) when viewed from the entire audience seat. When the user Y located at a certain seating position in the spectator seat photographs the performer Z to be watched on the stage with the mobile terminal device 10, the direction sensor 18 detects the presence direction (N0 °) of the stage. The relative orientation X ° of the performer Z with respect to is detected. The presence direction of the stage with respect to the seat position of the user Y at this time is known in the fixed space coordinates in the hall, and the mobile terminal device 10 itself has such a reference direction (the presence direction of the stage) and the gaze position. For example, it can be determined whether or not the azimuth X ° detected by the azimuth sensor 18 faces the performer Z.

  In the example of this embodiment, the existence direction C ° of the stage center position 106 viewed from each seat is set as the respective reference direction, and each mobile terminal device 10 acquires this reference direction together with the gaze position from the server 103 separately. deep. Thereby, each portable terminal device 10 determines whether or not the shooting direction of the camera 17 faces the performer Z by comparing the relative direction D ° of the detection direction of the direction sensor 18 with respect to the reference direction and the gaze position. it can.

  In FIG. 3B, the presence elevation angle Y ° of the center of the stage (center area) viewed from each seat is set as each reference elevation angle, and each mobile terminal device 10 acquires this reference elevation angle from the server 103 separately. As shown in the example, when the gaze position of the performer Z does not change significantly in the height direction, the elevation angle detected by the three-dimensional acceleration sensor 19 does not change greatly, so the detected elevation angle substantially matches the reference elevation angle. Whether or not the shooting direction of the camera 17 faces the performer Z can be determined based on whether or not it is present.

  As described above, if the posture information that combines the detected orientation of the orientation sensor 18 at the time of image capturing and the detected elevation angle of the three-dimensional acceleration sensor 19 and the time information at the time of image capturing are known, it corresponds to the seat position of the user. It is possible to determine whether or not the captured image is a photograph of the performer Z by collating the reference azimuth, the reference elevation angle separately acquired in advance, and the gaze position acquired separately corresponding to the time information. .

  Note that, as described in FIG. 3A, in the spectator seat, an area where adjacent seats are gathered, that is, an area where a similar image may be shot in the same time zone, in particular, the shooting block 105 And

  Then, the mobile terminal device 10 transmits the posture information, the seat position information, and the time information corresponding to the captured image determined to have captured the gaze position to the server 103 via the wireless communication device. Receiving this, the server 103 determines the usefulness of the captured image corresponding to the posture information based on the position information, posture information, and time information of the seat. This usefulness is determined to be low if a similar seat position, posture, and time taken image has already been acquired, and is useful if it is a new image that has not yet been acquired. Is determined to be high. That is, the transmission priority for avoiding acquisition of overlapping images as much as possible is evaluated for each captured image. As a result, among the transmission candidate images narrowed down on the mobile terminal device 10 side, it is possible to narrow down only those more appropriate for acquisition on the server 103 side.

  Then, the server 103 transmits to the portable terminal device 10 transmission permission information for the captured image that is determined to have a usefulness higher than a certain level. In the example of the present embodiment, the transmission permission information is transmitted in a list format indicating the usefulness of each of a plurality of captured images that are permitted to be transmitted. Receiving this, the mobile terminal device 10 transmits a plurality of captured images permitted to be transmitted in the transmission permitted image list to the server 103 in order of transmission priority, and the server 103 transmits the captured images from the mobile terminal device 10. The data are sequentially received in order of priority and stored in the mass storage device 103a.

  At this time, the mobile terminal device 10 and the server 103 automatically narrow down an appropriate image and the server automatically selects only the carefully selected photographed images, without any particular attention to operations other than taking images freely. 103 can be transmitted and stored. As a result, the number of captured images transmitted is greatly reduced, so that even a captured image with a large capacity can be transmitted and received smoothly.

  Details of the information transmission / reception between the mobile terminal device 10 and the server 103 described above are shown in the sequence chart of FIG. Note that the change of the gaze position during the performance is not shown or described because it is only necessary to transmit or receive gaze position information from the server 103 to the mobile terminal device 10 as necessary or sequentially.

  First, when a spectator (user Y) enters the concert hall venue 100 (before the performance), the user Y activates a dedicated application of the mobile terminal device 10 and inputs a seat number. The portable terminal device 10 transmits the seat position information to the server 103, and the server 103 that has received the seat information receives seat information including a reference azimuth and a reference elevation angle corresponding to the seat position from a venue database (see FIG. 6 described later). Search and return to the mobile terminal device 10. The seat information may be received at that point if the seat number reserved when the dedicated application is downloaded is known, or the dedicated application itself may store the venue database.

  Then, after the performance starts, the user can take an image freely at an arbitrary orientation and at an arbitrary timing, and the captured image is associated with the time information and posture information (detection orientation, detected elevation angle), and a nonvolatile storage device 14 is recorded. At the same time, it is determined whether or not the photographed image is obtained by photographing the gaze position at that time, and the result is also recorded.

  Here, in the present embodiment, the mobile terminal device 10 performs wireless communication with the server 103 in a communicable time zone assigned for each group of predetermined distinction modes. That is, a large number of mobile terminal devices 10 existing in the venue 100 do not send and receive information to and from the server 103 all at once, but send and receive information to and from the server 103 by providing a time difference for each group (in a time division manner). In the example of the present embodiment, the grouping is different from the shooting block 105 (for example, reservation order, horizontal arrangement of spectator seats or vertical arrangement), but is the same as the shooting block 105. Also good. The communicable time zone is set a plurality of times so that the plurality of groups rotate during the performance, and the communication start time is also included in the seat information and received before the performance.

  The mobile terminal device 10 automatically starts transmitting / receiving information to / from the server 103 via wireless communication at the start time of the communicable time zone assigned during the performance, and automatically at the end time. End communication. Since the communication with the server 103 in this communicable time zone is performed by background processing of a dedicated application, the user does not need to be particularly aware of the operation.

  In the communicable time zone, the mobile terminal device 10 determines that the gaze position has been shot so far, and each image ID, time information (shooting time), and posture information of the shot image that has not been transmitted to the server 103. (Detection azimuth, detection elevation angle) and seat position are collectively transmitted to the server 103 as metadata (see FIG. 5 described later). The server 103 sorts images estimated to be similar to each other from the received metadata, and determines the transmission priority (usefulness) of each image so as to avoid redundant acquisition as much as possible. Then, a transmission permission image list is created and returned for each portable terminal device 10 based on the transmission priority of each image. Receiving this, the mobile terminal device 10 continues to transmit captured images to the server 103 in the order of transmission priority in the transmission-permitted image list, and continues until the communicable time zone at that time ends. In the communication unavailable time zone, only the determination of image shooting and recording and gaze position shooting is repeated.

  A captured image database stored on the mobile terminal device 10 side is shown in FIG. This is information stored in the non-volatile storage device 14 and is associated with the image ID given in the order of image shooting after the performance starts, and the shooting time, detection direction (X °), detection elevation angle (Y °), A determination flag, a transmitted flag, and binary data of the captured image are stored. Among these, in addition to the image ID, shooting time, detection direction, and detection elevation angle, the position information of the seat position of the user is transmitted to the server 103 as the above-described metadata. The determination flag stores 1 when it is determined that the gaze position is captured, and 0 when it is not determined. Further, the transmitted flag stores 1 in the case where the image data is transmitted to the server 103 and 0 in the case where the image data is not transmitted. Although not particularly illustrated, in the transmission-permitted image list received from the server 103, a plurality of image IDs permitted to be transmitted are associated with transmission priorities.

  The venue database stored on the server 103 side is shown in FIG. This is information stored in the mass storage device 103a of the server 103, and is set for each venue. In this venue database, a reference azimuth toward the center of the stage, a reference elevation angle, a communication start time (plurality), and a distance to the center of the stage are stored in association with each seat number. Information for one seat is transmitted to the mobile terminal device 10 as the seat information described above before the performance.

  FIG. 7 shows a collected image database stored on the server 103 side. This is information stored in the large-capacity storage device 103a of the server 103, and is associated with the image ID given from each mobile terminal device 10 in the order of reception as a transmission-permitted photographed image, its photographing time, detection direction (X °), detected elevation angle (Y °), seat number, estimated stage area, and binary data of the transmission-permitted photographed image. Regarding the stage area, in the example of this embodiment, the area on the stage is divided into three by center (center), left, and right, and when the server 103 receives each metadata, each seat position By referring to the information and the posture information, it is estimated and recorded which stage area the captured image of the metadata has captured. Note that the stage area may be estimated together with the gaze position shooting determination on the mobile terminal device 10 side and recorded in the captured image database.

  The processing contents executed by the CPUs of the mobile terminal device 10 and the server 103 when the image acquisition system 1 processes various types of information as described above will be described with reference to the flowcharts of FIGS.

  FIG. 8 is a flowchart showing the processing contents executed by the CPU 11 of the mobile terminal device that has activated the dedicated application, and the execution is started when the dedicated application is activated.

  In FIG. 8, first, in step SK105, the CPU 11 of the mobile terminal device inputs a seat number via an operation input (or other information input) from the user.

  Next, the process proceeds to step SK110, and the CPU 11 of the portable terminal device transmits the position information of the seat number input in step SK105 to the server 103 via wireless LAN communication with the in-venue access point 101.

  Next, moving to step SK110, the CPU 11 of the mobile terminal device determines whether or not seat information has been received from the server 103, in other words, whether or not retrieval of seat information has been successful. If the search for seat information fails, the determination is not satisfied, and the procedure returns to step SK105 and the same procedure is repeated.

  On the other hand, if the search for seat information is successful, the determination is satisfied, and the routine goes to Step SK120.

  In step SK120, the CPU 11 of the mobile terminal device stores the received seat information in the nonvolatile storage device 14.

  Next, the process proceeds to step SK125, where the CPU 11 of the mobile terminal device displays a screen for image capturing (not shown) on the touch panel display 16, and accepts an operation input for image capturing.

  Next, the process proceeds to step SK130, and the CPU 11 of the mobile terminal device determines whether or not an operation for ending the dedicated application is input through an operation input from the user. If the end operation is input, the determination is satisfied, and this flow ends.

  On the other hand, if the end operation has not been input, the determination is not satisfied and the routine goes to Step SK135.

  In step SK135, the CPU 11 of the portable terminal device determines whether or not an image is captured by operating a photographing button on the touch panel display 16 (not shown). If an image shooting operation has been performed, the determination is satisfied, and the routine goes to Step SK200.

  In step SK200, the CPU 11 of the portable terminal device records the photographing time, camera orientation, and camera elevation angle detected together with the image photographing by the camera 17, and the photographed image photographs the gaze position at that time. An image shooting and recording process is executed to determine whether or not (see FIG. 9 described later). Then, the process returns to step SK130 and the same procedure is repeated.

  On the other hand, if it is determined in step SK135 that no image shooting operation has been performed, the determination is not satisfied, and the routine goes to step SK140.

  In step SK140, the CPU 11 of the mobile terminal device determines whether or not the next communication start time has been reached. If the communication start time has not been reached, the determination is not satisfied, and the procedure returns to step SK130 and the same procedure is repeated.

  On the other hand, when the communication start time is reached, the determination is satisfied, and the routine goes to Step SK300.

  In step SK300, the CPU 11 of the mobile terminal device starts execution of an image transmission process for transmitting a captured image by transmitting / receiving information to / from the server 103 as a background process. That is, the execution of the image transmission process is started internally in parallel with returning to step SK130 and repeating the same procedure. As a result, the user can continue to take an arbitrary image without being aware of the image transmission process.

  FIG. 9 is a flowchart showing the detailed processing contents of the image capturing / recording processing shown in FIG.

  In FIG. 9, first, in step SK205, the CPU 11 of the portable terminal device captures an image with the camera 17, and records the image data in the captured image database shown in FIG. At this time, image IDs are generated in the order of shooting, and image data is recorded in association with the image IDs.

  Next, the process proceeds to step SK210, and the CPU 11 of the portable terminal device detects the shooting time, shooting direction, and shooting elevation angle at the time of shooting in step SK205, and records them in the shot image database in association with the same image ID. At this time, the corresponding transmitted flag is recorded as 0.

  Next, the process proceeds to step SK215, and the CPU 11 of the portable terminal device analyzes whether the image captured in step SK205 is an image of the gaze position at that time. The analysis at this time is performed by the method described in FIG. 3, that is, the reference direction of the seat information separately acquired in advance with respect to the posture information of the detection direction and the detection elevation angle and the time information of the photographing time. And the reference elevation angle and the gaze position separately acquired corresponding to the time information (sequentially acquired from the server 103 during the performance) are analyzed to determine whether the captured image is the image of the performer Z.

  Next, the process proceeds to step SK220, and the CPU 11 of the portable terminal device determines whether or not it has been analyzed that the gaze position is photographed in step SK215. If it is analyzed that the gaze position is captured, the determination is satisfied, and the routine goes to Step SK225.

  In step SK225, the CPU 11 of the mobile terminal device records 1 in the determination flag corresponding to the image ID of the captured image in the captured image database. Then, this flow ends.

  On the other hand, in the determination of step SK220, if it is analyzed that the gaze position is not photographed, the determination is not satisfied and the process proceeds to step SK230.

  In step SK230, the CPU 11 of the mobile terminal device records 0 in the determination flag corresponding to the image ID of the captured image in the captured image database. Then, this flow ends.

  FIG. 10 is a flowchart showing the detailed processing contents of the image transmission processing shown in FIG. Note that, as described above, this image transmission process is performed as a background process without any display on the touch panel display 16 of the mobile terminal device 10.

  In FIG. 10, first, in step SK <b> 305, the CPU 11 of the mobile terminal device transmits information for requesting the start of wireless LAN communication connection with the server 103 via the in-venue access point 101.

  Next, the process proceeds to step SK310, and the CPU 11 of the mobile terminal device determines whether or not the connection of the wireless LAN communication is successful. If the connection fails, the determination is not satisfied, and the process returns to step SK305 and the same procedure is repeated.

  On the other hand, if the connection is successful, the determination is satisfied, and the routine goes to Step SK315.

  In step SK315, the CPU 11 of the mobile terminal device extracts metadata of the image ID (including the position information of the seat position) whose determination flag is 1 and whose transmission flag is 0 in the captured image database, Are all transmitted to the server 103.

  Next, the process proceeds to step SK320, and the CPU 11 of the portable terminal device receives the transmission permitted image list from the server 103 as a reply to the metadata transmitted in step SK315.

  Next, the process proceeds to step SK325, and the CPU 11 of the portable terminal device transmits one image data of the image ID specified in the transmission permitted image list received in step SK320 in the order of transmission priority.

  Next, the process proceeds to step SK330, and the CPU 11 of the mobile terminal device records 1 in the transmitted flag corresponding to the image data transmitted in step SK325 in the photographed image database.

  Next, the process proceeds to step SK335, and the CPU 11 of the mobile terminal device determines whether or not the communication available time period has ended, in other words, a specified time (2 minutes in this example) has elapsed from the communication start time of the time. It is determined whether or not. If the communication end time has not been reached, the determination is not satisfied and the routine goes to Step SK340.

  In step SK340, the CPU 11 of the portable terminal device determines whether there is an untransmitted image among the transmission permitted images, in other words, the transmitted flag corresponding to the image ID specified in the transmission permitted image list is still 0. Determine if there is something. If there is an untransmitted image, the determination is satisfied, and the process returns to step SK325 to repeat the same procedure.

On the other hand, if it is determined at step SK335 that the communication end time has been reached, the determination is satisfied, and the routine goes to step SK345.

On the other hand, if there is no untransmitted image in the determination at step SK340, the determination is not satisfied and the routine goes to step SK345.

  In step SK345, the CPU 11 of the mobile terminal device disconnects the wireless LAN connection and ends the wireless communication with the server 103. At this time, the wireless communication is forcibly ended even if the image data of the transmission-permitted image is still being transmitted even when the communication end time is reached. It should be noted that all the transmitted flags of the image IDs that have not been transmitted at this time should be recorded in all other than 0 and 1 (for example, 2) so that the metadata extraction and transmission will not be duplicated after the next time. Then, this flow ends.

  In the above, the procedure of step SK105 corresponds to the position input means described in each claim, the procedure of step SK210 corresponds to the captured image storage means described in each claim, and the procedure of step SK215 corresponds to the image determination described in each claim. The entire image transmission processing in step SK300 corresponds to the image transmission means described in each claim, and the procedure in step SK120 corresponds to the allocated time storage means described in each claim.

  FIG. 11 is a flowchart showing the processing contents executed by the CPU (not shown) of the server 103 for each communicable time zone, and the execution is started every time the communication start time is reached. The seat information retrieval process before the performance will be omitted.

  In FIG. 11, first, in step SS <b> 105, the CPU of the server 103 receives the gaze position captured image metadata sent from each mobile terminal device 10 via wireless LAN communication. Here, although not particularly illustrated, with reference to the seat position information and posture information (detected azimuth and detected elevation angle) of each received metadata, the stage areas (center, left, Light). The estimation of the stage area may be performed on the mobile terminal device 10 side. In this case, the estimation result of the stage area is recorded in the captured image database corresponding to each captured image, and this is also included in the metadata. May be transmitted.

  Next, the process moves to step SS110, and the CPU of the server 103 extracts and distributes all the metadata received in step SS105 by referring to the respective seat position information and belonging to the same photographing block 105.

  Next, the process proceeds to step SS115, and the CPU of the server 103 further extracts and distributes images taken in the same time zone with reference to the shooting time from the sorting in step SS110.

  Next, the process proceeds to step SS120, and the CPU of the server 103 extracts and distributes what is presumed to have been taken for the same stage area from the distribution in step SS115.

  Next, the process proceeds to step SS125, and the CPU of the server 103 sets, for example, a transmission priority in a five-step evaluation for each metadata (not shown). That is, since the metadata distributed in steps SS110 to SS120 is likely to have similar contents of the corresponding captured images (untransmitted), the transmission priority is calculated from the number of data, Set the transmission priority of metadata.

  Next, the process proceeds to step SS130, and the CPU of the server 103 returns to step SS110 and repeats the same procedure until the transmission priority is set for all additionally received metadata. When all the metadata is received in the communicable time zone at that time and the transmission priority is set for each metadata, the process proceeds to the next step SS135.

  In step SS135, the CPU of the server 103 organizes all metadata and transmission priority settings for each portable terminal device 10, creates each as a transmission-permitted image list, and transmits it to the corresponding portable terminal device 10.

  Next, the process proceeds to step SS140, and the CPU of the server 103 receives a transmission-permitted photographed image (image data) from the mobile terminal device 10 as a reply to the transmission-permitted image list transmitted in step SS135. Then, an image ID for the server 103 is assigned to the received image data in the order of reception, and is recorded in the collected image database together with the metadata of the image data and the stage area.

  Next, the process proceeds to step SS145, in which the CPU of the server 103 determines whether or not the communication available time period has ended, in other words, whether or not the specified time (2 minutes in this example) has elapsed from the communication start time of that time. Determine whether or not. If the communication end time has not been reached, the determination is not satisfied and the routine goes to Step SS150.

  In step SS150, the CPU of the server 103 determines whether there is an unreceived transmission permission image. If there is an unreceived image, the determination is satisfied, and the process returns to step SS140 and the same procedure is repeated.

  On the other hand, if it is determined in step SS145 that the communication end time has been reached, the determination is satisfied, and the routine goes to step SS155.

  On the other hand, if there is no unreceived image in the determination at step SS150, the determination is not satisfied and the routine goes to step SS155.

  In step SS155, the CPU of the server 103 disconnects the wireless LAN connection and ends the wireless communication with the mobile terminal device 10. At this time, the wireless communication is forcibly ended even if the image data of the transmission-permitted image is still being received even when the communication end time is reached. Then, this flow ends.

  In the above, the procedure of step SS125 corresponds to the usefulness determining means described in each claim, and the procedure of step SS140 corresponds to the image receiving means described in each claim.

  As described above, in this embodiment, the user automatically narrows down an appropriate image by the mobile terminal device 10 and the server 103 without being particularly aware of the operation other than taking an image freely, and the selected image is carefully selected. Only captured images can be automatically transmitted to the server 103 for storage. As a result, the number of captured images transmitted is greatly reduced, so that even a captured image with a large capacity can be transmitted and received smoothly. As a result, only appropriate captured images can be automatically and smoothly transmitted and received from a large number of portable terminal devices 10 in the same venue 100.

  In the present embodiment, in particular, the mobile terminal device 10 stores the seat information including the communication start time information in the nonvolatile storage device 14 in the procedure of step SK120, and the mobile terminal device 10 in the image transmission process of step SK300. Information is transmitted to and received from the server 103 via the in-venue access point 101 only during the communicable time zone assigned to the group to which the user belongs. Thereby, since a large number of mobile terminal devices 10 existing in the venue 100 perform wireless communication with a time difference for each group, the traffic load of the access point 101 in the venue can be reduced.

  As another method for reducing the traffic load on the access point 101 in the venue as described above, for example, as shown in FIG. 12, the entire audience seat is divided into a plurality of areas, and each area can be wirelessly communicable 104A to 104D. A plurality of in-venue access points 101A to 101D may be provided. In this case, since the number of mobile terminal devices 10 with which the in-venue access points 101A to 101D communicate with each other is small, information can be transmitted and received constantly by wireless communication. In the example shown in the figure, information is processed by one server 103 for four in-venue access points 101A to 101D, but in addition to this, individual access points 101A to 101D are individually associated with the in-venue access points 101A to 101D. Servers 103A to 103D may be provided. In this case, the collected image database shown in FIG. 7 is stored in each of the servers 103A to 103D, and an image ID is assigned individually. Thereby, the traffic load of wireless communication can be reduced and information can be transmitted and received more smoothly.

  In this embodiment, an example in which a still image is handled as image data has been described. However, a moving image may be handled as image data. In addition, the transmission permitted image list may list only photographed images whose transmission priority is determined to be higher than a certain level. In the present embodiment, an image shooting screen (not shown) is displayed on the touch panel display 16 of the mobile terminal device 10 during the performance. This image shooting screen is displayed while the shooting direction of the camera 17 is directed to the gaze position. You may make it display on that in real time. In this case, the gazing point shooting analysis of step SK215 is always performed without waiting for the image shooting operation, and only when the shooting direction of the camera 17 is determined to be directed to the gazing position, the fact is displayed on the image shooting screen. May be displayed. This makes it easy for the user to consciously capture the gaze position.

  In the present embodiment, the position information of the mobile terminal device 10 is specified by the user's seat position. In addition, the mobile terminal device 10 has a built-in GPS sensor, and the position information is expressed by the latitude and longitude of the user's position. You may specify. In the present embodiment, it is assumed that the gaze position on the stage is the position of one performer Z who moves, but in addition to this, a fixed position or a plurality of positions may be set.

  In addition to those already described above, the methods according to the above-described embodiments and modifications may be used in appropriate combination.

  In addition, although not illustrated one by one, the present invention is implemented with various modifications within a range not departing from the gist thereof.

DESCRIPTION OF SYMBOLS 1 Image acquisition system 10 Portable terminal device 11 CPU
12 ROM
13 RAM
DESCRIPTION OF SYMBOLS 14 Nonvolatile memory | storage device 15 Clock 16 Touch panel display 17 Camera 18 Direction sensor 19 Three-dimensional acceleration sensor 20 Wireless LAN communication part 100 Concert hall venue 101 Access point 102 in a hall 102 Wired LAN
103, 103A, servers 103B, 103C,
103D
103a Mass storage device (received image storage means)
104, 104A, wireless communication range 104B, 104C,
104D
105 Shooting block 106 Stage center position Y User Z Performer

Claims (5)

A portable terminal device that is possessed by a user at a predetermined position in a predetermined area and can transmit and receive information to and from a predetermined server via wireless communication with a wireless communication apparatus in which the area in the area is a wireless communicable range. And
Position input means for inputting position information of the predetermined position by an operation of the user;
Image photographing means for photographing an image;
A captured image storage unit that stores, together with the captured image captured by the image capturing unit, posture information and time information of the mobile terminal device detected when the image is captured in association with the captured image;
Image determining means for determining, based on the position information and the posture information, whether the captured image is an image of a predetermined gaze position in the region;
The posture information, the position information, and the time information corresponding to the captured image determined to have captured the gaze position by the image determination unit are transmitted to the server via the wireless communication device, and the reply An image transmitting means for transmitting the captured image to the server via the wireless communication device when transmission is permitted by the received transmission permission information. The portable terminal device
Allocated time storage means for storing a communicable time zone allocated for each group;
With
2. The portable device according to claim 1, wherein the image transmission unit transmits and receives information to and from the server via the wireless communication device only during a communicable time zone assigned to a group to which the portable terminal device belongs. Terminal device. A mobile terminal program executable by a computer provided in a mobile terminal device capable of transmitting and receiving information to and from a predetermined server via wireless communication with a wireless communication device having a predetermined area as a wireless communicable range ,
A position input step of inputting position information of a predetermined position in the area of the user possessing the mobile terminal device;
An image capturing step for capturing an image;
A captured image storage step for storing, together with the captured image captured in the image capturing step, the posture information and time information of the mobile terminal device detected when the image is captured, in association with the captured image;
An image determination step for determining whether the captured image is obtained by capturing a predetermined gaze position in the region based on the position information and the posture information;
The posture information, the position information, and the time information corresponding to the photographed image determined to have photographed the gaze position in the image judging step are transmitted to the server via the wireless communication device, and the reply An image transmission step of transmitting the captured image to the server via the wireless communication device when transmission is permitted by the received transmission permission information;
A program for a mobile terminal, characterized in that A server capable of transmitting and receiving information to and from a mobile terminal device possessed by a user at a predetermined position in the area through wireless communication with a wireless communication apparatus in which a predetermined area within the predetermined area is wirelessly communicable,
Based on the position information, the posture information, and the time information of the predetermined position received from the mobile terminal device via the wireless communication device, the usefulness of the captured image corresponding to the posture information and the time information is determined. Usefulness determination means;
Transmission permission information for a captured image that has been determined to have a usefulness that is higher than a certain level by the usefulness determination unit is transmitted to the portable terminal device via the wireless communication device, and the captured image is transmitted from the portable terminal device as a reply. Image receiving means for receiving;
Received image storage means for storing the captured image received by the image receiving means;
A server characterized by comprising: A portable terminal device possessed by a user at a predetermined position in a predetermined area;
A wireless communication device capable of wirelessly communicating with the mobile terminal device using the region within the region as a wireless communicable range;
A server connected to the wireless communication device so as to be able to send and receive information;
An image acquisition system comprising:
The portable terminal device
Position input means for inputting position information of the predetermined position by an operation of the user;
Image photographing means for photographing an image;
A captured image storage unit that stores, together with the captured image captured by the image capturing unit, posture information and time information of the mobile terminal device detected when the image is captured in association with the captured image;
Image determining means for determining, based on the position information and the posture information, whether the captured image is an image of a predetermined gaze position in the region;
The posture information, the position information, and the time information corresponding to the photographed image determined to have photographed the gaze position by the image judging means are transmitted to the server via the wireless communication device, and the reply Image transmission means for transmitting the captured image to the server via the wireless communication device when transmission is permitted by the transmission permission information received as:
The server
Based on the position information, the posture information, and the time information received from the mobile terminal device via the wireless communication device, the usefulness of the captured image corresponding to the posture information and the time information is determined. Usefulness determination means;
Transmission permission information for a captured image that has been determined to have a usefulness that is higher than a certain level by the usefulness determination unit is transmitted to the portable terminal device via the wireless communication device, and the captured image is transmitted from the portable terminal device as a reply. Image receiving means for receiving;
Received image storage means for storing the captured image received by the image receiving means;
An image acquisition system comprising:

Images