Vision App is a sample iOS application to automatically tag images and detect faces by using IBM visual recognition technologies.

Take a photo or select an existing picture, let the application generate a list of tags and detect people, buildings, objects in the picture. Share the results with your network.

Built using IBM Bluemix, the application uses:

• Watson Visual Recognition
• OpenWhisk
• Cloudant

The application sends the picture to a Cloudant database. Then it calls an OpenWhisk action that will analyze the picture and send back the results of the analysis.

This application is one example use case. Equipped with the OpenWhisk action implemented in this example, an other use case could be to automatically classify images in a library to improve search capabilities: the same OpenWhisk action but used in a different context. Indeed with this action, we created a microservice for image analysis in the cloud, without deploying or managing a single server.

• IBM Bluemix account. Sign up for Bluemix, or use an existing account.
• IBM Bluemix OpenWhisk early access. Sign up for Bluemix OpenWhisk.
• XCode 8.1, iOS 10, Swift 3.0

• Clone the app to your local environment from your terminal using the following command:

  git clone https://github.com/IBM-Bluemix/openwhisk-visionapp.git
  

• or Download and extract the source code from this archive

1. Open the IBM Bluemix console

2. Create a Cloudant NoSQL DB service instance named cloudant-for-vision

3. Open the Cloudant service dashboard and create a new database named openwhisk-vision

4. Create a Watson Visual Recognition service instance named visualrecognition-for-vision

Note: if you have existing instances of these services, you don't need to create new instances. You can simply reuse the existing ones.

1. Ensure your OpenWhisk command line interface is property configured with:

wsk list

1. Create the action using the following command line replacing the placeholders with the credentials obtained from the respective service dashboards in Bluemix:

wsk action create -p cloudantUrl [URL] -p cloudantDbName openwhisk-vision -p watsonApiKey [123] vision-analysis analysis.js

To configure the iOS application, you need the credentials of the Cloudant service created above, your OpenWhisk authorization key.

1. Open vision.xcworkspace with XCode

2. Open the file vision/vision/model/ServerlessAPI.swift

3. Set the value of the constant CloudantUrl to the Cloudant service credentials url.

4. Set the value of the constants WhiskAppKey and WhiskAppSecret to your OpenWhisk credentials. You can retrieve them from the iOS SDK configuration page or you can retrieve the key and secret with the following CLI command:

wsk property get --auth

whisk auth kkkkkkkk-kkkk-kkkk-kkkk-kkkkkkkkkkkk:tttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttt

The strings before and after the colon are your key and secret, respectively.

1. Save the file

1. Start the application from XCode with iPhone 6s as the target

1. Select an existing picture

Note: To add pictures to the simulator, go to the home screen (Cmd+Shift+H). Drag and drop images from the Finder to the simular window. This will open the Photos app and you should see your images.

1. The picture is sent for analysis and results are returned:

Results are made of the faces detected in the picture and of tags returned by Watson. The tags with the highest confidence score are pre-highlighted. The highlighted tags will be used when sharing the picture. You can tap tags to toggle their state.

1. Press the Share button. This opens the standard iOS sharing screen.

Note: to configure a Twitter account, go to the Settings app on the simulator. Under Twitter, add your account (no need for the Twitter app to be installed). You can go back to the home screen with Cmd+Shift+H

1. Pick Twitter as example.

The picture and the highlighted tags are included in the message. The message can be edited before posting.

analysis.js holds the JavaScript code to perform the image analysis:

1. It retrieves the image data from the Cloudant document. The data has been attached by the iOS app as an attachment named "image.jpg".
2. It saves the image file locally.
3. If needed, it resizes the image so that it matches the requirements of the Watson service
4. It calls Watson
5. It returns the results of the analysis

The action runs asynchronously.

Please create a pull request with your desired changes.

Polling activations is good start to debug the OpenWhisk action execution. Run

wsk activation poll

and submit a picture for analysis.

A typical activation log when everything goes fine will look like:

Activation: vision-analysis (123fb4230902822202029fff436a94be745)
2016-02-23T16:17:53.955350233Z stdout: [ 49382920fdb022039403934b3bd33d00 ] Processing image.jpg from document
2016-02-23T16:17:59.847872226Z stdout: [ 49382920fdb022039403934b3bd33d00 ] OK

The application prints several statements to the console as it uploads, analyzes and updates the user interface. Make sure you correctly updated the constants in ServerlessAPI.swift.

The application uses:

• Alamofire (License)
• AlamofireImage (License)
• SwiftyJSON (License)
• TagListView (License)
• JGProgressHUD (License)
• RDHCollectionViewGridLayout (License)

See License.txt for license information.