Experiment #102 How to Deploy and Integrate Azure Cognitive Services: Computer Vision and Text Analytics ๐Ÿ’ป๐Ÿ“‘

1.ย ย ย ย  Create the Computer Vision Resource

Computer Vision is an Azure Cognitive Service that allow us to get information from an image. It analyzes the image environment, detects objects inside them and classifies all of them by assigning tags and labels. We can find this resource in our Azure Portal under the โ€œAI + Machine Learningโ€ group:

 

 

After deploying the service, we must look and write down at the subscription key and the endpoint to integrate the service in our project.

 

2.ย ย ย ย  Create the Text Analytics Resource

Text Analytics is an Azure Cognitive Service that allow us to get information from a text. It analyzes the text context, detects subjects and topics and classifies all of them by assigning tags and labels. We can also find this resource in our Azure Portal under the โ€œAI + Machine Learningโ€ group:

 

 

After deploying the service, we must look and write down at the subscription key and the endpoint to integrate the service in our project.

 

3.ย  ย  ย Get caption and tags from Images

Now, we are going to integrate the HTTP requests to the Computer Vision API to send it an image and get caption and tags:

 

subscription_key = 'xxxxxxxxx'
assert subscription_key

vision_endpoint = 'https://xxxxxxxxx.cognitiveservices.azure.com/'
vision_base_url = vision_endpoint + "vision/v1.0/"
vision_analyze_url = vision_base_url + "analyze"

images_folder = "https://raw.githubusercontent.com/idiWork/Experiment_102/master/images/idiwork_experiment_102_sample_image_"
resort_pool = images_folder + "01.jpg"
resort_room = images_folder + "02.jpg"
resort_diner = images_folder + "03.jpg"
hotel_diner = images_folder + "04.jpg"
hotel_pool = images_folder + "05.jpg"
hotel_room = images_folder + "06.jpg"

image_url = resort_room

from IPython.display import Image, display
display(Image(image_url))

import requests
headers  = {'Ocp-Apim-Subscription-Key': subscription_key }
params   = {'visualFeatures': 'Categories,Description,Tags,Color'}
data     = {'url': image_url}
response = requests.post(vision_analyze_url, headers=headers, params=params, json=data)
response.raise_for_status()
analysis = response.json()
analysis

caption = analysis["description"]["captions"][0]["text"].capitalize()
caption

topTags = analysis["description"]["tags"][0:4]
topTags

 

4.ย ย ย ย  Get sentiment from Text

Then, we are going to implement the HTTP requests to the Text Analytics API to send it a text and get the overall sentiment:

 

text_analytics_subscription_key = 'xxxxxxxxx'
assert text_analytics_subscription_key

text_analytics_base_url = 'https://xxxxxxxxx.cognitiveservices.azure.com/text/analytics/v2.0/'
sentiment_api_url = text_analytics_base_url + "sentiment"

neg_sent = """
This is the worst stay I have every had! I have nothing positive to say about the diner and am extremely angry.
Poor service, bad food and too expensive drinks. I will not recommend it to anyone.
"""
pos_sent = """
I would like to thank you very much for a wonderful stay at the hotel. 
The room we stayed in was very nice and had plenty of room for the whole family and the beds were especially comfortable.
"""
neutral_sent = """
The kids go to the swimming pool all day while we take sunbaths near them.
We have an umbrella and a deck chair for each of us if we go early in the morning. 
"""
long_review = """
We arrived in the evening and the reception was very busy, 
so, we decided to wait until it was quieter. 
Unfortunately, we could not charge anything to the room yet as we were not checked in. 
We could not use the Wi-Fi either so had to use the guest Wi-Fi which was very slow and poor quality.
Once we checked in, the room was very nice and had fantastic views, but the bathroom was quite dirty.  
We called reception and they sent housekeeping who were very quick to clean the bathroom properly and even gave us some free bath toys for the kids which they loved. 
Overall the stay was pleasant and the service from staff was excellent. 
"""

review_text = long_review

documents = {'documents' : [
    {'id': '1', 'language': 'en', 'text': review_text}
]}
headers   = {"Ocp-Apim-Subscription-Key": text_analytics_subscription_key}
response  = requests.post(sentiment_api_url, headers=headers, json=documents)
sentiments = response.json()
sentiments

score = sentiments['documents'][0]['score']
score

score_interpretation = "Neutral"
if (score < 0.4): 
    score_interpretation = "Negative"
elif (score >= 0.6):
    score_interpretation = "Positive"
score_interpretation

 

5.ย ย ย ย  Invoke the Deployed Services

In this step we are going to invoke the services previously created and deployed to use them in combination with the Azure Cognitive Services:

 

def invoke_service(ml_service_key, ml_service_scoring_endpoint, ml_service_input):
    headers   = {"Authorization": "Bearer " + ml_service_key}
    response  = requests.post(ml_service_scoring_endpoint, headers=headers, json=ml_service_input)
    result = response.json()
    return result

classifier_service_key = "" 
classifier_service_scoring_endpoint = 'http://xxx-xxx-xxx-xxx-xxx.westeurope.azurecontainer.io/score'
classifier_service_input = [review_text]

classifier_result = invoke_service(classifier_service_key, classifier_service_scoring_endpoint, classifier_service_input)
classifier_result

classification = 'Room Review'
if classifier_result == 1:
    classification = 'Diner Review'
elif classifier_result == 2:
    classification = 'Pool Review'
classification

summarizer_service_key = "" 
summarizer_service_scoring_endpoint = 'http://xxx-xxx-xxx-xxx-xxx.westeurope.azurecontainer.io/score'
summarizer_service_input = review_text

summarizer_result = invoke_service(summarizer_service_key, summarizer_service_scoring_endpoint, summarizer_service_input)
formatted_result =  summarizer_result[0].replace("\\n", " ").strip() if len(summarizer_result) > 0 else "N/A"
formatted_result

 

6.ย  ย  ย Summarize the Results

Finally, we will create and apply an HTML template to display the results:

 

from IPython.core.display import HTML

displayTemplate = """
<div><h3>Claim Summary</h3></div>
<div> </div>
<div><b>Classification:</b> {}</div>
<div><b>Caption:</b> {}</div>
<div><b>Tags:</b> {}</div>
<div><b>Sentiment:</b> {}</div>
<div> </div>
<div><img src='{}' width='200px'></div>
<div> </div>
<div><b>Summary:</b></div>
<div>{}</div>
<div> </div>
<div><b>Claim:</b></div>
<div>{}</div>
"""

displayTemplate = displayTemplate.format(classification, caption, ' '.join(topTags), score_interpretation, image_url, formatted_result, review_text)

display(HTML(displayTemplate))

 

In the following animation you will see the processes and outputs of running these steps in Azure Notebooks with Jupyter:

 

 

Step by step

Idiwork has arrived and we invite you to join us!
Raspberry Pi4 – All of a sudden!
Experiment #101 How to set up an IoT device in Azure IoT Hub
Start your own video game with Unity3D and CreatorKits
Experiment #101 How To Create a Blockchain Workbench App
XR World: Minecraft Earth
Experiment #101 Architectural Diagram
Cyberpunk 2077 and The Future
Experiment #101 How to create an Azure Function App to record telemetry readings
An Introduction to neural networks
Microsoft Mixer, streaming your life !
Introduction to Azure Notebooks applying Cognitive Services with Jupyter
AR [T] Walk
Return of Age of Empires
Idiwork and Uno Platform partnership!!
The differences between Augemented Reality, Mixed Reality and Virtual Reality !
Experiment #102 How to Create an Azure Notebooks Project and Deploy a Summarization Service
Uno Platform Conference
YouTube Video: Creator Kits, learn how to create a RPG with Unity3D !! ๐Ÿค–
Experiment #102 How to construct and train a Deep Neural Network using Keras and deploy the model as an Azure Web Service ๐Ÿง โ˜๏ธ
Uno Platform Conference Recap ๐Ÿ˜Ž
How to create a Uno Platform project in Visual Studio ๐Ÿ’ป๐Ÿค“
What is a Neural Network? by Alberto Pinedo ๐Ÿง ๐Ÿ’ป
Next stop: Madrid Games Week ๐Ÿ‘‰๐Ÿค–
Our Madrid Games Week experience ! ๐Ÿ˜Ž๐Ÿค–
Experiment #102 How to use Microsoft Flow to send an email when an event occurs ๐Ÿ“ง๐Ÿ“Œ
How to create your own controllers for Mixed Reality ToolKit ๐ŸŽฎ๐Ÿค“
Experiment #102 Architectural Diagram ๐Ÿ“
The magic of Hytale โš”๏ธ
Start programming your own app in Uno Platform ๐Ÿ“ฒ๐Ÿ‘จโ€๐Ÿ’ป
Avalonia, a big candidate to create cross-platform apps with XAML ๐Ÿ˜Ž๐Ÿ“ฒ
The value of Blockchain for business by Pablo Junco โ›“๏ธ๐Ÿข
Living in the night: Bloodlines 2 ๐Ÿง›๐ŸŽฎ
Uno Platform Workshop Recap ๐Ÿ’ป๐Ÿ“Œ
EasyRPC. Be proud of your APIs (First Part) ๐Ÿค“๐Ÿš€
Paralives, reimagine your virtual life ๐Ÿ‘พ๐Ÿ‘ฉโ€๐Ÿ’ป
Experiment #103 How to Modify the Project to Work with Face Cognitive Service and Servo Motor ๐Ÿ‘ฉโ€๐Ÿ’ป๐Ÿ“™
Experiment #103 How to Set Up the IoT Device Hardware: Peripherals and Electronics ๐Ÿ”ˆ๐Ÿ’ก
We are going to be in the GDG DevFest in Mรกlaga! ๐Ÿ˜Ž๐Ÿ‘พ
EasyRPC. Be proud of your APIs (Second Part) ๐Ÿค“๐Ÿš€
Experiment #103 How to Build a 3D Printed Box to Pack and Run the IoT Project ๐Ÿš€
MediEvil, remember the fear๐Ÿ’€
Never forget the Fallen Order ๐Ÿน๐ŸŽฎ
Experiment #103 Architectural Diagram ๐Ÿ“
Grace: The perfect DI IoC container [part 1] ๐Ÿ’ป๐Ÿค“
Death Stranding: a story of death and connection โ˜ ๐Ÿน
Experiment #204 How to assembly the 3D printed parts and servo motors of the robotic arm ๐Ÿฆพ๐Ÿค–
TemTem: a dream come true ๐Ÿน๐Ÿ’ฅ
Grace: The perfect DI IoC container [part 2] ๐Ÿ’ป๐Ÿค“
Exploring the Outer Worlds ๐ŸŽ‡โš”
Our review from Hololens 2 ๐Ÿ‘“๐Ÿ‘ท
Learn to code on your iPhone for FREE ๐Ÿ“ฒ๐Ÿš€
New life, new horizons in Animal Crossing ๐ŸŽฎ๐Ÿฑ
Whatโ€™s WinUI? ๐Ÿ’ป๐Ÿš€
Half Life Alyx โ˜ ๐ŸงŸโ€โ™‚
How to deploy a Censorship Resistant Website for FREE ๐Ÿ”“๐ŸŒŽ
Superpower: The parser combinator library [Part 1] ๐Ÿ’ป๐Ÿค“
UnoPlatform and WinUI, what to expect? ๐Ÿš€๐Ÿ“ฒ
Happiness begins in Stardew Valley ๐ŸŽ๐Ÿ“
Science fiction in Assassinโ€™s Creed ๐Ÿ‘ฝ
The Cyber Attack Lifecycle ๐Ÿ•ตโ€โ™€๐Ÿ’ป
Superpower: The parser combinator library [Part 2] ๐Ÿ’ป๐Ÿค“
Learn more about UnoPlatform! ๐Ÿ˜œ๐Ÿ“ข
Create cybernetically enhanced web apps with SvelteJS ๐Ÿ’ป๐Ÿ‘‡
Six Fun Drag & Drop ๐Ÿงฉ Programming Languages To Learn How To Code! ๐Ÿ’ป
Riot and Hytale unite! ๐Ÿ‘พ๐Ÿ˜‰
A new home called EVE Online ๐Ÿ‘ฝ๐Ÿ‘‡
Rejoice with UnoPlatform! May 2020 ๐Ÿ“ฒ๐Ÿ‘‡
UnoPlatform arrives to macOS ๐Ÿ’ป๐Ÿ‘ˆ
Your new empire in Civilization VI ๐ŸŽฎ๐Ÿน
Play Station 5 is near!
Build and defend your city, this is Manor Lords
Customization, landscape generation and more in Hytale๐Ÿง๐Ÿปโ€โ™€๏ธ๐Ÿง๐Ÿผ
Welcome, MAUI! ๐Ÿ’ฃ๐Ÿ“ฒ
Zombies will be zombies (TLOU2) ๐Ÿ‘ฝ๐ŸงŸโ€โ™‚
Rejoice with UnoPlatform! August 2020 ๐Ÿ“ฒ๐Ÿ‘‡
Final Fantasy Crystal Chronicles is back๐Ÿง๐Ÿปโ€โ™€๏ธโš”๏ธ
Bolt is now free in Unity3D ๐ŸŽฎ
Feel the Flutter! ๐ŸŒ
Microsoft buys ZeniMax Media (including Bethesda!)
Intel and Microsoft team up to empower AI on Edge
Linux and WSL2 - Part 1 Linux and WSL2 – Part 1: How do you run Windows applications on Linux? Or vice versa.
Guide how to Run Windows applications on Linux Linux and WSL2 – Part 2: How do you run Windows applications on Linux? Or vice versa.
Domain-driven design Domain-Driven Design: the elephant in the room
azure object anchors Azure Object Anchors: the third tool
Experiment #205 Step by step 1 Experiment #205 Applied Artificial Intelligence, the real one ๐Ÿค–๐Ÿ“น
AI Assembling the system Experiment #205 Applied Artificial Intelligence – Assembling the system โš™๏ธ๐Ÿฆพ
Redit Conquer all the APIs Refit – Conquer all the APIs
Experiment #205 Applied AI: the Information analysis Experiment #205 Applied Artificial Intelligence – Analysis of the information
Brand Presence step by step 1 Blog Experiment #206 Brand Presence
Experiment #206 Brand Presence - The analysis Experiment #206 Brand Presence – The analysis

Stay up to date!



Leave a comment