Using Microsoft Computer Vision API with curl

-
Microsoft provides a powerful online Computer Vision API which allows to analyze images and provide the JSON data with the following features:
  • Objects on the image with bounding boxes and confidence:
    "rectangle":{"x":25,"y":132,"w":954,"h":776},"object":"fountain","confidence":0.561
  • Tags describing the image:
    "name":"nature","confidence":0.999956488609314
  • Categories with general description:
    "name":"outdoor_water","score":0.9921875
  • Captions with text description of the image:"text":"a large waterfall over a rocky cliff", "confidence":0.91645835840611234
The full list of features is the following:
https://westus.dev.cognitive.microsoft.com/docs/services/5adf991815e1060e6355ad44/operations/56f91f2e778daf14a499e1fa
  • Adult - detects if the image is pornographic in nature (depicts nudity or a sex act). Sexually suggestive content is also detected.
  • Brands - detects various brands within an image, including the approximate location. The Brands argument is only available in English.
  • Categories - categorizes image content according to a taxonomy defined in documentation.
  • Color - determines the accent color, dominant color, and whether an image is black&white.
  • Description - describes the image content with a complete sentence in supported languages.
  • Faces - detects if faces are present. If present, generate coordinates, gender and age.
  • ImageType - detects if image is clipart or a line drawing.
  • Objects - detects various objects within an image, including the approximate location. The Objects argument is only available in English.
  • Tags - tags the image with a detailed list of words related to the image content. 
 
To run it in from browser to process your image:
https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/ 
Here you can upload image and analyze it and copy description from browser manually.


If you need to process many images, you can do it using remote calls to the server.

The simplest way is to use REST API with curl:
https://docs.microsoft.com/en-us/azure/cognitive-services/computer-vision/quickstarts/curl-analyze 

(curl is a powerful command line tool for uploading and downloading data from servers using POST method.)
  
Here we describe using it for Windows.

1. Install curl - https://curl.haxx.se/windows/
https://cloud.mail.ru/public/4j97/5yEvJVRqy
 
2. Obtain a key by Creating a Cognitive Services account
- it's free for 7 days and allows to process 5000 images freely with rate 20 images per second.

(To check you key you may use this online console:

https://westcentralus.dev.cognitive.microsoft.com/docs/services/5adf991815e1060e6355ad44/operations/56f91f2e778daf14a499e1fa/console )


Now, create BAT-file with the command:
 
curl -k -H "Ocp-Apim-Subscription-Key: <your_key_here>" -H "Content-Type: application/json" "https://westcentralus.api.cognitive.microsoft.com/vision/v2.0/analyze?visualFeatures=Categories,Description,Objects,Tags&details=Landmarks&language=en" -d "{\"url\":\"http://upload.wikimedia.org/wikipedia/commons/3/3c/Shaki_waterfall.jpg\"}" > 0001.txt

  • -k  here is used to disable secure errorconsidering Microsoft certificate
  • <your_key_here> - please write your key here
  • Categories,Description,Objects,Tags - features to analyze (see above the full list of supported features)
  • http://upload.wikimedia.org/wikipedia/commons/3/3c/Shaki_waterfall.jpg\ - image to process.
  • > 0001.txt - this create file 0001.txt with result
Running this, you obtain the file 0001.txt:

{"categories":[{"name":"outdoor_water","score":0.9921875,"detail":{"landmarks":[]}}],
"tags":[{"name":"nature","confidence":0.999956488609314},{"name":"water","confidence":0.99656963348388672},{"name":"waterfall","confidence":0.99627882242202759},{"name":"outdoor","confidence":0.99618977308273315},{"name":"rock","confidence":0.97543877363204956},{"name":"mountain","confidence":0.9326665997505188},{"name":"rocky","confidence":0.82317483425140381},{"name":"cascade","confidence":0.60189425945281982},{"name":"hillside","confidence":0.314908504486084}],"description":{"tags":["nature","water","waterfall","outdoor","rock","mountain","rocky","grass","hill", "covered","hillside","standing","side","group","walking","white","man", "large","snow","grazing","forest","slope","herd","river","giraffe","field"], 
"captions":[{"text":"a large waterfall over a rocky cliff", "confidence":0.91645835840611234}]}, 
"objects":[{"rectangle":{"x":25,"y":132,"w":954,"h":776},"object":"fountain","confidence":0.561}],"requestId":"2ed01477-5307-4f27-91cd-e3d948ed3137","metadata":{"width":1280,"height":959,"format":"Jpeg"}}


Now, you can change BAT file to process image sequences (with pause 3 seconds because of limit 20 images per second).

Note: to apply it for your own images, you need to upload the images on some web-hosting. 
I using jino.ru, uploading there files using FTP (Total Commander Ctrl+F) and the use them.

Comments

Post a Comment

Popular posts from this blog

Computing ray origin and direction from Model View Projection matrices for raymarching

-
When performing raymarching (as well a raycasting and raytracing) using fragment shaders in OpenGL, it's required to compute ray's origin and direction. Here we will give solution for finding it from the Model View Projection matrices in OpenGL, implemented in openFrameworks. Using it, you can combine raymarching/raycasting/raytracing with forward OpenGL rendering, so obtain hybrid rendering modes. Also, you can use it for creating VE applications based on raymarching or hybrid rendering. (Actually, we checked what this approach is corrent by creating VR rendering app in HTC Vive). The algorithm can be used on any OpenGL platform with any language supporting matrix inversion computation (in our case it's C++ with GLM library, included in openFrameworks 0.10.1). The approach is based on this hint by  GClements in this discussion on the topic: https://community.khronos.org/t/ray-origin-through-view-and-projection-matrices/7257
Read more

Forward and backward alpha blending for raymarching

-
When implementing raymatching by scanning a ray to accumulating scene's colors, containing semi-transparent colors (for example, scene with clouds), it's required to properly combine colors using alpha blending method. Here we discuss how to do it and obtain corresponding formula. (So, here we propose that objects has "uniform" ambient lighting and some transparency and all we need to accumulate their colors). The forward alpha blending formula, used "everywhere" - in OpenGL, VJ software, photo and video editors, can be formulated in the following way: Formula for "forward" alpha blending ---------------------------------- Let C is a current RGB color, c is a new RGB color and alpha is a new color's opaqueness (alpha is a float value from 0 to 1). Then updating rule for C is the following:     C = (1- alpha )* C + alpha * c    ---------------------------------- So, the more alpha , the more c affec
Read more

Forward, Deferred and Raytracing rendering in openFrameworks and web

-
Image
Currently, there are three basic approaches for rendering computer graphics scenes: forward rendering, deferred rendering, raytracing. 1. Forward rendering It's that OpenGL does, you create vertices, then make faces, and rasterize them, all passing via shaders. The rendering process is straightworward and fast, but rendering shadows, making degree-of-field effect and drawing clouds is hard here. Most of openFrameworks examples use forward rendering. 2. Deferred rendering It's the same as forward rendering, but additionally some extra data is rendered for computing shadows. The lighting pass is performed after the scene geometry is drawn. See detailed explanation here: https://learnopengl.com/Advanced-Lighting/Deferred-Shading Shadows, degree-of-field effect and some kind of reflections are possible with deferred rendering. Unreal Engine 4 and Unity currently (2019) are based on deferred rendering. openFrameworks: https://github.com/perevalovds
Read more