Posts

Showing posts from April, 2019

Morphing point clouds algorithm

In some sense, point cloud is an universal medium for representing 2D and 3D vector objects and raster images/volumes. Namely, each 2D or 3D raster (image/volume) can be treated at least in two opposite ways: as a grid of points, where each point posess the color and opacity, without overlapping. as a set of points, where each point has pure grayscale, reg, green or blue color, and almost transparent. So many points overlapped additively and get required colors.  Of course, there are possibilities between this polar cases. Each 2D or 3D vector object (curve/surface) can be filled points so dense, that when viewing it on a rester display, it looks like just vector object. Such representation of an objects from a "sand" points gives opportunities of free morphing objects. So, consider the following problem. Let A and B are two point clouds of size N of equal color and transparency. It's required to create bijective mapping k_i, that {A_i}->{B_

Raymarching in Shadertoy, openFrameworks and Unreal Engine

Raymarching is a special method of rendering 3D objects or 3D scene: It's used for rendering ocean water, clouds, metaballs in Unreal Engine 4 and other engines and 3D editors. It's a main technique used for rendering 3D in Shadertoy projects. Examples Shadertoy projects of rendering generative volumes, clouds, ocean and skybox reflections: Sculpture III by iq: https://www.shadertoy.com/view/XtjSDK , see more at http://www.iquilezles.org/www/articles/raymarchingdf/raymarchingdf.htm https://www.shadertoy.com/view/MsfGRr https://www.shadertoy.com/view/lss3zr https://www.shadertoy.com/view/XslGRr https://www.shadertoy.com/view/llsXD2 https://www.shadertoy.com/view/XsB3Rm stochastic ray trace by Otavio Good https://www.shadertoy.com/view/WlfXRr  Metaballs and volumetric clouds in Unreal Engine 4: https://www.youtube.com/watch?v=ZbLCIcTHup4 https://www.youtube.com/watch?v=hWNX9jGEt8k Claybook game is made entirely in similar technique: https://www.youtube.com/

Raytracing APIs

Raytracing is a most powerful technique for making photorealistic 3D images. It's based on literal tracing light's rays path, and requires a lot of computtional resources. See theory of raytracing in a sample chapter of the book Real-Time Rendering, Fourth Edition . Currently NVidia RTX videocards are dedicated to work with raytracing in realtime, and so realtime raytracing becomes the main tool for 3D graphics. There are two main directions for raytracing API: using NVidia RTX cards and Intel CPU. Withoud such special API, raytracing (and reymarching and raycasting) can be implemented directly, for example, in fragment shader, for example, see such shaders on Shadertoy site. NVidia RTX  NVidia RTX videocards allows to work with NVidia's OptiX or Microsoft DirectX: https://developer.nvidia.com/rtx Raytracing with NVidia OptiX raytracing engine: https://developer.nvidia.com/optix https://developer.nvidia.com/rtx/raytracing See tutorial: https:/

Programming VR with openFrameworks

To work with VR using openFrameworks, you can use ofxOpenVR addon: https://github.com/perevalovds/ofxOpenVR It's implementation of Valve Software's OpenVR API, which was originally developed by smallfly https://github.com/smallfly/ofxOpenVR , next I forked it into Kuflex repo, and finally to my repo. The most important fix comparing original version is about problem of one eye's vertical shift due matrices computations errors. This addon lets create VR applications (using HTC Vive) on openFrameworks, Windows 10. It's using oF's programmable render. Requirements HTC Vive (though, it is not required to compile projects, just to deploy) and Steam VR Visual Studio 2017 openFrameworks 10.1 ofxOpenVR addon folder, which should be placed to openFrameworks/addons/ofxOpenVR. Running examples To check addons is working, compile and run its examples located in openFrameworks/addons/ofxOpenVR . There are two examples: exam

Compact platforms for computer vision and AI computations

Currently there are two main brands for compact AI: Intel and NVidia. Intel Myriad Site: https://www.movidius.com/ "Intel's Myriad ™ X VPU features a fully tune-able ISP pipeline for the most demanding image and video applications. The Intel ® Movidius™ Myriad ™ X VPU also features hardware based encode for up to 4K video resolution, meaning the VPU is a single-chip solution for all imaging, computer vision and CNN workloads." Particular products: Compact systems: https://up-shop.org/home/285-up-squared-ai-vision-x-developer-kit.html Intel neural compute stick https://software.intel.com/en-us/neural-compute-stick Example of using with Raspberry Pi: https://www.intel.ai/practical-applications-of-deep-learning-build-a-diy-smart-security-camera-using-the-intel-movidius-neural-compute-stick/#gs.7g7fgl NVidia Jetson Site: https://developer.nvidia.com/buy-jetson "NVIDIA® Jetson™ systems provide the performance and power efficien

IBM Watson AI service details

IBM Watson is powerful online AI service, which performs image recognition,text translation and understanding, and text-to-speech and back conversion. Also, IBM Watson allows to create and use customizable models for their algorithms. Information below was copied and from IBM Watson site and is given here for quick reference purposes. Topics: Visual Recognition - Classify an image with the built-in or custom classifiers - Detect faces in an image   Language Translator - Translate   - Identify language - Translate document Natural Language Classifier - Classify a phrase Natural Language Understanding- Categories - Concepts - Emotion - Entities - Keywords - Metadata - Relations - Semantic roles - Sentiment Personality Insights Get profile Speech to Text - Recognize audio Text to Speech - Synthesize audio - Get pronunciation Tone Analyzer - Analyze general tone - Analyze customer engagement tone 1. Visual Recognition Quickly and accurately tag, clas

AI online services list

https://encreative.blogspot.com/2019/09/using-microsoft-computer-vision-api.html Today exists several online AI-services, providing image recognition, text translation an other AI tasks by sending requests to their servers. So, you is able to experiment with and make projects using most advanced AI algorithms available today. The advantages of using online AI-services comparing offline libraries - is that they are represent state-of-art of AI and are very lightweight to work with it - you don't need install libraries to your computer, just call services. The disadvantages is that you need internet connection, there is bigger time delay between request and also, this is paid services. But normally you can make 1000 requests month free of charge. So, this services are great for you if you want to make several experiments with advanced AI technologies, your project in't sensitive to delay in AI answer, or you have more serious project and budget on running it on distant ser

Software tools for making creative projects

There are many tools you may use for creative coding: create video art, sound art, interactive installations and so on. Some projects require using several tools for production. When choosing the tool, take into account your requirements such as time for development (and learning the new tool if required), stability, flexibility and a price. Note: machine learning and artificial intelligence tools are described in our other posts. Non-programming tools  These tools doesn't require programming, and they are ready to use for implementing specific tasks. Normally, thay are robust and doing well they are made for, but are not very flexible for making very "custom" tasks. Normally, such tools are used as basics for the project for routing tasks such as getting data from cameras/sensors and output video to several screens. Resolume - VJ software, simple to start working with (commercial). VDMX - VJ software, with very sophisticated possibilities to create you