Images and Videos Become Context Sensitive

What’s really happening in the digital market? What sort of developments are happening?

The web is growing increasingly visual, comprising mostly of images and videos. This is probably because visual data has become very ubiquitous, thanks to cameras becoming very cheap and available in many appliances and smartphones. As such devices have Internet access, people tend to upload whatever images they shoot to the web.

Keeping this proliferation in mind, Graymatics started with the vision that images and videos should become as easy to use for applications that require contextual understanding as text is today. Such is the pace of this growth that though we started our operations in Silicon Valley only 3 years ago, we quickly started an office in Singapore and now we see tremendous growth in Asia.

Please elaborate on what you mean by contextual understanding of images and videos by apps.

Let’s look at what applications require contextual understanding of say text for example. One is search. When we do a google search, it tries to understand what the user is trying to get to by putting in a query keyword. That’s a typical example of contextual understanding. Then, if we talk about contextual advertising, which has been a big component of Google’s revenue, it’s largely around placing ads which are contextual to what a user is looking for or looking at. So whether the user is trying to search for a specific piece of info on the web, or is reading an article, within different online sites, the contextual advertising that Google allows for is based on text.

So here, the intent of Graymatics is that images and videos should be as easy to use as text is. We’ve developed a platform that transforms images and videos into a very rich set of readable data based on the content inside images and videos.

The benefit of doing this is that the eye ball attention of a user when they go to an online site tends to be more around images and videos than just around the text. So if you have a very contextual ad that appears next to such videos or images, then there’s higher likelihood of users clicking on that ad.

How do you ensure accuracy of the image?

We’ve developed an extremely versatile learning platform, where the system keeps learning, downloading images across the WWW, and keeps training, re-learning different objects inside images and videos. It’s an ongoing learning process, like humans. The versatile learning platform keeps getting smarter as you keep feeding videos and photos to it.

We’ve also developed a tool that gives some kind of accuracy as to what the likely accuracy is when it detects an object. If the accuracy tends to be border line, it automatically goes into another repository, where it’s viewed for accuracy, and gets into a training process.

The accuracy level is well over 85%, with many of them even going over 90%. There may be specific areas that require even higher levels of accuracy, like certain large Internet portals that need the ability to detect if there’s any racy, suggestive content. This requires very high level of accuracy, well over 95%. In such cases, we’ve designed it to have that high level of accuracy by feeding it very large number of images and videos in addition to optimizing the algorithms.

How does contextual advertising work for images and videos? Please give some scenarios.

The Graymatics platform extracts all data from within images and videos. Then, different companies that we work with will use it in different ways. For instance, companies focused around enhancing their search will utilize this information to provide much better/contextual images or videos based on the query/keyword that the user has placed.

Next to it, there’ll be a contextual advertisement. Since the user experience tends to be better, chances of user clicks will be higher. We work with publishers, ad networks, e-commerce companies, ad exchanges, etc. who use our various plug-ins. So if Priyanka Chopra appears on one of the publishers’ sites wearing say a specific lehnga, the platform will identify this lehnga with a specific platform.

Will it differentiate between different types of apparels?

It not only recognizes the clothing, but even different their attributes—the neckline, sleeve length, pattern, texture, and gives a very high level of detail that’s extracted from the image.

What about e-commerce companies? How can they benefit?

We’re working closely with a number of them, and very excited about a major e-commerce company we’re working with in India well. One is for the catalog of products that they have, very often, some of the products in the catalog may not be properly labelled. They’ll use our platform to obtain the deeper data. So if a user wants to search a catalog with a very specific requirement, then it will enable them to. In addition to that, there are some other ways that we work with e-commerce companies--one is with smartphone becoming so ubiquitous, and being used in a few different ways as an integral part of the shopping experience. A user can take a picture of a clothing that he really likes at a party or a social event, can take its photo through the e-commerce application, and the Graymatics platform will analyze all the detailed attributes, and associate them with the appropriate e-commerce product and show the results.

What about electronics products, where competing products could pop up?

That’s true, in many ways, we view Graymatics to be more of a tool that extracts all the data and how the data is used to provide the final recommendation/ad could be how the ad exchange might want to utilize.

How does it really work in case of an online portal? Will it crawl the site?

It depends. When we work with an online publisher/portal, we’ll provide a JavaScript code for the site. Each time there’s a new image/video, it will automatically be analyzed by our platform, all its data will be extracted, and a contextual recommendation will automatically be associated.

If you’re working with Ad exchanges, with every new publisher who subscribes to the ad exchange, the ad exchange will have structured so that Graymatics JavaScript code automatically appears on each of those publishers’ sites based on some options that they need to tick. It then operates in a similar workflow.

What about social media companies?

We’re working with a number of social media companies. Few of the carriers are deploying Graymatics for some new, visual/social media products where users can upload photos that have products of interest to them. Graymatics analyzes what’s inside the photo, what are the users’ interest, and they’ll be pointed to contextual news articles, and contextual applications and so on.

But in addition to that, we’re working with a number of social media companies, one very significant pain point for one of them, users upload photos in an uncontrolled environment, there are many cases of abusive content, and we have a product called ImageAssurance, which detects if there’s any nudity, objectionable content. This is becoming very important for social media sites.

What are your plans for India?

A lot of very exciting collaborations are currently going on. There will be some very big launches in the next few weeks/months. We’re working with some large carriers and very large e-commerce companies, and are very excited about at least one of them. In addition to that, we’re also working with a few digital marketing agencies, which are looking to use Graymatics products in different ways to enhance the user appeal to their marketing campaigns.

In fact, we’re actively looking at opening an office in India. We do have a couple of people in Bengaluru, and we’re looking into expanding our presence in India. So currently we have offices in California, Singapore, and definitely, India would be the most natural to have a significant R&D office.