Imagine if your fridge could shoot you a text message telling you that you've ran out of milk, or that you need to stock up on Tabasco sauce. Seem far fetched? Microsoft has teamed up with the appliance division of Liebherr to build a Cortana fridge device that does just that.

Both companies are working on the latest version of the SmartDeviceBox. You can integrate it into any Liebherr refrigerator or freezer, connecting them to the Internet. It will include a camera that will intelligently identify the contents, and inform the user when a particular food is running low.

Microsoft is keeping mum about how much this will cost. They say that the device is just a prototype, albeit one that shows a lot of promise. However, Microsoft says that it'll be modular, serviceable, and upgradable.

Already Microsoft has confirmed the existence of a voice module, which will allow users to add groceries to a smartphone-based shopping list by just speaking.

Powered by Deep Learning and Artificial Neural Networks

Microsoft isn't the first company to enter the smart fridge space. It's certainly a niche field, but one that has been in existence for almost 16 years. It started back when LG launched the first Internet refrigerator - the Internet Digital DIOS. A number of companies are currently battling for dominance, including Electrolux, LG, Whirlpool, and Samsung.

Some of the most interesting devices have come from Samsung. At the start of the year, it announced the Tizen-based Samsung Family Hub refrigerator. Its $5,000 pricetag isn't the only thing that's massive about it; it packs a 21.5 inch 1080p touchscreen, Amazon Alexa integration, and three integrated cameras that allow the user to see the contents from a companion application.

But there's something palpably exciting about this Cortana fridge effort from Redmond.

How SmartDeviceBox Works

While the Samsung Family Hub Refrigerator requires that you manually check how much stuff you've got in your fridge, Microsoft's SmartDeviceBox does this for you. It does this through its deep- learning-powered object recognition technology, which allows it to identify each individual item.

Microsoft's deep learning technology depends on artificial neural networks. These are computer programs that are inspired by biological processes of the human brain, and most importantly, can learn and improve with experience.

These sound like science fiction, but YouTube is already using it to identify video content, and even to create works of art.

Developers build artificial neural networks in a series of layers. The theory is that more layers would lead to better results; in this case, it would lead to more accurate recognition of items. But there's a problem with this. As you add more layers, there's a kind of "signal degradation" and data gets lost.

It's kind of like when you build a complicated computer network using really cheap routers and switches. As a packet travels through each hop, there's a chance it might get dropped.

Microsoft research manager Jian Sun said that:

"...researchers were excited when they could successfully train a 'deep neural network' system with eight layers three years ago, and thrilled when a 'very deep neural network' with 20 to 30 layers delivered results last year."

But with a bit of trial and error, Microsoft stumbled on something it calls "deep residual networks". These are artificial neural networks that have an unprecedented number of layers. The one Microsoft built for the ImageNet contest, which was a competition for the best program that can identify physical objects, had a colossal 152 layers.

Artificial Networks Need Work

But for artificial neural networks to be useful, users need to train them. While developing the SmartDeviceBox Cortana fridge device, Microsoft showed the program literally millions of generic images of foodstuffs - milk cartons, ketchup bottles, pickle jars, and so on. When something is placed in the refrigerator outfitted with the Cortana fridge device, the program will try to identify it from what it has seen previously.

The accuracy of the artificial neural network will get better as more and more people use it. According to Microsoft's Allison Linn:

"One key advantage of neural networks is that they get better at one task when they are given another. For example, with Skype Translator, a neural network that is designed to translate from English to German gets better at translating German once it has been trained for the additional task of translating Chinese."

We're going to see these deep-learning artificial neural networks used in more context and applications. This technology is already found in a number of products, including the Microsoft Cognitive Services Computer Vision, which can look at images and annotate them. So, if you show them a photo of you at the beach, it should be able to give it a category, determine if it's an adult image, and even work out your gender and age.

Perhaps most excitingly, Microsoft has open-sourced their deep-learning algorithms as part of its Computational Network Toolkit (CNTK), meaning that third-party developers will be able to integrate it into their programs.

The Smart Fridge Hits The Prime-Time

It's not uncommon to hear people refer to smart fridges in a derisory way. Perhaps justifiably. Many see them as an example of the excessive and unnecessary computerization of ordinary household appliances. Who honestly wants to check their calendar, or watch a movie from their refrigerator? Who would want to do the weekly grocery shopping while standing next to 200 pounds of whirring chromium?

But I actually can see the point of this Cortana fridge device. Rather than focusing on doing stuff, Microsoft's SmartDeviceBox recognizes the importance of data, and how it can empower users. It wants to tell you stuff.

It's still early days, and it's still very much a prototype, but given the nature of consumer technologies become obsolete after a few years, I'm glad that the SmartDeviceBox is upgradable and modular. I'm glad that it's just a box that fits into an ordinary fridge, rather than an integral part of one.

What do you think? Will you be getting one when the SmartDeviceBox eventually hits the mainstream, or are you still a determined smart fridge cynic? Let me know in the comments below. 

Image Credits: Ljupco Smokovski/Shutterstock