While Samsung’s initial rollout of its virtual assistant left many users underwhelmed, the full launch of Bixby Voice has piqued the interest of consumers once again.
The company promises that its artificial intelligence (AI) assistant is a different breed from the others, thanks to contextual awareness and the ability to learn. But is this just marketing hype?
We take a look at five surprising things that Bixby Voice can do, as well as one thing it doesn’t quite get right.
1. Understand Context
A welcome feature of Bixby Voice is the assistant’s contextual awareness. This means that the AI is able to interpret what you’re referring to depending on which app you have open.
So, if you have the Samsung internet browser open, you can simply say “Share this with Bob” and the AI will know you’re referring to your current webpage. It will then copy the link and allow you to choose which app to share it through if you haven’t specified this already.
Meanwhile, if you open a photo in your gallery using the command “Share this to Twitter,” Bixby Voice will understand you are referring to the picture.
Considering that many AI assistants haven’t nailed down this kind of context means its a surprising and unique feature. Bixby Voice also switches from native mode to contextual mode depending on the app you’re in.
So, if you’re on your home screen and say “Show me my most recent message,” Bixby will show you the last message in your default messaging app. However, if you use the exact same command in Whatsapp, the assistant loads your most recent Whatsapp message.
“Bixby is contextually aware and it is multi-modal,” Rui Brui, the Mobile Head of Product for Samsung South Africa told MakeUseOf.
He attributes this feature as the reason the assistant can understand commands that other AI assistants can’t. Add to this the ability for the AI to learn, and you have a recipe for a very user-friendly assistant that understands humans, Brui says.
Even Google is moving towards more contextual awareness in the latest Assistant update.
This kind of awareness is also part of the second surprising thing Bixby can do.
2. Allow Extremely Specific Commands
Bixby Voice, thanks to its compatibility with Samsung apps and many third-party apps, is able to follow through on very detailed, app-specific commands.
Nowhere is this more obvious than when you access “Voice Apps and Services” through Bixby and click on one of the listed apps.
The camera app has some of the most detailed commands, including “Set the timer for the rear camera to two seconds and turn on [the] Take 3 shots [feature].” Through Bixby Voice, you can also swap between the rear and front camera, select which filter you would like to use and adjust settings such as HDR and shutter speed.
Furthermore, these commands aren’t only limited to the app that you currently have open. You can tell Bixby to take a selfie and send it to a contact through Whatsapp, going through multiple app functions all in one breath.
But the assistant isn’t infallible when it comes to these commands. Sometimes the Bixby app hangs and quietly crashes, simply not fulfilling your command. If you’ve strung too many commands into one sentence, it also sometimes misinterprets and leaves out parts of the instructions.
For example, while in the camera app, you can tell Bixby to switch to the front camera, take a selfie and send it. However, try to add in a change in the colour mode as well, and Bixby tends to ignore it.
Luckily, this is where the third feature comes in handy.
3. Learn From Its Mistakes
While Bixby Voice has improved from its beta stage, it still has a few hiccups, including often misinterpreting pronunciation or not understanding a command.
But the intrepid AI can actually learn from its mistakes.
For most commands, you can provide feedback as to whether Bixby interpreted it correctly. You can also access your history in the My Bixby menu to go over past commands and correct any errors the AI made. Bixby literally asks you to teach it.
In fact, based on how many interactions you’ve had and what you’ve taught the assistant, Bixby actually “levels up.”
“The big thing is that it’s contextually aware, as you use it it will learn and improve,” Brui says. The AI is also supposed to become more accustomed to your accent, personal manner of speaking and your favorite commands.
4. Customize Commands
This is my favorite Bixby Voice feature: the ability to create custom or quick commands.
In addition to the slew of commands Bixby already has enabled, users can create their own unique commands. This is great for efficiency, but it also provides a way to have fun with the assistant.
Want a quick command that will make Bixby load pictures of your favorite feline? You could do this by creating the voice prompt “Take me to my happy place.”
I currently have Bixby answering the question “What do you believe in?” by loading and playing I Believe in a Thing Called Love by The Darkness.
I made Bixby do a thing @SamsungMobileSA
— Megan Ellis, Tech Harpy ????? (@Megg_Ellis) September 8, 2017
But aside from these party tricks, it actually is a very useful feature that allows you to create quick prompts for very detailed functions.
You could set up a command as simple as “Wake up” to get Bixby to tell you the weather, show you your calendar, and even play your morning music.
5. Work With Your Smart Home
Long before Samsung was in the AI assistant game, it had its foot firmly planted in the smart home industry. In recent years, the company acquired SmartThings, launched intelligent appliances, and made third-party device integration relatively easy.
With this ecosystem primed and ready, Samsung is making Bixby central to their smart home products. The company has an advantage as many of its smart appliances have been launched in developing markets where products like Google Home and Amazon Echo are not available.
Meanwhile, the SmartThings hub also allows consumers to make their “dumb” devices smart. But how does Bixby Voice come into the picture?
Bixby Voice works with Samsung Connect, the central app used to manage users’ smart homes. Through Connect, users will be able to use voice commands to control their smart devices.
If your devices are compatible with Samsung Connect, you can use Bixby Voice to control them. And considering over 100 devices are compatible with the app, it makes for a promising ecosystem base.
Brui confirmed that Bixby will be central to Samsung’s smart home ecosystem moving forward, so you can expect to see the assistant more often.
But What Can’t Bixby Voice Do?
There is one thing that is deterring users from using Bixby Voice: voice recognition. The virtual assistant’s voice capabilities were officially launched in over 200 countries near the end of August. But it only understands one English accent: U.S. English.
This is not to say that if you deviate from the typical American accent that Bixby won’t comprehend you at all, but the AI failing to recognize words becomes a pretty frequent occurrence. This is especially noticeable when it comes to the differences in pronunciation for the “a” vowel between U.S. and non-U.S. English speakers. Many English speakers use a long vowel in words like “last”, “cast”, “can’t”, etc. But the American pronunciation of the vowel is much shorter.
When I try to use these words with Bixby Voice, the AI struggles to understand what I’m saying. Every time I say “last,” Bixby thinks I said “lost.” When I say “cast,” Bixby hears “cost.”
But it’s not only my oh-so-South African accent that is misunderstood by Bixby. There are also strange interpretations of words from the assistant that seem to come out of nowhere. Some examples include “Cheer me up” being interpreted as “Jimmy up” and “Share me up.” It even sometimes interprets completely different consonants from what has been said. I’ve had “last” interpreted as “most” by the assistant. “Do a beatbox” became “Der beatbox” at one point.
This is an issue that will likely improve over time as Bixby learns and grows more accustomed to other accents.However, with the high frequency of interpretation errors for non-U.S. English speakers, you can’t blame some users for becoming exasperated.
You can teach Bixby to interpret common misinterpreted words differently, but the AI doesn’t always learn immediately.
Was Bixby Voice Worth the Wait?
While it still has some issues to sort through, Bixby Voice has definitely made Samsung’s virtual assistant a more appealing and useful product. Contextual awareness and the ability to give Bixby very specific commands is a welcome feature that makes it stand out.
However, the problems with voice recognition may tire some users and result in them simply turning the voice feature off. Bixby as a whole also requires some improvements to become more competitive.
Samsung has created a great base from which to expand the AI assistant’s capabilities, and the future of its integration into the company’s smart device ecosystem is promising. But considering the assistant still has some very noticeable issues to work out, it might be a while before it sees unrestrained praise.
Bixby has a lot more to offer than just Bixby Voice. Check out how to use other features like Bixby Home.
Image Credit: Syda_Productions/Depositphotos