Data is information, but that's only part of the story. One detail about an event or a factoid about human health isn't much data to work with. It's the collection, organization, and storage of information that we think of when we talk about data.

In the internet age, companies and organizations around the world have collected so much data that we're now talking about matters on an exponentially larger scale. Now there's big data, and it's having a huge impact on all of our lives.

What Is Big Data?

Big data is a data set so large that our traditional means of managing information isn't up for the job. This collection can take many forms.

Examples of Big Data

  • The tweets stored on Twitter's servers
  • The information Google gets from tracking car rides
  • A country's full set of local and national election results, going back as far as records have been kept
  • What health insurance companies know about who gets what treatments done at which hospitals
  • The types of purchases and places that appear on credit cards
  • What people watch on Netflix, when, where, and how long

What Is Big Data Technology?

Our PCs can manage quite a bit of data. Just imagine all the information it's possible to cram into a single spreadsheet. Database software is capable of handling even higher volumes of information. These tools can cram onto a single hard drive data that would otherwise require shelves filled with boxes packed with notebooks and folders.

But these tools are insufficient to handle all the volume of information we refer to as big data. For that, we've developed new methods. Cloud computing offloads the job from our PCs onto distant servers. From there, there are any number of ways to access and utilize the information.

Notable Uses for Big Data

Big data didn't come about on its own. Several trends have encouraged its existence.

Internet of Things

The internet you currently know is the internet of people. This is where people interact with one another, with machines facilitating that communication. You view sites that people design. You read words that people typed.

The Internet of Things is one where devices communicate directly with one another without human involvement. One device monitors the weather. A smart thermostat accesses that information and makes adjustments to the temperature in your home.

Big data and the Internet of Things are interdependent. These devices are able to take actions on their own thanks to all the data that is available to them. The more devices function this way, the more data that gets generated.

Machine Learning

Machine learning refers to a computer's ability to learn from data. This is how Pandora radio stations mold to your particular style. Machine learning is also behind content recommendations on YouTube and Netflix.

These predictions are due to algorithms. Google's search algorithm? The algorithm that determines what you see in Facebook's news feed? This is all machine learning at work.

These are but a few examples of how machine learning algorithms are influencing our everyday experiences.

Artificial Intelligence

Artificial intelligence is the next step after machine learning. Here, not only is a computer learning from data, but it's using that information to make its own decisions and shape its own behavior.

Microsoft and Google have both shown off efforts to create humanoid robots. Facebook is using artificial intelligence to help prevent suicides. The technology is progressing at a rate where there have been several instances where a computer's thinking has outperformed a human's.

What Is Big Data Analytics?

Sources of big data don't tell us anything on their own. Someone has to make sense of all of that information. This is what the field of big data analytics is all about: looking at incomprehensibly large volumes of information and seeing what we can learn.

Today, more organizations are starting new big data projects, and companies are competing to offer their particular form of big data analysis across many different fields. It's through these actions that big data is having an effect on your life, even if you're a modern day Luddite.

Why are people doing this? Because with the right insight, big data can do a lot of good.

The Benefits of Big Data

People are racing to utilize big data to improve our lives. Here are some of the areas where big data is at work.

Big Data in Healthcare

The healthcare industry isn't the fastest at adopting new technologies. Some providers are still migrating from paper to digital means of storage. Nonetheless, there are areas where big data is making a difference. One is the area of integration. Insurers and providers are working on combining data from different sources, such as claims, X-rays, doctors' notes, and prescriptions.

Doctor's Office Notes and Pen - what is big data
Image Credit: MGDboston

Many believe that if healthcare data was better integrated, this could provide better care at a lower cost. When Amazon, Berkshire Hathaway, and JP Morgan announced earlier this year that they were working together on healthcare, they cited technology as their area of focus, as The Guardian covers.

Big Data in Finance

The finance industry is all-in on the idea of making decisions based on computer analysis. Wall Street flash crashes are due to automated trading, with machines rapidly selling off stock without human intervention, based on what's going on in the market. This is called high frequency trading.

Now financial data scientists use big data to predict which stocks will succeed and when future crashes are likely to occur. Banks also see big data as a way to increase their revenue.

Big Data in Ecommerce and Marketing

We generate a lot of information when we shop. In store, credit cards and loyalty cards track our every purchase. Some shops use cameras or even track our phones to see which part of the store hold our attention the longest. Online, we have to create accounts before shopping, allowing sites not only to track what we buy, but every item that we view.

Stores base their layouts around consumer interest and behavior. Online sellers decide what we see based on demographic information and other metrics. Amazon's new brick-and-mortar stores are an example of the two worlds merging together.

There is big demand for the kind of insight that comes from monitoring our interests and online behavior. Facebook and Google are profitable tech giants because of their ability to sell ads that are better able to target specific consumer groups than other advertisement methods and platforms. They are able to do this thanks to all the information we provide when we use their services.

Is Big Data Dangerous?

Big data comes with promise, but it also comes with risk. First is the erosion of privacy. More people know more about each of us than at any point in human history. It's not only easy to find where we live, but where we go, who we love, how we live, and what we think.

This makes individuals and societies more open to manipulation. We can be tricked into giving up our passwords and credit card numbers or influenced to vote for candidates we wouldn't otherwise support. More data offers more ways for advertisers and media companies to shape our desires and values.

There's more data about us than before, and that data is stored in more places. That creates more targets for attack. No longer is it enough to safeguard our own machines. Data breaches are now a regular occurrence, with what happens to our data being out of our control.

Even companies that may do a decent job of protecting our data from outside attack often do questionable things with that data themselves, as is the case with Facebook.

Then there's the risk of what people do with the information big data enables them to predict. Do we charge people with unhealthy eating habits more for health insurance? Should we increase policing in areas that we predict will have more crime? Do we increases prices for online shoppers who live in affluent areas?

Finding ways to keep our data safe, our privacy respected, and our values maintained will be ongoing challenges as the trend toward big data continues. Yet no matter how we feel about it, for better or worse, we all live in a world of big data.