What is Google Big Query[Big Query-Part1]

More and more companies are looking to unlock business insights from their data, but it can be hard to scalably ingest, store and analyze the data as it rapidly grows.

What is Big Query

Google Big Query is a data warehouse that was designed to make large-scale data analysis accessible to everyone. Big Query to help you to get data insights with ease.

Situation

If your business has small amount of data, you might be able to store it in a spreadsheet. But as your amount of data grows to gigabytes, terabytes or even petabytes, you start to need a more efficient system like a data warehouse. That is because all the data isn't very useful unless you have a way to analyze it. Traditionally, larger sets of data means longer times between asking your questions and getting answers.

needed to wait hours or days for an analytics report to be run.

The solution-Big Query

Big Query was designed to handle massive amounts of data such as log data from thousands of retail systems or IoT data from millions of vehicle sensors across the globe. It's a fully managed and serverless data warehouse

Empower you to focus on analytics instead of managing infrastructure.

Big Query helps you avoid the data silo problem which happens when you have individual teams in your company having their own independent data marts. This can create significant friction between analyzing data across team and cause challenges with data version control.

Thanks to the integration with Google Cloud's native identity and asset management, you can assign read or write permissions to specific users, groups, or projects and keep your sensitive data secure, while still collaborating across teams.

Work with Big Query

Working with data in Big Query involves 3 primary parts- Storage, Ingestion & Querying. Google handles running everything else.

Big Query is a fully managed service, which means that you don't need to set up or install anything and you don't require a database administrator. You can simply log in to your google cloud project from a browser and get started.

First, let's talk about data storage

Data Storage

Data is stored in a structured table, means that you can use standard SQL for easy querying and data analysis. Big Query is perfect for big data as Google manage all the storage and scaling operations automatically for you.

Data Ingestion

Of course ,storing the data doesn't matter if you can't get into Big Query in the first place. Thankfully there is a lot of ways to do that as Big Query is integrated with the rest of the data analytics platform from Google.

You can upload data from cloud storage and just streaming data from Cloud Dataflow, build an ETL pipeline using Cloud data fusion, import data from a variety of file formats, or use a combination of all of these.

Data Querying

Once your data is up in Big Query, you are ready to start answering those questions. Big Query supports the same Structured Query Language (SQL), that you may be familiar with if you worked with ANSI-compliant relational databases in the past.

There are also public datasets available in Big Query. These are third-party datasets which have been made public for anyone to query against. Google handles all the storage so you can focus on figuring out answers to questions.