Introduction to Big Data

Big data is data in a huge volume that cannot be handled by our traditional tools or databases. It is growing exponentially with time. Its size is so large and complexity is high that it is hard to use it efficiently with traditional data management tools.

Take an example of Facebook, in a day Facebook generates 500+ terabytes of data. whenever we like, comment on a photo or video it generates huge data. But, we want to use it efficiently.

TYPES OF DATA:

There are different types of data as follows:

1. Structured data: Data that can be stored or managed in fixed format is structured data. This data can be handled by traditional databases also like SQL, MySQL, etc. It is in the form of rows and columns and this data is easy to understand and process. An example for structured data is any database table like a student table that has Student id, name, class, age.
2. Unstructured data: Data that does not have any format and is difficult to understand. This data is generated by social media like Facebook, Instagram, Snapchat, etc. The size of unstructured data is huge and it is very difficult to process this data with traditional tools. The main challenge with this data is to derive value and it is in raw form.
3. Semi-structured data: Data that is structured as well as unstructured. XML files and JSON files are examples of Semi-structured data.

CHARACTERISTICS OF BIG DATA(3 V’s):

1. Volume: It is related to the size of data. The size of data plays a vital role to derive value out of it. Every day approx 2.5 quintillion bytes of data are created. And this data is kept on increasing which is all unstructured data.
2. Variety: It is all structured, unstructured data, and semi-structured data from different sources. Earlier we got data from databases, spreadsheets, etc, but now there is so much data come from social media, emails, photos, audios, videos which are all unstructured data.
3. Velocity: It is the speed of data that is generated at a very high speed and volume. Whenever we use Facebook and like photos or videos then huge data is generated. Every day we generate so much data without knowing which is all unstructured without any format.

ADVANTAGES OF BIG DATA:

1. Predictive Analysis: Most important advantage of big data is predictive analysis. Using Big data future analysis can be made. For example, it uses earlier data to predict the weather. Another example is using Twitter data, analysis can be made on how many users are following a particular trend.
2. Decision Making: All companies or businesses are using big data to see future trends. Based on these analyses final decisions are made. So, Big data helps businesses in decision making.
3. Social Media: Customer data is dug from social media. For example, whenever we search for shoes then social media shows us shoes on every site because it helps to know customer interest.
4. Market Basket Analysis: Under this analysis, whenever customers add one thing to the cart then recommendations are made based on products in the cart. Example: In the morning whenever customers add milk to the cart then butter and bread are recommended. Another example, on shopping websites, whenever we search for a top then matching jeans and shoes come below.

APPLICATIONS OF BIG DATA:

1. Healthcare: Based on patient data, predictive analysis is made which helps in the cure of disease. Example: Huge data of cancer patients is collected to find trends and treatments which have the highest rate of success.
2. IT: In IT companies, Big data is used for decision making. It helps to expand the businesses and minimize the risks in business operations. Using machine learning and artificial intelligence with Big data finding solutions to complex problems.
3. Banking: Big data prevents banking from fraud detection, misuse of credit/ debit cards. It is very useful for the banking sector for security purposes.
4. Transportation: Big data is used by transportation companies also to find out the best routes, manage traffic, enhance services, etc.

CASE STUDIES:

1. Uber: Using big data, uber analyses customer’s data. Based on demand and supply it provides charges to the customers. Example: If you are in a location with traffic then it will charge more and even if it’s raining then also it will charge double.
2. Netflix: Netflix uses big data for movie recommendations. Whenever you use Netflix you can see Popular on Netflix, this is also based on data collected by different users which movies are mostly seen. Even location-based analysis is done, if a user is in India it will show the top 10 movies in India. This is all based on big data.
3. Facebook: Facebook user’s data is used for analysis. Friend suggestions are based upon the friend list, they collect all the data of the user and on its basis suggest friends. Most of the unstructured data is generated by Facebook based on likes, comments, videos we watch. Even facial recognition of users is also used by Facebook. Suggestions are made for tags also.
4. Twitter: Sentiment Analysis is done using Twitter data. Suppose, how many Twitter users use the farmer support tag. It tells the user’s reaction to what they think about it. If they are in support of it or against it.
5. Walmart: Using Big data, Walmart can see the most frequently bought products. It helps to find the trends, how weather affects customers visiting, during festive days how much purchases increase. It helps Walmart to study past data to improve sales in the future. Even mapping applications are used in Walmart which helps find the location of every material in the store, even a soap bar.