Big Data

BIG DATA

Big Data constitutes a large volume of structured or unstructured data.

Big data is so huge that the traditional data processing system is inadequate to process.

• 90% of total data available on internet has been generated in the last one decade.

• By analysing this data, a useful decision can be made in various cases such as:

• Tracking Customer Spending Habit, Shopping Behaviour

• Smart Traffic System

• Auto Driving Car

• Virtual Personal Assistants

• Internet of Things etc.

6 V’s of Big Data

• Volume: Volume is an enormous quantity of data.

amount of the data is a critical factor in determining its worth. Data that is extremely huge in bulk is referred to as “Big Data.” This implies that the volume of data determines whether a given set of data can genuinely be categorized as a big data set or not.

• Velocity: The rapid rate at which data accumulates is referred to as velocity. Data comes in at Big Data velocity from various sources, including devices, networks, social media, mobile phones, etc.

• Variety: It refers to the characteristics of unstructured, semi-structured and structured data types.

• Veracity: It alludes to uncertainties and inconsistencies in data that is, readily available data might occasionally become jumbled and difficult to control in terms of correctness and quality. Due to the multiplicity of data dimensions arising from several dissimilar data kinds and sources, Big Data is also unpredictable.

• Value: Following the consideration of the four Vs, there is one more V: value! The majority of useless data is useless to the business unless it can be transformed into

something valuable.

• Variability: It establishes the rate at which the structure of the accessible data is changing.

Applications of Big Data

• Economy: Big data may be extremely beneficial to many different economic areas, including

• In the insurance industry, to enhance client satisfaction and guarantee their entitlement to claims

• To manage financial data in the banking sector

• To capture the production, price statistics, & calculate the resultant GDP

• Health Care: Big Data in health care has the following benefits:

• Diseases prediction,

• Medicines prescription,

• Optimizing treatment

• Digital Space:

• In the telecom arena, connecting the hinterland areas and bringing them together to the mainstream,

• On Social Media for targeting platform users,

• Artificial Intelligence – Controlling home appliances

• Agriculture:

• Seed Selection

• Geo-Tagging to keep the track record of agricultural assets in the country

• Weather Forecasting

• Irrigation & effective water management

• Governance

• Prevent cyber-attacks

•
Enhance security systems

• Detect card-related fraud cases

• Predict criminal activities, e.g – Crime and Criminal

Tracking Networks and Systems (CCTNS)

• In improving the quality of education.

Government Initiatives & Interventions

• Currently, NITI Aayog is collaborating with private partners to build a strategy to create the “National Data & Analytics Platform,” which would serve as a centralized source of sector-specific data for researchers, policymakers, and people.

• The “Big Data Management Policy,” which was created by the CAG to audit vast amounts of data produced by the states and union territories’ public sectors, is an ideal place to start.

• The Ministry of Statistics and Program Implementation has suggested creating a “National Data Warehouse on Official Statistics” to enhance the potential of macroeconomic aggregates by utilizing big data analytical methods and leveraging technology.

Significance

• With a population of around 1.4 billion, Big Data holds a significant position in the Indian context. As per the study conducted by NASSCOM, “the Indian analytics industry is predicted to reach $16 billion mark by 2025”.

• When it comes to job positions and roles, Big Data is one of the most versatile career options. As Analytics is a crucial tool used in many different fields, you get a host of job

titles to choose from including Big Data Engineer, Big Data Analyst, Big Data Analytics Architect etc.

• NITI Aayog is developing “National Data and Analytics Platform (NDAP)” with a vision to Democratize access to public Government data through a world-class user experience. Mission is to Standardize data across multiple Government sources to provide flexible analytics and make it easily accessible in formats conducive for research, innovation, policy making and public consumption.

Challenges

• Invasion of Privacy: Big data analytics brings to light the twin challenges of digitalization: data privacy and net neutrality.

• Data Security: Several Aadhaar data leak incidents have brought attention to the need for the government to improve the safety and security of the digital data it collects from its citizens.

• Technical Challenges: Big data has some inherent limitations, as its name suggests, such as

• Difficulties with computation and storage;

• Scalability and streaming issues;

• Inadequate infrastructure for organizing and collecting such vast volumes of data.

• Challenges related to Governance: To effectively utilize Big Data for policymaking, the government needs to adopt a methodical yet adaptable approach. If the necessary policy structures are adjusted flexibly and the resulting information is regularly reviewed, the advantages will eventually filter down to the lowest level.

Way Forward

• The government needs to set up properly equipped data centers so that the massive amounts of data at its disposal can be analyzed efficiently. Sorting the pertinent facts from the unimportant is crucial.

• To ensure that the vast amount of data that is accessible is almost completely safe, the government needs to improve its cybersecurity.

• The government must also create a data privacy policy and handle the moral dilemmas raised by big data analytics.