Data Preparation - Concepts

Loading...

Do curso por IBM

Data Science Methodology

291 classificações

IBM

Data Science Methodology

291 classificações

Curso 3 de 9 em Specialization IBM Data Science Professional Certificate

Despite the recent increase in computing power and access to data over the last couple of decades, our ability to use the data within the decision making process is either lost or not maximized at all too often, we don't have a solid understanding of the questions being asked and how to apply the data correctly to the problem at hand. This course has one purpose, and that is to share a methodology that can be used within data science, to ensure that the data used in problem solving is relevant and properly manipulated to address the question at hand. Accordingly, in this course, you will learn: - The major steps involved in tackling a data science problem. - The major steps involved in practicing data science, from forming a concrete business or research problem, to collecting and analyzing data, to building a model, and understanding the feedback after model deployment. - How data scientists think! LIMITED TIME OFFER: Subscription is only $39 USD per month for access to graded materials and a certificate.

Na lição

From Understanding to Preparation and From Modeling to Evaluation

In this module, you will learn what it means to understand data, and prepare or clean data. You will also lean about the purpose of data modeling and some characteristics of the modeling process. Finally, through a lab session, you will learn how to complete the Data Understanding and the Data Preparation stages as well as the Modeling and the Model Evaluation stages pertaining to any data science problem.

Data Understanding3:16

Data Preparation - Concepts3:02

Data Preparation - Case Study4:14

Conheça os instrutores

Alex Aklson
Ph.D., Data Scientist
IBM Developer Skills Network
Polong Lin
Data Scientist
IBM Developer Skills Network

Welcome toData Science Methodology 101 From Understanding to Preparation Data Preparation

- Concepts!

In a sense, data preparation is similar to washing freshly picked vegetables in so far

as unwanted elements, such as dirt or imperfections, are removed.

Together with data collection and data understanding, data preparation is the most time-consuming

phase of a data science project, typically taking seventy percent and even up to even

ninety percent of the overall project time.

Automating some of the data collection and preparation processes in the database, can

reduce this time to as little as 50 percent.

This time savings translates into increased time for data scientists to focus on creating

models.

To continue with our cooking metaphor, we know that the process of chopping onions to

a finer state will allow for its flavours to spread through a sauce more easily than

that would be the case if we were to drop the whole onion into the sauce pot.

Similarly, transforming data in the data preparation phase is the process of getting the data into

a state where it may be easier to work with.

Specifically, the data preparation stage of the methodology answers the question: What

are the ways in which data is prepared?

To work effectively with the data, it must be prepared in a way that addresses missing

or invalid values and removes duplicates, toward ensuring that everything is properly

formatted.

Feature engineering is also part of data preparation.

It is the process of using domain knowledge of the data to create features that make the

machine learning algorithms work.

A feature is a characteristic that might help when solving a problem.

Features within the data are important to predictive models and will influence the results

you want to achieve.

Feature engineering is critical when machine learning tools are being applied to analyze

the data.

When working with text, text analysis steps for coding the data are required to be able

to manipulate the data.

The data scientist needs to know what they're looking for within their dataset to address

the question.

The text analysis is critical to ensure that the proper groupings are set, and that the

programming is not overlooking what is hidden within.

The data preparation phase sets the stage for the next steps in addressing the question.

While this phase may take a while to do, if done right the results will support the project.

If this is skipped over, then the outcome will not be up to par and may have you back

at the drawing board.

It is vital to take your time in this area, and use the tools available to automate common

steps to accelerate data preparation.

Make sure to pay attention to the detail in this area.

After all, it takes just one bad ingredient to ruin a fine meal.

This ends the Data Preparation section of this course, in which we've reviewed key concepts.

Thanks for watching!

Explore nosso catálogo

Registre-se gratuitamente e obtenha recomendações, atualizações e ofertas personalizadas.

O Coursera proporciona acesso universal à melhor educação do mundo fazendo parcerias com as melhores universidades e organizações para oferecer cursos on-line.

© 2018 Coursera Inc. Todos os direitos reservados.

Baixar na App Store

Baixar no Google Play