Building intuition about the data

Loading...

Do curso por National Research University Higher School of Economics

How to Win a Data Science Competition: Learn from Top Kagglers

389 classificações

National Research University Higher School of Economics

How to Win a Data Science Competition: Learn from Top Kagglers

389 classificações

Curso 2 de 7 em Specialization Advanced Machine Learning

If you want to break into competitive data science, then this course is for you! Participating in predictive modelling competitions can help you gain practical experience, improve and harness your data modelling skills in various domains such as credit, insurance, marketing, natural language processing, sales’ forecasting and computer vision to name a few. At the same time you get to do it in a competitive context against thousands of participants where each one tries to build the most predictive algorithm. Pushing each other to the limit can result in better performance and smaller prediction errors. Being able to achieve high ranks consistently can help you accelerate your career in data science. In this course, you will learn to analyse and solve competitively such predictive modelling tasks. When you finish this class, you will: - Understand how to solve predictive modelling competitions efficiently and learn which of the skills obtained can be applicable to real-world tasks. - Learn how to preprocess the data and generate new features from various sources such as text and images. - Be taught advanced feature engineering techniques like generating mean-encodings, using aggregated statistical measures or finding nearest neighbors as a means to improve your predictions. - Be able to form reliable cross validation methodologies that help you benchmark your solutions and avoid overfitting or underfitting when tested with unobserved (test) data. - Gain experience of analysing and interpreting the data. You will become aware of inconsistencies, high noise levels, errors and other data-related issues such as leakages and you will learn how to overcome them. - Acquire knowledge of different algorithms and learn how to efficiently tune their hyperparameters and achieve top performance. - Master the art of combining different machine learning models and learn how to ensemble. - Get exposed to past (winning) solutions and codes and learn how to read them. Disclaimer : This is not a machine learning course in the general sense. This course will teach you how to get high-rank solutions against thousands of competitors with focus on practical usage of machine learning methods rather than the theoretical underpinnings behind them. Prerequisites: - Python: work with DataFrames in pandas, plot figures in matplotlib, import and train models from scikit-learn, XGBoost, LightGBM. - Machine Learning: basic understanding of linear models, K-NN, random forest, gradient boosting and neural networks.

Na lição

Exploratory Data Analysis

We will start this week with Exploratory Data Analysis (EDA). It is a very broad and exciting topic and an essential component of solving process. Besides regular videos you will find a walk through EDA process for Springleaf competition data and an example of prolific EDA for NumerAI competition with extraordinary findings.

Exploratory data analysis7:01

Building intuition about the data6:47

Exploring anonymized data15:38

Visualizations11:52

Dataset cleaning and other things to check7:35

Conheça os instrutores

Dmitry Ulyanov
Visiting lecturer
HSE Faculty of Computer Science
Alexander Guschin
Visiting lecturer at HSE, Lecturer at MIPT
HSE Faculty of Computer Science
Mikhail Trofimov
Visiting lecturer
HSE Faculty of Computer Science
Dmitry Altukhov
Visiting lecturer
HSE Faculty of Computer Science
Marios Michailidis
Research Data Scientist
H2O.ai

In this video, we'll go through and break down several important steps namely, the first,

getting domain knowledge step, second,

checking if data is intuitive,

and finally, understanding how the data was generated.

So let's start with the first step,

getting the domain knowledge.

If we take a look at the computations hosted in the Kaggle,

well, you'll notice, they are rather diverse.

Sometimes, we need to detect threats on three dimensional body scans,

or predict real estate price,

or classify satellite images.

Computation can be on a very specific topic which we know almost nothing about,

that is, we don't have a domain knowledge.

Usually, we don't need to go too deep inside the field

but it's preferable to understand what our aim is,

what data we have,

and how people usually tackle this kind of problems to build a baseline.

So, our first step should probably be searching for the topic,

Googling within Wikipedia, and making sure we understand the data.

For example, let's say we start

a new computation in which we need to predict advertisers cost.

Our first step is to realize that the computation is about web advertisement.

By looking and searching for the column names,

using any search engine,

we understand that the data was exported from Google AdWords system.

And after reading several articles about Google AdWords,

we get the meaning of the columns.

We now know that impressions column contained a number

of times a particular ad appeared before users,

and clicks column is how many times the ad was clicked by the users,

and of course, the number of clicks should be less

or equal than the number of impression.

In this video, we'll not go much further into the details about this data set,

but you can open the supplementary reading material for a more detailed exploration.

After we've learned some domain knowledge,

it is necessary to check if the values in the data set are intuitive,

and agree with our domain knowledge.

For example, if there is a column with age data,

we should expect the values rarely to be larger than 100.

And for sure, no one ever lived more than 200 years.

So, the values should be smaller than 200.

But for some reason, we find this super huge value 336.

Most probably, is just a typo but it should be 36 or 33,

and the best we can do is manually correct it.

But the other possibility is that its not a human age,

but some alien's age for which it's totally normal to live more than 300 years.

To check that, we should probably read the data description one more time, ask on forums.

Maybe the data is totally correct,

and then we just misinterpret it.

Now, take a look at our Google AdWords data set.

We understood that the values in the clicks variable should be

less or equal than the values in impressions column.

And in our case, in the first row,

we see zero impressions and three clicker.

That sounds like a bug, right?

In fact, it probably is,

but differently to the example of person's age,

it could be rather a regular error made by

either data exporting script or another kind of algorithm.

That is, the errors were made not at random,

but there is some kind of logic why there is an error in that particular place.

It means that these mistakes can be used to get a better score.

For example, in our case,

we could create a new feature,

is_incorrect, and mark all the rows that have errors.

Probably, our models will find this feature helpful.

It is also very important to understand how the data was generated.

What was the algorithm for sampling objects from the database?

Maybe, the host sample get objects at random,

or they over-sample the particular class, that is,

they generated more examples of that class.

For example, to make the data set more class balanced.

In fact, only if you know how the data was generated,

you can set up a proper validation scheme for models.

Coming down for the correct validation pipeline is crucial,

and we will discuss it later in this course.

So, what can we possibly find out about generation processes?

For example, we could find out

the train and test set were generated with different algorithms.

And if the test set is different to the train set,

we cannot use part of the train set as a validation set,

because this part will not be representative of test set.

And so, we cannot evaluate our models using it.

So once again, to set up a correct validation,

we need to know underlying data generation processes.

In the ad computation, we've discussed before,

that all the symptoms of different train test sampling.

Improving the model on validation set

didn't result into improved public leader-board score.

And more, the leader-board score was unexpectedly higher than the validation score.

I was also visualizing various things while trying to understand what's happening,

and every time, the plots for the train set were much different to the test set plots.

This also could not happen if the train and test set were similar.

And finally, it was suspicious that although

the train period was more than ten times larger than the test period,

the train set had much fewer rows.

it was not straight forward,

but this triangle on the left figure was the clue for me,

and the puzzle was solved.

I've adjusted the train set to match test set.

The validation score became reliable,

and the modeling could be commenced.

You can find the entire task description and investigation in the written materials.

So, in this video,

we've discussed several important exploratory steps.

First, we need to get domain knowledge about the task as

it helps to better understand the problem and the data.

Next, we need to check if the data is intuitive,

and agrees with our domain knowledge.

And finally, it is necessary to understand how

the data was generated by organizers because otherwise,

we cannot establish a proper validation for our models.

Explore nosso catálogo

Registre-se gratuitamente e obtenha recomendações, atualizações e ofertas personalizadas.

O Coursera proporciona acesso universal à melhor educação do mundo fazendo parcerias com as melhores universidades e organizações para oferecer cursos on-line.

© 2018 Coursera Inc. Todos os direitos reservados.

Baixar na App Store

Baixar no Google Play