Skip to main content

10 Steps to Data Wrangling for Data Analysis using Pandas

10 Steps to Data Wrangling for Data Analysis using Pandas

Step 1: Import Pandas

import pandas as pd

Step 2: Have a DataFrame created using pandas

df = pd.read_csv('sample_data.csv')

Step 3: Count null values

df.isnull().sum() # gives you count of null values in each column of the dataframe

Step 4: Plot on heatmap

import seaborn as sns
sns.heatmap(df.isnull(), yticklabels==False, cmap='viridis')

Step 5: Drop Column which has way too many null values

df.drop('Column name', axis=1, inplace=True)
axis=1 is required to delete column and not row.
inplace=True is required to update the dataframe.

Step 6: Drop all rows having null values

df.dropna(inplace=True)

Step 7: Recheck dataframe

df.isnull().sum() # should return 0 for all columns

Step8: Change categorical-string data into columns with binary values

For example sex/gender of a person can be changed to 0s and 1s and will still make sense.
Using pandas' dummies function this can be achieved.
sex=pd.dummies(df['sex'], drop_first=True)
Here we are using drop_first=True because the function returns two columns 'male', and 'female'. One is opposite of other, so we don't need both.
Do this for all categorical columns.

Step9: Club all dataframes

df = pd.concat(pd, sex)

Step10: Remove all non-numeric and non-binary columns

df.drop(['sex'], axis=1, inplace=True)

Comments

Post a Comment

Popular posts from this blog

Difference between .exec() and .execPopulate() in Mongoose?

Here I answer what is the difference between .exec() and .execPopulate() in Mongoose? .exec() is used with a query while .execPopulate() is used with a document Syntax for .exec() is as follows: Model.query() . populate ( 'field' ) . exec () // returns promise . then ( function ( document ) { console . log ( document ); }); Syntax for .execPopulate() is as follows: fetchedDocument . populate ( 'field' ) . execPopulate () // returns promise . then ( function ( document ) { console . log ( document ); }); When working with individual document use .execPopulate(), for model query use .exec(). Both returns a promise. One can do without .exec() or .execPopulate() but then has to pass a callback in populate.
1 comment
Read more

Two Ways of rendering in React Examples

Two Ways of rendering in React Examples 1 - Render twice HTML < div id = "p1" ></ div > < div id = "p2" ></ div > CSS . person { display : inline - block ; margin : 10px ; border : 1px solid # eee ; box - shadow : 0 2px 2px # ccc ; } JAVASCRIPT (Babel) function Person ( props ) { return ( < div className = ' person "> < h1 >{ props . name }</ h1 > < p > Age :{ props . age }</ p > </ div > ); } ReactDOM . render (< Person name = "VK" age = "29" />, document . querySselector ( '#p1' )); ReactDOM . render (< Person name = "HK" age = "28" />, document . querySselector ( '#p2' )); 2 Render all at once HTML < div id = "app" ></ div > CSS . person { display : inline - block ; margin : 1...
Post a Comment
Read more

Resolve: Uncaught TypeError: firebase.database is not a function

If you are getting the error: Uncaught TypeError: firebase.database is not a function Resolve it by including firebase-database.js in your html page as follows: <!-- The core Firebase JS SDK is always required and must be listed first --> <script defer src = "https://www.gstatic.com/firebasejs/6.2.4/firebase-app.js" ></script> <script defer src = "https://www.gstatic.com/firebasejs/3.1.0/firebase-database.js" ></script> That is it. Let me know if this was helpful.
Post a Comment
Read more