How to find missing values in Python

 Missing values is a very big problem in data science projects. Missing values can occur when no information is provided for one or more items or for a whole unit. It is necessary to find out whether there are missings.

The search for missings is usually one of the first steps in data analysis. At the beginning, the question is whether there are any missings at all and, if so, how many there are. As is often the case, Pandas offers several ways to determine the number of missings.

Missing values can be handled in different ways depending on if the missing values are continuous or categorical.

In Pandas missing data is represented by two value: None and NaN. Pandas treat None and NaN as essentially interchangeable for indicating missing or null values.

In order to check missing values in Pandas DataFrame, we use a function isnull() and notnull(). Both function help in checking whether a value is NaN or not. These function can also be used in Pandas Series in order to find null values in a series.

isnull()

In order to check null values in Pandas DataFrame, we use isnull() function this function return dataframe of Boolean values which are True for NaN values.

Example:

import pandas as pd 
import numpy as np 

di = {'Name': ['abi', 'hari', np.nan, 'dev'],
		'Age': [21, 20, 21, np.nan],
		'Grade': [np.nan, 'london', 'delhi', 'mumbai']}

df = pd.DataFrame(di)

print(df.isnull())

Output:

    Name    Age  Grade
0  False  False   True
1  False  False  False
2   True  False  False
3  False   True  False

 notnull()

In order to check null values in Pandas Dataframe, we use notnull() function this function return dataframe of Boolean values which are False for NaN values.

Example:

import pandas as pd 
import numpy as np 

di = {'Name': ['abi', 'hari', np.nan, 'dev'],
		'Age': [21, 20, 21, np.nan],
		'Grade': [np.nan, 'london', 'delhi', 'mumbai'],
		'Gender': [np.nan, np.nan, np.nan, np.nan]}

df = pd.DataFrame(di)

print(df.notnull())

Output:

    Name    Age  Grade  Gender
0   True   True  False   False
1   True   True   True   False
2  False   True   True   False
3   True  False   True   False