How to Fix: ValueError: Cannot mask with non-boolean array containing NA / NaN values

One error you may encounter when using pandas is:

ValueError: Cannot mask with non-boolean array containing NA / NaN values

This error usually occurs when you attempt to find the rows in a pandas DataFrame that contain a particular string, but the column you’re searching in has NaN values.

The following example shows how to fix this error in practice.

How to Reproduce the Error

Suppose we have the following pandas DataFrame:

import pandas as pd
import numpy as np

#create DataFrame
df = pd.DataFrame({'team': ['A', 'A', 'A', 'B', 'B'],
                   'position': ['Guard', 'Guard', np.nan, 'Guard', 'Forward'],
                   'points': [22, 28, 14, 13, 19]})

#view DataFrame
print(df)

  team position  points
0    A    Guard      22
1    A    Guard      28
2    A      NaN      14
3    B    Guard      13
4    B  Forward      19

Now suppose we attempt to access all rows in the DataFrame where the position column contains the string “Guard”:

#access all rows where position column contains 'Guard'
df[df['position'].str.contains('Guard')]

ValueError: Cannot mask with non-boolean array containing NA / NaN values

We receive an error because there is a NaN value in the position column.

How to Fix the Error

To avoid this error, we simply need to use the argument na=False within the str.contains() function:

#access all rows where position column contains 'Guard', ignore NaN
df[df['position'].str.contains('Guard', na=False)]

        team	position  points
0	A	Guard	  22
1	A	Guard	  28
3	B	Guard	  13

This time we’re able to access all rows that contain “Guard” in the position column without any errors.

Another way to avoid this error is to use .fillna(False) as follows:

#access all rows where position column contains 'Guard', ignore NaN
df[df['position'].str.contains('Guard').fillna(False)]

        team	position  points
0	A	Guard	  22
1	A	Guard	  28
3	B	Guard	  13

Once again we’re able to access all rows that contain “Guard” in the position column without any errors.

Additional Resources

The following tutorials explain how to fix other common errors in Python:

How to Fix KeyError in Pandas
How to Fix: ValueError: cannot convert float NaN to integer
How to Fix: ValueError: operands could not be broadcast together with shapes