Python

How to filter out na in R language

RA Asked by Rakesh · 22-02-2020
0 upvotes 417 views 0 comments
The question

Below is the code:

library(tidyverse)  df <- tibble(      ~col1, ~col2, ~col3,      1, 2, 3,       1, NA, 3,       NA, 2, 3  )

I can remove all NA abservations with the help of drop_na():

df %>% drop_na()

Or remove all NA in a single column (col1 for example):

df %>% drop_na(col1)

Why I cannot just use a regular != filter pipe ?

df %>% filter(col1 != NA)

Why do we use a special function from tidyr to remove NAs?

5 answers

0
MA
Answered on 11-02-2022
This is not directly related to dplyr::filter. However, any comparison involving NA, such as NA==NA, will yield NA as a result.
R is unaware of the context of your analysis.
Essentially, it does not permit comparison operators to treat NA as a valid value.
Are you considering a career in data analysis? Our Data Analyst Certification Course will provide you with the essential tools and techniques for success.
0
VE
Answered on 15-03-2022

Try this:

df %>% filter(!is.na(col1))
VI 19-04-2022

This was simple, direct and perfect...thank you!

BA 02-04-2022

Thanks, that worked

0
VE
Answered on 25-04-2022
In R, null values do not possess a concept of equality. Consequently, the expression NA == NA yields NA. In reality, comparing NA with any object in R will also result in NA. The filter function in dplyr necessitates a boolean argument. Therefore, when it processes col1 and checks for inequality using filter(col1 != NA), the command 'col1 != NA' consistently produces NA values for every row in col1. Since this does not constitute a boolean expression, the filter function fails to evaluate correctly.
0
CA
Answered on 15-05-2022
Is it possible to generate a list similar to the one below to identify the rows that contain no null values, and subsequently use this list with the dplyr function to filter rows based on their position values?

In this case, na.omit(airquality$Ozone) will yield the values that are not null.

Afterward, can we supply the list of positions to the filtering function?
0
JO
Answered on 24-08-2022
The dplyr package includes the 'filter()' function for filtering data, but its capabilities extend beyond that. With dplyr, you can execute filtering operations that may be challenging or complex to achieve using SQL or conventional business intelligence tools, all in a straightforward and intuitive manner. For instance, consider a flight dataset where we can eliminate NA values using the filter keyword.
flight %>%  select(FL_DATE, CARRIER, ORIGIN, ORIGIN_CITY_NAME, ORIGIN_STATE_ABR, DEP_DELAY, DEP_TIME, ARR_DELAY, ARR_TIME) %>%  filter(!is.na(ARR_DELAY))

Share your thoughts

Your email address will not be published. Required fields are marked (*)

Professional Counselling Session

Still have questions?
Schedule a free counselling session

Our experts are ready to help you with any questions about courses, admissions, or career paths. Get personalized guidance from industry professionals.

Request a Call Back

Search Online

We Accept

We Accept

Follow Us

"PMI®", "PMBOK®", "PMP®", "CAPM®" and "PMI-ACP®" are registered marks of the Project Management Institute, Inc. | "CSM", "CST" are Registered Trade Marks of The Scrum Alliance, USA. | COBIT® is a trademark of ISACA® registered in the United States and other countries.

Book Free Session