Filtering by factor variable using a double variable in R data.table

Question

Why can I filter a factor variable using a double variable in one case, but not in another?

Examples of data below:

dt <- data.table(id=1:9, var=factor(81:89)) # > dt # id var # 1: 1 81 # 2: 2 82 # 3: 3 83 # 4: 4 84 # 5: 5 85 # 6: 6 86 # 7: 7 87 # 8: 8 88 # 9: 9 89 

Why does it work ...

 dt[id %in% 1:7 & var %in% c(82, 84)] # id var # 1: 2 82 # 2: 4 84 

... but does this give an error?

 dt[var %in% c(82, 84)] # Error in bmerge(i, x, leftcols, rightcols, io <- FALSE, xo, roll = 0, : # x.'var' is a factor column being joined to i.'V1' which is type 'double'. # Factor columns must join to factor or character columns.` 

It seems like this is a little inconsistent and could be a mistake?

Answer 1

The difference is that the second example is optimized by automatic indexing, which causes this error. You can disable this feature as follows:

 dt[(var %in% c(82, 84))] # id var #1: 2 82 #2: 4 84 

Then the basic vector scan R is used and the usual enforcement rules are applied. From help("%in%") :

Factors, source vectors and lists are converted to character vectors and then x and the table are forced to a common type

 var <- factor(81:89) var %in% c(82, 84) #[1] FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 

The problem has been fixed in data.table version 1.9.7.