You can use mapply:
vec = mapply(function(u,v) A[,u]%*%B[,v], c('a','d'), c('f','h'))
If you want to use data.frame C:
vec = mapply(function(u,v) A[,u]%*%B[,v], as.character(C[,1]), as.character(C[,2]))
What really matters is the number of lines C, the number of lines in Aand Bmay not be a bottleneck:
v1=rnorm(1000000)
v2=rnorm(1000000)