geom_raster() will be the fastest for square tiles of the same shape.
You probably want geom_polygon() - here is Sierpinski:
don't forget to use coord_fixed(ratio=1) or the aspect ratio of the shape will scale to the shape of your viewer:
EDIT - sorry to understand that I did not give you Sierpinski (not sure what I thought) fixed

require(ggplot2) require(reshape2) th<-sin(2*pi/6) # eq triangle unit height sierpinski<-function(iter=3){ n<-2^iter points<-ldply((n-1):0,function(x){ data.frame( y=rep(nx-1,x)*th/n, x=seq((from=(0.5/n)+(nx)*(0.5/n)),by=1/n,length.out=x) ) }) points$id<-1:nrow(points) rbind( points, points+matrix(c((th/n),(-0.5/n),0),nrow(points),ncol=3,byrow=T), points+matrix(c((th/n),(0.5/n),0),nrow(points),3,byrow=T) ) } axiom<-data.frame(x=c(0,0.5,1),y=c(0,th,0)) iterations<-6 ggplot() + theme_bw() + coord_fixed(ratio=1) + geom_polygon(data=axiom,aes(x,y), fill="orange") + lapply(1:iterations,function(x){ geom_polygon(data=sierpinski(x),aes(x,y,group=id), fill="white") })
In addition, you should be aware of these very accurate recursive and serial models. Sometimes ggplot will not display the image exactly as you expect. For example, see below for the cantor dust diagram:
Using ggplot (using a raster) you can see that even at high resolution the “legs” look consistent, whereas mathematically you know what they are. If you upload an image and zoom in, you will see inconsistencies at the bottom.

In the code below, I showed how you can create your own accurate image by creating a raw png file on the fly. Do not be afraid to do this if you need precision! The result of the formation of the raw image below:
Good luck

# CANTOR