Well, web servers present an interesting problem, since they already have multithreading, serving concurrent requests. Thus, at a busy place, you cannot rely on the possibility of stealing a large number of cores for your purposes. Of course, if you expect only light traffic, and your site requires a lot of crunches (data processing, etc.), you can get some savings.
I would expect Parallel Extensions to work better on the client or specialized service applications where you can reasonably expect the kernels to be available for your use [ab].