The cost of garbage collection depends on the number of objects in the heap, in particular on the number of long-lived objects. The more you have, the more time will be spent on the GC.
Yes, the V8 can sometimes take some significant GC pauses, if the heap is big. The V8 team seems to be working to minimize the cost of each GC pause, spreading the work. You can see the cost of GC in your node programs by running it with --trace-gc.
GC . GC, V8, . , GC , .