As I can see, in deciding whether to use pointers against objects, there are a number of factors:
1. Do you or do not need polymorphism?
If you want to maintain a container of objects of a base class, but then store objects of various derived classes in it, you must use pointers, because calls to virtual functions would otherwise not be correctly resolved.
2. Size of stored objects and their suitability for copy operations
One of the main reasons pointers may be preferable for objects is that various operations performed in the container include creating copies of the objects stored in it. This applies to many storage operations (for example, std::vector<>::push_back() or std::map<>::insert() ), some search operations (for example, std::vector<>::operator[] , and then saving the object in a local variable) and some operations performed by the container “internally”, for example, redistributing a vector when it exceeds its capacity, or renaming std::unordered_map<> . Note that copy operations may be less significant depending on how you select the container and how to use it (for example, using std::vector<>::reserve() to allocate enough space using std::vector<>::emplace_back() for storing and never creating a local copy of the extracted item may mean that the copy is not made).
However, if you expect a large number of copies (or if profiling existing code shows that many copies have been made), using pointers instead of objects can obviously help, since pointers are small and well aligned in memory. Again, this will not make much sense if the objects you are storing are actually smaller than pointers.
3. Other operations that you perform on the container and its contents
Even if the objects you are dealing with are larger than pointers, and you expect a significant number of copy operations, using pointers is not necessarily preferable. Consider a situation where you store a large number of medium-sized objects (say, 16 bytes each), and you often need to sort through the entire container and perform some kind of statistical calculation. When you store these objects directly in a vector, you get more cache efficiency during iteration: as you retrieve a single object, the entire entire cache line will be retrieved from memory, which greatly speeds up the search for the next few objects. This is usually not the case when pointers are used; conversely, after retrieving an item, the pointer must be dereferenced, which will lead to another move operation from the memory area, which may not be cached.
Thus, it all depends on the type and size of the objects that you store, as well as the type and frequency of operations performed. If the objects you are dealing with are different types of windows, buttons, and menus in a GUI application, you will most likely want to use pointers and polymorphism. If, on the other hand, you are dealing with huge structures of compact elements that are the same in size and shape, and the operations you perform are associated with frequent iteration or bulk copying, and storing objects directly is unsurpassed. There may also be situations where a decision is difficult to make without trying to solve both issues and based on the results of memory and time tests.
As a final note, if you finish using pointers, consider whether the container you are building is the ultimate owner of the objects that you are allocating to the heap or just supporting temporary pointers. If the container is the owner of these objects, it will be useful for you to use smart pointers, not raw pointers.