Cloud storage client libraries offer a file-like object that Open () and Close () work with. If one operation can be transactional, then, theoretically, it should be possible to open one “lock file” for the duration of all other operations, only closing it when you finish will have all the other files.
In other words, you will need to write your processes in order to use the “lock file”, and thus, you could at least find out if all your files were written / read or if there were any errors. Whenever the next round of operations occurs, it simply looks for a lock file that matches the set of files written (you would need to organize your name, directory layout, etc., to make sense for this). If it exists, we can assume that the group of files was successfully recorded. If this does not exist, assume that something has happened (or that the process is not yet complete).
I really have not tested this. But I propose this as an idea for others who may be desperate to try.
source share