Can parquet support simultaneous write operations?

Can distributed parallel recordings be done in parquet format?

And is it possible to read parquet files while they are being written?

If there are methods for reading / writing at the same time, I would be interested to know.

Thanks in advance for your help.

+6
source share
1 answer

In the end, I got an answer from the Parquet developers: the answer to both questions does not answer:

The authors of the parquet are not thread safe and the files cannot be read or written by different readers or writers at the same time. Parquet does not expose the user to flash / synchronization (for good reason), so it is not a reliable way to do this in any case.

+4
source

All Articles