I pass the data to the bigquery table, for some reason the last few times I checked that the data is not available instantly. Streaming data is ultimately available for the request, sometimes it seems like it has been out of sync for more than an hour. But the data eventually appears after some delays.
Is this expected? If so, what are the ways around it? The table is divided by date and we do not upload data to any particular section, we transmit data only for the current day.
Update:
When I say that the data is not available, I mean that starting the next query does not match the date / time when the data was transferred to Bigquery.
select max(dateTime) from [nproj.reporting] ;
source
share