Google Cloud Function terminates but never calls callbacks

Question

I am trying to make ~ 25 queries to get some data and save the response to separate files. I am using the npm request module to make requests with the google cloud core function. Each request may take from 0.5 to 5 seconds to return a response. When the script is configured to run less than 10 requests, everything works as expected and saves everything to the files you need. However, nothing more, and I do not receive a response from any request.

for (i = 0; i < popIDS.length; i++) { for (j = 0; j < genreIDs.length; j++) { fetchAndSaveToFile(popIDS[i], genreIDs[j], date); } } 

fetchAndSaveToFile makes an npm request and uses pako to save a compressed file response in Google Cloud Storage. Initially, this is a problem due to which ~ 5 requests were requested, but I increased the timeout for this function and it started working, however I exceeded the timeout for the function to 9 minutes. All functions are currently in one file.

It is also worth noting that the value popIDS [i], genreIDs [j] is not used inside the callback, and “i” and “j” do not need to be written.

Do I need to make asynchronous requests differently for this to work? I see that there is an async request module that handles such things. Are there any parameter settings that I need to configure? Or do I need to split this into separate files so that I can set a 9 minute timeout for each request?

Answer 1

I was able to solve by updating to use Promise.all() and matching query calls to promise objects using new Promise . Returning Promise.all() to the main Google Cloud Function cumulus, let the function know in order to wait for all promises to complete before exiting the function. Implementing callbacks instead of promises may cause the Google Cloud feature to exit prematurely before all callbacks complete.