We have some data about Hadoop, which should be supplemented with data that is accessible only to us through the REST service. We use Spark to find and fix missing data. Despite the fact that there are many records for finding missing data, it is expected that the total number of calls to the service will be low, so it would be ideal to do all the work in Spark when we look through the data.
I don't see anything obvious in the API or on Google related to calling a REST from a Spark job. Is it possible?
thank
source
share