Geek Answers Handbook

Serve a Tensorflow Learning Model with a REST API using Flask?

I have a trained Tensorflow model and I want to use the prediction method using the REST API. I can think of using Flask to create a simple REST API that accepts JSON as input and then calls the prediction method in Tensorflow and then returns the predicted result on the client side.

I would like to know if there is any concern to do it this way, especially in a production environment?

Thank you very much!

+6
source share
2 answers

The first problem that comes to my mind is performance.

The TensorFlow team seems to have developed server / client usage. You may want to look into the tenorflow function . By default, gRPC is used for the communication protocol.

+2
source

We use Flask + TensorFlow, working at work. Our setup may not be the best way to service models, but it does its job, and it has worked perfectly for us so far.

The setting is as follows:

  • Since tfserving takes the build forever, we built the docker image (not GPU support or anything else, but it only works to serve the model, and it’s faster and better than servicing directly from the huge Python / Flask monolith). An image of the model server can be found here: https://hub.docker.com/r/epigramai/model-server/
  • Flask is then used to configure the API. To send requests to the model server, we need the grcp forecast client, so we built it in Python, which we can import directly into the flags API, https://github.com/epigramai/tfserving_predict_client/ .

It’s good that the model is not served by the Flask API. The docker image model server can be easily replaced with a model server running on a GPU compiled for the hardware of the machines, and not in the docker container.

0
source

More articles:


All Articles