Create and upload a zip file in the Ring web app in Clojure

Question

I have a Ring handler that should:

• Burn multiple files
• Zip stream for the client.

Now I have such work, but only the first record in zipped becomes streaming, and after that it stops / stops. I feel this has something to do with flushing / streaming, which is wrong.

Here is my handler (compojure):

(GET "/zip" {:as request} :query-params [order-id :- s/Any] (stream-lessons-zip (read-string order-id) (:db request) (:auth-user request))) 

Here is the stream-lessons-zip function:

 (defn stream-lessons-zip [] (let [lessons ...];... not shown {:status 200 :headers {"Content-Type" "application/zip, application/octet-stream" "Content-Disposition" (str "attachment; filename=\"files.zip\"") :body (futil/zip-lessons lessons)})) 

And I use streamed-stream-stream to stream like this:

 (defn zip-lessons "Returns an inputstream (piped-input-stream) to be used directly in Ring HTTP responses" [lessons] (let [paths (map #(select-keys % [:file_path :file_name]) lessons)] (ring-io/piped-input-stream (fn [output-stream] ; build a zip-output-stream from a normal output-stream (with-open [zip-output-stream (ZipOutputStream. output-stream)] (doseq [{:keys [file_path file_name] :as p} paths] (let [f (cio/file file_path)] (.putNextEntry zip-output-stream (ZipEntry. file_name)) (cio/copy f zip-output-stream) (.closeEntry zip-output-stream)))))))) 

So, I confirmed that the vector "lessons" contains as 4 entries, but the zip file contains only 1 entry. In addition, Chrome does not seem to “complete” the download, i.e. he thinks he is still loading.

How can i fix this?

Answer 1

It seems that creating a stateful thread using an IO lock is not supported by the http-kit. Potentialless flows can be performed as follows:

http://www.http-kit.org/server.html#async

PR to enter stateful streams using IO lock is not accepted:

https://github.com/http-kit/http-kit/pull/181

It seems like an opportunity to explore is to use ByteArrayOutputStream to completely map the zip file into memory, and then return the created buffer. If this endpoint is not heavily loaded and the generated zip file is small (<1 gb), then this may work.