Geek Answers Handbook

Connect Node.js as a Client to a Common Lisp Server

I have a small but heavy application with an alpha stage processor in node.js, this is a small game. I am facing performance issues and I need to speed it up at least 20 times to get to the beta. And since parallel execution would make me very far away, I decided that sharing a game map between processes or threads that would perform parallel operations on it would be a good start. This is pretty impossible to do in node, so I decided to write the meat parts in CL (SBCL + Linux) and connect to it through a unix domain socket.

Plan:

[players] <-> (node.js front server) <-> (SBCL performing game ticks)

The thing is, I need to pass fast messages between node.js and SBCL in a question like socket.io.

Here's what didn't work (you can skip this part)

On the Node side, I cannot use simple socket.io because it does not support Unix Domain Sockets, but the net module does, so I can at least make socket.write('raw data') - better than nothing at the moment,

On the CL side, I tried to start the woo web server (it supports local sockets), and I could connect to it from Node and send raw data, but there are all unnecessary parts of HTTP, and woo always works as a server; he is waiting for GET / HTTP/1.1 .... I did not find a way to initiate a message from woo in the first place. In addition, it is completely undocumented and commented out and includes many FF calls in C libs, which I am not familiar with at all.

So, I went through several more CL web servers that did not compile, did not support UNIX sockets, were left or undocumented, ended up moving to simple sb-bsd sockets and finally to iolib, but I still can’t find it out.

iolib looked promising, but I can't connect to it from node.

I have it:

 (with-open-socket (socket :address-family :local :type :datagram ;; :connect :passive :local-filename "/tmp/socket") (format t "Socket created") ;; (listen-on socket) ;; (bind-address socket (make-address "/tmp/socket")) (loop (let ((msg (receive-from socket :size 20))) (format t msg))))

and i get

 #<Syscall "recvfrom" signalled error EWOULDBLOCK(11) "Resource temporarily unavailable" FD=6> [Condition of type IOLIB/SYSCALLS:EWOULDBLOCK] Restarts: 0: [IGNORE-SYSCALL-ERROR] Ignore this socket condition 1: [RETRY-SYSCALL] Try to receive data again 2: [RETRY] Retry SLIME interactive evaluation request. 3: [*ABORT] Return to SLIME top level. 4: [ABORT] abort thread (#<THREAD "worker" RUNNING {10055169A3}>)

I don't know if I should first call something like accept-connection or listen-on on this socket. Everything I tried led to errors. Also, if I [RETRY-SYSCALL] in repl, the error disappears for about 10 seconds, but returns. During this time, Node is still unable to connect.

This seems more complicated than I thought. I already lost ~ 6 hours of work on iolib, and I did not even start parsing messages, learning how to create events from them, converting between JSON and s-exp, etc.

My questions are :

  • How to establish this connection in iolib so that Node network can connect?
  • Assuming I can choose which type of connection is best suited for transmitting events / messages? (datagram / stream)
  • Are there any work tools that I have not tried?
  • Also, are there some other libraries than iolib that are possibly higher level / better documented?
  • Are there any more efficient / lighter / faster approaches to this performance / concurrency issue?
  • Any other ideas?

I'm close to just dropping the CL idea and using something like in-memory mongo with multiple Node processes instead (... it really doesn't sound fast), but I like lisp, it would be great to have such things, like lparallel on the backend. I just haven't moved an inch since yesterday morning, I just can't understand the libraries. Perhaps I should learn clojure.

PS: Normally, I would not ask "write me a code", but if some good soul is around, I would really appreciate it, even in pseudo-code.

PPS: any radically different approaches are also welcome. Please give your opinion :)

Thank you for reading!

+5
source share
2 answers

So in the end, I figured it out ...

 (with-open-socket (socket :address-family :local :type :stream :connect :passive :local-filename "/tmp/node-sbcl.sock") (log-message :info "Waiting for client...") (setf *client* (accept-connection socket :wait t)) (log-message :info "Accepted client connection.") ;; ...lunch with *client* + the bits for parsing json and separating messages... )

I switched to :type :stream and most of the problems disappeared. accept-connection should be called on the socket, but listen-to should not. I had to write a way to divulge messages myself, but it was much easier than I thought. For some reason :type :datagram just didn't work, I don't know why.

And in node:

 var JsonSocket = require('json-socket'); var net = require('net'); var sbcl = new JsonSocket(net.Socket()); sbcl.connect("/tmp/node-sbcl.sock"); sbcl.on('connect', function(){ console.log('Connected to SBCL, YAY!'); console.log('Sending hi!'); sbcl.sendMessage({'cmd': "heyThere"}); sbcl.on('message', function(message){ if(!message.cmd) { console.log("We've received msg from SBCL with no CMD!!!'"); return; } switch(message.cmd){ case 'heyNode': console.log('SBCL says hi...'); break; } }); });

So this works if someone has similar chicken ideas using lisp and node together.

+2
source

If I understand your problem correctly, you need to install the server in Common Lisp. Let me reuse my previous answer , which uses USOCKET :

 (use-package :usocket) (defun some-server (hostname port) ;; create a server which establishes an event-loop (socket-server hostname ; a string port ; a number ;; tcp-handler (see API for details) ;; This function is called each time a client connects, ;; and provides a bidirectional stream to communicate with it. ;; The function executes in a context where some useful special ;; variables are bound. ;; The connection is closed automatically on exit. (lambda (stream) ;; format to stream to client (format stream "~D~%" ;; add all elements of the host, ;; a vector of 4 integers (reduce #'+ *remote-host*)))))
+5
source

More articles:


All Articles