Python web service with Twisted

Question

This is related to my previous question by the Python web service .

I will use Tornado to exchange information between the server and clients. There will be one server and N clients. Customers will periodically send information (disk usage, processes, etc.) (Every 2 minutes or so). Client-side data will be represented by custom classes / lists. It would be nice to have the same data on the other hand (server).

I have experience with SOAP, and it will probably be fine for this (with enough timeouts on the server), but would rather prefer to use something lighter and more pythonic. The message will be more or less the only client -> server.

Both the server and the client side are written in Python.

What should I look in the Twisted documentation for this?

edit: I am not asking how to serialize data (JSON or pickle or XML, etc.). I would like to know what Twisted options are for transferring data.

With SOAP, I would have these methods:

- sendDiskUsage(DiskUsage class instance) - sendProcesses(ProcessList class instance) - etc.. 

I would like to know what are the options with Twisted. One of them is XML-RPC, which will be fine, but this is not my favorite ...

edit2: the connection will be "two-way" - the client will read tasks from the server ...

Answer 1

I recommend AMP . This is a very simple key-value-based protocol, perfect for what you are doing. A promising broker is another alternative, but it is a bit complicated and usually not needed.

AMP works directly over TCP (why worry about HTTP?), The serialization format is minimal and logical. This makes it "light" and "pythonic" in the sense that you probably mean, but these terms can be interpreted in several ways.

Take a look at AMP examples for twisted code examples , they are pretty clear. AMP connections are bidirectional, so try changing the example so that the server asks the client for current disk usage. twisted.protocols.amp API documents will be useful here.

When you are ready to create a real application, read Twisted from Scratch or Evolution of Finger .

AMP was implemented in most popular languages, but if you are looking for something more "mainstream", the Google protobuf thing. IMO is more complicated and lacks some important functions, such as error feedback.

Answer 2

You can try the twisted “prospective broker”. It has some nice features, such as symmetry (there is no real difference between the server and the server after the connection is established). He performs serialization himself. This might not be the best choice if you need a simple one-way push update update.

Answer 3

To serialize data, if you use pickle to serialize instances of a custom class, make sure that both the server and the client have the same class definition in the global namespace.

For a twisted link, you can take a look at its documentation , especially the “Writing a TCP Server” and “Writing a TCP Client,” part.

Answer 4

If both the server and the client are written in Python, the easiest way is to simply perform an HTTP POST request from the client to the server having data serialized into pickled Python objects. Salt burns will carry the entire structure of an orignal object.

If the client is a non-Python process, then JSON is the way to go - you just serialize the objects into JSON. The difference between pickles and JSON is that JSON cannot carry classes, but only data as nested lists and dicts.

HTTP POST with JSON payload is what I would do.

Do not worry about SOAP. WSDL etc. It just adds extra complexity to the process, the very non-pythonic and Python SOAP libraries are not so reliable.

Swirling documentation does not cover this kind of thing, because this is a general Python problem, not related to Twisted.