Websocket Frame Size Limit

Question

I am sending huge chunks of JSON data through websockets. JSON can have more than 1000 entries. Due to frame size limitations, the Websocket protocol automatically splits JSON into frames, which is not possible. Since we cannot resize the websockets frame.

Problem:

When I try to evaluate my JSON using JSON.parse , it gives me a parsing error, which is obvious because frames are not full JSON objects. All this happens in the callback of the Websocket onmessage . How can I get huge JSON in different frames and still be able to parse it? I tried to execute frames in onmessage , but the error persists.

Side question:

How to connect broken JSON?

Answer 1

One WebSocket frame, per RFC-6455 basic framing , has a maximum size limit of 2 ^ 63 bytes (9,223,372,036,854,775,807 bytes ~ = 9.22 exabytes) ( @Sebastian fix)

However, a WebSocket message consisting of 1 or more frames does not impose restrictions on it from the protocol level.

Each WebSocket implementation will handle message and frame boundaries differently. Such as setting the maximum message size for the entire message (usually for reasons of memory consumption) or providing the ability to stream large messages for better memory use.

But in your case, it is likely that your chosen WebSocket implementation has an error and incorrectly breaks the JSON message into multiple messages, rather than multiple frames. You can use the network validation tool in Chrome or an external tool like Wireshark to confirm this behavior.

Answer 2

Since you are dealing with WS, which is low-level, you need to create an application protocol that processes data that is sent over several WS frames. You must concatenate the data that is in each WS frame (btw, do not concatenate frames ... concat the data in each frame).

Basically you invent a file transfer protocol.

Answer 3

 var wsServer = new websocket.server({ httpServer: server, maxReceivedFrameSize: 131072, maxReceivedMessageSize: 10 * 1024 * 1024, autoAcceptConnections: false }); 

Change the default value of maxFrameSize and MessageSize