I ran your code and it worked sometimes and sometimes it didn’t execute (about 90% of the time). It worked on my computer with 5 FPS . It will not play well on a mobile device, and I'm sure that you are targeting an iPad.
There are several problems in the code, but these are very serious problems.
1. Your image is not fully received before it is uploaded.
That's why your image looks so weird.
The biggest mistake people make when working with socket is to assume that everything you send will be received right away. It is not true. This is a way to encode your client. Read please .
This is the method I used in my answer:
A. Get the byte array of Texture2D .
B. Send the length of the byte array. Not an array of bytes, but a length.
C. The client will read the length first.
D. The client will use this length to read all the data / pixels of the texture to completion.
E. Convert received bytes to an array.
You can look at private int readImageByteSize(int size) and private void readFrameByteArray(int size) functions to read all bytes.
Of course, you should also know the length of the data length that is sent first. The length is stored in the int data type.
The value of max int is 2,147,483,647 , and it is longer than 10 . So I made an array the length of the array, which is first sent 15 as a protocol. This is a rule that must also be implemented on the client side.
Now how does it work:
Read an array of bytes from Texture2D , read the length of this array, send it to the client. The client follows the rule that the first 15 bytes are just the length. The client will then read that 15 bytes, convert it back to length, and then use that length in a loop to read the completed Texture2D from the server.
Length conversion is performed using the void byteLengthToFrameByteArray(int byteLength, byte[] fullBytes) and int frameByteArrayToByteLength(byte[] frameBytesLength) . Take a look at those who understand them.
2. We make the socket work in the main topic.
This is why FPS on my computer 5 .
Do not do this, as this will cause the frame rate to be low, just like it. I answered a lot of questions like this, but I won’t go deeper, because it looks like you know what you are doing, and you tried to use Thread , but you did it wrong.
A. You read from the main Thread when you did this: serverStream.Read(readBuffer, 0, readBuffer.Length); in the Update function.
You should have done it inside
Loom.RunAsync(() => {
B. You made the same mistake in the SendPng function when you sent data using stream.Write(pngBytes, 0, pngBytes.Length); in
Loom.QueueOnMainThread(() => {});
Everything you do inside Loom.QueueOnMainThread will be done in the main Thread .
You must Thread.Loom.RunAsync(() =>{}); to another Thread.Loom.RunAsync(() =>{});
Finally, listner = new TcpListener(port); is absolute. This did not cause any problems, but use listner = new TcpListener(IPAddress.Any, port); in the server code that should listen on the IP address.
The final FPS is over 50 on my computer after all these fixes have been completed. The code below can be improved a-lot. I will leave it for you.
You can use the online code comparison to see what has changed in each class.
SERVER
using UnityEngine; using System.Collections; using System.IO; using UnityEngine.UI; using System; using System.Text; using System.Net; using System.Net.Sockets; using System.Threading; using System.Collections.Generic; public class Connecting : MonoBehaviour { WebCamTexture webCam; public RawImage myImage; public bool enableLog = false; Texture2D currentTexture; private TcpListener listner; private const int port = 8010; private bool stop = false; private List<TcpClient> clients = new List<TcpClient>();
CLIENT
using UnityEngine; using System.Collections; using UnityEngine.UI; using System.Net.Sockets; using System.Net; using System.IO; using System; public class reciver : MonoBehaviour { public RawImage image; public bool enableLog = false; const int port = 8010; public string IP = "192.168.1.165"; TcpClient client; Texture2D tex; private bool stop = false; //This must be the-same with SEND_COUNT on the server const int SEND_RECEIVE_COUNT = 15; // Use this for initialization void Start() { Application.runInBackground = true; tex = new Texture2D(0, 0); client = new TcpClient(); //Connect to server from another Thread Loom.RunAsync(() => { LOGWARNING("Connecting to server..."); // if on desktop client.Connect(IPAddress.Loopback, port); // if using the IPAD //client.Connect(IPAddress.Parse(IP), port); LOGWARNING("Connected!"); imageReceiver(); }); } void imageReceiver() { //While loop in another Thread is fine so we don't block main Unity Thread Loom.RunAsync(() => { while (!stop) { //Read Image Count int imageSize = readImageByteSize(SEND_RECEIVE_COUNT); LOGWARNING("Received Image byte Length: " + imageSize); //Read Image Bytes and Display it readFrameByteArray(imageSize); } }); } //Converts the data size to byte array and put result to the fullBytes array void byteLengthToFrameByteArray(int byteLength, byte[] fullBytes) { //Clear old data Array.Clear(fullBytes, 0, fullBytes.Length); //Convert int to bytes byte[] bytesToSendCount = BitConverter.GetBytes(byteLength); //Copy result to fullBytes bytesToSendCount.CopyTo(fullBytes, 0); } //Converts the byte array to the data size and returns the result int frameByteArrayToByteLength(byte[] frameBytesLength) { int byteLength = BitConverter.ToInt32(frameBytesLength, 0); return byteLength; } /////////////////////////////////////////////////////Read Image SIZE from Server/////////////////////////////////////////////////// private int readImageByteSize(int size) { bool disconnected = false; NetworkStream serverStream = client.GetStream(); byte[] imageBytesCount = new byte[size]; var total = 0; do { var read = serverStream.Read(imageBytesCount, total, size - total); //Debug.LogFormat("Client recieved {0} bytes", total); if (read == 0) { disconnected = true; break; } total += read; } while (total != size); int byteLength; if (disconnected) { byteLength = -1; } else { byteLength = frameByteArrayToByteLength(imageBytesCount); } return byteLength; } /////////////////////////////////////////////////////Read Image Data Byte Array from Server/////////////////////////////////////////////////// private void readFrameByteArray(int size) { bool disconnected = false; NetworkStream serverStream = client.GetStream(); byte[] imageBytes = new byte[size]; var total = 0; do { var read = serverStream.Read(imageBytes, total, size - total); //Debug.LogFormat("Client recieved {0} bytes", total); if (read == 0) { disconnected = true; break; } total += read; } while (total != size); bool readyToReadAgain = false; //Display Image if (!disconnected) { //Display Image on the main Thread Loom.QueueOnMainThread(() => { displayReceivedImage(imageBytes); readyToReadAgain = true; }); } //Wait until old Image is displayed while (!readyToReadAgain) { System.Threading.Thread.Sleep(1); } } void displayReceivedImage(byte[] receivedImageBytes) { tex.LoadImage(receivedImageBytes); image.texture = tex; } // Update is called once per frame void Update() { } void LOG(string messsage) { if (enableLog) Debug.Log(messsage); } void LOGWARNING(string messsage) { if (enableLog) Debug.LogWarning(messsage); } void OnApplicationQuit() { LOGWARNING("OnApplicationQuit"); stop = true; if (client != null) { client.Close(); } } }