Develop Reference

Transfer real time live audio From microphone over TCP/IP Using C# .net

I want to make an app to transfer Audio from the microphone in a laptop or PC live, in real time, like a YouTube stream but without video. I will describe my process:
I transfer a normal file by converting it to byte then back again to origin.
I change the file type to MP3 or wav then use NAudio. Works fine also I can play the file if transferred or while receiving
I change the input file to Microphone and receive the audio.
Here is the problem: NAudio is unable to put the live stream from the mic then send it automatically. Always, the buffer gives me a null pointer exception while debugging ,then it gives me another error that the decoder of NAudio did not receive any data, "not acceptable by the way".
It should listen or keep receiving data until the port or connection closes.
I've tried to search about any library about VoIP but found nothing except Ozeki, but no tutorial to handle. All I found is old videos that do not work. I searched about that over a week but no result. I don't want a fully developed project because I already found one, but it is too complex -- about 2K lines of code. All I need is to know what to do or to be given the code that solves the problem.
This is client side code:
public void client()
{
try
{
//byte[] send_data = Audio_to_byte(); // ORIGINAL WORK CODE
byte[] send_data = new byte [BufferSize]; // 100M buffer size
TcpClient client = new TcpClient(serverip, port);
NetworkStream stream = client.GetStream();
// sourceStream and wavein is global vars
sourceStream = new NAudio.Wave.WaveIn();
sourceStream.DeviceNumber = 1;
sourceStream.WaveFormat = new NAudio.Wave.WaveFormat(44100, NAudio.Wave.WaveIn.GetCapabilities(1).Channels);
wavein = new NAudio.Wave.WaveInProvider(sourceStream);
//wavein.Read(send_data, 0, send_data.Length); // this buffer not work some times give me full buffer
BufferedWaveProvider pro = new BufferedWaveProvider(wavein.WaveFormat);
pro.AddSamples(send_data, 0, send_data.Length); // Empty buffer or error full buffer
stream.Write(send_data, 0, send_data.Length);
stream.Close();
client.Close();
}
catch (Exception e)
{
MessageBox.Show("Server is offline" + e, "Error");
// Here the message is buffer is full or send it empty then the decoder did not receive anything give exception error or of them happen first
}
}
and this server side code with MP3 reader code
IPAddress ip = Dns.GetHostEntry(serverip).AddressList[0];
TcpListener server_obj = new TcpListener(ip,port);
TcpClient client = default(TcpClient);
try
{
server_obj.Start();
while (true)
{
// accept all client
client = server_obj.AcceptTcpClient();
// make byte storage from network
byte[] received_buffer = new byte[BufferSize];
//get data from cst
NetworkStream stream = client.GetStream();
//save data from network to memory till finish then save with playing
MemoryStream ms = new MemoryStream();
int numBytesRead = 0;
while ((numBytesRead = stream.Read(received_buffer, 0, received_buffer.Length)) > 0)
{
// THIS STEP TO RECEIVE ALL DATA FROM CLIENT
ms.Write(received_buffer, 0, numBytesRead);
//receive sound then play it direct
WaveOut(ms.ToArray());
}
Byte_to_audio(ms.ToArray()); // YOU can make or allow override
}
}
catch(Exception e)
{
MessageBox.Show("Error Message : " + e, "Error");
}
}
this is Method that read stream receiving data from network
private void WaveOut(byte[] mp3Bytes)
{
// MP3 Format
mp3Stream = new MemoryStream(mp3Bytes);
mp3FileReader = new Mp3FileReader(mp3Stream);
wave32 = new WaveChannel32(mp3FileReader, 0.3f, 3f);
ds = new DirectSoundOut(); // but declration up global
ds.Init(wave32);
ds.Play(); // work code*/
}

I recommend using UDP, if it has to be in real time.
As I use Naudio with vb.net, I based this post.
Client Example:
waveIn = new WaveIn();
waveIn.BufferMilliseconds = 50; //Milissecondes Buffer
waveIn.DeviceNumber = inputDeviceNumber;
waveIn.WaveFormat = HEREWaveFormatHERE;
waveIn.DataAvailable += waveIn_DataAvailable; //Event to receive Buffer
waveIn.StartRecording();
void waveIn_DataAvailable(object sender, WaveInEventArgs e)
{
//e -> BUFFER
stream.Write(send_data, 0, send_data.Length);
stream.Close();
client.Close();
}
RECEIVE:
1 - Create WaveOut and BufferProvider Global
WOut = New WaveOut(WaveCallbackInfo.FunctionCallback());
BufferedWaveProvider pro = new BufferedWaveProvider(HEREWaveFormatHERE);
pro.BufferLength = 20 * 1024; //Max Size BufferedWaveProvider
pro.DiscardOnBufferOverflow = True;
WOut.Init(pro);
WOut.Play();
As long as there is no audio BufferedWaveProvider will provide silence for WaveOut or other outputs, it will also queue everything that arrives, for continuous playback.
2 - Play and Enqueue
while ((numBytesRead = stream.Read(received_buffer, 0, received_buffer.Length)) > 0)
{
pro.AddSamples(received_buffer, 0, numBytesRead);
}
My knowledge of naudio is limited to that
English from google translator

Unity .net socket + protobuf issues

Currently I'm trying to develop a multiplayer game with Unity. I checked the Unity's built-in Network and NetworkManager but I need the server dedicated. Hence I developed a java server and implements the protolbuf which is provided by Google. It works with my cocos project but not with this Unity one.
Two problems, first, no matter what did I send to server, the server side has a InvalidProtolBufferException :
While parsing a protocol message, the input ended unexpectedly in the middle > of a field. This could mean either than the input has been truncated or that > embedded message misreported its own length.
second, the stream.read method always makes my unity not responding. Here is my client-side code:
public class SocketClient : MonoBehaviour {
public Text send;
public Text read;
const string ipAddress = "192.168.0.233";
const int port = 8080;
TcpClient socket;
NetworkStream stream;
BinaryWriter bw;
BinaryReader br;
// Use this for initialization
void Start () {
SetupSocket();
}
void SetupSocket() {
socket = new TcpClient(ipAddress, port);
stream = socket.GetStream();
bw = new BinaryWriter(socket.GetStream());
br = new BinaryReader(socket.GetStream());
}
// Update is called once per frame
void Update () {
ReadMessage();
}
public void SendMessage() {
//NetworkStream stream = new NetworkStream(socket.Client);
MessagesProto msg = new MessagesProto();
msg.id = int.Parse(send.text);
using (MemoryStream ms = new MemoryStream())
{
Serializer.Serialize<MessagesProto>(ms, msg);
ms.Position = 0;
byte[] data = ms.ToArray();
ms.Position = 0;
//foreach (byte d in data) {
// Debug.Log(d);
//}
stream.Write(data, 0 , data.Length);
}
}
public void ReadMessage() {
if (stream.CanRead) {
//byte[] receiveData = new byte[socket.ReceiveBufferSize];
byte[] receiveData = new byte[4];
socket.GetStream().Read(receiveData, 0, 4);
Debug.Log("Loading...");
using (MemoryStream ms = new MemoryStream()) {
ms.Write(receiveData, 0, receiveData.Length);
ms.Position = 0;
var msg = Serializer.Deserialize<MessagesProto>(ms);
read.text = msg.data + "";
}
}
}
}
I tried to set the stream.CanRead to stream.DataAvailable, no more crash but not reading anything either, these .Net sockets problems drive me crazy, anyone can help me with this please?

It is not responding because your socket code is synchronous(blocking). You can solve this by either using asynchronous version of the socket functions or use Thread to do all your receiving stuff. This has been answered many times and I will just link to the answers.
TCP Server in Unity
Just port the code to Client.
UDP.

Sending and receiving images via a socket

I have a C# desktop app. It connects to another PC on my network which is a UWP C# app.
I am trying to send an image or 2 to my listening socket and to test this I get the listening socket to send me the image back.
The trouble is that even though my server recieves all the bytes that were orginally sent the recieved image back to the client is not of the same size.
To make this even more weird is sometimes the returned bytes are correct and I get the whole image and when I attempt to send 2 images the 1st one is OK and the 2nd one is not.
Then it will/can revert back to no images being sent back correctly.
I think is maybe to do with the async/await parts bit I am not sure how.
This is my server code:
using (IInputStream input = args.Socket.InputStream)
{
byte[] data = new byte[BufferSize];
IBuffer buffer = data.AsBuffer();
uint dataRead = BufferSize;
while (dataRead == BufferSize)
{
await input.ReadAsync(buffer, BufferSize, InputStreamOptions.Partial);
requestInBytes.AddRange(data.Take((int) buffer.Length));
dataRead = buffer.Length;
}
}
var ct = requestInBytes.Count;
I then trip out the header info:
int counter = 0;
counter = requestCommand[0].Length;
counter = counter + requestCommand[1].Length;
counter = counter + requestCommand[2].Length;
counter = counter + requestCommand[3].Length;
counter = counter + requestCommand[4].Length;
counter = counter + requestCommand[5].Length;
counter = counter + 6;
Now I extract the image:
var imgBody = new byte[totalBytes.Length- counter];
System.Buffer.BlockCopy(totalBytes, counter, imgBody, 0, imgBody.Length);
byteArray = imgBody;
And send just the image back:
using (IOutputStream output = args.Socket.OutputStream)
{
using (Stream response = output.AsStreamForWrite())
{
MemoryStream stream = new MemoryStream(byteArray);
await response.WriteAsync(byteArray, 0, byteArray.Length);
await response.FlushAsync();
}
}
This is my client code:
StringBuilder sb = new StringBuilder();
foreach (var gallery in Shared.CurrentJobGallery)
{
try
{
sb.Clear();
sb.Append(GeneralTags.ACTION_ADD);
sb.Append(Shared.DLE);
sb.Append("GALLERY");
sb.Append(Shared.DLE);
sb.Append(Shared.CurrentClientId);
sb.Append(Shared.DLE);
sb.Append(gallery.Title);
sb.Append(Shared.DLE);
sb.Append(gallery.Description);
sb.Append(Shared.DLE);
sb.Append(jobRef);
sb.Append(Shared.DLE);
byte[] galleryHdr = Encoding.UTF8.GetBytes(sb.ToString());
byte[] byteArray = new byte[galleryHdr.Length + gallery.ImageData.Length];
Buffer.BlockCopy(galleryHdr, 0, byteArray, 0, galleryHdr.Length);
Buffer.BlockCopy(gallery.ImageData, 0, byteArray, galleryHdr.Length, gallery.ImageData.Length);
List<byte> requestInBytes2 = new List<byte>();
System.Diagnostics.Debug.WriteLine("SENT: " + gallery.ImageData.Length.ToString());
using (TcpClient clientSocket = new TcpClient())
{
await clientSocket.ConnectAsync(GeneralTags.RASPBERRY_PI_IP_ADDRESS, GeneralTags.RASPBERRY_PI_PORT);
using (NetworkStream serverStream = clientSocket.GetStream())
{
List<byte> requestInBytes = new List<byte>();
serverStream.Write(byteArray, 0, byteArray.Length);
serverStream.Flush();
int i;
Byte[] bytes = new Byte[1024];
do
{
i = serverStream.Read(bytes, 0, bytes.Length);
byte[] receivedBuffer = new byte[i];
Array.Copy(bytes, receivedBuffer, i);
requestInBytes2.AddRange(receivedBuffer);
} while (serverStream.DataAvailable);
}
}
using (MemoryStream ms = new MemoryStream())
{
System.Diagnostics.Debug.WriteLine("BACK: " + requestInBytes2.Count.ToString());
ms.Write(requestInBytes2.ToArray(), 0, requestInBytes2.ToArray().Length);
Shared.ViewImage(Image.FromStream(ms, true));
}
}
catch (Exception ex)
{
System.Diagnostics.Debug.WriteLine(ex.ToString());
}
}

Your problem is that TCP sockets are based around streams, not packets. It's true that "on the wire" everything is a packet, but when you're using TCP, you have no control over how the data is split up into packets or is reassembled into a stream.
In particular, this line of code is incorrect:
await input.ReadAsync(buffer, BufferSize, InputStreamOptions.Partial);
According to the docs, you must use the buffer returned from ReadAsync. Also note that this buffer may be a partial image, and it's up to your code to detect that situation, read more if necessary, and append those blocks together. Also, the buffer may contain part of one image and part of the next image; it's also up to your code to detect that and handle it correctly.
For this reason, most TCP applications use some form of message framing (described in more detail on my blog). Note that getting this right is surprisingly hard.
I strongly recommend that you use SignalR instead of raw TCP sockets. SignalR handles message framing for you, and it is capable of self-hosting (i.e., it does not require ASP.NET).

NetworkStream BeginRead only reads once?

I have some pretty simple code that reads lines from a network stream that it connects to. In the code example only one line is every read and it doesn't carry on getting more from the server.
What is wrong?
byte[] readBuffer = new byte[1024];
byte[] tempBuff = new byte[1024];
int tempBuffSize = 0;
private void btnConnect_Click(object sender, EventArgs e)
{
TcpClient tcpClient = new TcpClient("192.168.1.151", 5505);
NetworkStream stream = tcpClient.GetStream();
stream.BeginRead(readBuffer, 0, 1024, readHandler, tcpClient);
}
void readHandler(IAsyncResult result)
{
TcpClient tcpClient = (TcpClient)result.AsyncState;
int dataLen = tcpClient.GetStream().EndRead(result);
int currStart = 0;
int currEnd = -1;
for (int i = 0; i < dataLen; i++)
{
if (readBuffer[i] == '\r' && i < (readBuffer.Length - 1) &&
readBuffer[i + 1] == '\n')
{
// Set the end of the data
currEnd = i - 1;
// If we have left overs from previous runs:
if (tempBuffSize != 0)
{
byte[] joinedData = new byte[tempBuffSize + (currEnd - currStart + 1)];
Array.Copy(tempBuff, 0, joinedData, 0, tempBuffSize);
Array.Copy(readBuffer, currStart, joinedData, tempBuffSize, (currEnd - currStart + 1));
System.Text.Encoding enc = System.Text.Encoding.ASCII;
string myString = enc.GetString(joinedData);
System.Diagnostics.Debug.Write(myString);
tempBuffSize = 0;
}
else
{
System.Text.Encoding enc = System.Text.Encoding.ASCII;
string myString = enc.GetString(readBuffer);
System.Diagnostics.Debug.Write(myString);
// HandleData(readBuffer, currStart, currEnd);
}
// Set the new start - after our delimiter
currStart = i + 2;
}
}
// See if we still have any leftovers
if (currStart < dataLen)
{
Array.Copy(readBuffer, currStart, tempBuff, 0, dataLen - currStart);
tempBuffSize = dataLen - currStart;
}
}

Why do you expect it to read the whole information in the first place? I am not an expert but it seems to me that neither the synchronous nor the asynchronous methods guarantee reading all the data (whatever that means because as long as a socket is open more data can arrive). After the code in your EndRead method you should call Read or BeginRead again if you expect more data. You should know if more data is expected based on the protocol you've established with the client.

I have faced similar issues when i developed tcp apps for an embedded device. In my case the problem was the device was giving out data in a delayed time and hence before the rest of the data can come in the control moves to the next line in the program fetching only the initial data from the server. I got around this by introducing a delay.
Just after the line where you read data from the server introduce a delay and for that reason it would be better to run this on a separate thread
thread.sleep(3000)
This should be your problem most probably.

maybe your stream object got disposed when it got out of scope, before the readHandler could be called again. try promoting tcpClient and stream to class scope instead of method scope, or move the reading to a separate thread that exits when the operation has finished.

Frustrating TCP Serialization Exception: Binary stream '0' does not contain a valid BinaryHeader

I posted a question on how to send large objects over TCP and it seems like the primary issue is solved, but now frequently I get another exception:
Binary stream '0' does not contain a
valid BinaryHeader. Possible causes
are invalid stream or object version
change between serialization and
deserialization.
The issue is still in my Receive method:
public Message Receive()
{
if (_tcpClient == null || !_tcpClient.Connected)
{
throw new TransportException("Client Not Connected");
}
// buffers
byte[] msgBuffer;
byte[] sizeBuffer = new byte[sizeof(int)];
// bites read
int readSize = 0;
// message size
int size = 0;
MemoryStream memStream = new MemoryStream();
NetworkStream netStream = _tcpClient.GetStream();
BinaryFormatter formatter = new BinaryFormatter();
try
{
// Read the message length
netStream.Read(sizeBuffer, 0, sizeof(int));
// Extract the message length
size = BitConverter.ToInt32(sizeBuffer, 0);
msgBuffer = new byte[size];
// Fill up the message msgBuffer
do
{
// Clear the buffer
Array.Clear(msgBuffer, 0, size);
// Read the message
readSize += netStream.Read(msgBuffer, 0, _tcpClient.ReceiveBufferSize);
// Write the msgBuffer to the memory streamvb
memStream.Write(msgBuffer, 0, readSize);
} while (readSize < size);
// Reset the memory stream position
memStream.Position = 0;
// Deserialize the message
return (Message)formatter.Deserialize(memStream); // <-- Exception here
}
catch (System.Exception e)
{
if (_tcpClient == null || !_tcpClient.Connected)
{
throw new TransportException("Client Not Connected");
}
else
{
throw e;
}
}
}
The rest of the code relevant to this example can be found in my original question.
Does anybody know what is causing this exception and how I can avoid it?
Update
Changed the Read to read a maximum of _tcpClient.ReceiveBufferSize bytes at a time, rather than trying to read the full message size (which can be larger than the buffer size) and while the frequency of the Exception decreased slightly, it's still occurring quite often.

Let me suggest you a slight simplification of your code:
public Message Receive()
{
try
{
if (_tcpClient == null || !_tcpClient.Connected)
{
throw new TransportException("Client Not Connected");
}
using (var stream = _tcpClient.GetStream())
using (var reader = new BinaryReader(stream))
{
int size = reader.ReadInt32();
byte[] buffer = reader.ReadBytes(size);
using (var memStream = new MemoryStream(buffer))
{
var formatter = new BinaryFormatter();
return (Message)formatter.Deserialize(memStream);
}
}
}
catch (System.Exception e)
{
if (_tcpClient == null || !_tcpClient.Connected)
{
throw new TransportException("Client Not Connected");
}
throw e;
}
}
Also if you are doing this for fun and/or education purposes then it's ok, but in a real project you should probably consider WCF in order to transmit objects over the wire.

WCF not so good in client-server. Th polling duplex is quite raw technology.