Develop Reference

Calling Functions on Unity-Application embedded in Winforms-Application [duplicate]

This question already has answers here:
Send message from one program to another in Unity
(2 answers)
Embed Unity3D app inside WPF application
(1 answer)
Closed 5 years ago.
I am currently developing a simple prototype for an editor. The editor will use WinForms (or WPF, if possible) to provide the main user interface and will also embed a Unity 2017 standalone application to visualise data and provide additional control elements (e.g. zoom, rotate, scroll, ...).
Thanks to this nice post below, getting an embedded Unity application to work within a WinForms-application was shockingly easy.
https://forum.unity.com/threads/unity-3d-within-windows-application-enviroment.236213/
Also, there is a simple example application, which you may access here:
Example.zip
Unfortunately, neither the example, nor any posts I could find answered a very basic question: how do you pass data (or call methods) from your WinForms-application in your embedded Unity-application (and vice versa)?
Is it possible for your WinForms-application to simply call MonoBehaviour-scripts or static methods in your Unity-application? If so, how? If not, what would be a good workaround? And how could the Unity-to-WinForms communication work in return?
Update:
Used the duplicate-pages mentioned by Programmer (link) to implement a solution, which uses named pipes for communication between the WinForms- and Unity-application.
Both applications use BackgroundWorkers, the WinForms-application acts as server (since it is started first and needs an active connection-listener, before the client is started), while the embedded Unity-application acts as client.
Unfortunately, the Unity-application throws a NotImplementedException, stating "ACL is not supported in Mono" when creating the NamedPipeClientStream (tested with Unity 2017.3 and Net 2.0 (not the Net 2.0 subset)). This exception has already been reported in some comments in the post mentioned above, but its unclear, if it has been solved. The proposed solutions of "make sure, that the server is up and running before the client tries to connect" and "start it in admin mode" have been tried, but failed so far.
Solution:
After some more testing, it became clear, that the "ACL is not supported in Mono" exception was caused the TokenImpersonationLevel-parameter used when creating the NamedPipeClientStream-instance. Changing it to TokenImpersonationLevel.None solved the issue.
Here the code used by the WinForms-application, which acts as a named pipe server. Make sure, this script is executed, BEFORE the Unity-application client tries to connect! Also, make sure, that you have build and released the Unity-application before you start the server. Place the Unity executable of your Unity-application in the WinForms-applications folder and name it "Child.exe".
using System;
using System.ComponentModel;
using System.Runtime.InteropServices;
using System.Threading;
using System.Windows.Forms;
using System.Diagnostics;
using System.IO.Pipes;
namespace Container
{
public partial class MainForm : Form
{
[DllImport("User32.dll")]
static extern bool MoveWindow(IntPtr handle, int x, int y, int width, int height, bool redraw);
internal delegate int WindowEnumProc(IntPtr hwnd, IntPtr lparam);
[DllImport("user32.dll")]
internal static extern bool EnumChildWindows(IntPtr hwnd, WindowEnumProc func, IntPtr lParam);
[DllImport("user32.dll")]
static extern int SendMessage(IntPtr hWnd, int msg, IntPtr wParam, IntPtr lParam);
/// <summary>
/// A Delegate for the Update Log Method.
/// </summary>
/// <param name="text">The Text to log.</param>
private delegate void UpdateLogCallback(string text);
/// <summary>
/// The Unity Application Process.
/// </summary>
private Process process;
/// <summary>
/// The Unity Application Window Handle.
/// </summary>
private IntPtr unityHWND = IntPtr.Zero;
private const int WM_ACTIVATE = 0x0006;
private readonly IntPtr WA_ACTIVE = new IntPtr(1);
private readonly IntPtr WA_INACTIVE = new IntPtr(0);
/// <summary>
/// The Background Worker, which will send and receive Data.
/// </summary>
private BackgroundWorker backgroundWorker;
/// <summary>
/// A Named Pipe Stream, acting as the Server for Communication between this Application and the Unity Application.
/// </summary>
private NamedPipeServerStream namedPipeServerStream;
public MainForm()
{
InitializeComponent();
try
{
//Create Server Instance
namedPipeServerStream = new NamedPipeServerStream("NamedPipeExample", PipeDirection.InOut, 1);
//Start Background Worker
backgroundWorker = new BackgroundWorker();
backgroundWorker.DoWork += new DoWorkEventHandler(backgroundWorker_DoWork);
backgroundWorker.WorkerReportsProgress = true;
backgroundWorker.RunWorkerAsync();
//Start embedded Unity Application
process = new Process();
process.StartInfo.FileName = Application.StartupPath + "\\Child.exe";
process.StartInfo.Arguments = "-parentHWND " + splitContainer.Panel1.Handle.ToInt32() + " " + Environment.CommandLine;
process.StartInfo.UseShellExecute = true;
process.StartInfo.CreateNoWindow = true;
process.Start();
process.WaitForInputIdle();
//Embed Unity Application into this Application
EnumChildWindows(splitContainer.Panel1.Handle, WindowEnum, IntPtr.Zero);
//Wait for a Client to connect
namedPipeServerStream.WaitForConnection();
}
catch (Exception ex)
{
throw ex;
}
}
/// <summary>
/// Activates the Unity Window.
/// </summary>
private void ActivateUnityWindow()
{
SendMessage(unityHWND, WM_ACTIVATE, WA_ACTIVE, IntPtr.Zero);
}
/// <summary>
/// Deactivates the Unity Window.
/// </summary>
private void DeactivateUnityWindow()
{
SendMessage(unityHWND, WM_ACTIVATE, WA_INACTIVE, IntPtr.Zero);
}
private int WindowEnum(IntPtr hwnd, IntPtr lparam)
{
unityHWND = hwnd;
ActivateUnityWindow();
return 0;
}
private void panel1_Resize(object sender, EventArgs e)
{
MoveWindow(unityHWND, 0, 0, splitContainer.Panel1.Width, splitContainer.Panel1.Height, true);
ActivateUnityWindow();
}
/// <summary>
/// Called, when this Application is closed. Tries to close the Unity Application and the Named Pipe as well.
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
private void Form1_FormClosed(object sender, FormClosedEventArgs e)
{
try
{
//Close Connection
namedPipeServerStream.Close();
//Kill the Unity Application
process.CloseMainWindow();
Thread.Sleep(1000);
while (process.HasExited == false)
{
process.Kill();
}
}
catch (Exception ex)
{
throw ex;
}
}
private void MainForm_Activated(object sender, EventArgs e)
{
ActivateUnityWindow();
}
private void MainForm_Deactivate(object sender, EventArgs e)
{
DeactivateUnityWindow();
}
/// <summary>
/// A simple Background Worker, which sends Data to the Client via a Named Pipe and receives a Response afterwards.
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
private void backgroundWorker_DoWork(object sender, DoWorkEventArgs e)
{
//Init
UpdateLogCallback updateLog = new UpdateLogCallback(UpdateLog);
string dataFromClient = null;
try
{
//Don't pass until a Connection has been established
while (namedPipeServerStream.IsConnected == false)
{
Thread.Sleep(100);
}
//Created stream for reading and writing
StreamString serverStream = new StreamString(namedPipeServerStream);
//Send to Client and receive Response (pause using Thread.Sleep for demonstration Purposes)
Invoke(updateLog, new object[] { "Send Data to Client: " + serverStream.WriteString("A Message from Server.") + " Bytes." });
Thread.Sleep(1000);
dataFromClient = serverStream.ReadString();
Invoke(updateLog, new object[] { "Received Data from Server: " + dataFromClient });
Thread.Sleep(1000);
Invoke(updateLog, new object[] { "Send Data to Client: " + serverStream.WriteString("A small Message from Server.") + " Bytes." });
Thread.Sleep(1000);
dataFromClient = serverStream.ReadString();
Invoke(updateLog, new object[] { "Received Data from Server: " + dataFromClient });
Thread.Sleep(1000);
Invoke(updateLog, new object[] { "Send Data to Client: " + serverStream.WriteString("Another Message from Server.") + " Bytes." });
Thread.Sleep(1000);
dataFromClient = serverStream.ReadString();
Invoke(updateLog, new object[] { "Received Data from Server: " + dataFromClient });
Thread.Sleep(1000);
Invoke(updateLog, new object[] { "Send Data to Client: " + serverStream.WriteString("The final Message from Server.") + " Bytes." });
Thread.Sleep(1000);
dataFromClient = serverStream.ReadString();
Invoke(updateLog, new object[] { "Received Data from Server: " + dataFromClient });
}
catch(Exception ex)
{
//Handle usual Communication Exceptions here - just logging here for demonstration and debugging Purposes
Invoke(updateLog, new object[] { ex.Message });
}
}
/// <summary>
/// A simple Method, which writes Text into a Console / Log. Will be invoked by the Background Worker, since WinForms are not Thread-safe and will crash, if accessed directly by a non-main-Thread.
/// </summary>
/// <param name="text">The Text to log.</param>
private void UpdateLog(string text)
{
lock (richTextBox_Console)
{
Console.WriteLine(text);
richTextBox_Console.AppendText(Environment.NewLine + text);
}
}
}
}
Attach this code to a GameObject within your Unity-application. Also make sure to reference a GameObject with a TextMeshProUGUI-component (TextMeshPro-Asset, you'll find it in your Asset Store) to the 'textObject'-member, so the application doesn't crash and you can see some debug information.
Also (as stated above) make sure you build and release your Unity-application, name it "Child.exe" and put it in the same folder as your WinForms-application.
using UnityEngine;
using System.Collections;
using UnityEngine.UI;
using System;
using System.IO.Pipes;
using System.Security.Principal;
using Assets;
using System.ComponentModel;
using TMPro;
/// <summary>
/// A simple Example Project, which demonstrates Communication between WinForms-Applications and embedded Unity Engine Applications via Named Pipes.
///
/// This Code (Unity) is considered as the Client, which will receive Data from the WinForms-Server and send a Response in Return.
/// </summary>
public class SendAndReceive : MonoBehaviour
{
/// <summary>
/// A GameObject with an attached Text-Component, which serves as a simple Console.
/// </summary>
public GameObject textObject;
/// <summary>
/// The Background Worker, which will send and receive Data.
/// </summary>
private BackgroundWorker backgroundWorker;
/// <summary>
/// A Buffer needed to temporarely save Text, which will be shown in the Console.
/// </summary>
private string textBuffer = "";
/// <summary>
/// Use this for initialization.
/// </summary>
void Start ()
{
//Init the Background Worker to send and receive Data
this.backgroundWorker = new BackgroundWorker();
this.backgroundWorker.DoWork += new DoWorkEventHandler(backgroundWorker_DoWork);
this.backgroundWorker.WorkerReportsProgress = true;
this.backgroundWorker.RunWorkerAsync();
}
/// <summary>
/// Update is called once per frame.
/// </summary>
void Update ()
{
//Update the Console for debugging Purposes
lock (textBuffer)
{
if (string.IsNullOrEmpty(textBuffer) == false)
{
textObject.GetComponent<TextMeshProUGUI>().text = textObject.GetComponent<TextMeshProUGUI>().text + Environment.NewLine + textBuffer;
textBuffer = "";
}
}
}
/// <summary>
/// A simple Background Worker, which receives Data from the Server via a Named Pipe and sends a Response afterwards.
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
private void backgroundWorker_DoWork(object sender, DoWorkEventArgs e)
{
try
{
//Init
NamedPipeClientStream client = null;
string dataFromServer = null;
//Create Client Instance
client = new NamedPipeClientStream(".", "NamedPipeExample", PipeDirection.InOut, PipeOptions.None, TokenImpersonationLevel.None);
updateTextBuffer("Initialized Client");
//Connect to Server
client.Connect();
updateTextBuffer("Connected to Server");
//Created stream for Reading and Writing
StreamString clientStream = new StreamString(client);
//Read from Server and send Response (flush in between to clear the Buffer and fix some strange Issues I couldn't really explain, sorry)
dataFromServer = clientStream.ReadString();
updateTextBuffer("Received Data from Server: " + dataFromServer);
client.Flush();
updateTextBuffer("Sent Data back to Server: " + clientStream.WriteString("Some data from client.") + " Bytes.");
dataFromServer = clientStream.ReadString();
updateTextBuffer("Received Data from Server: " + dataFromServer);
client.Flush();
updateTextBuffer("Sent Data back to Server: " + clientStream.WriteString("Some more data from client.") + " Bytes.");
dataFromServer = clientStream.ReadString();
updateTextBuffer("Received Data from Server: " + dataFromServer);
client.Flush();
updateTextBuffer("Sent Data back to Server: " + clientStream.WriteString("A lot of more data from client.") + " Bytes.");
dataFromServer = clientStream.ReadString();
updateTextBuffer("Received Data from Server: " + dataFromServer);
client.Flush();
updateTextBuffer("Sent Data back to Server: " + clientStream.WriteString("Clients final message.") + " Bytes.");
//Close client
client.Close();
updateTextBuffer("Done");
}
catch (Exception ex)
{
//Handle usual Communication Exceptions here - just logging here for demonstration and debugging Purposes
updateTextBuffer(ex.Message + Environment.NewLine + ex.StackTrace.ToString() + Environment.NewLine + "Last Message was: " + textBuffer);
}
}
/// <summary>
/// A Buffer, which allows the Background Worker to save Texts, which may be written into a Log or Console by the Update-Loop
/// </summary>
/// <param name="text">The Text to save.</param>
private void updateTextBuffer(string text)
{
lock (textBuffer)
{
if (string.IsNullOrEmpty(textBuffer))
{
textBuffer = text;
}
else
{
textBuffer = textBuffer + Environment.NewLine + text;
}
}
}
}
Also, both scripts need an additional class, which encapsulates the pipe stream, so sending and receiving text becomes much easier.
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
namespace Assets
{
/// <summary>
/// Simple Wrapper to write / read Data to / from a Named Pipe Stream.
///
/// Code based on:
/// https://stackoverflow.com/questions/43062782/send-message-from-one-program-to-another-in-unity
/// </summary>
public class StreamString
{
private Stream ioStream;
private UnicodeEncoding streamEncoding;
public StreamString(Stream ioStream)
{
this.ioStream = ioStream;
streamEncoding = new UnicodeEncoding();
}
public string ReadString()
{
int len = 0;
len = ioStream.ReadByte() * 256;
len += ioStream.ReadByte();
byte[] inBuffer = new byte[len];
ioStream.Read(inBuffer, 0, len);
return streamEncoding.GetString(inBuffer);
}
public int WriteString(string outString)
{
byte[] outBuffer = streamEncoding.GetBytes(outString);
int len = outBuffer.Length;
if (len > UInt16.MaxValue)
{
len = (int)UInt16.MaxValue;
}
ioStream.WriteByte((byte)(len / 256));
ioStream.WriteByte((byte)(len & 255));
ioStream.Write(outBuffer, 0, len);
ioStream.Flush();
return outBuffer.Length + 2;
}
}
}
If you read the post down to this point: thank you :) I hope it will help you on your journey to a successfull developer!
Final result of my prototype:

If nothing else, you could fall back on basic file I/O to communicate between the two.

JumpList in C# application (recent files)

At the moment I am developing an application and want to add a Windows (7) JumpList. I followed several tutorials and studied documentation, but I can't figure out how to get the job done. In short: I want a recent list of last choosen files. So after closing the app, the user can easily open a recent file with my application. I already implemented some file association mechanism.
Is it possible to share some code/ tutorial how I can solve above problem?
Thank you in advance!
*I already tried the next few projects/ tutorials:
http://www.codeproject.com/Articles/103913/How-to-Create-a-Custom-Jumplist-with-Custom-Events
http://channel9.msdn.com/coding4fun/articles/Windows-7-Jump-Lists
http://csharp-tricks-en.blogspot.nl/2011/10/create-jumplist-using-c.html
*The code of Coding 4 Fun works, but I don't know how to develop a recent file list.

You can check out this article. Instead of showing result in WPF you need to show it in jumplist.
Why dont you try storing the recently opened file names in a database or an xml file and read it to set the jumplist. For eg.
private void ReportUsage()
{
XmlDocument myXml = new XmlDocument();
myXml.Load(historyXml);
string list = historyXml;
jumpList.ClearAllUserTasks();
foreach (XmlElement el in myXml.DocumentElement.ChildNodes)
{
string s = el.GetAttribute("url");
JumpListLink jll = new JumpListLink(Assembly.GetEntryAssembly().Location, s);
jll.IconReference = new IconReference(Path.Combine("C:\\Program Files\\ACS Digital Media\\TOC WPF Browser\\Icon1.ico"), 0);
jll.Arguments = el.GetAttribute("url");
jumpList.AddUserTasks(jll);
}
jumpList.Refresh();
}
Or a beginners solution will be retain all the file paths into a Queue of given maximum capacity and adding them at run-time into a menuItem. Sorry I didnt have time to write the whole code.

As it is described in your second article, your application must be registered as a handler for targeted file extension, otherwise the recent category for your Jumplist won't show up. You can find more details about file association registration there.
You can either manually register your application but you will need admin right so it is not recommended, or create a setup project for your application like described in the coding 4 fun article, or you can let the user associate the file extension.
Here is a sample which works for me under Windows Seven without registration, by just right clicking the text file I want to load and choosing "Open With" and browse to my application.
The sample require Windows API Code Pack
public partial class Form1 : Form
{
[STAThread]
static void Main(string[] args)
{
var file = args != null && args.Length > 0 ? args[0] : string.Empty;
Application.EnableVisualStyles();
Application.SetCompatibleTextRenderingDefault(false);
Application.Run(new Form1(file));
}
public Form1()
: this(string.Empty)
{
}
public Form1(string file)
{
InitializeComponent();
Open(file);
}
[DllImport("user32.dll")]
private static extern uint RegisterWindowMessage(string message);
private uint wmTBC;
/// <summary>
/// Registers the window message for notification when the taskbar button is created.
/// </summary>
/// <param name="e">The event args.</param>
protected override void OnLoad(EventArgs e)
{
base.OnLoad(e);
wmTBC = RegisterWindowMessage("TaskbarButtonCreated");
}
/// <summary>
/// Handles the window message for notification of the taskbar button creation.
/// </summary>
/// <param name="m">The window message.</param>
protected override void WndProc(ref Message m)
{
base.WndProc(ref m);
if (m.Msg == wmTBC)
{
OnTaskbarButtonCreated();
}
}
/// <summary>
/// Override this method to recieve notification when the taskbar button is created on Windows 7 machines and above.
/// </summary>
protected void OnTaskbarButtonCreated()
{
if (TaskbarManager.IsPlatformSupported)
{
jumpList = JumpList.CreateJumpList();
jumpList.KnownCategoryToDisplay = JumpListKnownCategoryType.Recent;
jumpList.Refresh();
}
}
JumpList jumpList;
private void openToolStripMenuItem1_Click(object sender, EventArgs e)
{
using (OpenFileDialog ofd = new OpenFileDialog())
{
if (ofd.ShowDialog() == DialogResult.OK)
{
Open(ofd.FileName);
}
}
}
private void Open(string file)
{
try
{
if (!string.IsNullOrEmpty(file) && File.Exists(file))
{
textBox1.Text = File.ReadAllText(file);
if (TaskbarManager.IsPlatformSupported)
{
jumpList.AddToRecent(file);
}
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
/// <summary>
/// Required designer variable.
/// </summary>
private System.ComponentModel.IContainer components = null;
/// <summary>
/// Clean up any resources being used.
/// </summary>
/// <param name="disposing">true if managed resources should be disposed; otherwise, false.</param>
protected override void Dispose(bool disposing)
{
if (disposing && (components != null))
{
components.Dispose();
}
base.Dispose(disposing);
}
#region Windows Form Designer generated code
/// <summary>
/// Required method for Designer support - do not modify
/// the contents of this method with the code editor.
/// </summary>
private void InitializeComponent()
{
this.textBox1 = new System.Windows.Forms.TextBox();
this.menuStrip1 = new System.Windows.Forms.MenuStrip();
this.openToolStripMenuItem = new System.Windows.Forms.ToolStripMenuItem();
this.openToolStripMenuItem1 = new System.Windows.Forms.ToolStripMenuItem();
this.menuStrip1.SuspendLayout();
this.SuspendLayout();
//
// textBox1
//
this.textBox1.Location = new System.Drawing.Point(12, 27);
this.textBox1.Multiline = true;
this.textBox1.Name = "textBox1";
this.textBox1.Size = new System.Drawing.Size(796, 306);
this.textBox1.TabIndex = 0;
//
// menuStrip1
//
this.menuStrip1.Items.AddRange(new System.Windows.Forms.ToolStripItem[] {
this.openToolStripMenuItem});
this.menuStrip1.Location = new System.Drawing.Point(0, 0);
this.menuStrip1.Name = "menuStrip1";
this.menuStrip1.Size = new System.Drawing.Size(820, 24);
this.menuStrip1.TabIndex = 1;
this.menuStrip1.Text = "menuStrip1";
//
// openToolStripMenuItem
//
this.openToolStripMenuItem.DropDownItems.AddRange(new System.Windows.Forms.ToolStripItem[] {
this.openToolStripMenuItem1});
this.openToolStripMenuItem.Name = "openToolStripMenuItem";
this.openToolStripMenuItem.Size = new System.Drawing.Size(37, 20);
this.openToolStripMenuItem.Text = "File";
//
// openToolStripMenuItem1
//
this.openToolStripMenuItem1.Name = "openToolStripMenuItem1";
this.openToolStripMenuItem1.Size = new System.Drawing.Size(152, 22);
this.openToolStripMenuItem1.Text = "Open";
this.openToolStripMenuItem1.Click += new System.EventHandler(this.openToolStripMenuItem1_Click);
//
// Form1
//
this.AutoScaleDimensions = new System.Drawing.SizeF(6F, 13F);
this.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font;
this.ClientSize = new System.Drawing.Size(820, 345);
this.Controls.Add(this.textBox1);
this.Controls.Add(this.menuStrip1);
this.MainMenuStrip = this.menuStrip1;
this.Name = "Form1";
this.Text = "Form1";
this.menuStrip1.ResumeLayout(false);
this.menuStrip1.PerformLayout();
this.ResumeLayout(false);
this.PerformLayout();
}
#endregion
private System.Windows.Forms.TextBox textBox1;
private System.Windows.Forms.MenuStrip menuStrip1;
private System.Windows.Forms.ToolStripMenuItem openToolStripMenuItem;
private System.Windows.Forms.ToolStripMenuItem openToolStripMenuItem1;
}

Naudio record microphone for X amount of seconds

Currently I can record the microphone input using the Naudio .dll with this:
public void recordInput()
{
Console.WriteLine("Now recording...");
waveSource = new WaveIn();
waveSource.WaveFormat = new WaveFormat(16000, 1);
waveSource.DataAvailable += new EventHandler<WaveInEventArgs>(waveSource_DataAvailable);
waveSource.RecordingStopped += new EventHandler<StoppedEventArgs>(waveSource_RecordingStopped);
//string tempFile = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString() + ".wav");
string tempFile = Path.Combine(#"C:\Users\Nick\Desktop", "test.wav");
waveFile = new WaveFileWriter(tempFile, waveSource.WaveFormat);
waveSource.StartRecording();
myTimer.Interval = 5000;
myTimer.Tick += new EventHandler(myTimer_Tick);
myTimer.Start();
}
Currently I used an eventhandler to wait for 5 seconds and then onTick I stop recording. However this is causing problems because I am having trouble waiting for the recording task to finish in the main part of the code where I call:
public string voiceToText()
{
recordInput();
//Somehow wait here until record is done then proceed.
convertToFlac();
return "done";
}
I've tried putting the Naudio in a thread and using a waitOne(); but waveSource.StartRecording(); is put in an eventhandler and thread an expection. I've also tried using a Thread.Sleep(5000) and stop recording after that thread finishes but the audio only records the first 500mS of audio for some reason.
I'm pretty new to c# and don't fully understand threading so any help or seperate approach is welcome.

I know this is an old topic, but here is some code that I made for one of my projects and that records audio for x seconds (it uses NAudio.Lame to convert the wave file to mp3):
public class Recorder
{
/// <summary>
/// Timer used to start/stop recording
/// </summary>
private Timer _timer;
private WaveInEvent _waveSource;
private WaveFileWriter _waveWriter;
private string _filename;
private string _tempFilename;
public event EventHandler RecordingFinished;
/// <summary>
/// Record from the mic
/// </summary>
/// <param name="seconds">Duration in seconds</param>
/// <param name="filename">Output file name</param>
public void RecordAudio(int seconds, string filename)
{
/*if the filename is empty, throw an exception*/
if (string.IsNullOrEmpty(filename))
throw new ArgumentNullException("The file name cannot be empty.");
/*if the recording duration is not > 0, throw an exception*/
if (seconds <= 0)
throw new ArgumentNullException("The recording duration must be a positive integer.");
_filename = filename;
_tempFilename = $"{Path.GetFileNameWithoutExtension(filename)}.wav";
_waveSource = new WaveInEvent
{
WaveFormat = new WaveFormat(44100, 1)
};
_waveSource.DataAvailable += DataAvailable;
_waveSource.RecordingStopped += RecordingStopped;
_waveWriter = new WaveFileWriter(_tempFilename, _waveSource.WaveFormat);
/*Start the timer that will mark the recording end*/
/*We multiply by 1000 because the Timer object works with milliseconds*/
_timer = new Timer(seconds * 1000);
/*if the timer elapses don't reset it, stop it instead*/
_timer.AutoReset = false;
/*Callback that will be executed once the recording duration has elapsed*/
_timer.Elapsed += StopRecording;
/*Start recording the audio*/
_waveSource.StartRecording();
/*Start the timer*/
_timer.Start();
}
/// <summary>
/// Callback that will be executed once the recording duration has elapsed
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
private void StopRecording(object sender, ElapsedEventArgs e)
{
/*Stop the timer*/
_timer?.Stop();
/*Destroy/Dispose of the timer to free memory*/
_timer?.Dispose();
/*Stop the audio recording*/
_waveSource.StopRecording();
}
/// <summary>
/// Callback executed when the recording is stopped
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
private void RecordingStopped(object sender, StoppedEventArgs e)
{
_waveSource.DataAvailable -= DataAvailable;
_waveSource.RecordingStopped -= RecordingStopped;
_waveSource?.Dispose();
_waveWriter?.Dispose();
/*Convert the recorded file to MP3*/
ConvertWaveToMp3(_tempFilename, _filename);
/*Send notification that the recording is complete*/
RecordingFinished?.Invoke(this, null);
}
/// <summary>
/// Callback executed when new data is available
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
private void DataAvailable(object sender, WaveInEventArgs e)
{
if (_waveWriter != null)
{
_waveWriter.Write(e.Buffer, 0, e.BytesRecorded);
_waveWriter.Flush();
}
}
/// <summary>
/// Converts the recorded WAV file to MP3
/// </summary>
private void ConvertWaveToMp3(string source, string destination)
{
using (var waveStream = new WaveFileReader(source))
using(var fileWriter = new LameMP3FileWriter(destination, waveStream.WaveFormat, 128))
{
waveStream.CopyTo(fileWriter);
waveStream.Flush();
}
/*Delete the temporary WAV file*/
File.Delete(source);
}
}
And here is how to use it in your code:
var rec = new Recorder();
/*This line allows us to be notified when the recording is complete and the callback 'OnRecordingFinished' will be executed*/
rec.RecordingFinished += OnRecordingFinished;
rec.RecordAudio(seconds, recPath);
private void OnRecordingFinished(object sender, RecordingContentArgs e)
{
//Put your code here to process the audio file
}

If you aren't running in a Windows Forms or WPF application, then you should use WaveInEvent instead, so that a background thread is set up to handle the callbacks. The default constructor for WaveIn uses Windows messages.

Kinect for Windows Input Stream

I am basically trying to take input as RGB Stream from Kinect for Windows using the same code as given in the SDK (ColorBasics Example).
The code of the Example in SDK is as follow
public partial class MainWindow : Window
{
/// <summary>
/// Active Kinect sensor
/// </summary>
private KinectSensor sensor;
/// <summary>
/// Bitmap that will hold color information
/// </summary>
private WriteableBitmap colorBitmap;
/// <summary>
/// Intermediate storage for the color data received from the camera
/// </summary>
private byte[] colorPixels;
/// <summary>
/// Initializes a new instance of the MainWindow class.
/// </summary>
public MainWindow()
{
//InitializeComponent();
}
/// <summary>
/// Execute startup tasks
/// </summary>
/// <param name="sender">object sending the event</param>
/// <param name="e">event arguments</param>
private void WindowLoaded(object sender, RoutedEventArgs e)
{
// Look through all sensors and start the first connected one.
// This requires that a Kinect is connected at the time of app startup.
// To make your app robust against plug/unplug,
// it is recommended to use KinectSensorChooser provided in Microsoft.Kinect.Toolkit
foreach (var potentialSensor in KinectSensor.KinectSensors)
{
if (potentialSensor.Status == KinectStatus.Connected)
{
this.sensor = potentialSensor;
break;
}
}
if (null != this.sensor)
{
// Turn on the color stream to receive color frames
this.sensor.ColorStream.Enable(ColorImageFormat.RgbResolution640x480Fps30);
// Allocate space to put the pixels we'll receive
this.colorPixels = new byte[this.sensor.ColorStream.FramePixelDataLength];
// This is the bitmap we'll display on-screen
this.colorBitmap = new WriteableBitmap(this.sensor.ColorStream.FrameWidth, this.sensor.ColorStream.FrameHeight, 96.0, 96.0, PixelFormats.Bgr32, null);
// Set the image we display to point to the bitmap where we'll put the image data
this.Image.Source = this.colorBitmap;
// Add an event handler to be called whenever there is new color frame data
this.sensor.ColorFrameReady += this.SensorColorFrameReady;
// Start the sensor!
try
{
this.sensor.Start();
}
catch (IOException)
{
this.sensor = null;
}
}
if (null == this.sensor)
{
this.statusBarText.Text = Properties.Resources.NoKinectReady;
}
}
/// <summary>
/// Execute shutdown tasks
/// </summary>
/// <param name="sender">object sending the event</param>
/// <param name="e">event arguments</param>
private void WindowClosing(object sender, System.ComponentModel.CancelEventArgs e)
{
if (null != this.sensor)
{
this.sensor.Stop();
}
}
/// <summary>
/// Event handler for Kinect sensor's ColorFrameReady event
/// </summary>
/// <param name="sender">object sending the event</param>
/// <param name="e">event arguments</param>
private void SensorColorFrameReady(object sender, ColorImageFrameReadyEventArgs e)
{
using (ColorImageFrame colorFrame = e.OpenColorImageFrame())
{
if (colorFrame != null)
{
// Copy the pixel data from the image to a temporary array
colorFrame.CopyPixelDataTo(this.colorPixels);
// Write the pixel data into our bitmap
this.colorBitmap.WritePixels(
new Int32Rect(0, 0, this.colorBitmap.PixelWidth, this.colorBitmap.PixelHeight),
this.colorPixels,
this.colorBitmap.PixelWidth * sizeof(int),
0);
}
}
}
/// <summary>
/// Handles the user clicking on the screenshot button
/// </summary>
/// <param name="sender">object sending the event</param>
/// <param name="e">event arguments</param>
private void ButtonScreenshotClick(object sender, RoutedEventArgs e)
{
if (null == this.sensor)
{
this.statusBarText.Text = Properties.Resources.ConnectDeviceFirst;
return;
}
// create a png bitmap encoder which knows how to save a .png file
BitmapEncoder encoder = new PngBitmapEncoder();
// create frame from the writable bitmap and add to encoder
encoder.Frames.Add(BitmapFrame.Create(this.colorBitmap));
string time = System.DateTime.Now.ToString("hh'-'mm'-'ss", CultureInfo.CurrentUICulture.DateTimeFormat);
string myPhotos = Environment.GetFolderPath(Environment.SpecialFolder.MyPictures);
string path = Path.Combine(myPhotos, "KinectSnapshot-" + time + ".png");
// write the new file to disk
try
{
using (FileStream fs = new FileStream(path, FileMode.Create))
{
encoder.Save(fs);
}
this.statusBarText.Text = string.Format("{0} {1}", Properties.Resources.ScreenshotWriteSuccess, path);
}
catch (IOException)
{
this.statusBarText.Text = string.Format("{0} {1}", Properties.Resources.ScreenshotWriteFailed, path);
}
}
}
}
And the code in my application is as follows
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Data;
using System.Windows.Documents;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Navigation;
using System.Windows.Shapes;
using Microsoft.Kinect;
namespace VideoKinect
{
/// <summary>
/// Interaction logic for MainWindow.xaml
/// </summary>
public partial class MainWindow : Window
{
private KinectSensor sensor;
private WriteableBitmap colorBitmap;
private byte[] colorPixels;
public MainWindow()
{
//InitializeComponent();
}
private void WindowLoaded(object sender, RoutedEventArgs e)
{
foreach (var potentialSensor in KinectSensor.KinectSensors)
{
if (potentialSensor.Status == KinectStatus.Connected)
{
this.sensor = potentialSensor;
break;
}
}
if (null != this.sensor)
{
// Turn on the color stream to receive color frames
this.sensor.ColorStream.Enable(ColorImageFormat.RgbResolution640x480Fps30);
// Allocate space to put the pixels we'll receive
this.colorPixels = new byte[this.sensor.ColorStream.FramePixelDataLength];
// This is the bitmap we'll display on-screen
this.colorBitmap = new WriteableBitmap(this.sensor.ColorStream.FrameWidth, this.sensor.ColorStream.FrameHeight, 96.0, 96.0, PixelFormats.Bgr32, null);
// Set the image we display to point to the bitmap where we'll put the image data
this.ColorImage.Source = this.colorBitmap;
// Add an event handler to be called whenever there is new color frame data
this.sensor.ColorFrameReady += this.SensorColorFrameReady;
// Start the sensor!
//try
//{
this.sensor.Start();
// }
// catch (IOException)
//{
this.sensor = null;
// }
}
if (null == this.sensor)
{
MessageBox.Show("No Kinect Available");
}
}
private void SensorColorFrameReady(object sender, ColorImageFrameReadyEventArgs e)
{
using (ColorImageFrame colorFrame = e.OpenColorImageFrame())
{
if (colorFrame != null)
{
// Copy the pixel data from the image to a temporary array
colorFrame.CopyPixelDataTo(this.colorPixels);
// Write the pixel data into our bitmap
this.colorBitmap.WritePixels(
new Int32Rect(0, 0, this.colorBitmap.PixelWidth, this.colorBitmap.PixelHeight),
this.colorPixels,
this.colorBitmap.PixelWidth * sizeof(int),
0);
}
}
}
}
}
But I am getting the Error at "IOException" in my code and the Image "ColorImage" is not being detected even though I named the image in my xaml file as the same.

I was able to compile and execute your code above with the expected behaviour - the only change was uncommenting the InitializeComponent(); and adding <Image x:Name="ColorImage"/> to the MainWindow.xaml.
What exactly does the IOException say? Can you double check that you Kinect sensor is connected properly via at least a USB 2.0 bus - 1.1 has insufficient bandwidth? Are you also sure that the sensor has sufficient power as you will need the kinect power supply cable. You could always check to see if the SDK is installed properly, have a look at this post.
With regards to it not finding your ColorImage rebuilding the solution should resolve the issue, assuming there is not a spelling mistake on the x:Name of the Image in XAML.

You messed up when you commented out the try ... catch statement, you left sensor = null; in there, after you started it, meaning you had no sensor. You should have commented that out.
// What it should be:
// try
// {
this.sensor.Start();
// }
// catch (IOException)
// {
// this.sensor = null; <- What it should be
// }
// What it is:
// try
// {
this.sensor.Start();
// }
// catch (IOException)
// {
this.sensor = null; //turns sensor null, then SensorColorFrame
// never gets called, etc.
// }
That line will ultimately destroy any Kinect program you ever write. Hope this helps!

you left out a reference. you need to include:
using System.IO;
at the very top. Hope This helps!

NullReferenceException when Firing an Event [duplicate]

This question already has answers here:
What is a NullReferenceException, and how do I fix it?
(27 answers)
Closed 8 years ago.
The problem is: I get a NullReferenceException but I can't seem to find out why. (I just start with C#) and have read the C# for kids from Microsoft. It explained a lot but this I do not understand.
But I don't get the piece of code which throw the exception to my head.
Code is:
using System;
using System.Collections.Generic;
using System.Linq; using System.Text;
using System.IO.Ports;
using PlugwiseLib.BLL.BC;
using plugwiseLib.BLL.BPC;
using System.Text.RegularExpressions;
using System.Threading;
using System.IO;
using PlugwiseLib.BLL.BPC;
using PlugwiseLib.UTIL;
namespace PlugwiseLib {
public class plugwiseControl
{
private SerialPort port;
private PlugwiseActions currentAction;
public delegate void PlugwiseDataReceivedEvent(object
sender, System.EventArgs e,
List<PlugwiseMessage> data);
public event PlugwiseDataReceivedEvent
DataReceived;
private PlugwiseReader reader;
/// <summary>
/// Constructor for the Plugwise Control class
/// </summary>
/// <param name="serialPort">The serial port name that the plugwise stick takes</param>
public plugwiseControl(string serialPort)
{
try
{
port = new SerialPort(serialPort);
port.DataReceived += new SerialDataReceivedEventHandler(port_DataReceived);
port.BaudRate = 115200;
currentAction = PlugwiseActions.None;
reader = new PlugwiseReader();
}
catch (Exception e)
{
throw new Exception("Could not connect to plug.");
}
}
/// <summary>
/// This is the method that sends a command to the plugwise plugs.
/// </summary>
/// <param name="mac">The mac adress of the plug that needs to perform the action</param>
/// <param name="action">The action that has to be performed</param>
public void Action(string mac,PlugwiseActions action)
{
try
{
string message = "";
switch (action)
{
case PlugwiseActions.On:
currentAction = PlugwiseActions.On;
message = PlugwiseSerialPortFactory.Create(PlugwiseSerialPortRequest.on,mac);
break;
case PlugwiseActions.Off:
currentAction = PlugwiseActions.Off;
message = PlugwiseSerialPortFactory.Create(PlugwiseSerialPortRequest.off, mac);
break;
case PlugwiseActions.Status:
currentAction = PlugwiseActions.Status;
message = PlugwiseSerialPortFactory.Create(PlugwiseSerialPortRequest.status, mac);
break;
case PlugwiseActions.Calibration:
currentAction = PlugwiseActions.Calibration;
message = PlugwiseSerialPortFactory.Create(PlugwiseSerialPortRequest.calibration, mac);
break;
case PlugwiseActions.powerinfo:
currentAction = PlugwiseActions.powerinfo;
message = PlugwiseSerialPortFactory.Create(PlugwiseSerialPortRequest.powerinfo,mac);
break;
case PlugwiseActions.history:
message = "";
break;
}
if (message.Length > 0)
{
port.WriteLine(message);
Thread.Sleep(10);
}
}
catch (Exception e)
{
throw e;
}
}
/// <summary>
/// This is the method that sends a command to the plugwise plugs that retrieves the history power information
/// </summary>
/// <param name="mac">The mac adress of the plug that needs to perform the action</param>
/// <param name="logId">The id of the history message that has to be retrieved</param>
/// <param name="action">The action that has to be performed this MUST be history</param>
public void Action(string mac,int logId,PlugwiseActions action)
{
string message = "";
switch(action)
{
case PlugwiseActions.history:
currentAction = PlugwiseActions.history;
message = PlugwiseSerialPortFactory.Create(PlugwiseSerialPortRequest.history, MessageHelper.ConvertIntToPlugwiseLogHex(logId), mac);
break;
}
if (message.Length > 0)
{
port.WriteLine(message);
Thread.Sleep(10);
}
}
private void port_DataReceived(object sender, SerialDataReceivedEventArgs e)
{// Event for receiving data
string txt = port.ReadExisting();
List<PlugwiseMessage> msg = reader.Read(Regex.Split(txt, "\r\n"));
DataReceived(sender, new System.EventArgs(), msg);
}
/// <summary>
/// This method Opens the connection to the serial port
/// </summary>
public void Open()
{
try
{
if (!port.IsOpen)
{
port.Open();
}
}
catch (System.IO.IOException ex)
{
throw ex;
}
}
/// <summary>
/// This method closes the connection to the serial port
/// </summary>
public void Close()
{
try
{
if (port.IsOpen)
{
port.Close();
}
Thread.Sleep(5);
}
catch (IOException ex)
{
throw ex;
}
}
}
}
I get the exception at this piece (in Visual C# 2008 express)
private void port_DataReceived(object sender, SerialDataReceivedEventArgs e)
{// Event for receiving data
string txt = port.ReadExisting();
List<PlugwiseMessage> msg = reader.Read(Regex.Split(txt, "\r\n"));
DataReceived(sender, new System.EventArgs(), msg);
I checked 'msg' but this is filled with so far i can see valid data.
So I don't know why I get the exception.

Do you have any subscribers for plugwiseControl.DataReceived event? A common way to raise event is
var handler = DataReceived;
if(handler != null) handler(sender, EventArgs.Empty, msg);

I think DataReceived event is not initialized, by the calling method.

In .NET, if you raise an event and there are no registered listeners for that event (nothing has done to your event what you do to your port's DataReceived event in the second line of your try {} block in your constructor), it shows up as null, and raises that exception. In your Watch list screenshot, DataReceived is null, which makes me think this is the problem. So:
if (DataReceived != null)
DataReceived(...arguments here...);