I love electronic music and I am interested in how it all ticks.
I've found lots of helpful questions on Stack Overflow on libraries that can be used to play with audio, filters etc. But what I am really curious about is what is actually hapening: how is the data being passed between effects and oscillators? I have done research into the mathematical side of dsp and I've got that end of the problem sussed but I am unsure what buffering system to use etc. The final goal is to have a simple object heirarchy of effects and oscillators that pass the data between each other (maybe using multithreading if I don't end up pulling out all my hair trying to implement it). It's not going to be the next Propellerhead Reason but I am interested in how it all works and this is more of an exercise than something that will yeild an end product.
At the moment I use .net and C# and I have recently learnt F# (which may or may not lead to some interesting ways of handling the data) but if these are not suitable for the job I can learn another system if necessary.
The question is: what is the best way to get the large amounts of signal data through the program using buffers? For instance would I be better off using a Queue, Array,Linked List etc? Should I make the samples immutable and create a new set of data each time I apply an effect to the system or just edit the values in the buffer? Shoud I have a dispatcher/thread pool style object that organises passing data or should the effect functions pass data directly between each other?
Thanks.
EDIT: another related question is how would I then use the windows API to play this array? I don't really want to use DirectShow because Microsoft has pretty much left it to die now
EDIT2: thanks for all the answers. After looking at all the technologies I will either use XNA 4(I spent a while trawling the internet and found this site which explains how to do it) or NAudio to output the music... not sure which one yet, depends on how advanced the system ends up being. When C# 5.0 comes out I will use its async capabilities to create an effects architecture on top of that. I've pretty much used everybody's answer equally so now I have a conundrum of who to give the bounty to...
Have you looked at VST.NET (http://vstnet.codeplex.com/)? It's a library to write VST using C# and it has some examples. You can also consider writing a VST, so that your code can be used from any host application (but even if you don't want, looking at their code can be useful).
Signal data is usually big and requires a lot of processing. Do not use a linked list! Most libraries I know simply use an array to put all the audio data (after all, that's what the sound card expect).
From a VST.NET sample:
public override void Process(VstAudioBuffer[] inChannels, VstAudioBuffer[] outChannels)
{
VstAudioBuffer audioChannel = outChannels[0];
for (int n = 0; n < audioChannel.SampleCount; n++)
{
audioChannel[n] = Delay.ProcessSample(inChannels[0][n]);
}
}
The audioChannel is a wrapper around an unmanaged float* buffer.
You probably store your samples in an immutable array. Then, when you want to play them, you copy the data in the output buffer (change the frequency if you want) and perform effects in this buffer. Note you can use several output buffers (or channels) and sum them at the end.
Edit
I know two low-level ways to play your array: DirectSound and WaveOut from Windows API. C# Example using DirectSound. C# example with WaveOut. However, you might prefer use an external higher-level library, like NAudio. NAudio is convenient for .NET audio manipulation - see this blog post for sending a sine wave to the audio card. You can see they are also using an array of float, which is what I recommend (if you do your computations using bytes, you'll end up with a lot of aliasing in the sound).
F# is probably a good choice here, as it's well fitted to manipulate functions. Functions are probably good building blocks for signal creation and processing.
F# is also good at manipulating collections in general, and arrays in particular, thanks to the higher-order functions in the Array module.
These qualities make F# popular in the finance sector and are also useful for signal processing, I would guess.
Visual F# 2010 for Technical Computing has a section dedicated to Fourier Transform, which could be relevant to what you want to do. I guess there is plenty of free information about the transform on the net, though.
Finally, to play samples, you can use XNA. I think the latest version of the API (4.0) also allows recording, but I have never used that. There is a famous music editing app for the Xbox called ezmuse+ Hamst3r Edition that uses XNA, so it's definitely possible.
With respect to buffering and asynchrony/threading/synchronization issues I suggest you to take a look at the new TPL Data Flow library. With its block primitives, concurrent data structures, data flow networks, async message prcessing, and TPL's Task based abstraction (that can be used with the async/await C# 5 features), it's a very good fit for this type of applications.
I don't know if this is really what you're looking for, but this was one of my personal projects while in college. I didn't truly understand how sound and DSP worked until I implemented it myself. I was trying to get as close to the speaker as possible, so I did it using only libsndfile, to handle the file format intricacies for me.
Basically, my first project was to create a large array of doubles, fill it with a sine wave, then use sf_writef_double() to write that array to a file to create something that I could play, and see the result in a waveform editor.
Next, I added another function in between the sine call, and the write call, to add an effect.
This way you start playing with very low-level oscillators and effects, and you can see the results immediately. Plus, it's very little code to get something like this working.
Personally, I would start with the simplest possible solution you can, then slowly add on. Try just writing out to a file and using your audio player to play it, so you don't have to deal with the audio apis. Just use a single array to start, and modify-in-place. Definitely start off single-threaded. As your project grows, you can start moving to other solutions, like pipes instead of the array, multi-threading it, or working with the audio API.
If you're wanting to create a project you can ship, depending on exactly what it is, you'll probably have to move to more complex libraries, like some real-time audio processing. But the basics you learn by doing the simple way above will definitely help when you get to this point.
Good luck!
I've done quite a bit of real-time DSP, although not with audio. While either of your ideas (immutable buffer) vs (mutable buffer modified in place) could work, what I prefer to do is create a single permanent buffer for each link in the signal path. Most effects don't lend themselves well to modification in place, since each input sample affects multiple output samples. The buffer-for-each-link technique works especially well when you have resampling stages.
Here, when samples arrive, the first buffer is overwritten. Then the first filter reads the new data from its input buffer (the first buffer) and writes to its output (the second buffer). Then it invokes the second stage to read from the second buffer and write into the third.
This pattern completely eliminates dynamic allocation, allows each stage to keep a variable amount of history (since effects need some memory), and is very flexible as far as enabling rearranging the filters in the path.
Alright, I'll have a stab at the bounty as well then :)
I'm actually in a very similar situation. I've been making electronic music for ages, but only over the past couple of years I've started exploring actual audio processing.
You mention that you have researched the maths. I think that's crucial. I'm currently fighting my way through Ken Steiglitz' A Digital Signal Processing Primer - With Applications to Digital Audio and Computer Music. If you don't know your complex numbers and phasors it's going to be very difficult.
I'm a Linux guy so I've started writing LADSPA plugins in C. I think it's good to start at that basic level, to really understand what's going on. If I was on Windows I'd download the VST SDK from Steinberg and write a quick proof of concept plugin that just adds noise or whatever.
Another benefit of choosing a framework like VST or LADSPA is that you can immediately use your plugins in your normal audio suite. The satisfaction of applying your first home-built plugin to an audio track is unbeatable. Plus, you will be able to share your plugins with other musicians.
There are probably ways to do this in C#/F#, but I would recommend C++ if you plan to write VST plugins, just to avoid any unnecessary overhead. That seems to be the industry standard.
In terms of buffering, I've been using circular buffers (a good article here: http://www.dspguide.com/ch28/2.htm). A good exercise is to implement a finite response filter (what Steiglitz refers to as a feedforward filter) - these rely on buffering and are quite fun to play around with.
I've got a repo on Github with a few very basic LADSPA plugins. The architectural difference aside, they could potentially be useful for someone writing VST plugins as well. https://github.com/andreasjansson/my_ladspa_plugins
Another good source of example code is the CSound project. There's tonnes of DSP code in there, and the software is aimed primarily at musicians.
Start with reading this and this.
This will give you idea on WHAT you have to do.
Then, learn DirectShow architecture - and learn HOW not to do it, but try to create your simplified version of it.
You could have a look at BYOND. It is an environment for programmatic audio / midi instrument and effect creation in C#. It is available as standalone and as VST instru and effect.
FULL DISCLOSURE I am the developer of BYOND.
Related
Introduction
So, let me introduce a problem. Currently, I'm writing a program in C# that has a lot of computations in it (more precisely it's a neural network lib) and by far I've used standard arrays to store matrixes, but I thought of it's better to create a 2d, 3d matrix class to encapsulate all matrix operations I need and then clean loops in my code.
As you may know, it's pretty easy to accomplish with basic operators overloading, but I came around another problem, it would be slower than regular for loops over arrays, as in the case you have a big equation, intermediate classes which are produced by overloading of operators may cause overhead. I googled it and found the article that turned out to be very useful for me. In short, the writer uses additional classes to first create an equation tree, second compile it in a C# method with the use of MSIL (Microsoft Intermediate Language) solves the equation at once.
But then I thought of the possibility of running matrix calculations on my GPU, as it will be even faster. I came around a NuGet package Cloo that uses OpenCL and a wrapper for it (I'd like it to work on any video card not only NVidia with its CUDA) to run C code on your GPU, but it as I've just said it uses C code that has to be written as a string.
Question
Finally, my question. Is it a good idea to generate the C code string dynamically, from the equation tree, to calculate my optimized equations on a GPU or there are other ways to accomplish that.
First of all, right answer for your question - implement both ways and write the benchmark. Because we don't know, what GPU is used, what CPU is used, what matrix size, etc.
Theoretically, GPU should be faster if you are able to use SIMD/SIMT approach without branches (e.g. without ifs). So if you can write planar code which operates internal arrays, then GPU (event embedded) will work faster. However, the main word here is theoretically.
Practically:
.Net-based (and JVM-based) code is much simpler to support.
OpenCL code works different for NVidia/AMD/Intel GPUs (because sometimes you can store a lot of data in the local memory, sometimes you couldn't; sometimes you can rely on fast GDDR6, sometimes videocard (for example - embedded) just shares computer RAM).
Some GPU memory profiling requires video freeze (to mitigate fluctuations). And you will have many other interesting items during the GPU development.
However to help you:
Try Tensorflow Matrix multiplication first. It has .Net bindings and it can do mathematics operations on both CPU and GPU.
Compare .Net code with native, for example - Rust/Kotlin Native/C++. Probably you can just move all computations into the native part (all these options are much simpler than OpenCL coding and supporting).
From my prospective (no proofs here, sorry) it is much simpler to write code on multiple languages than generate code at language X from language Y.
Here is an example, I'll try to make it as simple as possible.
I have 2 .wav/.mp3 sound file "A" and "B".
"A" sounds like this: "asdfasdafasfsaf DATA sdafsfdafsa".
"B" sounds like this: "DATA".
Now my question is, how to check if "A" contains "B"?
I'm using Visual Studio 2015 and coding in C#. Is it even possible without a 3rd party sofware?
Thanks in advance.
This is not a trivial matter however:
If both wav files have the same sample frequency, you're good to go. If not, you have to resample to correspond (either with a tool or programmatically)
Figure out where the actual data starts for both files (this can vary greatly, read about wav header sizes and how to detect them)
Read data for both files into byte arrays
The above is pretty much step 0.
Your "sample" that you'll be looking for, has a length. That length will be a sliding window over your target byte array.
If you're content to search for the exact same sample (some part of A sounds EXACTLY like B) then you can do a dumb compare of your sliding window content to your reference sample (byte compare). If, however, you want to look for your sample in a mixdown target wav (mixed in with other sounds or layers so some part of A has B in it but it's just one of x things you hear at the time indices where B occurs) then from what I understand you're heading for Fast and/or Discrete Fourier Transforms, Hidden Markov Models or any number of algorithms for pattern recognition. There are undoubtedly countless people here who are way smarter than me who can actually say sensible things when it comes to math. Just... not me.
Up until the math part, it's pretty straightforward but the real magic starts when you implement the algorithm(s) you settle upon. You can, of course, take this as far as you want, depending on your requirements and your math background / skillset / patience / ...
A small side note: "is this possible without 3rd party software" is rarely a really pertinent question. 3rd party software is not some black magic, it's just software written by people like you and me. The difference is, that it's sometimes written by very smart people (often plural) and often developed and tested and perfected over an extended period of time so if the question is "can it be done" then almost always "yes". If the question is "can I reproduce a complete, complex and high-performance library by myself in a small amount of time" ...well... maybe I guess.
Real-time is not necessarily required, however I am creating a game for my final year project and I wish to use the power of audio to create dynamic levels based solely on a music track that is playing. I aim to create this game for the PS Vita using playstation mobile and C#, but if i want i can switch to C++ and PSP.
I can use a WAV file, and hopefully extract the amplitude of the waveform, as well as calculating other characteristics like average frequency and approximate BPM from this data to create a level.
I have no qualms about trying to work with this raw data, I just want to know a way I can actually GET that information first. If i can extract the samples and assertain different characteristics of these samples, I can store them and work out changes in loudness, pitch and more to create notes etc.
I am using C#, but if at all possible i can either use p/invoke or switch my project to another device that uses C++ instead of C#.
I'm panicking a bit here, cos I really am a bit stumped.
Many thanks guys.
Unfortunately i don't think you'll be able to use C# to do this - AFAIK, there is no JIT compiler for it. I remember reading about something for Mono, which would make it available to use with C#, but i'm not sure right now.
That said - i would go with c++. If you go that way, you can make use of a vast amount of audio analysis libraries, like CLAM (http://clam-project.org/).
Don't panic (imagine big, friendly letters.) Envision the necessary parts for the project step by step, tackle one by one, and you'll be done in no time. =)
The problem you describe here is one of music/audio feature extraction and a substantial body of academic work exists that you can draw on. Another useful term of art with which to search is Music Information Retrieval (MIR).
The list of 'features' that researchers have attempted to retrieve from recordings is large and varied, from deterministic things such as pitch and key through emotional characteristics, such as 'energy'.
Most of these turn out to be more difficult than you might imagine, and typically only about 60-70% accurate - although for your requirements, this is probably adequate.
A good entry point might be download Sonic Visualiser, for which a large number of feature extraction plug-ins exist, and are open-source. You'll at least get a feel for what's possible.
Update: Another useful term of art is Onset detection - this is typically used to describe beat detection algorithms.
Aubio is a C/C++ library that does pitch tracking, onset detection and bpm tracking, among other things.
As for "extracting the amplitude of the waveform", the waveform is amplitude, i.e., you could just pick the audio sample with the greatest absolute value every n samples and use that value to do the "amplitude" part of the visualization.
Here's some code that might help you get started reading WAVE data in C#.
Here's some information about writing a C# wrapper for the FFTW library.
Hi I need some help on getting started with creating my first algorithm; I want to create a NN/Genetic Algorithm for use as an Intrusion detection system.
But I’m struggling with some points (never written an algorithm before.)
I want to develop in C# would it be possible as a console app? If so, as a precursor how big would the programme roughly be, at its most simplistic form. Is it even possible in c#?
How to connect the program to read in data from the network? Also how packets can be converted to readable data for the algorithm.
How to get the programme to write rules for snort or some other form of firewall and block what the programme deems as a potential threat. (i.e it spots a threat from No.2 then it writes a rule into the snort rules page blocking that specific traffic)
How to track the data. (what its blocked what its observing how it came to that conclusion)
Where to place it on the network? (can the programme connect to other algorithms and share data on the same network, would that be beneficial)
If anyone can help start me off in the right direction or explain what other alternatives there are like fuzzy logic etc and why is it deemed as a black box?
Yes, a console app, and C#, can be used to create a Neural Network. Of course, if you want more visual aspects to the UI, you'll want to use WinForms/WPF/Silverlight etc.. It's impossible to tell how big the program will be as there's not enough information on what you want to do. Also, the size shouldn't really be a problem as long as it's efficient.
I assume this is some sort of final year project? What type of Neural Network are you using? You should read some academic papers /whitepapers on using NN with intrusion detection to get an idea. For example, this PDF has some information that might help.
You should take this one step at a time. Creating a Neural Network is separate from creating a new rule in Snort. Work on one topic at a time otherwise you'll just get overwhelmed. Considering the hard part will most likely be the NN, you should focus on that first.
It's unlikely anyone's going to go through each step with you as it's quite a large project. Show what you've done and explain where you need help.
My core realization when I started learning about neural networks is that they are just function approximators. I think that's a crucial thing to keep in mind. Whether you're using genetic algorithms or neural nets (or combining them as mentioned by #Ben Voigt, even though neural networks are typically associated with other training techniques) - what you get in the end is a function where you put in a number of real values and get out a single value.
Keeping this in mind, you can design your program and just think of the network as a black box providing those predictions, on the testing part. During training, think of another black box where you put in pairs of input and output pairs and assume it's gonna get better the more pairs you show to it.
Maybe you find this trivial, but with all the theory and mystic behaviour that's associated with this type of algorithms, I found it reassuring (though a bit disappointing ;) to reduce them to those kinds of boxes.
Is there a SDK that can be used in managed code to shred files securely?
EDIT: This is the only link i could find in google that helps me
EDIT: Either SDK or some kind of COM based component.
This code from codeproject may be a good starting point.
Eraser has been around for years, you could call out to it by using System.Diagnostics.Process, or at least review the algorithm there.
Take a look at Windows.WinAny.Helper at the CodePlex. It has SecureDelete extension which allows you to shredd files with different algorithms like Gutmann, DoD-7, DoD-3, Random or Quick.
Technology has changed in the past few years so when I happened to see this answer (why wasn't an answer accepted again?) I wanted to provide an update for others with similar questions.
Please note that shredding is very much filesystem and media dependent. Attempting to "shred" a file on a log based filesystem or a filesystem stored on smart (write leveling) flash isn't going to get you very far. You would have to, at a minimum, write enough data to complete fill the device to hope that the old data might be overwritten one time.
More likely you would have to write several smaller files and when you get FS full, delete one and then keep writing a new one, to ensure that all reserved space has been overwritten as well. Then you will probably be fairly safe. Probably.
I say probably because the storage media/FS could decide that a block was failing (or used too much relatively) and map it away substituting some other part of the disk instead. This is a per-block thing of course, so any much larger file is unlikely to be reconstructed.