We're going to do some experiments on perceptual thresholds and want to show an image for a very short time. I'm speaking about less then 10 ms (our screen supports 144 Hz, that is a new image every 6.94 ms).
But all our approaches until now failed. We tried it with C#: WinForms were much too slow, WPF was faster, but we were still able to see the image, and even showing a texture with XNA framework did not work for us.
Do you have any suggestions for us? We're allowed to use C++ too, but we prefer using C#, so if your suggestions works with C# we would really much appreciate.
I would highly recommend using the XNA Framework for this. Some might consider it overkill but the fact is it is first of all designed to handle high throughput of frames and secondly has a relatively small learning curve.
I'm not a game developer but started by reading an article, modifying a simple project and creating one from scratch.
This would probably be your best bet in terms of the technologies you mentioned in your question.
UPDATE: Regarding measuring the actual time an image is shown, XNA would again be your closest guess short of dealing with specialized hardware and resorting to low-level programming.
Related
I'm creating a small application that requires a live feed of phonemes to be output as the user speaks into their microphone. In my case, the speed of the recognition output is the number 1 priority, even over accuracy. Using C# is the preference, but if a better speed can be accomplished using a different language and/or library (Like CMUSphinx), I would switch.
Using System.Speech.Recognition, along with DictationGrammar("grammar:dictation#pronunciation"), I've been able to create a simple and effective phoneme recognizer that does output phonemes as you speak into the mic, with generally impressive accuracy (subscribing to the SpeechRecognitionEngine.SpeechHypothesized event allows me to see live output). The problem is, it has a minimum delay of around .5s between the user speaking and the output which is too much to work well with the project. I know that in general this is a fairly high speed, especially considering the good accuracy, but I really need something faster, even if the accuracy takes a big hit. Is there any way to configure a SpeechRecognitionEngine to throw accuracy out the window in order to spew out hypothesis faster? I found some exposed settings using SpeechRecognitionEngine.UpdateRecognizerSetting, but they seem to have little effect on the output for phoneme recognition.
I've also looked into CMUSphinx, a free speech recognition project that looked promising. Sphinx4 was easy to compile and set up a test is Java, but I couldn't figure out how to configure it to live output phonemes, and it's word recognition was relatively slow. Here, I found some notes about phoneme recognition using their other project, pocketsphinx. I was able to also download and compile it, but unable to run any tests successfully. Has anyone use CMUSphinx or Pocketsphinx with phonemes? Is it capable of high, live output speeds? Or perhaps there is even more alternatives? I really am looking for something extremely basic, but fast.
Edit: Was able to get pocketsphinx recognizing phonemes, but it was too slow to use in the project
The following picture show that my WPF lines are not as exact like a similar drawing in a non WPF application. So? Where is the problem? antialiasing? or is this a WPF feature? What am I supposed to do?
the below barchart is a simple Shape.Line.
I am not sure I would say "better line quality". Subpixel rendering is a feature in many many cases - actually in most. Financial charts are a special case - but saying WPF lines are bad in general is really an oversimplification. Especially as it points (as some have pointed out) to the usual lack of having read the documentation and learned the technology, as pixel exact rendering was reintroduced some time ago.
What you run into is the main problem that WPF is device independent and allows arbitrary zooming - so everything happens in it's own coordinate system. Which may not run down to exact a pixel. As I said - generally a feature.
Now there are edge cases (like the financial chart you point out nicely) and for that in .NET 3.0 (ages ago, seriously) pixel snapping was made available.
http://msdn.microsoft.com/en-us/library/aa970908(v=vs.85).aspx
has some explanations. It basically works by means of the SnapsToDevicePixels property.
As a matter of fact this is not the first time this question has arrived, as can be seen at - Rendering sharp lines in WPF.
There are a lot of features in WPF and it is quite mandatory to read the documentation. Especially in financial applications - which are not really static, if you deal with real time data. I suggest you do so - you can gain tremendous performance benefits by for example pre-caching parts of the UI in the graphics card memory, which is a simple other trick, easy to do.
I program in Java doing a lot of web related stuff but I've been toying with the idea of creating a very simple DAW in some language. I considered C# but it doesn't seem to support Direct X anymore (Though there are some libraries that work with differing degrees of success). I was curious if anyone out there had an opinion on playing a lot of multi-channel sounds through Java. I would also at some point need to hack in some VST support (which would probably not be trivial. I'm really afraid that my only option will be C++, and that would be unpleasant enough to make me not actually work on it (know some C++, but not really enough to write something this intense).
Anyone have some ideas? Thanks
VST support in Java may be reasonably easy after all; I've heard of positive experiences with http://github.com/mhroth/jvsthost (that is to say, someone I had a conversation with on a forum seemed to be up and running with it pretty quickly, running a number of different synths successfully).
An aside: Personally, I'm developing some software in Java that uses SuperCollider as an audio backend (disclaimer: my actual experience of Java sound is limited). While it would probably be just about possible to build a DAW around SuperCollider, I wouldn't really recommend it as the tool for that job. However, I also don't quite understand why you want to build a DAW in the first place... should you decide you want to explore alternative means of making music with computers, you might give SC a look (also ChucK I found very easy to get started with and quite a lot of fun) :-)
Anyway, back to the question... while I tend to refer to Java specifically, much of this will go for C# as well:
Traditionally, garbage collection has been a source of concern doing anything where time is of the essence in Java; in a DAW, for example, this may manifest itself as inaccurate timing or clicks in the output where the GC interrupts the program long enough that it is not able to process a complete buffer. This will be particularly true if you want to use small buffers for low latency, and/or are not careful about the amount of garbage generated. However, I don't want to spread FUD about Java sound: as I mentioned, I haven't really used it heavily myself, and in any case I believe these issues are improving. It is certainly an issue you will need to be aware of, but probably not a show-stopper.
I imagine that a big bottleneck in any DAW will be file IO, which shouldn't suffer through Java as long as proper care is taken.
If you start doing intense DSP on many channels simultaneously, then it may be that Java computation performance isn't totally optimal (although probably not bad really); however if you mostly do basic mixing in your DAW code and any DSP with VSTs, then his should be a non-issue anyway.
In terms of actual audio IO, I see that there are also ASIO implementations for Java, should you be interested. I don't even have indirect experience of those, so I really won't vouch for them. Java 1.7 is supposed to have improved low-latency audio support, FWIW (although from what I've read, the applications they have in mind are not things like DAWs). DirectX support I don't think should be a major factor for a DAW. In that sense, you might not want to dismiss C#, as it is a very nice language.
There are already some DAWs that are using the java plattform (frinika or javaDAW
for example). So I think it's a reasonable option.
I'm working on something similar so I would have to say it is possible, my laptop was stolen and I've has to start over but I've rebuilt most of it. So far the track threads have been lining up pretty well but I'm considering implementing something like LWJGL's timer for better precision. Tritonus is a very helpful library and you can find it at jsresources.org as well as some very helpful examples. I've learned a lot there. I'd you send me an e-mail I'd be happy to share my code with you.
I would like to build my own signal processing library, and possibly another one about graphs algorithm. I find C# very useful and robust in regards of possible bugs associated with memory allocation, pointers, threading etc...
But I was wondering how much am I going to lose in terms of performance. Is it going to be something acceptable?
Thanks
When I started my DSIP course I was a pure C# developer . After looking around for a while, I ended up using C++libraries and learning C++ which at end it was to my benefit since I was doing real-time image processing and there is no way C# can match the performance.
In fact, you can run a quick test and run a mathematical equation consisting of a few multiplication in C# and C++ a million times and see the huge difference that there is doing calculations with floating point numbers.
If you are lucky, you will get wrappers in C# which is the best of both worlds, fast calculation in C++ and easy to use in C#. There is a C# wrapper for OpenCV and it seems to be quite good (http://www.emgu.com/wiki/index.php/Main_Page).
I highly recommend OpenCV especially for 2D signal processing. It is great in performance and made my project possible. Have a look at the demo here: http://www.youtube.com/watch?v=NeQvcdRPxoI and http://www.youtube.com/watch?v=NqYPJELHkhA
With the right hardware, you shouldn't have an issue.
Something to think about though: I have no trouble whatsoever trying to find a library to do signal processing in C++. However, I can't easily find much information at all dealing with signal processing for C#.
If you can find a way to make it work, you might have just found a way to get into a niche area of development that many other people can benefit from.
Just something to think about. Unless you are doing this on some mission critical system it probably doesn't matter either way. I'd just go with whatever you think will benefit you more in the long term.
It's not so much about the performance loss in C# but the unpredictability of when garbage collection is performed. During GC all managed threads are frozen until the GC is completed. During that time you cannot do any processing, which for 99% of applications won't matter, but for signal processing is critical. This is i.e. why you don't see a Microsoft sanctioned managed version of DirectShow or other "real time" signal processing applications.
I develop a utility that behaves like "Adobe photoshop". user can draw rectangle,circle,... with mouse pointer and then move or resize it. for this functionality, I assume each shape is a object that stored in a generic collection in a container object. when user wants to change anything I recognise that where he clicked and in behind of scence I select the target object and so on...
this way have a problem when objects in screen is lot or user loads a picture with high resolution.
What's your opinion?
How can I solve it?
This makes sense because the larger the data set, the more RAM and CPU will be required to handle it.
While performance issues are important to solve, a lot of it may be perceieved performance so something like a threading issue - where you have one thread trying to process the information and you block the UI thread which makes it look like the system is freezing.
There is a lot of information on StackOverflow that you may want to look at
C# Performance Optimization
C# Performance Best Practices
C# Performance Multi threading
C# Performance Collections (Since you said you were using a collection)
Use a profiler such as dotTrace and find out which method is the one most called and the one that takes the most amount of time to process. Those are the ones you want to try to optimize. Other than that, you may have to go down to the GPU to try to speed things up.
About these kind of problem, think about parallel extensions :
http://msdn.microsoft.com/en-us/concurrency/default.aspx
The more cpu you have, the faster your program is running.
The thing is that in hi resolution the computer needs to use more the processor, then this occurs, remember that this also occurs in The Gimp, even in Adobe Photoshop.
Regards.
Look into using a performance analyzing tool (such as ANTS Profiler) to help you pinpoint exactly where the bottle necks are occurring. True graphical computations on a high res photo require alot of resources, but I would assume the logic you are using to manage and find your objects require some tuning up as well.
I high-resolution image takes up a lot of memory (more bits-per-pixel). As such, any operation that you do to it means more bits to manipulate.
Does your program utilise "layers"?
If not, then I'm guessing you are adding components directly to the image - which means each operation has to manipulate the bits. So if you aren't using layers, then you should definitely start. Layers will allow you to draw operations to the screen but only merge them into the base high-resolution image once - when you save!
What library from Windows are you using to open the image?
If you are using System.Drawing then you are actually using GDI+ as it is a wrapper on top of it. GDI+ is nice for a lot of things because it simplies tons of operations, however it isn't the fastest in the world. For example using the [Get|Set]Pixel methods are MUCH slower than working directly on the BitmapData. As there are tons of articles on speeding up operations on top of GDI+ I will not re-iterate them here.
I hope the information I've provided answer some of your questions causes new ones. Good luck!