Real-time is not necessarily required, however I am creating a game for my final year project and I wish to use the power of audio to create dynamic levels based solely on a music track that is playing. I aim to create this game for the PS Vita using playstation mobile and C#, but if i want i can switch to C++ and PSP.
I can use a WAV file, and hopefully extract the amplitude of the waveform, as well as calculating other characteristics like average frequency and approximate BPM from this data to create a level.
I have no qualms about trying to work with this raw data, I just want to know a way I can actually GET that information first. If i can extract the samples and assertain different characteristics of these samples, I can store them and work out changes in loudness, pitch and more to create notes etc.
I am using C#, but if at all possible i can either use p/invoke or switch my project to another device that uses C++ instead of C#.
I'm panicking a bit here, cos I really am a bit stumped.
Many thanks guys.
Unfortunately i don't think you'll be able to use C# to do this - AFAIK, there is no JIT compiler for it. I remember reading about something for Mono, which would make it available to use with C#, but i'm not sure right now.
That said - i would go with c++. If you go that way, you can make use of a vast amount of audio analysis libraries, like CLAM (http://clam-project.org/).
Don't panic (imagine big, friendly letters.) Envision the necessary parts for the project step by step, tackle one by one, and you'll be done in no time. =)
The problem you describe here is one of music/audio feature extraction and a substantial body of academic work exists that you can draw on. Another useful term of art with which to search is Music Information Retrieval (MIR).
The list of 'features' that researchers have attempted to retrieve from recordings is large and varied, from deterministic things such as pitch and key through emotional characteristics, such as 'energy'.
Most of these turn out to be more difficult than you might imagine, and typically only about 60-70% accurate - although for your requirements, this is probably adequate.
A good entry point might be download Sonic Visualiser, for which a large number of feature extraction plug-ins exist, and are open-source. You'll at least get a feel for what's possible.
Update: Another useful term of art is Onset detection - this is typically used to describe beat detection algorithms.
Aubio is a C/C++ library that does pitch tracking, onset detection and bpm tracking, among other things.
As for "extracting the amplitude of the waveform", the waveform is amplitude, i.e., you could just pick the audio sample with the greatest absolute value every n samples and use that value to do the "amplitude" part of the visualization.
Here's some code that might help you get started reading WAVE data in C#.
Here's some information about writing a C# wrapper for the FFTW library.
I am trying to use a MCS (Multi classifier system) to do some better work on limited data i.e become more accurate.
I am using K-means clustering at the moment but may choose to go with FCM (Fuzzy c-means) with that the data is clustered into groups (clusters) the data could represent anything, colours for example. I first cluster the data after pre-processing and normalization and get some distinct clusters with a lot in between. I then go on to use the clusters as the data for a Bayes classifier, each cluster represents a distinct colour and the Bayes classifier is trained and the data from the clusters is then put through separate Bayes classifiers. Each Bayes classifier is trained only in one colour. If we take the colour spectrum 3 - 10 as being blue 13 - 20 as being red and the spectrum in between 0 - 3 being white up to 1.5 then turning blue gradually through 1.5 - 3 and same for blue to red.
What I would like to know is how or what kind of aggregation method (if that is what you would use) could be applied so that the Bayes classifier can become stronger, and how does it work? Does the aggregation method already know the answer or would it be human interaction that corrects the outputs and then those answers go back into the Bayes training data? Or a combination of both? Looking at Bootstrap aggregating it involves having each model in the ensemble vote with equal weight so not quite sure in this particular instance I would use bagging as my aggregation method? Boosting however involves incrementally building an ensemble by training each new model instance to emphasize the training instances that previous models mis-classified, not sure if this would be a better alternative to bagging as im unsure how it incrementally builds upon new instances? And the last one would be Bayesian model averaging which is an ensemble technique that seeks to approximate the Bayes Optimal Classifier by sampling hypotheses from the hypothesis space, and combining them using Bayes' law, however completely unsure how you would sample hypotheses from search space?
I know that usualy you would use a competitive approach to bounce between the two classification algorithms one says yes one says maybe a weighting could be applied and if its correct you get the best of both classifiers but for keep sake I dont want a competitive approach.
Another question is using these two methods together in such a way would it be beneficial, i know the example i provided is very primitive and may not apply in that example but can it be beneficial in more complex data.
I have some issues about the method you are following:
K-means puts in each cluster the points that are the most near to it. And then you train a classifier using the output data. I think that the classifier may outperform the clustering implicit classification, but only by taking into account the number of samples in each cluster. For example, if your training data after clustering you have typeA(60%), typeB(20%), typeC(20%); your classifier will prefer to take ambiguous samples to typeA, to obtain less classification error.
K-means depends on what "coordinates"/"features" you take from the objects. If you use features where the objects of different types are mixed, the K-means performance will decrease. Deleting these kind of features from the feature vector may improve your results.
Your "feature"/"coordinates" that represent the objects that you want to classify may be measured in different units. This fact can affect your clustering algorithm since you are implicitly setting a unit conversion between them through the clustering error function. The final set of clusters is selected with multiple clustering trials (that were obtained upon different cluster initializations), using an error function. Thus, an implicit comparison is made upon the different coordinates of your feature vector (potentially introducing the implicit conversion factor).
Taking into account these three points, you will probably increase the overall performance of your algorithm by adding preprocessing stages. For example in object recognition for computer vision applications, most of the information taken from the images comes only from borders in the image. All the color information and part of the texture information are not used. The borders are substracted from the image processing the image to obtain the Histogram of Oriented Gradients (HOG) descriptors. This descriptor gives back "features"/"coordinates" that separate better the objects, thus, increasing classification (object recognition) performance. Theoretically descriptors throw information contained in the image. However, they present two main advantages (a) the classifier will deal with lower dimensionality data and (b) descriptors calculated from test data can be more easily matched with training data.
In your case, I suggest that you try to improve your accuracy taking a similar approach:
Give richer features to your clustering algorithm
Take advantage of prior knowledge in the field to decide what features you should add and delete from your feature vector
Always consider the possibility of obtaining labeled data, so that supervised learning algorithms can be applied
I hope this helps...
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
Akismet does an amazing job at detecting spam comments. But comments are not the only form of spam these days. What if I wanted something like akismet to automatically detect porn images on a social networking site which allows users to upload their pics, avatars, etc?
There are already a few image based search engines as well as face recognition stuff available so I am assuming it wouldn't be rocket science and it could be done. However, I have no clue regarding how that stuff works and how I should go about it if I want to develop it from scratch.
How should I get started?
Is there any open source project for this going on?
This is actually reasonably easy. You can programatically detect skin tones - and porn images tend to have a lot of skin. This will create false positives but if this is a problem you can pass images so detected through actual moderation. This not only greatly reduces the the work for moderators but also gives you lots of free porn. It's win-win.
#!python
import os, glob
from PIL import Image
def get_skin_ratio(im):
im = im.crop((int(im.size[0]*0.2), int(im.size[1]*0.2), im.size[0]-int(im.size[0]*0.2), im.size[1]-int(im.size[1]*0.2)))
skin = sum([count for count, rgb in im.getcolors(im.size[0]*im.size[1]) if rgb[0]>60 and rgb[1]<(rgb[0]*0.85) and rgb[2]<(rgb[0]*0.7) and rgb[1]>(rgb[0]*0.4) and rgb[2]>(rgb[0]*0.2)])
return float(skin)/float(im.size[0]*im.size[1])
for image_dir in ('porn','clean'):
for image_file in glob.glob(os.path.join(image_dir,"*.jpg")):
skin_percent = get_skin_ratio(Image.open(image_file)) * 100
if skin_percent>30:
print "PORN {0} has {1:.0f}% skin".format(image_file, skin_percent)
else:
print "CLEAN {0} has {1:.0f}% skin".format(image_file, skin_percent)
This code measures skin tones in the center of the image. I've tested on 20 relatively tame "porn" images and 20 completely innocent images. It flags 100% of the "porn" and 4 out of the 20 of the clean images. That's a pretty high false positive rate but the script aims to be fairly cautious and could be further tuned. It works on light, dark and Asian skin tones.
It's main weaknesses with false positives are brown objects like sand and wood and of course it doesn't know the difference between "naughty" and "nice" flesh (like face shots).
Weakness with false negatives would be images without much exposed flesh (like leather bondage), painted or tattooed skin, B&W images, etc.
source code and sample images
This was written in 2000, not sure if the state of the art in porn detection has advanced at all, but I doubt it.
http://www.dansdata.com/pornsweeper.htm
PORNsweeper seems to have some ability to distinguish pictures of people from pictures of things that aren't people, as long as the pictures are in colour. It is less successful at distinguishing dirty pictures of people from clean ones.
With the default, medium sensitivity, if Human Resources sends around a picture of the new chap in Accounts, you've got about a 50% chance of getting it. If your sister sends you a picture of her six-month-old, it's similarly likely to be detained.
It's only fair to point out amusing errors, like calling the Mona Lisa porn, if they're representative of the behaviour of the software. If the makers admit that their algorithmic image recogniser will drop the ball 15% of the time, then making fun of it when it does exactly that is silly.
But PORNsweeper only seems to live up to its stated specifications in one department - detection of actual porn. It's half-way decent at detecting porn, but it's bad at detecting clean pictures. And I wouldn't be surprised if no major leaps were made in this area in the near future.
I would rather allow users report on bad images. Image recognition development can take too much efforts and time and won't be as much as accurate as human eyes. It's much cheaper to outsource that moderation job.
Take a look at: Amazon Mechanical Turk
"The Amazon Mechanical Turk (MTurk) is one of the suite of Amazon Web Services, a crowdsourcing marketplace that enables computer programs to co-ordinate the use of human intelligence to perform tasks which computers are unable to do."
Bag-of-Visual-Words Models for Adult Image Classification and Filtering
What is the best way to programatically detect porn images?
A Brief Survey of Porn-Detection/Porn-Removal Software
Detection of Pornographic Digital Images (2011!)
BOOM! Here is the whitepaper containing the algorithm.
Does anyone know where to get the source code for a java (or any language) implementation?
That would rock.
One algorithm called WISE has a 98% accuracy rate but a 14% false positive rate. So what you do is you let the users flag the 2% false negatives, ideally with automatic removal if a certain number of users flag it, and have moderators view the 14% false positives.
Nude.js based on the whitepaper by Rigan Ap-apid from De La Salle University.
There is software that detects the probability for porn, but this is not an exact science, as computers can't recognize what is actually on pictures (pictures are only a big set of values on a grid with no meaning). You can just teach the computer what is porn and what not by giving examples. This has the disadvantage that it will only recognize these or similar images.
Given the repetitive nature of porn you have a good chance if you train the system with few false positives. For example if you train the system with nude people it may flag pictures of a beach with "almost" naked people as porn too.
A similar software is the facebook software that recently came out. It's just specialized on faces. The main principle is the same.
Technically you would implement some kind of feature detector that utilizes a bayes filtering. The feature detector may look for features like percentage of flesh colored pixels if it's a simple detector or just computes the similarity of the current image with a set of saved porn images.
This is of course not limited to porn, it's actually more a corner case. I think more common are systems that try to find other things in images ;-)
The answer is really easy: It's pretty safe to say that it won't be possible in the next two decades. Before that we will probably get good translation tools. The last time I checked, the AI guys were struggling to identify the same car on two photographs shot from a slightly altered angle. Take a look on how long it took them to get good enough OCR or speech recognition together. Those are recognition problems which can benefit greatly from dictionaries and are still far from having completely reliable solutions despite of the multi-million man months thrown at them.
That being said you could simply add an "offensive?" link next to user generated contend and have a mod cross check the incoming complaints.
edit:
I forgot something: IF you are going to implement some kind of filter, you will need a reliable one. If your solution would be 50% right, 2000 out of 4000 users with decent images will get blocked. Expect an outrage.
A graduate student from National Cheng Kung University in Taiwan did a research on this subject in 2004. He was able to achieve success rate of 89.79% in detecting nude pictures downloaded from the Internet. Here is the link to his thesis: The Study on Naked People Image Detection Based on Skin Color It's in Chinese therefore you may need a translator in case you can't read it.
short answer: use a moderator ;)
Long answer: I dont think there's a project for this cause what is porn? Only legs, full nudity, midgets etc. Its subjective.
Add an offensive link and store the md5 (or other hash) of the offending image so that it can automatically tagged in the future.
How cool would it be if somebody had a large public database of image md5 along with descriptive tags running as a webservice? Alot of porn isn't original work (in that the person who has it now, didn't probably make it) and the popular images tend to float around different places, so this could really make a difference.
If you're really have time and money:
One way of doing it is by 1) Writing an image detection algorithm to find whether an object is human or not. This can be done by bitmasking an image to retrieve it's "contours" and see if the contours fits a human contour.
2) Data mine a lot of porn images and use data mining techniques such as the C4 algorithms or Particle Swarm Optimization to learn to detect pattern that matches porn images.
This will require that you identify how a naked man/woman contours of a human body must look like in digitized format (this can be achieved in the same way OCR image recognition algorithms works).
Hope you have fun! :-)
Seems to me like the main obstacle is defining a "porn image". If you can define it easily, you could probably write something that would work. But even humans can't agree on what is porn. How will the application know? User moderation is probably your best bet.
I've seen a web filtering application which does porn image filtering, sorry I can't remember the name. It was pretty prone to false positives however most of the time it was working.
I think main trick is detecting "too much skin on the picture :)
Detecting porn images is still a definite AI task which is very much theoretical yet.
Harvest collective power and human intelligence by adding a button/link "Report spam/abuse". Or employ several moderators to do this job.
P.S. Really surprised how many people ask questions assuming software and algorithms are all-mighty without even thinking whether what they want could be done. Are they representatives of that new breed of programmers who have no understanding of hardware, low-level programming and all that "magic behind"?
P.S. #2. I also remember that periodically it happens that some situation when people themselves cannot decide whether a picture is porn or art is taken to the court. Even after the court rules, chances are half of the people will consider the decision wrong. The last stupid situation of the kind was quite recently when a Wikipedia page got banned in UK because of a CD cover image that features some nakedness.
Two options I can think of (though neither of them is programatically detecting porn):
Block all uploaded images until one of your administrators has looked at them. There's no reason why this should take a long time: you could write some software that shows 10 images a second, almost as a movie - even at this speed, it's easy for a human being to spot a potentially pornographic image. Then you rewind in this software and have a closer look.
Add the usual "flag this image as inappropriate" option.
The BrightCloud web service API is perfect for this. It's a REST API for doing website lookups just like this. It contains a very large and very accurate web filtering DB and one of the categories, Adult, has over 10M porn sites identified!
I've heard about tools which were using very simple, but quite effective algorithm. The algorithm calculated relative amount of pixels with color value near to some predefined "skin" colours. If that amount is higher than some predefined value then image is considered to be of erotic/pornographic content. Of course that algorithm will give false positive results for close-up face photos and many other things.
Since you are writing about social networking there will be lots of "normal" photos with high amount of skin colour on it, so you shouldn't use this algorithm to deny all pictures with positive result. But you can use it provide some help for moderators, for example flag these pictures with higher priority, so if moderator want to check some new pictures for pornographic content he can start from these pictures.
This one looks promising. Basically they detect skin (with calibration by recognizing faces) and determine "skin paths" (i.e. measuring the proportion of skin pixels vs. face skin pixels / skin pixels). This has decent performance.
http://www.prip.tuwien.ac.at/people/julian/skin-detection
Look at file name and any attributes. There's not nearly enough information to detect even 20% of naughty images, but a simple keyword blacklist would at least detect images with descriptive labels or metadata. 20 minutes of coding for a 20% success rate isn't a bad deal, especially as a prescreen that can at least catch some simple ones before you pass the rest to a moderator for judging.
The other useful trick is the opposite of course, maintain a whitelist of image sources to allow without moderation or checking. If most of your images come from known safe uploaders or sources, you can just accept them bindly.
I shall not today attempt further to
define the kinds of material I
understand to be embraced within that
shorthand description ["hard-core
pornography"]; and perhaps I could
never succeed in intelligibly doing
so. But I know it when I see it, and
the motion picture involved in this
case is not that.
— United States Supreme Court Justice Potter Stewart, 1964
You can find many whitepapers on the net dealing with this subject.
It is not rocket science. Not anymore. It is very similar to face recognition. I think that the easiest way to deal with it is to use machine learning. And since we are dealing with images, I can point towards neuronal networks, because these seem to be preferred for images. You will need training data. And you can find tons of training data on the internet but you have to crop the images to the specific part that you want the algorithm to detect. Of course you will have to break the problem into different body parts that you want to detect and create training data for each, and this is where things become amusing.
Like someone above said, it cannot be done 100% percent. There will be cases where such algorithms fail. The actual precision will be determined by your training data, the structure of your neuronal networks and how you will choose to cluster the training data (penises, vaginas, breasts, etc, and combinations of such). In any case I am very confident that this can be achieved with high accuracy for explicit porn imagery.
This is a nudity detector. I haven't tried it. It's the only OSS one I could find.
https://code.google.com/p/nudetech
There is no way you could do this 100% (i would say maybe 1-5% would be plausible) with nowdays knowledge. You would get much better result (than those 1-5%) just checking the image-names for sex-related-words :).
#SO Troll: So true.
I have a question. I want to write a chess like program applying the rules as follows:
It should have just a king and a queen on one side and the other side should have just a king.
The first side should mate the second side with the lowest number of moves possible.
I want to know your thoughts about how to make this project. For example I want to know about which way of writing code is easier (object oriented or structured, ...) (I have a little information about object oriented) and can say me about writing its algorithm? For example from where I should begin to write the codes?
The good news here is that your problem is quite restricted in scope, as you only have three pieces to contend with. You're not really implementing a game so much here, as solving a logical puzzle. I'd approach it like this:
Figure out how to represent the three pieces in a simple way. You really don't need a UI here (other than for testing), since you're just trying to solve a puzzle. The simplest way is probably a simple Row,Column position for each of the three pieces.
If you haven't written an object-oriented program before, you'll probably want to stick with a procedural model and simply define variables for the data you'll need to represent. The problem scope is small, so you can get away with this. If you have some OOP experience, you can split up the problem appropriately, though you probably won't need any inheritance relationships.
Write the code for generating possible moves and determine whether a given move makes any sense at all. A legal King move is any move that does not check the King. Most queen moves should be permissible, but you probably also want to exclude moves that would allow the enemy King to take the Queen.
Now you need to determine a strategy for how to put together a sequence of moves that will solve the puzzle. If you need to find the true optimal solution (not merely a good solution), you may need to do a brute-force search. This may be feasible for this problem. You'll probably want to perform a depth-first search (if you don't know what this means, that's your first topic to research), as once you find a possible solution, that limits the depth at which all other solutions must be considered.
If you can get brute force functional and need to make things faster, consider if there are moves you can prove will have no benefit. If so, you can exclude these moves immediately from your search, saving on the number of branches you need to consider. You can also work to optimize your evaluation functions, as a faster evaluation is very beneficial when you are doing billions of them. Finally, you might come up with some heuristics to evaluate which of the branches to try first. The faster you can converge to a 'good' solution, the less cases you need to consider to find the optimal solution.
One side note I realized is that the problem is very different if you assume that the enemy King is trying to avoid checkmate. The simple depth-first pruning only works if you are allowed to move the enemy King in the way that best checkmates it. If the enemy King is attempting to avoid checkmate, that complicates the problem, as you have conflicting optimization goals (you want it to occur in as few moves as possible, yet your enemy King wants to postpone as long as possible.) You might be limited to characterizing a range of possibilities (say, 3 moves best case if King is perfectly cooperative, 8 moves best worst-case if King is perfectly evasive.)
Take a look at this SO question (Programming a chess AI).
From the answers to that question, I think this C# Chess Game Starter Kit would be a good start, but I would also look at the other articles referenced as well for some interesting history/information.
This is the simplest possible example of an endgame database. There are less than 64^3 = 262144 positions, so you can easily store the score of each position. In this case, we can define the score as the number of moves to checkmate, for a winning position; or 255 for a drawn position. Here is an outline:
Set all scores to 255.
Look for all checkmate positions, and score them as 0.
Set depth = 1.
For each drawn position (score=255), see if a move exists into a won position (more precisely, see if a move exists into a position from which all of the opponent's moves are losing.) If so, set its score to depth.
If no new position was found in step 4, you're done.
Increment depth, and go to Step 4.
Now you have a 250k table that you can save to disk (not that it should take many seconds to generate it from scratch). If space is important, you can reduce this significantly by various tricks. Wikipedia has a nice article on all this -- seach for "Endgame tablebase".
A poster here suggest that Stockfish would be a good start, but it is a C++ project, whereas you are asking for C#.
The solution depends on your requirement. If you are interested in "just make it work", you could complete the project without writing more than 200 lines of code. You could embed an open source C# project, and ask the engine to report you the number of moves to mate. If the open source project is UCI supported, the following command will do the job:
go mate x
where x is the number of moves to mate.
However, if you need to do the thinking yourself. You will need to choose between efficient bitboard or object-oriented representation. Bitboard is a much better representation, it is very fast but harder to program. All chess engines use bitboard. In your project, represenation efficiency is not too much of concern, so you could choose OO represenation.
I like using the vector drawing feature of Expression Blend to create icons. That way I can change
colors easily on my icons without having to resort to an image editor.
But my question is... Say I have a treeview control that has an icon next to each tree element and say I have hundreds of elements. Do you think using images is faster - performance wise than using vector icons? B/c I'd rather use vectors but I'm wondering about performance concerns.
Whenever you produce vector graphics that are static, use CacheMode to at least reduce the level of math needed to handle the said graphics (given Silverlight doesn't use GPU and relies on software rendering to perform tasks like this).
That being said, go for the Bitmap approach as much as you can as it's less tax on the CPU but obviously more tax in download - balancing act :(
A good article on this can be found here:
http://mtaulty.com/CommunityServer/blogs/mike_taultys_blog/archive/2009/04/16/silverlight-3-and-gpu-acceleration.aspx
Scott Barnes
(Former Silverlight Product Manager)
Yes it's probably faster to have little bitmap icons. But it's also probably negligibly faster. WPF itself uses little vectors for things like the + and - in tree views if you break down the templates. It's designed to handle things like this. I also prefer to use the vector icons.
The only other trouble you'll probably encounter is "fuzziness", that's just the nature of vector graphics but with icons it's possible to explicitly draw every pixel and get them exactly right and thus be a bit less fuzzy. The downside is, of course, if you need it to be displayed at any other size it looks much much worse.
Just have to weigh the trade-offs.