Develop Reference

Speech Recognition Game - Unity

Good Afternoon everyone,
I am currently working on a uni project on accessibility in video games. My game uses eye tracking and speech recognition. It consists of 2 small level : a shooting game and a running level. The game i offline. The Eye tracking part works fine but I encountered an issue with the speech recognition. I am using the phrase recognizer from unity speech: https://learn.microsoft.com/en-us/windows/mixed-reality/develop/unity/voice-input-in-unity .
The problem is that there is a delay of a second to a second and a half from the moment I speak to the recognition. It happens before my onphrase recognizer is called (before my functions are called). The delay is still present when I take down wifi and cortana and I am wondering if there is any way to shorten it as it is pretty bad in a video game...
Here is the code in question:
//Speech recognition Initialization
private KeywordRecognizer keywordRecognizer;
private Dictionary<string, System.Action> actions = new Dictionary<string, System.Action>();
[...]
void Start()
{
//we add the jump function to the dictionnary
actions.Add("jump", () => Up(1.25f));
//we set the speech recognition function and start it
keywordRecognizer = new KeywordRecognizer(actions.Keys.ToArray(), ConfidenceLevel.Low);
keywordRecognizer.OnPhraseRecognized += RecognizedSpeech;
keywordRecognizer.Start();
}
private void RecognizedSpeech(PhraseRecognizedEventArgs speech)
{
Debug.LogWarning("jump");
actions[speech.text].Invoke();
}
public void EndListening()
{
actions.Clear();
//keywordRecognizer.Stop();
}
[...]"
Would anyone have a lead or an advice or is working/ has worked on something similar ?
Thank you for your time.

Unity mobile device 30fps locked

Since yesterday, my unity game is locked 30fps on an android device. Before it was like 150fps.. I disabled v-sync. This is the only script I added yesterday:
public static class SaveLoadProgress {
public static void Save()
{
LevelManager levelManager = GameObject.Find ("GameManager").GetComponent<LevelManager> ();
BinaryFormatter bf = new BinaryFormatter();
FileStream file = File.Create (Application.persistentDataPath + "/savedProgress.1912");
bf.Serialize(file, levelManager.levelReached);
file.Close();
}
public static void Load()
{
LevelManager levelManager = GameObject.Find ("GameManager").GetComponent<LevelManager> ();
if(File.Exists(Application.persistentDataPath + "/savedProgress.1912"))
{
BinaryFormatter bf = new BinaryFormatter();
FileStream file = File.Open(Application.persistentDataPath + "/savedProgress.1912", FileMode.Open);
levelManager.levelReached = (int)bf.Deserialize(file);
file.Close();
}
}
}
I don't this can be the problem.
I added this to the GameManager:
void Awake()
{
Application.targetFrameRate = -1;
QualitySettings.vSyncCount = 0;
}
The profiler shows this:
What should I do to unlock it from 30fps?

By default, Unity locks fps for specific values, according to the current platform. Documentation says that the default for Android is 30, which is used by most Android devices. Also it says:
The default targetFrameRate is a special value -1, which indicates that the game should render at the platform's default frame rate.
Which is 30 for mobile as it's said in the article.
Also, you can have as much fps as possible while running your game in editor, it maybe 150 or more, but it's really bad practice to have more than 60 (max value for iPhones and some Androids). If you have more than 60 fps you can't actually see the difference because device's display can't actually render more than 60 frames per second, but it takes battery and CPU resources and all those calculations (for extra 60+ fps) are useless and even bad.
To make your fps more than 30, you can set it for 60 as you are doing in your code:
Application.targetFrameRate = 60;
Also v-sync is mostly don't work on mobile, so you don't need to bother with this.

Not an answer but more of an extended comment to #vmchar's answer:
I disagree that VSync doesn't work on mobile. We had Application.targetFrameRate set to 60, however, we still had an issue with the CPU trashing the GPU. Setting VSync forces it to wait. Framerate went from https://imgur.com/a/lE0LedF to https://imgur.com/a/EPq6q7T.
Unity don't recommend using both at once though:
Application.targetFrameRate is a "soft" vsync and is useful to limit simulation rate without hardware vsync set (e.g. on servers). It is not desirable to use targegFrameRate with vSyncCount set, as CPU frame might drift and cause rendercommands to be scheduled later and miss next vsync.
VSync settings aren't respected in the Editor though, so you can still have issues there.
Full thread: https://forum.unity.com/threads/gfx-presentframe-taking-4x-as-long-but-only-for-intermittent-periods.538254

As stated in other answer, if Unity locks your framerate to 30 or something. Mine was locked at 30 FPS, Just add Application.targetFrameRate = 60; in your Start()method, or whenever you want to set the Framerate in the application lifetime.
void Start()
{
Application.targetFrameRate = 60;
}

If you have the Adaptive Performance Samsung Android package installed it will override any changes to Application.targetFrameRate.
If you want to keep the package installed then you will need to play with the settings. To access the settings: Edit > Project Settings > Adaptive Performance > Samsung (Android)
Unchecking everything removed the cap set by Adaptive Performance for me. You can then re-enable the settings you need from there.

Song doesn't play all the time in Monogame

Heey,
We created a game with Monogame but we got the following problem.
We got a themesong that plays when you have loaded the game now is the problem that the themesong sometimes plays but sometimes just doesn't. We convert it by the XNA pipeline to a wma and load it into our game with the .xnb together but just sometimes the music doesn't wanna start.
We just use the standard code for starting a song and all of this code does fire.
internal void PlayBackgroundMusic()
{
if (MediaPlayer.GameHasControl)
{
MediaPlayer.Volume = mainVolume;
MediaPlayer.Play(backgroundMusic);
MediaPlayer.IsRepeating = true;
}
}
We also use SoundEffects but these work 100% of the time it's only the song that won't play everytime you start. Windows RT runs it fine by the way.

Make sure that the debugger gets into the if statement through debugging (or remove the statement temporarily). Another possibility might be that the function is running before the game is fully initialized. You could try delaying the function until the game has been fully loaded.
PS: I can't comment on questions yet so here's an answer.
Edit:
Alright, after some messing around with the Song class and looking in the implementation in MonoGame I came to the conclusion that the SoundEffect class is easier to use and better implemented.
backgroundSound = Content.Load<SoundEffect>("alarm");
protected override void BeginRun()
{
// I created an instance here but you should keep track of this variable
// in order to stop it when you want.
var backSong = backgroundSound.CreateInstance();
backSong.IsLooped = true;
backSong.Play();
base.BeginRun();
}
I used this post: using BeginRun override to play the SoundEffect on startup.

If you want to play a song, use the Song class. But make sure you are using ".mp3" instead of ".wma" before converting them into ".xnb".
backgroundMusic = Content.Load<Song>("MainTheme");
MediaPlayer.Play(backgroundMusic);
MediaPlayer.IsRepeating = true;
See MSDN.

SharpDX fixed time step issue

I'm using SharpDX 2.5.0 and I have a game class, and I try to disable the fixed time step, however it doesn't seem to work, I still get 59-60 FPS. I'm only drawing a utah teapot, so I'm pretty sure it must work with a lot more (like 1000) FPS. Here is what I do:
protected override void LoadContent()
{
// ...
// Disabling fix time step.
this.IsFixedTimeStep = false;
// ...
base.LoadContent();
}
Do I forget something? Do I have to apply this change somehow? Or am I doing it in the wrong place (I also tried doing it elsewhere without any success)? Thanks for the answers:
Muad'Dib

You need to disable both vsync and fixed timestep, try to add this to the game constructor:
// GraphicsDeviceManager is mandatory for a Toolkit Game
_graphicsDeviceManager = new GraphicsDeviceManager(this);
// disable vsync
_graphicsDeviceManager.SynchronizeWithVerticalRetrace = false;
// disable fixed timestep
this.IsFixedTimeStep = false;

Have you also tried disabling vsync? If vsync is enabled and your monitor is running at 60Hz (very likely) then you will also see this behaviour. I'm not sure about the Game class but I usually do it in the PresentationParameters when creating the device.
new PresentParameters(width, height) {
PresentationInterval = PresentInterval.Immediate
}
"Immediate" indicates that present will not wait for the monitor to refresh.
This is assuming D3D9, which version of DirectX are you using?

XNA 4.0 runs at 30 fps with GF 640M but works fine with integrated Intel card

Every time I run my game using integrated graphics card it works fine at 60 FPS. But sometimes it lags too much when there're a lot of particles on screen.
So I switched to Nvidia GeForce 640M, but framerate keeps at 30 instead of 60. I tried both Reach and HiDef but neither of them could fix the framerate problem. I also tried
this.TargetElapsedTime = TimeSpan.FromMilliseconds(15);
but FPS was still 30.
Also tried this but didn't work:
public Game1()
{
graphics = new GraphicsDeviceManager(this);
Content.RootDirectory = "Content";
graphics.PreparingDeviceSettings += new EventHandler<PreparingDeviceSettingsEventArgs>(graphics_PreparingDeviceSettings);
}
private void graphics_PreparingDeviceSettings(object sender, PreparingDeviceSettingsEventArgs e)
{
e.GraphicsDeviceInformation.PresentationParameters.MultiSampleCount = 4;
e.GraphicsDeviceInformation.PresentationParameters.PresentationInterval = PresentInterval.One;
}
What can I do to fix the framerate on Nvidia graphics card?

Have you tried changing the PresentationParameters.PresentationInterval?
If I recall correctly, if you set this to PresentInterval.One, it "pulls out all the stops" and tries to match the maximum refresh rate.
Been a while since I tinkered with XNA tho...

May have something to do with your vsync settings. Try the following
graphics.SynchronizeWithVerticalRetrace = false;