Attempting to get a filtered stream using the Tweetinvi library via c#.
I am trying to do something similar to the search, which I have been able to successfully receive tweets:
public List<ITweet> SearchTweets(string query)
{
List<ITweet> tweets = new List<ITweet>();
TwitterCredentials.ExecuteOperationWithCredentials(Credentials, () =>
{
var searchParam = Search.GenerateSearchTweetParameter(query);
searchParam.Lang = Language.English;
searchParam.MaximumNumberOfResults = 100;
tweets = Search.SearchTweets(searchParam);
});
return tweets;
}
but the limitation here is that I can only get a max 100 tweets. I want to leverage the stream in a similar way and get a stream of tweets matching the tracks that I pass. I have tried this:
public void FilterStream(string query)
{
IFilteredStream tweets;
TwitterCredentials.ExecuteOperationWithCredentials(Credentials, () =>
{
var filteredStream = Tweetinvi.Stream.CreateFilteredStream();
filteredStream.AddTrack(query);
filteredStream.AddTrack("#" + query);
filteredStream.MatchingTweetReceived += (sender, args) => { Debug.WriteLine(args.Tweet.Text); };
filteredStream.StartStreamMatchingAllConditions();
});
}
but problem is it seems to run in an infinite loop and i'm unsure where to stop or limit the number of tweets I received from the stream to make it stop. The library's documentation is quite unclear and I have been unable to achieve the behavior I am seeking. I'm sure I am on the right route, just unsure how to stop the stream and store all the tweets I've received in a List<ITweet> construct.
To stop the stream use:
filteredStream.StopStream();
You can keep a count of the tweets globally then stop when the number of tweets you need is received.
I had the same problem and ended up doing this:
var task = filteredStream.StartStreamMatchingAllConditionsAsync();
while (true)
{
Thread.Sleep(500);
if (task.IsCanceled || task.IsCompleted || task.IsFaulted)
{
break;
}
if (ShouldStop)
{
filteredStream.StopStream();
break;
}
}
task.Wait();
StartStreamMatchingAllConditionsAsync() will start the task and return immediately, then the endless while loop will check for a condition to stop the stream and break the loop.
ShouldStop is a static boolean variable that is flipped to True when the code should stop streaming from twitter.
Then we Wait() for the stream task to finish.
That code smells but it seems to work! I'd be curious to see if someone comes up with a better implementation...
You should try to put a condition inside the MatchingTweetReceived method like this :
public void FilterStream(string query)
{
IFilteredStream tweets;
TwitterCredentials.ExecuteOperationWithCredentials(Credentials, () =>
{
var filteredStream = Tweetinvi.Stream.CreateFilteredStream();
filteredStream.AddTrack(query);
filteredStream.AddTrack("#" + query);
filteredStream.MatchingTweetReceived += (sender, args) => {
Debug.WriteLine(args.Tweet.Text);
if (condition)
stream.StopStream();
};
filteredStream.StartStreamMatchingAllConditions();
});
}
This worked fine for me.
I know it's an old question, but I've just done this:
public void Start()
{
_stream = Stream.CreateFilteredStream(_creds);
// Do whatever stream setup you want to do
Task.Factory.StartNew(() => {
Debug.WriteLine("Thread started");
_stream.StartStreamMatchingAllConditions();
Debug.WriteLine("Thread stopped");
});
}
public void Stop()
{
_stream.StopStream();
}
Related
I am working with a dnp3 nuget package and I want to implement it in a WPF, and I achieve it by referring to this example https://github.com/stepfunc/dnp3/blob/master/ffi/bindings/dotnet/examples/master/Program.cs
I managed to implement it by an MVVM method where it dynamically shows the messages it receives (in the gif it shows an error because it did not create another program that will connect but the important thing is that it is being updated).
but I created another much simpler project to do the demonstration here on github , in my project create a class called protocol and paste all the master example eliminating the main function and changing the private functions for public functions, then inside my MainWindow.xaml.cs start the code
{
public MainWindow()
{
InitializeComponent();
// ANCHOR: logging_init
// Initialize logging with the default configuration
// This may only be called once during program initialization
Logging.Configure(
new LoggingConfig(),
new Protocolo.ConsoleLogger()
);
// ANCHOR_END: logging_init
// ANCHOR: runtime_init
var runtime = new Runtime(new RuntimeConfig { NumCoreThreads = 4 });
// ANCHOR_END: runtime_init
// ANCHOR: create_master_channel
var channel = MasterChannel.CreateTcpChannel(
runtime,
LinkErrorMode.Close,
Protocolo. GetMasterChannelConfig(),
new EndpointList("127.0.0.1:20000"),
new ConnectStrategy(),
new Protocolo.TestListener()
);
// ANCHOR_END: create_master_channel
Task.Run(() =>
{
try
{
Protocolo.RunChannel(channel).GetAwaiter().GetResult();
}
finally
{
// ANCHOR: runtime_shutdown
runtime.Shutdown();
// ANCHOR_END: runtime_shutdown
}
}
);
}
}
I did performance profiles and RunChannel is what they demand a lot from the CPU
public static async Task RunChannel(MasterChannel channel)
{
// ANCHOR: association_create
var association = channel.AddAssociation(
1024,
GetAssociationConfig(),
new TestReadHandler(),
new TestAssocationHandler()
);
// ANCHOR_END: association_create
// ANCHOR: add_poll
var poll = channel.AddPoll(association, Request.ClassRequest(false, true, true, true), TimeSpan.FromSeconds(5));
// ANCHOR_END: add_poll
// start communications
channel.Enable();
while (true)
{
//Here there was a switch that read the commands that were entered into the console but it is unnecessary, with the empty while it continues to update
}
}
I don't know why but without the While the received messages are not updated, (these messages arrive in the Logger) I have to keep this function always alive but I don't know how to do it without consuming so much CPU
to see the messages in the output you have to change Console.WriteLine to Debug.WriteLine
class ConsoleLogger : ILogger
{
public void OnMessage(LogLevel level, string message)
{
Console.Write($"{message}");
//this is inside the master example in the github link
}
}
As commented before (answering about "performance issue"):
Your empty while loop running without any pause or job. You need to fill loop with some job or simple pause (Thread.Sleep(10)/await Task.Delay(10)).
Your RunChannel method have no any await, so you should get CS1998 warning "This async method lacks 'await'" and, obviously, put it into your while loop.
On DNP3 Github example, there is await GetInputAsync() inside while loop, which waits for user input with Console.ReadLine() and returns inputed string into switch statement.
So:
public MainWindow()
{
// InitializeComponent() and other stuff...
Task.Run(async () => // Make it async
{
try
{
// You really didn't want GetAwaiter().GetResult().
await Protocolo.RunChannel(channel);
}
finally
{
runtime.Shutdown();
}
});
}
public class Protocolo
{
public static async Task RunChannel(MasterChannel channel)
{
// Associations and polls stuff...
while (true)
{
await Task.Delay(10); // Set some delay or...
await SomeJob(); // ...or do some job or ...
switch (await GetInputAsync()) // ...or take github's example
{
case "x":
return;
default:
Console.WriteLine("Unknown command");
break;
}
}
}
public static Task SomeJob()
{
return Task.Run(() =>
{
for (int i = 0; i < 5; i++)
{
Thread.Sleep(200); // 200 x 5 = Total task for 1 second
}
});
}
public static Task<string> GetInputAsync()
{
return Task.Run(() => Console.ReadLine());
}
}
I am trying to recreate a C# app with Angular, but I am struggling to get the code to run synchronously as it would in c#.
Here is an example:
private void doChecks()
{
if (isInvoiced())
return;
Console.WriteLine("Done");
}
private bool isInvoiced()
{
var invID = Server.GetInvoice(mAccID);
if (invID <= 0)
return false;
else
return someOtherFunction(invID);
}
When I try implementing this in Angular I am not sure how to accomplish the same flow without resorting to async await methods.
async doChecks() {
const doAllChecks = await this.service.DoChecks(this.mAcc).toPromise();
if (await this.isInvoiced()) {
return;
}
Console.log("Done");
}
async isInvoiced() {
const invID = await this.service.GetInvoice(this.mAcc).toPromise();
if (invID <= 0)
return false;
else {
const data = await someOtherFunction(invID); // now this function is also going to have to be async to I can await inside of that for its http request to finish.
return data;
}
}
Is there a way to accomplish this in a simpler way ?
You could run it one after the other inside "then statement". Nested Promises. If you are using Angular 2 and above, you can try nesting your logic in subscribers instead of promises.
this.MyPromise1().then(result => {
this.MyPromise2().then(result2 => {
this.MyPromise3().then(result3 => {
and so on..
})
})
});
I have some software with an event-based networking protocol for control which is using a IObservable<Event> for handling in bound messages.
In many cases a sent message will expect a specific response (or sequence, such as to report progress). In order to not potentially miss the response a task is being set up in advance with FirstAsync and ToTask, however these appear to leak if the task never completes.
It is also not allowed to simply put evtTask in a using block as trying to dispose the incomplete task is not allowed.
var jobUuid = Guid.NewGuid();
var evtTask = Events.FirstAsync((x) => x.Action == Action.JobComplete && x.JobUuid == jobUuid).ToTask();
// e.g. if this throws without ever sending the message
await SendMessage($"job {jobUuid} download {url}");
var evt = await evtTask;
if (evt.Success)
{
...
}
Does the library provide a simple means for this use-case that will unsubscribe on leaving the scope?
var jobUuid = Guid.NewGuid();
using(var evtTask = Events.FirstAsync((x) => x.Action == Action.JobComplete && x.JobUuid == jobUuid)
.ToDisposableTask())) // Some method like this
{
// e.g. if this throws without ever sending the message
await SendMessage($"job {jobUuid} download {url}");
var evt = await evtTask;
if (evt.Success)
{
...
}
} // Get rid of the FirstAsync task if leave here before it completes for any reason
Disposing Task will not help, since it does nothing useful (in most situations, including this one). What will help though is cancelling task. Cancelling disposes underlying subscription created by ToTask and so, resolves this "leak".
So it can go like this:
Task<Event> evtTask;
using (var cts = new CancellationTokenSource()) {
evtTask = Events.FirstAsync((x) => x.Action == Action.JobComplete && x.JobUuid == jobUuid)
.ToTask(cts.Token);
// e.g. if this throws without ever sending the message
try {
await SendMessage($"job {jobUuid} download {url}");
}
catch {
cts.Cancel(); // disposes subscription
throw;
}
}
var evt = await evtTask;
if (evt.Success)
{
...
}
Of course you can wrap that in some more convenient form (like extension method). For example:
public static class ObservableExtensions {
public static CancellableTaskWrapper<T> ToCancellableTask<T>(this IObservable<T> source) {
return new CancellableTaskWrapper<T>(source);
}
public class CancellableTaskWrapper<T> : IDisposable
{
private readonly CancellationTokenSource _cts;
public CancellableTaskWrapper(IObservable<T> source)
{
_cts = new CancellationTokenSource();
Task = source.ToTask(_cts.Token);
}
public Task<T> Task { get; }
public void Dispose()
{
_cts.Cancel();
_cts.Dispose();
}
}
}
Then it becomes close to what you want:
var jobUuid = Guid.NewGuid();
using (var evtTask = Events.FirstAsync((x) => x.Action == Action.JobComplete && x.JobUuid == jobUuid).ToCancellableTask()) {
await SendMessage($"job {jobUuid} download {url}");
var evt = await evtTask.Task;
if (evt.Success) {
...
}
}
You can either use TPL Timeout (as referenced by #Fabjan), or the Rx/System.Reactive version of Timeout.
using sounds nice, but doesn't make sense. Using is the equivalent of calling .Dispose on something at the end of the using block. The problem here, I'm assuming, is that your code never gets past await evtTask. Throwing all of that in a hypothetical using wouldn't change anything: Your code is still waiting forever.
At a higher level, your code is more imperative than reactive, you may want to refactor it to something like this:
var subscription = Events
.Where(x => x.Action == Action.JobComplete)
.Subscribe(x =>
{
if(x.Success)
{
//...
}
else
{
//...
}
});
I'm using dispatchers to update a bound collection from an event. I just ran into a nasty issue where I had two different dispatchers in the same event and it wasn't working. Using the debugger it was completely skipping over the code in the first dispatcher. Putting the entire event in a single dispatcher fixed it. I assume it's because of how the compiler handles it, can anyone confirm this - only one dispatcher per event, at least when dealing with the same elements?
Here is the code, when it gets to the await after (line == 0), it exits the function completely. Later, when line !=0 it runs the "Old style menu" fine. If I put all of the code in a single dispatcher, everything works fine.
private async void ProcessNLS(string parameters) // NET/USB List Info
{
if (parameters.Substring(0, 1) == "A" || (parameters.Substring(0, 1) == "U")) // ASCII x08/2010 Only
{
int line = Convert.ToInt32(parameters.Substring(1, 1));
string text = parameters.Substring(3);
// New Menu, Clear Old - Use Last Received/Holding Menu: See NLT bug
if (line == 0)
{
await CoreApplication.MainView.CoreWindow.Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
{
State.Menu.ServiceType = State.holdingMenu.ServiceType;
...
State.Menu.Items.Clear();
});
OnMenuTitleInfoChanged(new MenuTitleInfoChangedArgs(State.Menu));
// Replace Network Top with custom menu
if (State.Menu.LayerInfo == LayerTypes.NetworkTop)
{
await CoreApplication.MainView.CoreWindow.Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
{
State.Menu.Items.Clear();
});
...
}
// Get 1st Advanced Menu
if (Device.SupportsAdvancedMenus & State.Menu.LayerInfo != LayerTypes.NetworkTop)
{
...
}
}
// Old style menu
if (!Device.SupportsAdvancedMenus && State.Menu.LayerInfo != LayerTypes.NetworkTop)
{
NetworkMenuItem menuItem = new NetworkMenuItem(line, text);
await CoreApplication.MainView.CoreWindow.Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
{
State.Menu.Items.Add(menuItem);
});
OnMenuLoading(new MenuLoadingArgs(menuItem));
}
}
// C - Track Cursor
if (parameters.Substring(0,1) == "C")
{
if (parameters.Substring(1, 1)== "-")
{
// No Cursor
// Sent when entering player screen
await CoreApplication.MainView.CoreWindow.Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
{
...
State.Menu.Items.Clear();
OnMenuTitleInfoChanged(new MenuTitleInfoChangedArgs(State.Menu));
}
}
});
}
Like this it would just jump over the dispatcher for no apparent reason. If I put the entire thing in a single dispatcher it works fine.
A second question, if I have another event with a dispatcher, something like this:
foreach (xxx)
{
if (xxx == yyy)
{
await CoreApplication.MainView.CoreWindow.Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, async () =>
{
State.Menu.Items.Add(menuItem);
});
}
}
Would it be preferable to instead wrap the entire foreach loop in a dispatcher rather then calling it when needed each iteration?
Since my original question has changed I've made a new post with more specifics and another possible solution by just wrapping the original socket listener task in a dispatcher
Possible solution to issue with multiple UI dispatchers in the same method?
*** Update:
I think Raymond is on the right track, though adding Task didn't fix it, I noticed although it starts processing line "0" of the menu, before it sets up the new menu it tries to process the next line "1" command which is ignored because it doesn't have the right menu state yet, it still hasn't been set by the previous command yet.
I'm not sure how to fix it, it seems like I have to do an await at a lower level so be sure sure it full finishes one command before starting the next (and not sure why putting the whole ProcessNLS in UI dispatcher works), it's a little complicated since I go through multiple levels but here is the flow:
socket = new StreamSocket();
try
{
await socket.ConnectAsync(new HostName(HostName), Port);
OnConnect(new EventArgs());
await Task.Factory.StartNew(WaitForMessage);
}
catch (Exception ex)
{
OnConnectionFail(new EventArgs());
}
Goes to:
private async void WaitForMessage()
{
...
foreach (var message in messages)
{
if (string.IsNullOrWhiteSpace(message))
continue;
ProcessMessage(message);
}
}
Goes to
private void ProcessMessage(string message, string optionalFlags = "")
{
...
case "NLS": // NET/USB List Info
ProcessNLS(parameters);
break;
}
to finally
private async void ProcessNLS(string parameters) // NET/USB List Info
My alternate solution is to put to ProcessMessage call under WaitForMessage in a UI dispatcher
*** Update #2
I think this may be working, here is the updated flow, have to await multiple steps, use task instead of void
private async void WaitForMessage()
{
...
foreach (var message in messages)
{
if (string.IsNullOrWhiteSpace(message))
continue;
await ProcessMessage(message);
}
}
}
catch (Exception ex)
{
Debug.WriteLine("WaitForMessage Error: " + ex.Message);
OnDisconnect(new EventArgs());
}
}
to
private async Task ProcessMessage(string message, string optionalFlags = "")
{
...
case "NLS": // NET/USB List Info
await ProcessNLS(parameters);
break;
}
to
private async Task ProcessNLS(string parameters) // NET/USB List Info
The problem is here:
private async void ProcessNLS(...)
^^^^^^^^^^
You declared an async void function, which means "When the first await happens, return from the function immediately, and let the rest of the work run asynchronously." If you want the caller to be able to await on completion of your function, change the signature to private async Task ProcessNLS(...).
Ok, Trying to understand Rx, kinda of lost here.
FromAsyncPattern is now deprecated so I took the example from here (section Light up Task with Rx), and it works, I just made a few changes, not using await just wait the observable and subscribing.....
What I don't understand is Why is called Twice the function SumSquareRoots?
var res = Observable.FromAsync(ct => SumSquareRoots(x, ct))
.Timeout(TimeSpan.FromSeconds(5));
res.Subscribe(y => Console.WriteLine(y));
res.Wait();
class Program
{
static void Main(string[] args)
{
Samples();
}
static void Samples()
{
var x = 100000000;
try
{
var res = Observable.FromAsync(ct => SumSquareRoots(x, ct))
.Timeout(TimeSpan.FromSeconds(5));
res.Subscribe(y => Console.WriteLine(y));
res.Wait();
}
catch (TimeoutException)
{
Console.WriteLine("Timed out :-(");
}
}
static Task<double> SumSquareRoots(long count, CancellationToken ct)
{
return Task.Run(() =>
{
var res = 0.0;
Console.WriteLine("Why I'm called twice");
for (long i = 0; i < count; i++)
{
res += Math.Sqrt(i);
if (i % 10000 == 0 && ct.IsCancellationRequested)
{
Console.WriteLine("Noticed cancellation!");
ct.ThrowIfCancellationRequested();
}
}
return res;
});
}
}
The reason that this is calling SumSquareRoots twice is because you're Subscribing twice:
// Subscribes to res
res.Subscribe(y => Console.WriteLine(y));
// Also Subscribes to res, since it *must* produce a result, even
// if that result is then discarded (i.e. Wait doesn't return IObservable)
res.Wait();
Subscribe is the foreach of Rx - just like if you foreach an IEnumerable twice, you could end up doing 2x the work, multiple Subscribes means multiple the work. To undo this, you could use a blocking call that doesn't discard the result:
Console.WriteLine(res.First());
Or, you could use Publish to "freeze" the result and play it back to > 1 subscriber (kind of like how you'd use ToArray in LINQ):
res = res.Publish();
res.Connect();
// Both subscriptions get the same result, SumSquareRoots is only called once
res.Subscribe(Console.WriteLine);
res.Wait();
The general rule you can follow is, that any Rx method that doesn't return IObservable<T> or Task<T> will result in a Subscription(*)
* - Not technically correct. But your brain will feel better if you think of it this way.