Develop Reference

How to parallelize asynchronous integration work that includes JSON deserialization?

One question about parallel programming.
I need to POST 2 APIs (one after another) to get a Order list. First one is for getting token..
After I got the list, I need to POST 3 APIs (one after another) to integrate these Orders.
These 3 APIs don't accept arrays, so I need to send one by one. I cannot send batch data. With 1 thread it only integrates 10 orders in 1 minute. I need more performance. How can I run foreach part in parallel?
using System.Net.Http;
using System.Text;
using System.Text.Json;
namespace Order_Integrator
{
public class Program
{
static readonly HttpClient client = new HttpClient();
static async Task Main()
{
//Auth
var connectResponse = await client.PostAsync(connectUrl, connectContent);
var connectResponseString = await connectResponse.Content.ReadAsStringAsync();
var connect = JsonSerializer.Deserialize<connectResponse>(connectResponseString);
var token = connect.Token;
//Get Order List
var orderResponse = await client.PostAsync(orderUrl, orderContent);
var orderResponseString = await orderResponse.Content.ReadAsStringAsync();
var orders = JsonSerializer.Deserialize<orderResponse>(orderResponseString);
foreach (var order in orders)
{
//Get Order Details
//Generate getOrderDetailsContent with order
var getOrderDetailsResponse = await client.PostAsync(getOrderDetailsUrl, getOrderDetailsContent);
var getOrderDetailsResponseString = await getOrderDetailsResponse.Content.ReadAsStringAsync();
var getOrderDetails = JsonSerializer.Deserialize<getOrderDetailsResponse>(getOrderDetailsResponseString);
//Create Order
//Generate createOrderContent with GetOrderDetails
var createOrderResponse = await client.PostAsync(createOrderUrl, createOrderContent);
var createOrderResponseString = await createOrderResponse.Content.ReadAsStringAsync();
var createOrder = JsonSerializer.Deserialize<createOrderResponse>(createOrderResponseString);
//Create Log
//Generate createLogContent with CreateOrderResponse
var createLogResponse = await client.PostAsync(createLogUrl, createLogContent);
var createLogResponseString = await createLogResponse.Content.ReadAsStringAsync();
var createLog = JsonSerializer.Deserialize<createLogResponse>(createLogResponseString);
}
}
}
}

You can use Parallel.ForEachAsync
var options = new ParallelOptions()
{
MaxDegreeOfParallelism = 20
};
await Parallel.ForEachAsync(orders, options, async (OrderNumber, ct) => {
var getOrderDetailsResponse = await client.PostAsync(getOrderDetailsUrl, getOrderDetailsContent);
});

Since you're using asynchronous code, you should be using asynchronous concurrency, not (multithreaded) parallelism. Parallel.ForEachAsync will do both, but it's not available on older runtimes.
You can do just asynchronous concurrency by using Select and then Task.WhenAll:
var tasks = orders.Select(async order =>
{
//Get Order Details
//Generate getOrderDetailsContent with order
var getOrderDetailsResponse = await client.PostAsync(getOrderDetailsUrl, getOrderDetailsContent);
var getOrderDetailsResponseString = await getOrderDetailsResponse.Content.ReadAsStringAsync();
var getOrderDetails = JsonSerializer.Deserialize<getOrderDetailsResponse>(getOrderDetailsResponseString);
//Create Order
//Generate createOrderContent with GetOrderDetails
var createOrderResponse = await client.PostAsync(createOrderUrl, createOrderContent);
var createOrderResponseString = await createOrderResponse.Content.ReadAsStringAsync();
var createOrder = JsonSerializer.Deserialize<createOrderResponse>(createOrderResponseString);
//Create Log
//Generate createLogContent with CreateOrderResponse
var createLogResponse = await client.PostAsync(createLogUrl, createLogContent);
var createLogResponseString = await createLogResponse.Content.ReadAsStringAsync();
var createLog = JsonSerializer.Deserialize<createLogResponse>(createLogResponseString);
return createLog;
}).ToList();
await Task.WhenAll(tasks);

How to use findOne from mongoDB in C#

I used to use this command FindOne to query the result from mongoDB with C# in the past and it can be used at that time. But now I use the same code but it doesn't work. What should I use instead of FindOne?
My code is like this:
var connectionString = "mongodb://localhost";
var client = new MongoClient(connectionString);
var database = client.GetDatabase("extend");
var collection = database.GetCollection<Entity>("user");
var query = Query<Entity>.EQ(e => e.user_id, int.Parse(targetUser.CurrentUser));
var entity_TargetUser = collection.FindOne(query);
When I try to run it, I got this error
Error CS1061: 'IMongoCollection<Entity>' does not contain a definition for 'FindOne' and no extension method 'FindOne' accepting a first argument of type 'IMongoCollection<Entity>' could be found (are you missing a using directive or an assembly reference?) (CS1061)
What command that I can use instead of FindOne?

You are dealing with an id, so I assume each one is unique - if they aren't, they should be. Assuming user_id is unique then I would do the following
public static class MongoDataService
{
public static async Task<List<BsonDocument>> GetDocumentCollectionAsync(
MongoClient client, FilterDefinition<BsonDocument> filter,
string databaseName, string collectionName, CancellationToken token,
int? limit = null)
{
return await Task.Run(async () =>
{
long i = 1;
List<BsonDocument> items = new List<BsonDocument>();
var collection = GetCollection<BsonDocument>(client, databaseName, collectionName);
using (var cursor = await collection.FindAsync(filter))
{
while (await cursor.MoveNextAsync())
{
var batch = cursor.Current;
foreach (var doc in batch)
{
items.Add(doc);
if (token.IsCancellationRequested || i == limit)
return items;
i++;
}
}
}
return items;
}, token);
}
}
This method with the correct filter will return single documents, or can be used to return batches of documents again according to the imposed filter. Calling this method for your case, you can do
var filterBuilder = Builders<BsonDocument>.Filter;
var filter = filterBuilder.Eq("user_id", int.Parse(targetUser.CurrentUser));
var documents = await MongoDataService.GetDocumentCollectionAsync(client, filter, "extend", "user", token, null);
There are other methods to do what you want, but this should do what you want.
Note, I am assuming you are using the offical MongoDB.Driver.

you can do something like this:
var connectionString = "mongodb://localhost";
var client = new MongoClient(connectionString);
var database = client.GetDatabase("extend");
var collection = database.GetCollection<Entity>("user");
var query = Query<Entity>.EQ(e.user_id,int.Parse(targetUser.CurrentUser));
var entity_TargetUser = collection.AsQueryable().where(query).single();
or
var connectionString = "mongodb://localhost";
var client = new MongoClient(connectionString);
var database = client.GetDatabase("extend");
var collection = database.GetCollection<Entity>("user");
var entity_TargetUser = collection.AsQueryable().where(e=>e.user_id ==
int.Parse(targetUser.CurrentUser)).single();

Test connection with TLSharp

I'm trying to send a message with TLSharp but cant,i dont get errors,it just execute the code and do nothing;
This is my method.
public virtual async Task SendMessageTest()
{
string NumberToSendMessage = "+55199999999";
if (string.IsNullOrWhiteSpace(NumberToSendMessage))
throw new Exception("TESTE");
// this is because the contacts in the address come without the "+" prefix
var normalizedNumber = NumberToSendMessage.StartsWith("+") ?
NumberToSendMessage.Substring(1, NumberToSendMessage.Length - 1) :
NumberToSendMessage;
var client = NewClient();
var tsk = client.ConnectAsync();
await client.ConnectAsync();
var result = await client.GetContactsAsync();
var user = result.users.lists
.OfType<TLUser>()
.FirstOrDefault(x => x.phone == normalizedNumber);
if (user == null)
{
throw new System.Exception("Number was not found in Contacts List of user: " + NumberToSendMessage);
}
await client.SendTypingAsync(new TLInputPeerUser() { user_id = user.id });
Thread.Sleep(3000);
await client.SendMessageAsync(new TLInputPeerUser() { user_id = user.id }, "TEST");
}
This is my code,is says is wait for activation,what should i do?
I'm trying to use this method also,but it doenst return nothing too.
I'm new to TelegramApi,what i'm doing wrong?
await client.ConnectAsync();

You should authorize first! And only after that you can call other methods. Look at the examples here.
In general you should write something like this:
var hash = await client.SendCodeRequestAsync(NotRegisteredNumberToSignUp);
var code = Console.ReadLine(); //Input the code, that was sent to your phone
var loggedInUser = await client.MakeAuthAsync(NotRegisteredNumberToSignUp, hash, code);

Load text from web during xml parsing in Windows 8 Async

I have a unique issue, I want to get the name of an application from it's AppID while I convert an XML file into objects. This is the code I'm using at present:
if (xdoc.Element("Application") != null)
{
var data = from query in xdoc.Descendants("AppID")
select new APP
{
AppID = query.Value,
AppName = GetName(query.Value).ToString(),
};
itemGridView.DataContext = data;
}
This is the code I'm using to convert the GUID into an app name using Microsoft's Store API. I can confirm that it does return the app name. I'm just unsure how I can get this to display.
private async Task<string> GetName(string guid)
{
try
{
string link = "https://services.apps.microsoft.com/browse/6.2.9200-1/615/en-NZ_en-NZ/c/NZ/cp/10005001/Apps/{0}";
string url = string.Format(link, guid);
var httpClient = new HttpClient();
var httpRequestMessage = new HttpRequestMessage(HttpMethod.Get, url);
var response = await httpClient.SendAsync(httpRequestMessage);
var xmlString = await response.Content.ReadAsStringAsync();
XmlDocument NameXML = new XmlDocument();
NameXML = await XmlDocument.LoadFromUriAsync(new Uri(url));
string sAppName = NameXML.GetElementsByTagName("T")[0].ChildNodes[0].NodeValue.ToString();
return sAppName;
}
catch(Exception)
{
return guid;
}
}
I think my problem is with the async / await tasks. I've just been exposed to it now... how would I load up the App Name alongside the AppID when I parse the xml file?
The output that's being displayed when I run the app is "System.Threading.Tasks.Task[System.String]" (The objects load and the links and everything works fine, its just that the above is displayed instead of the app name).
I've been debugging using breakpoints, it appears that the GetName method only seems to be triggered later on, I'm not sure however.

Try to change this line :
AppName = GetName(query.Value).ToString(),
To this :
AppName = await GetName(query.Value),
GetName will return Task<string> instead of string when not awaited. And the method where above code resides required to be async because of using await inside that method :
private async void SomeMethod()
{
....
if (xdoc.Element("Application") != null)
{
var data = from query in xdoc.Descendants("AppID")
select new APP
{
AppID = query.Value,
AppName = await GetName(query.Value),
};
itemGridView.DataContext = data;
}
....
}
UPDATE :
As you already noticed, LINQ has very limited support for async/await currently. So to workaround this limitation, we can use normal for loop to avoid calling async function inside LINQ :
private async void SomeMethod()
{
....
if (xdoc.Element("Application") != null)
{
var query = from query in xdoc.Descendants("AppID")
select query.Value;
var data = new List<App>();
foreach (var q in query)
{
data.Add(new App{ AppId = q, AppName = await GetName(q) });
}
itemGridView.DataContext = data;
}
....
}

C# Google drive sdk. How to get a list of google drive folders?

I'm writing a program to allow a user to upload files to their Google Drive account. I have the upload part working and am using OAuth2. The issue I'm currently having is getting a list of folders from the users Drive account.
I found some code that is supposed to do this using the .setUserCredentials method, but it doesn't work:
DocumentsService service1 = new DocumentsService("project");
service1.setUserCredentials("user","pass");
FolderQuery query1 = new FolderQuery();
// Make a request to the API and get all documents.
DocumentsFeed feed = service1.Query(query1);
// Iterate through all of the documents returned
foreach (DocumentEntry entry in feed.Entries)
{
var blech = entry.Title.Text;
}
Nothing is returned. Ideally, I want to use OAuth2 to do this. I've been trying with the following code, trying to set the authentication token, but I always get denied access:
String CLIENT_ID = "clientid";
String CLIENT_SECRET = "secretid";
var docprovider = new NativeApplicationClient(GoogleAuthenticationServer.Description, CLIENT_ID, CLIENT_SECRET);
var docstate = GetDocAuthentication(docprovider);
DocumentsService service1 = new DocumentsService("project");
service1.SetAuthenticationToken(docstate.RefreshToken);
FolderQuery query1 = new FolderQuery();
DocumentsFeed feed = service1.Query(query1); //get error here
// Iterate through all of the documents returned
foreach (DocumentEntry entry in feed.Entries)
{
// Print the title of this document to the screen
var blech = entry.Title.Text;
}
..
private static IAuthorizationState GetDocAuthentication(NativeApplicationClient client)
{
const string STORAGE = "storagestring";
const string KEY = "keystring";
string scope = "https://docs.google.com/feeds/default/private/full/-/folder";
// Check if there is a cached refresh token available.
IAuthorizationState state = AuthorizationMgr.GetCachedRefreshToken(STORAGE, KEY);
if (state != null)
{
try
{
client.RefreshToken(state);
return state; // Yes - we are done.
}
catch (DotNetOpenAuth.Messaging.ProtocolException ex)
{
}
}
// Retrieve the authorization from the user.
state = AuthorizationMgr.RequestNativeAuthorization(client, scope);
AuthorizationMgr.SetCachedRefreshToken(STORAGE, KEY, state);
return state;
}
Specifically, I get "Execution of request failed: https://docs.google.com/feeds/default/private/full/-/folder - The remote server returned an error: (401) Unauthorized".
I've also tried:
var docauth = new OAuth2Authenticator<NativeApplicationClient>(docprovider, GetDocAuthentication);
DocumentsService service1 = new DocumentsService("project");
service1.SetAuthenticationToken(docauth.State.AccessToken);
but "State" is always null, so I get a null object error. What am I doing wrong and how is this done?

You should use the Drive SDK, not the Documents List API, which allows you to list folders. You can use "root" as a folderId if you want to list the root directory.

I actually implemented the v3 version of the GDrive SDK for .NET and needed to search for folders as well.
I prefer requesting uniquely all folders instead of getting all files and then performing a LinQ query to keep just the folders.
This is my implementation:
private async Task<bool> FolderExistsAsync(string folderName)
{
var response = await GetAllFoldersAsync();
return response.Files
.Where(x => x.Name.ToLower() == folderName.ToLower())
.Any();
}
private async Task<Google.Apis.Drive.v3.Data.FileList> GetAllFoldersAsync()
{
var request = _service.Files.List();
request.Q = "mimeType = 'application/vnd.google-apps.folder'";
var response = await request.ExecuteAsync();
return response;
}
You could request the name on the Q this way as well:
request.Q = $"mimeType = 'application/vnd.google-apps.folder' and name = '{folderName}'";
Which would lead and simplify things to (obviating null checking):
private async Task<bool> FolderExistsAsync(string folderName)
{
var response = await GetDesiredFolder(folderName);
return response.Files.Any();
}
private async Task<FileList> GetDesiredFolder(string folderName)
{
var request = _service.Files.List();
request.Q = $"mimeType = 'application/vnd.google-apps.folder' and name = '{folderName}'";
var response = await request.ExecuteAsync();
return response;
}

private IEnumerable<DocumentEntry> GetFolders(string id) {
if (IsLogged) {
var query = new FolderQuery(id)
{
ShowFolders = true
};
var feed = GoogleDocumentsService.Query(query);
return feed.Entries.Cast<DocumentEntry>().Where(x => x.IsFolder).OrderBy(x => x.Title.Text);
}
return null;
}
...
var rootFolders = GetFolders("root");
if (rootFolders != null){
foreach(var folder in rootFolders){
var subFolders = GetFolders(folder.ResourceId);
...
}
}
where GoogleDocumentsService is a instance of DocumentsService and IsLogged is a success logged flag.

I got this way to get list of folders from google drive
FilesResource.ListRequest filelist= service.Files.List();
filelist.Execute().Items.ToList().Where(x => x.MimeType == "application/vnd.google-apps.folder").ToList()