Develop Reference

RavenDB: How to properly update a document at the same time? (two simultaneous API request to the same endpoint)

I have a C# REST API with an upload endpoint that has the sole purpose to process a binary file and add its metadata (as an Attachment model) to a List<Attachment> property of a different entity.
When I call the endpoint from my web application in a sequential manner like below (pseudo code), the endpoint does as intended and processes each binary file and adds a single Attachment to the provided entity.
const attachments = [Attachment, Attachment, Attachment];
for(const attachment of attachments) {
await this.api.upload(attachment);
}
But when I try to upload the attachments in a parallel manner like below (pseudo code), each binary file gets processed properly, but only one Attachment metadata object gets added to the entity.
const attachments = [Attachment, Attachment, Attachment];
const requests = attachments.map((a) => this.api.upload(a));
await Promise.all(requests);
The endpoint basically does the following (simplified):
var attachment = new Attachment()
{
// Metadata is collected from the binary (FormFile)
};
using (var session = Store.OpenAsyncSession())
{
var entity = await session.LoadAsync<Entity>(entityId);
entity.Attachments.Add(attachment);
await session.StoreAsync(entity);
await session.SaveChangesAsync();
};
I suspect that the problem is that the endpoint is called at the same time. Both request open (at the same time) a database session and query the entity into memory. They each add the Attachment to the entity and update it in the database. The saved attachment you see in the database is from the request that finishes last, e.g. the request that takes the longest.
I've tried to recreate the issue by creating this example. When you open the link, the example runs right away. You can see the created entities on this database server.
Open the Hogwarts database and after that open the contact Harry Potter and you see two attachments added. When you open the contact Hermione Granger you only see the one attachment added (the Second.txt), although it should also have both attachments.
What is the best approach to solve this issue? I prefer not having to send the files as a batch to the endpoint. Appreciate any help!
PS: You might need to run the example manually by clicking on Run. If the database doesn't exist on the server (as the server gets emptied automatically) you can create it manually with the Hogwarts name. And because it looks like a race condition, sometimes both Attachment items are added properly. So you might need to run the example a few times.

That is a a fairly classic example of a race condition in writing to the database, you are correct.
The sequence of event is:
Req 1 load doc Attachments = []
Req 1 load doc Attachments = []
Req 1 Attachments.Push()
Req 2 Attachments.Push()
Req 1 SaveChanges()
Req 2 SaveChanges()
The change in 5 overwrites the change in 4, so you are losing data.
There are two ways to handle this scenario. You can enable optimistic concurrency for this particular scenario, see the documentation on the topic:
https://ravendb.net/docs/article-page/4.2/csharp/client-api/session/configuration/how-to-enable-optimistic-concurrency#enabling-for-a-specific-session
Basically, you can do session.Advanced.UseOptimisticConcurrency = true; to cause the transaction to fail if the document was updated behind the scenes.
You can then retry the transaction to make it work (make sure to create a new session).
Alternatively, you can use the patching API, which will allow you to add an item to the document concurrently safely.
Here is the relevant documentation:
https://ravendb.net/docs/article-page/4.2/csharp/client-api/operations/patching/single-document#add-item-to-array
Note that there is a consideration here, you shouldn't care what the order of the operations are (because they can happen in any order).
If there is a business usecase behind the order, you probably cannot use the patch API easily and need to go with the full transaction route.

Using Force.com for arbitrary REST API methods

NB: This is a duplicate of my question on https://github.com/developerforce/Force.com-Toolkit-for-NET/issues/357. I'll sync both threads after this is resolved.
I'm using the Force.com toolkit for .NET. SOQL queries are working fine for me, but I'm trying to also make a simple REST call similar to one I can make with the REST Explorer tool at https://workbench.developerforce.com/restExplorer.php.
My code is:
using (var queryClient = new ForceClient(authClient.InstanceUrl, authClient.AccessToken, authClient.ApiVersion))
{
// Works fine
string soql = GetMyQuery();
QueryResult<dynamic> result = await queryClient.QueryAsync<dynamic>(soql);
// Throws exception. Changing version # doesn't seem to have any effect.
dynamic restResult = await queryClient.ExecuteRestApiAsync<dynamic>("/services/data/v46.0/sobjects/Task/describe");
When I get to ExecuteRestApiAsync, I get the exception:
Salesforce.Common.ForceException
HResult=0x80131500
Message=Could not find a match for URL
Source=Salesforce.Common
StackTrace:
at Salesforce.Common.JsonHttpClient.<HttpGetAsync>d__4`1.MoveNext()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
[...]
I assume that my argument "/services/data/v46.0/sobjects/Task/describe", which works fine in the REST Explorer tool, needs to change. But I've guessed a bunch of different options (different versions, with or without leading URL string, etc.) and so far haven't gotten anything to work.
Can someone tell me how to use ForceClient to get the same results I get back from the Salesforce Workbench REST Explorer?

I ended up looking more closely at the ForceClient code. ExecuteRestApiAsync was not the correct call for this. For my specific example, there are 3 ways I found that work:
dynamic restResult = await queryClient.DescribeAsync<dynamic>("Task");
restResult = await queryClient.BasicInformationAsync<dynamic>("Task/describe/");
restResult = await queryClient.BasicInformationAsync<dynamic>("Task/describe");
(All 3 of the above lines return the same result)
This will work fine for sobjects. I don't think it's currently possible to use this library directly for things like reports which use "analytics" in their REST API definitions, but there are a couple workarounds that don't require recompiling the code. The cleanest and most performant of these that I've found is to pass your own HttpClient instances into the ForceClient ctor and use those directly instead of the methods on the ForceClient. An inspection of the ForceClient code will reveal that most of these methods are just thin string.Format() wrappers on HttpClient methods anyway.

Is it possible to have more than one parse client configured in a single app?

So I have a few different parse servers setup.
One server is just to capture error logs from various applications (I have a LOT out there) in nice uniformed database.
So I might have a specific standalone data migration tool that if it encounters an error, will write out the exception into this Error_log parse table/class. No problem there.
But, if I have an app that uses a Parse Database for itself, I have not been able to figure out how to let it work on it's own parse server configuration for it's own stuff, but write out error logs to this other Parse server instance.
Yes... I could go through the trouble of writing out something via the REST api just for writing out logs,but I am I trying to avoid that and stick with native parse APIs for the particular platform I am on because of the benefits that the APIs give over REST (like save eventually for the none .NET stuff).
EDIT
Some clarification was requested so here I go...
On the app side of things (c# for this example but the same holds true for iOS etc)… I do the usual initialization of the Parse client as such …
ParseClient.Initialize(new ParseClient.Configuration
{
ApplicationId = "MyAppID",
WindowsKey = "MyDotNetKey",
Server = "www.myparseserver.com/app1"
});
So for all calls to save a parse object go through that parse client connection
But what I need to do would be something like this ….
//Main App cloud database
ParseClient1.Initialize(new ParseClient.Configuration
{
ApplicationId = "MyAppID",
WindowsKey = "MyDotNetKey",
Server = "www.myparseserver.com/app1"
});
ParseClient2.Initialize(new ParseClient.Configuration
{
ApplicationId = "MyAppID",
WindowsKey = "MyDotNetKey",
Server = "www.myparseserver.com/errorcollection"
});
try{
ParseConfig config = null;
config = await ParseConfig.GetAsync().ParseClient1;
} catch (Exception ex){
ParseObject MyError = new ParseObject("Error_Log");
MyError["Application"] = "My First App-App2";
MyError["Error"] = ex.message;
await MyError.Save().ParseClient2;
}
Yes - this is all fake code... my point is I want to be able to have multiple ParseClient instances in one app.
Now... I can simply write a routine that writes out errors that resets the ParseClient.Initialization to the error parse server instance and then redo it back to the original (primary app data) instance when it's done... but that is just asking for trouble in a multi threaded environment and will cause conflicts if some other thread in the app goes to write out parse data right at the moment the error method resets the init.
If ParseClient were IDisposable I could probably do that using :
ParseClient ParseErrorServer = new ParseClient();
ParseErrorServer.ApplicationId = "hmmm";
ParseErrorServer.WindwosKey= "hmmm";
ParseErrorServer.Server= "www.hmmm.com/errorcollection";
using ParseErrorServer {
//Do The Work
}
Is that clear as mud yet? ;P

Without alteration I believe none of the Parse SDKs have the ability to initialise multiple instances.
In the iOS SDK for example, is possible to make a new instance (say with a different server url) upon restarting the app but you cannot have multiple. There has also been discussion on the iOS SDK about being able to change the configuration without restart but no one has implemented this yet.
We would happily review a PR for this, however it would require a major and complex overhaul as you would have to manage cache, users etc across multiple instances.

Simplified LDAP/AD Server on C#

I've searched without much success to the simplest (and yet working) example of an LDAP/AD Server for C#. Many libraries exist to connect to LDAP servers, but not the LDAP Server by itself (on C#).
I found however some information about it and even a post requesting a simple LDAP server that was answered "LDAP isn't simple"; and yet i read a lot of the RFC4511 and this sample code at GitHub Flexinet LDAP Server, but unfortunatly i don't have yet the knowledge to complete it's code.
My goal is not to make a fully functional LDAP server, but one that can at least do:
Serve as a login pool for softwares that allow it's users to be
registered and log on a AD/LDAP server (just check for login and
password for authentication).
Allow softwares like Outlook and Thunderbird to get a list of users (without passwords) with first and last name, e-mail address, phone number and department for contact list model.
No delete, add (or create), move, and other
functions are required since the main software that i aim to
integrate it with will do all the user and group management.
UPDATE
I'm trying to implement the Flexinet sample and adjust to that functionalities; as form of a question what should i do to change this function to prevent it from causing an exception (on the "var filter = searchRequest.ChildAttributes[6];" line it always breaks) when i call from a LDAP client software:
private void HandleSearchRequest(NetworkStream stream, LdapPacket requestPacket)
{
var searchRequest = requestPacket.ChildAttributes.SingleOrDefault(o => o.LdapOperation == LdapOperation.SearchRequest);
var filter = searchRequest.ChildAttributes[6];
if ((LdapFilterChoice)filter.ContextType == LdapFilterChoice.equalityMatch && filter.ChildAttributes[0].GetValue<String>() == "sAMAccountName" && filter.ChildAttributes[1].GetValue<String>() == "testuser") // equalityMatch
{
var responseEntryPacket = new LdapPacket(requestPacket.MessageId);
var searchResultEntry = new LdapAttribute(LdapOperation.SearchResultEntry);
searchResultEntry.ChildAttributes.Add(new LdapAttribute(UniversalDataType.OctetString, "cn=testuser,cn=Users,dc=dev,dc=company,dc=com"));
searchResultEntry.ChildAttributes.Add(new LdapAttribute(UniversalDataType.Sequence));
responseEntryPacket.ChildAttributes.Add(searchResultEntry);
var responsEntryBytes = responseEntryPacket.GetBytes();
stream.Write(responsEntryBytes, 0, responsEntryBytes.Length);
}
var responseDonePacket = new LdapPacket(requestPacket.MessageId);
responseDonePacket.ChildAttributes.Add(new LdapResultAttribute(LdapOperation.SearchResultDone, LdapResult.success));
var responseDoneBytes = responseDonePacket.GetBytes();
stream.Write(responseDoneBytes, 0, responseDoneBytes.Length);
}
The code is on the github link.

Finally i made a fork of the Flexinet LDAP Server on #Sammuel-Miranda/LdapServerLib and with the author's support and some changes and adaptations i completed this implementation. It responds to the bind and search calls and works perfectly for Outlook and Thunderbird to use as a shared address book.
I did not implemente however any ADD/MODIFY/DELETE request (but would not be hard to do) since i don't need then.

I found on the RFC4511 the explanation on how the search works ... and i'm "kind" of understanding it, not very well - and i see that the method implemented on the GitHub from Flexinet LDAP Server only answer to bind and search requests of one single user (since it's only a example implementation).
The client is requesting diferent calls to verify capabilities, structure and other info before making the search request itself. So i'll implement it all, one by one.
Still, if any other lib (in C#) exists, and anyone know about, would be better than writing a hole new server. If my implementation works, i'll fork it on github and share.

Microsoft Dynamics CRM SDK CRMServiceClient connection string cache bug

Caching behavior of the last Dynamics SDK is driving me crazy.
First, if you want to use CrmServiceClient to access different environments you have to use the parameter RequireNewInstance=True; in the connection string. If not, every instance of CrmServiceClient will use the same connection, even if you create and dispose instances to different environments.
Now, even if you use the RequireNewInstance=True; in the connection string I found that cache still occurs in some scenarios.
var client1 = new CrmServiceClient("RequireNewInstance=True;
Url=https://myCompany.crm.dynamics.com;
Username=myUser#myCompany.onmicrosoft.com; Password=myPassowrd;
AuthType=Office365");
//Now, client 2 points to a url that doesn’t exists:
var client2 = new CrmServiceClient("RequireNewInstance=True;
Url=https://xxx.crm.dynamics.com; Username=myUser#myCompany.onmicrosoft.com;
Password=myPassowrd; AuthType=Office365");
The client2 keeps using the first connection string, so you cannot determine if the new connection string is correct.
Any ideas how to test Dynamics Crm connections strings correctly in my asp.net application?

Late reply, but the behavior you're seeing is because when you're specifying an erroneous URL the discovery service is used to ascertain which instance to connect to.
To prevent this specify SkipDiscovery=True in your connection string:
var connectionString2 = #"AuthType=Office365;Url=https://FAKE.crm.dynamics.com;Username=USERNAME;Password=PASSWORD;RequireNewInstance=True;SkipDiscovery=True;";
Edit: SkipDiscovery is true by default starting with 9.0.7, kudos to #mwardm

I think I found the problem. It seems to only happens on Dynamics 365 online trials, that was the reason we were getting inconsistent results depending on the environment.
Apparently, the url doesn't need to be completely valid to establish a connection to a CRM online trial environment, as long as the credentials are valid and the url structure is kept.
Let's consider the following example:
var client1 = new CrmServiceClient("RequireNewInstance=True;
Url=https://fake.crm.dynamics.com;
Username=myUser#myCompany.onmicrosoft.com; Password=myPassowrd;
AuthType=Office365");
In this case I can substitute the "fake" part of the url with whatever I want, but still execute requests correctly using the CrmServiceClient service.
If I try to do this with another environment (e.g. 2015, on premise, not-trial crm online, etc.), the IsReady property of the CrmServiceClient would return false and I would get an error in the LastCrmError property.
Very weird behavior, and difficult to pinpoint.
Now that I think I understand the inconsistent behavior I know that finally it will not affect our clients, so I will mark this response as the answer even if I still do not know why we have different behavior between a trial and a normal environment..

I agree choosing to reuse the existing connection if you don't include RequireNewInstance=true seems counter-intuitive, but I can't reproduce what you're seeing. If I try the following from LinqPad crmSvcClient2 will print out errors and then throw a null ref on the Execute call (8.2.0.2 version of the SDK). With this version of the SDK you'll want to always check LastCrmError after connecting to see if the connection failed.
var connectionString = #"AuthType=Office365;Url=https://REAL.crm.dynamics.com;Username=USERNAME;Password=PASSWORD;RequireNewInstance=True;";
var connectionString2 = #"AuthType=Office365;Url=https://FAKE.crm.dynamics.com;Username=USERNAME;Password=PASSWORD;RequireNewInstance=True;";
using (var crmSvcClient = new CrmServiceClient(connectionString))
{
"crmSvcClient".Dump();
crmSvcClient.LastCrmError.Dump();
((WhoAmIResponse)crmSvcClient.Execute(new WhoAmIRequest())).OrganizationId.Dump();
crmSvcClient.ConnectedOrgFriendlyName.Dump();
}
using (var crmSvcClient2 = new CrmServiceClient(connectionString2))
{
"crmSvcClient2".Dump();
crmSvcClient2.LastCrmError.Dump();
((WhoAmIResponse)crmSvcClient2.Execute(new WhoAmIRequest())).OrganizationId.Dump();
crmSvcClient2.ConnectedOrgFriendlyName.Dump();
}