Develop Reference

Dynamics Crm Bulk Update records in a transaction

Requirement
I have the requirement where I want to update few fields on account
and create contact for it reading data from an API.
The number of records to be updated is around 100,000 so I want to
use either ExecuteTransactionRequest or ExecuteMultipleRequest so that I can execute all in batches.
Since I want the contact record to be created for the account updated I used the ExecuteTransactionRequest.
Problem -
The problem is batch size.
I have added the condition if request batch size count equals 500 then execute all the requests. However my batch can include
Update request for account and
Create request for contact
So it may happen that the batch may not be exact 500 and it would skip the execute request. How can I do this and make sure that contact record is created for each Updated Account correctly.
Any help would be appreciated. Thanks in Advance
Below is my code --
var requests = new ExecuteTransactionRequest
{
Requests = new OrganizationRequestCollection(),
ReturnResponses = returnResponses
};
foreach (var customer in customerList)
{
string custNo = customer.GetAttributeValue<string>("customernumber");
// Gets customer details from another api
var custInfo = await CustomerService.Get(custNo);
// Update the values on customer table
Entity cust = new Entity("account");
cust.Id = customer.Id;
cust["companytypecode"] = custInfo.EntityTypeCode;
cust["companytypedescription"] = custInfo .EntityTypeDescription;
var roles = custInfo.Roles.Where(c => c.RoleStatus == "ACTIVE").ToArray();
//create contact for each account
foreach(var role in roles)
{
Entity contact = new Entity("contact");
contact["FirstName"] = role.RolePerson?.FirstName;
contact["MiddleName"] = role.RolePerson?.MiddleNames;
contact["LastName"] = role.RolePerson?.LastName;
contact["AccountId"] = new EntityReference("account", customer.Id);
CreateRequest createRequest = new CreateRequest { Target = contact };
requests.Requests.Add(createRequest);
}
UpdateRequest updateRequest = new UpdateRequest { Target = cust };
requests.Requests.Add(updateRequest);
if (requests.Requests.Count == 500) // Problem is the batch size will not execute per account since it also has create request of contact. How can i make sure that each request is executed correctly
{
service.Execute(requests);
requests.Requests.Clear();
}
}
// For the last remaining accounts
if (requests.Requests.Count > 0)
{
service.Execute(requests);
}

Thank you for helping out. I resolved this with below solution. Happy to be corrected.
EntityCollection requestsCollection = new EntityCollection();
foreach (var customer in customerList)
{
string custNo= customer.GetAttributeValue<string>("customernumber");
var custInfo = await businessService.Get(custNo);
Entity cust = new Entity("account");
cust.Id = customer.Id;
cust["companytypecode"] = custInfo.EntityTypeCode;
cust["companytypedescription"] = custInfo .EntityTypeDescription;
requestsCollection.Entities.Add(cust);
var roles = custInfo.Roles.Where(c => c.RoleStatus == "ACTIVE").ToArray();
foreach(var role in roles)
{
Entity contact = new Entity("contact");
contact["FirstName"] = role.RolePerson?.FirstName;
contact["MiddleName"] = role.RolePerson?.MiddleNames;
contact["LastName"] = role.RolePerson?.LastName;
contact["AccountId"] = new EntityReference("account", customer.Id);
requests.Entities.Add(contact);
}
if (requestsCollection.Entities.Count > 500)
{
ExecuteBulkUpdate(requestsCollection);
requestsCollection = new EntityCollection();
}
}
private void ExecuteBulkUpdate(EntityCollection requestsCollection)
{
var requests = new ExecuteTransactionRequest
{
Requests = new OrganizationRequestCollection(),
ReturnResponses = returnResponses
};
foreach (var request in requestsCollection.Entities)
{
if (request.Id != Guid.Empty)
{
UpdateRequest updateRequest = new UpdateRequest { Target = request };
requests.Requests.Add(updateRequest);
}
else
{
CreateRequest createRequest = new CreateRequest { Target = request };
requests.Requests.Add(createRequest);
}
}
try
{
var responseForCreateRecords = (ExecuteTransactionResponse)service.Execute(requests);
int i = 0;
// Display the results returned in the responses.
foreach (var responseItem in responseForCreateRecords.Responses)
{
if (responseItem != null)
log.LogInformation(responseItem.Results["id"].ToString());
i++;
}
requests.Requests.Clear();
}
catch (FaultException<OrganizationServiceFault> ex)
{
log.LogInformation("Request failed for the {0} and the reason being: {1}",
((ExecuteTransactionFault)(ex.Detail)).FaultedRequestIndex + 1, ex.Detail.Message);
throw;
}
}

ASP.NET Core Large Data Set Processing

I have an Asp.net core 2.1 application with a Postgresql database.
A large file (~100K lines) gets uploaded to the database and then each line is processed.
As each line is processed there are approximately 15 sub queries per line taking up about 10 seconds in total per line. I have tried optimizing the queries and loading certain datasets in memory to reduce database calls which has seen a slight improvement. I have also tried reducing the batch size but that has also had minimal reduction.
Are there any other ways I can speed this up? The queries are written in lambda.
The high level function:
public void CDR_Insert(List<InputTable> inputTable, IQueryable<Costing__Table> CostingTable, List<Patterns> PatternsTable, IQueryable<Country_Calling_Codes> CallingCodesTable, IQueryable<Country> CountryTable, string voicemail_number)
{
foreach (var row in inputTable)
{
//if (!string.IsNullOrEmpty(row.finalCalledPartyPattern))
//{
var rowexists = _context.CDR.Where(m => m.InputId == row.id).Count();
if (rowexists == 0)
{
if (ValidateIfOneNumberIsForCountry(row.callingPartyNumber, row.finalCalledPartyNumber, PatternsTable, CountryTable) == true)
{
var CallRingTime = TimeSpan.FromSeconds(Ring_Time(row.dateTimeOrigination, row.dateTimeConnect));
var Call_Direction = CallDirection(row.callingPartyNumber, row.finalCalledPartyNumber, row.finalCalledPartyPattern, voicemail_number);
var CallReason = "Destination Out Of Order";
if (row.dateTimeConnect != "0")
{
CallReason = Call_Reason(row.callingPartyNumber, row.originalCalledPartyNumber, row.finalCalledPartyNumber, voicemail_number);
}
var CallOutcome = Outcome(row.duration, row.dateTimeOrigination, row.dateTimeConnect, row.callingPartyNumber, row.originalCalledPartyNumber, row.finalCalledPartyNumber, voicemail_number);
var CallCategory = GetCategory(row.callingPartyNumber, row.finalCalledPartyNumber, PatternsTable, CallingCodesTable, CountryTable);
var CallCost = 0.00;
if (CallOutcome != "Not Connected" || CallOutcome != "No answer")
{
CallCost = Costing(row.callingPartyNumber, row.finalCalledPartyNumber, row.dateTimeConnect, row.dateTimeDisconnect, row.finalCalledPartyPattern, row.duration, Call_Direction, CostingTable, PatternsTable, voicemail_number, CallingCodesTable, CountryTable);
}
CDR cDR = new CDR()
{
Date_Time = UnixTimeStampToDateTimeConvertor2(row.dateTimeOrigination),
Call_Direction = Call_Direction,
Calling_Party = row.callingPartyNumber_uri,
Calling_Digits = row.callingPartyNumber,
Called_Party = row.finalCalledPartyNumber_uri,
Called_Digits = row.finalCalledPartyNumber,
Duration = TimeSpan.FromSeconds(Convert.ToDouble(row.duration)),
Cost = CallCost,
Reason = CallReason,
Outcome = CallOutcome,
Country_Code = "MUS",//_context.Country.Where(o => o.CountryCode == _CDRExtension.CountryCodeId(_CDRExtension.Call_Type_Id(row.callingPartyNumber))).Select(o => o.CountryCode).FirstOrDefault(),
Second_Ring_Time = CallRingTime,
Category = CallCategory,
CiscoInputId = row.id
};
_context.CDR.Add(cDR);
}
row.Processed = true;
row.DateAdded = DateTime.Now;
_context.Cisco_Input.Update(row);
// }
_context.SaveChanges();
}
}
}

ASP.NET stripe.net Cannot charge a customer that has no active card

protected void ChargePayment(object sender, EventArgs e)
{
StripeCustomer CurrentCustomer = GetCustomer();
if (CurrentCustomer == null)
{
return;
}
var myCharge = new StripeChargeCreateOptions();
myCharge.Currency = "dkk";
myCharge.CustomerId = CurrentCustomer.Id;
myCharge.Description = "KPHF Prorated Charge";
string key = "sk_test_P6GjMq1OVwmxZv5YNozAX6dY";
var chargeService = new StripeChargeService(key);
try
{
chargeService.Create(myCharge);
}
catch (StripeException ex)
{
exError.Text = ex.Message;
}
}
private StripeCustomer GetCustomer()
{
MembershipUser CurrentUser = Membership.GetUser();
var myCustomer = new StripeCustomerCreateOptions();
var myCustomer2 = new StripeCreditCardOptions();
myCustomer.Email = cMail.Text;
myCustomer2.TokenId = CreditCard.Text;
myCustomer2.ExpirationMonth = CardMonth.SelectedItem.Text;
myCustomer2.ExpirationYear = CardYear.SelectedItem.Text;
myCustomer2.Cvc = cvc.Text;
myCustomer.PlanId = "1";
var customerService = new StripeCustomerService("sk_test_P6GjMq1OVwmxZv5YNozAX6dY");
try
{
StripeCustomer result = customerService.Create(myCustomer);
return result;
}
catch (StripeException ex)
{
exError.Text = ex.Message;
return null;
}
After entering credit card info I do get customer created in the stripe system, but he's not being charged and I get following exception: "Cannot charge a customer that has no active card". Any help or tips?

you are not attaching the card with the customer. You have created the card object but not attached with the customer. Follow this syntax to add the card
var myCustomer = new StripeCustomerCreateOptions();
myCustomer.Email = "pork#email.com";
myCustomer.Description = "Johnny Tenderloin (pork#email.com)";
// setting up the card
myCustomer.SourceCard = new SourceCard()
{
Number = "4242424242424242",
ExpirationYear = "2022",
ExpirationMonth = "10",
Cvc = "1223" // optional
};
myCustomer.PlanId = *planId*; // only if you have a plan
myCustomer.TaxPercent = 20; // only if you are passing a plan, this tax percent will be added to the price.
myCustomer.Coupon = *couponId*; // only if you have a coupon
myCustomer.TrialEnd = DateTime.UtcNow.AddMonths(1); // when the customers trial ends (overrides the plan if applicable)
myCustomer.Quantity = 1; // optional, defaults to 1
var customerService = new StripeCustomerService();
StripeCustomer stripeCustomer = customerService.Create(myCustomer)
you can read more about it here stripe .net

CRM Dynamics 2013 SDK Update Current Accounts With 2 Values

I have a scenario in CRM where I need to update existing accounts with their Vat and Registration number. There is well over 30 thousand accounts in the system. I am trying to update using the CRM SDK API but I am battling to figure out how to perform the actual update. The vat number and reg have been provided to me in a spreadsheet with their corresponding number, please note that the accounts are already in CRM so I just need to update the correct account with its Vat and Reg number, How can I do this in CRM, please advice on my code below:
public static void UpdateAllCRMAccountsWithVATAndRegistrationNumber(IOrganizationService service)
{
QueryExpression qe = new QueryExpression();
qe.EntityName = "account";
qe.ColumnSet = new ColumnSet("account", "new_vatno", "new_registrationnumber");
qe.Criteria.AddCondition("accountnumber", ConditionOperator.In,"TA10024846", "TA10028471", "TA20014015", "TA4011652", "TA4011557");
EntityCollection response = service.RetrieveMultiple(qe);
foreach (var acc in response.Entities)
{
acc.Attributes["new_vatno"] = //this is where I am struggling to figure out how I am gong to match the records up,
acc.Attributes["new_registrationnumber"] = //this is where I am struggling to figure out how I am gong to match the records up,
service.Update(acc);
}
}
How am I going to ensure that I update the correct records. I have the vat and reg numbers for the accounts in a spreadsheet, please see example image below. Can I please get advised here. Thanks.

I would load the list of VAT updates from the spreadsheet into a dictionary and then load the 30k record from CRM into memory. Then I would match them up and use ExecuteMultipleRequest to do the updates. Alternatively, you could query CRM using the account numbers (if the list is small enough.) I made the assumption you had thousands of updates to do across the record set of 30k. Note, if the Account record size was very large and couldn't be loaded into memory you would need to do account number queries.
Here is the rough code for the basic solution (I haven't tested, method should be split up, and there is minimal error handling):
public class VatInfo
{
public string RegistrationNumber;
public string TaxNumber;
public static Dictionary<string, VatInfo> GetVatList()
{
//TODO: Implement logic to load CSV file into a list. Dictionary key value should be Account Number
throw new NotImplementedException();
}
}
public class UpdateVatDemo
{
public const int maxBatchSize = 100;
public static void RunVatUpdate(IOrganizationService conn)
{
var vats = VatInfo.GetVatList();
var pagingQuery = new QueryExpression("account");
pagingQuery.ColumnSet = new ColumnSet("accountnumber");
Queue<Entity> allEnts = new Queue<Entity>();
while (true)
{
var results = conn.RetrieveMultiple(pagingQuery);
if (results.Entities != null && results.Entities.Any())
results.Entities.ToList().ForEach(allEnts.Enqueue);
if (!results.MoreRecords) break;
pagingQuery.PageInfo.PageNumber++;
pagingQuery.PageInfo.PagingCookie = results.PagingCookie;
}
ExecuteMultipleRequest emr = null;
while (allEnts.Any())
{
if (emr == null)
emr = new ExecuteMultipleRequest()
{
Settings = new ExecuteMultipleSettings()
{
ContinueOnError = true,
ReturnResponses = true
},
Requests = new OrganizationRequestCollection()
};
var ent = allEnts.Dequeue();
if (vats.ContainsKey(ent.GetAttributeValue<string>("accountnumber")))
{
var newEnt = new Entity("account", ent.Id);
newEnt.Attributes.Add("new_vatno", vats[ent.GetAttributeValue<string>("accountnumber")].TaxNumber);
newEnt.Attributes.Add("new_registrationnumber", vats[ent.GetAttributeValue<string>("accountnumber")].RegistrationNumber);
emr.Requests.Add(new UpdateRequest() { Target = newEnt });
}
if (emr.Requests.Count >= maxBatchSize)
{
try
{
var emResponse = (ExecuteMultipleResponse) conn.Execute(emr);
foreach (
var responseItem in emResponse.Responses.Where(responseItem => responseItem.Fault != null))
DisplayFault(emr.Requests[responseItem.RequestIndex],
responseItem.RequestIndex, responseItem.Fault);
}
catch (Exception ex)
{
Console.WriteLine($"Exception during ExecuteMultiple: {ex.Message}");
throw;
}
emr = null;
}
}
}
private static void DisplayFault(OrganizationRequest organizationRequest, int count,
OrganizationServiceFault organizationServiceFault)
{
Console.WriteLine(
"A fault occurred when processing {1} request, at index {0} in the request collection with a fault message: {2}",
count + 1,
organizationRequest.RequestName,
organizationServiceFault.Message);
}
}

Updating the fetched entity is bound to fail because of its entity state, which would not be null.
To update the fetched entities, you need to new up the entity:
foreach (var acc in response.Entities)
{
var updateAccount = new Entity("account") { Id = acc.Id };
updateAccount .Attributes["new_vatno"] = null; //using null as an example.
updateAccount .Attributes["new_registrationnumber"] = null;
service.Update(acc);
}

Code below shows how I managed to get it righy. forst let me explain. I imported my records into a seperate SQL table, in my code I read that table into a list in memory, I then query CRM accounts that need to be updated, I then loop though each account and check if the account number in CRM matches the account number from my sql database, if it matches, I then update the relevant Reg no and Vat no, See code below:
List<Sheet1_> crmAccountList = new List<Sheet1_>();
//var crmAccount = db.Sheet1_.Select(x => x).ToList().Take(2);
var crmAccounts = db.Sheet1_.Select(x => x).ToList();
foreach (var dbAccount in crmAccounts)
{
CRMDataObject modelObject = new CRMDataObject()
{
ID = dbAccount.ID,
Account_No = dbAccount.Account_No,
Tax_No = dbAccount.Tax_No.ToString(),
Reg_No = dbAccount.Reg_No
//Tarsus_Country = dbAccount.Main_Phone
};
}
var officialDatabaseList = crmAccounts;
foreach (var crmAcc in officialDatabaseList)
{
QueryExpression qe = new QueryExpression();
qe.EntityName = "account";
qe.ColumnSet = new ColumnSet("accountnumber", "new_vatno", "new_registrationnumber");
qe.Criteria.AddCondition("accountnumber", ConditionOperator.In,'list of account numbers go here'
EntityCollection response = service.RetrieveMultiple(qe);
foreach (var acc in response.Entities)
{
if (crmAcc.Account_No == acc.Attributes["accountnumber"].ToString())
{
//acc.Attributes["new_vatno"] = crmAcc.VAT_No.ToString();
acc.Attributes["new_registrationnumber"] = crmAcc.Reg_No.ToString();
service.Update(acc);
}
}
}

Foreach is not checking properly in mvc when doing a workflow

Am doing a workflow cheching in which i have 2 values and the when the foreach condition is checked only one time it enters the loop and exits out without going to the next one.
public CustomBusinessServices InvokeWorkFlowPermissionBusinessRule(dynamic workFlowImplemented, out string serviceName, out int permissionId)
{
try
{
List<WorkflowEligibilityMapping> workFlowPermissionService = new List<WorkflowEligibilityMapping>();// to handle null values
int current_ControllerId = Convert.ToInt32(workFlowImplemented); //ControllerId
using (var db = new AdminDb())
{
//to select services against this controller
workFlowPermissionService = (from definition in db.WorkFlowDefinition.AsNoTracking()
join model in db.WorkFlowModel.AsNoTracking()
on definition.WorkFlowDefinitionId equals model.WorkFlowDefinitionId
join permission in db.WorkFlowPermission.AsNoTracking()
on model.WorkFlowDefinitionId equals permission.WorkFlowDefinitionId
where model.ControllerNameId.Equals(current_ControllerId)
select new WorkflowEligibilityMapping
{
Service = permission.Service,
WorkFlowPermissionId = permission.WorkFlowPermissionId
}).ToList();
}
int[] workFlowServiceDetails = workFlowPermissionService.Select(x => x.WorkFlowPermissionId).ToArray();
//to Login userId
var userId = Assyst.PanERP.Common.AppSession.Common.UserID;
/*******************Issue in foreach i think**************************************/
foreach (int workFlowServiceDetail in workFlowServiceDetails)
/*******workFlowServiceDetails have 2 valus********/
{
using (var db = new AdminDb())
{
string workFlowServiceDtl = (from perm in db.WorkFlowPermission.AsNoTracking()
where perm.WorkFlowPermissionId == workFlowServiceDetail
select perm.Service).FirstOrDefault();
//to select eligibility rules against this service
string eligibility = (from definition in db.WorkFlowDefinition.AsNoTracking()
join model in db.WorkFlowModel.AsNoTracking()
on definition.WorkFlowDefinitionId equals model.WorkFlowDefinitionId
join permission in db.WorkFlowPermission.AsNoTracking()
on model.WorkFlowDefinitionId equals permission.WorkFlowDefinitionId
where model.ControllerNameId.Equals(current_ControllerId) && permission.WorkFlowPermissionId == workFlowServiceDetail
select permission.EligibilityRule).FirstOrDefault();
if (eligibility == null)
{
string validationMessage = "";
validationMessage = "Please set eligibility for workflow permission";
serviceName = null;
permissionId = 0;
return new CustomBusinessServices() { strMessage = validationMessage };
}
string[] strTxt = workFlowServiceDtl.Split(';'); //split the service name by ';' and strore it in an array
string serviceUrl = string.Empty;
string workFlowServiceName = string.Empty;
string classpath = string.Empty;
workFlowServiceName = strTxt[0].ToString();
workFlowServiceName = workFlowServiceName.Replace(" ", "");//get the service name by removing empty blank space for the word
classpath = strTxt[1].ToString();
//Invoke REST based service (like Node.Js service)
if (strTxt.Length == 4)
{
serviceUrl = strTxt[3].ToString();
}
//Invoke c# based service
else
{
serviceUrl = string.Empty;
}
var userLists = PermissionCallMethod(classpath, workFlowServiceName, new[] { workFlowImplemented, eligibility }, serviceUrl);
if (userLists.UserList.Contains(userId))
{
serviceName = strTxt[0].ToString() + ";Assyst.PanERP.Common.WorkFlowNotificationServices;" + strTxt[2].ToString();
permissionId = workFlowServiceDetail;
return userLists;
}
}
}
serviceName = string.Empty;
permissionId = 0;
return null;
}
catch (Exception ex)
{
throw ex;
return null;
}
}
workFlowServiceDetails have 2 values and the workFlowServiceDetail takes the first one and checks for it.goes through the loop and mapes the role for the first one to the user list at the end and the without checking the for the second vale it moves out of the loop. Please help me to make the loop work for 2 values.Is it some problem in the return part...?

if (eligibility == null)
{
string validationMessage = "";
validationMessage = "Please set eligibility for workflow permission";
serviceName = null;
permissionId = 0;
return new CustomBusinessServices() { strMessage = validationMessage };
}
if (userLists.UserList.Contains(userId))
{
serviceName = strTxt[0].ToString() + ";Assyst.PanERP.Common.WorkFlowNotificationServices;" + strTxt[2].ToString();
permissionId = workFlowServiceDetail;
return userLists;
}
If any of the above if statements evaluates to true, your loop will exit without looping through the second item in your array. The reason for this is that you are in your first conditional check do the following:
return new CustomBusinessServices() { strMessage = validationMessage };
And in your second:
return userLists;
The return statement will exit your method, and therefore terminate the foreach as well.
Try building your object first, and after your loop has walked through each item, do a return statement returning your object.