Develop Reference

C# NetSuite WebServices: Get value from custom field in saved search (ItemSearchAdvanced)

I'm using C# MVC to connect to NetSuite using their WebServices API. I have some current code that calls a saved search of inventory items. Here is the current code that is working perfectly:
ItemSearchAdvanced searchItems = new ItemSearchAdvanced();
searchItems.savedSearchId = "128";
SearchResult result = netSuiteSvc.search(searchItems);
int totalRecords = 0;
int processedRecords = 0;
UpdateNetsuitePriceListModel returnObj = new UpdateNetsuitePriceListModel();
returnObj.NewPriceList = new List<NetSuitePriceListRecord>();
if (result.status.isSuccess)
{
SearchRow[] searchRows = result.searchRowList;
if (searchRows != null && searchRows.Length >= 1)
{
for (int i = 0; i < searchRows.Length - 1; i++)
{
ItemSearchRow itemRow = (ItemSearchRow)searchRows[i];
if (itemRow.basic.itemId != null && itemRow.basic.mpn != null && itemRow.basic.basePrice != null && itemRow.basic.salesDescription != null)
{
returnObj.NewPriceList.Add(new NetSuitePriceListRecord()
{
ItemId = itemRow.basic.itemId[0].searchValue,
ManufacturerPartNumber = itemRow.basic.mpn[0].searchValue,
ContractPrice = Convert.ToDecimal(itemRow.basic.basePrice[0].searchValue),
Cost = CalculateProductCostForIngram(Convert.ToDecimal(itemRow.basic.basePrice[0].searchValue)),
Description = itemRow.basic.salesDescription[0].searchValue
});
processedRecords++;
}
totalRecords++;
}
}
else
{
throw new Exception("NetSuite Part List Blank");
}
}
else
{
throw new Exception("NetSuite Part List Search Failure");
}
Now I have need to pull the itemId from a custom added field rather than the default itemId field.
Obviously since this is a custom field it isn't a property of ItemSearchRowBasic. It looks like instead of the property I can choose "customFieldList" which is an array of "SearchColumnCustomField". If I choose an index for the array I can see that SearchColumnCustomField contains:
customLabel
internalId
scriptId
I imagine I should be able to get the internalId of the SearchColumnCustomField and somehow use that to get the search value for that custom column but I've had some trouble finding any examples that fit so far.
This custom field is a free form text field added to all inventory items.

Try setting scriptId with the ID of the field ("custitem_xyz"). That should work.
Before 2013 one would use internalId, but since then it changed to scriptId.

You would need to loop over the CustomRecord items in the customFieldList. I then usually check for a specific type so I can cast to the correct object, but with some reflection you could probably avoid that.
foreach (Record r in mySearchResponse.recordList){
foreach (CustomFieldRef cr in ((CustomRecord)r).customFieldList){
if (cr.GetType().Name == "SelectCustomFieldRef"){
if (((SelectCustomFieldRef)cr).scriptId == "my_custom_field"){
internalID = ((CustomRecord)r).internalId;
}
}
}
}

Remove duplicates and put the list into a listbox

This is an uni assignment and I am having problem with part of it. This is the code;
namespace Assignment_1
{
public partial class Classifier : System.Web.UI.Page // We are using a web form as stated
{
protected void Page_Load(object sender, EventArgs e) // No variables are initiated for the beginning
{
}
protected void ButtonClassify_Click(object sender, EventArgs e)
{
if (this.TextBox1.Text != "")
{
List<string> numbersText = this.TextBox1.Text.Split(',').ToList<string>();
foreach (var item in numbersText)
{
int num = int.Parse(item);
if (RadioButtonList1.SelectedValue == "Both")
{
if (num % 2 == 0)
{
if (CheckBoxDuplicate.Checked == true)
{
List<int> evenNumbers = new List<int>();
evenNumbers.Add(num);
List<int> distinctEvenNumbers = evenNumbers.Distinct().ToList();
ListBoxEvenNumbers.DataSource = distinctEvenNumbers;
}
else
{
//Put the results into the respective boxes
ListBoxEvenNumbers.Items.Add(num.ToString());
}
}
else
{
//Put the results into the respective boxes
ListBoxOddNumbers.Items.Add(num.ToString());
}
}
if (RadioButtonList1.SelectedValue == "Even")
{
if (num % 2 == 0)
{
//Put the results into the respective boxes
ListBoxEvenNumbers.Items.Add(num.ToString());
}
}
if (RadioButtonList1.SelectedValue == "Odd")
{
if (num % 2 == 1)
{
//Put the results into the respective boxes
ListBoxOddNumbers.Items.Add(num.ToString());
}
}
Let me explain the question and what I have done. User inserts list of numbers into a text box and then has 3 options (radiolistbutton). He can list even, odd or both type of numbers. They display in even and odd listboxes(2 listboxes). I have done up to this part.
There is a checkbox to remove duplicates and the user can check it if he wishes to. If the button is checked, the code should remove the duplicates. I tried to do this part in the 4th "if-else" "if (CheckBoxDuplicate.Checked == true)". The way I understand it, I check if the number is even and then check the CheckboxDuplicate button. if it is checked I put the values in a new list and then delete repeated values. Then put into EvenNumbers listbox. For some reason, this part doesn't work.
If you would like to help me, please don't post just your answer. This is my first project in C# and it is difficult for me to understand an elegant solution yet. If you have time, please check my code and let me know where I made a mistake. Thanks for your time in advance.

Sprinkle a bit of linq magic on it, and you're done.
var my_list = new List<int>{1,2,3,4,5,5,6,7};
var pair = my_list.Where (n => n%2 ==0); // 2,4,6
var odd = my_list.Where (n => n%2 ==1); // 1,3,5,5,7
var unique_pair = pair.Distinct(); // stays the same
var unique_odd = odd.Distinct(); // 1,3,5,7
From here is just adding it to your appropriate calls and GUI containers
From your comment, here are a couple of things:
Change the if to if - else if, since only one will apply.
you can do it the way you do, but it's not the most efficient. If you go that way, you'll have to figure out which numbers you've added in order to not have duplicates.
alternatively, you can simply create the lists like I've done in the code above, and then assign them at the end. It'll save you time and code.
Here's some more help, with no code, since I believe i covered it.
Step 1: get the user input, and create a list of ints. (call it: input_list).
Step 2: According to what he chose (even, odd, both), you want to assign to each listbox, a list of numbers. Look at my above code, it'll do that bit for you.
Step 3: If user choses unique, you pass to those listboxes the Distinct list, again, look at my above code for that.
You can apply the unique on the event of the checkbox being selected if you prefer.
Notes:
Keep the list of ints (the input_list) as a variable, so you don't need to parse it whenever he changes his selection.

public enum PairOddEnum
{
Evens,
Odds,
Both
}
public void BindControl(PairOddEnum type)
{
if (this.textBox1.Text != "")
{
List<string> numbersText = this.textBox1.Text.Split(',').ToList<string>();
var evens = numbersText.Where(t => int.Parse(t) % 2 == 0).Distinct();
var odds = numbersText.Where(t => int.Parse(t) % 2 == 1).Distinct();
if (type == PairOddEnum.Evens)
{
ListBoxEvenNumbers.DataSource = evens.ToList();
}
else if (type == PairOddEnum.Odds)
{
ListBoxOddNumbers.DataSource = odds.ToList();
}
else
{
ListBoxEvenNumbers.DataSource = evens.ToList();
ListBoxOddNumbers.DataSource = odds.ToList();
}
}
}
protected void ButtonClassify_Click(object sender, EventArgs e)
{
if (RadioButtonList1.SelectedValue == "Both")
{
BindControl(PairOddEnum.Both);
}
if (RadioButtonList1.SelectedValue == "Even")
{
BindControl(PairOddEnum.Evens);
}
if (RadioButtonList1.SelectedValue == "Odd")
{
BindControl(PairOddEnum.Odds);
}
}

Add to exisiting Database instance from MVC Controller

Private BookingDB db = new BookingDB();
Private MonthDb mdb = new MonthDB();
if (ModelState.IsValid)
{
String date = (booking.Start_Date).ToString();
var check = from b in mdb.months
where b.BookedDays.Contains(date)
select b;
if (check != null)
{
return View(booking);
}
else
{
booking.Reservation_Owner = User.Identity.Name;
//Add booking.Start_Date to mdb.Entry(check).BookedDays
mdb.SaveChanges();
db.bookings.Add(booking);
db.SaveChanges();
return RedirectToAction("Index");
}
}
I've got this code that on creation of a new booking, will check that no exisiting bookings have already been made on or around that specific day.
if the day to be booked is not already been booked (ie exists under BookedDays in mdb.months) then i wish to add the Start_Date of the booking, to the BookedDays string in the mdb.months database (the mdb.month database is just a list of the 12 months)
at first i tried using mdb.Entry() to add to that specific month instance, however i cannot get it to work.
the error is:
the model does not have a definition for BookedDays
what do?

Your checking that check is null
if (check != null)
{
return View(booking);
}
else
{
and then using check anyway:
check.BookedDays
check is null and therefore does not contain any BookedDays. I'm guessing your null check is the wrong way around and should be
if (check == null)
{
return View(booking);
}
else
{
That said your problem is not well explained so I'm not sure.

How can I make this code faster?? List<custom_class>().Find(lambda_expression)

I need to match email sends with email bounces so I can find if they were delivered or not. The catch is, I have to limit the bounce to within 4 days of the send to eliminate matching the wrong send to the bounce. Send records are spread over a period of 30 days.
LinkedList<event_data> sent = GetMyHugeListOfSends(); //for example 1M+ records
List<event_data> bounced = GetMyListOfBounces(); //for example 150k records
bounced = bounced.OrderBy(o => o.event_date).ToList(); //this ensures the most accurate match of bounce to send (since we find the first match)
List<event_data> delivered = new List<event_data>();
event_data deliveredEmail = new event_data();
foreach (event_data sentEmail in sent)
{
event_data bounce = bounced.Find(item => item.email.ToLower() == sentEmail.email.ToLower() && (item.event_date > sentEmail.event_date && item.event_date < sentEmail.event_date.AddDays(deliveredCalcDelayDays)));
//create delivered records
if (bounce != null)
{
//there was a bounce! don't add a delivered record!
}
else
{
//if sent is not bounced, it's delivered
deliveredEmail.sid = siteid;
deliveredEmail.mlid = mlid;
deliveredEmail.mid = mid;
deliveredEmail.email = sentEmail.email;
deliveredEmail.event_date = sentEmail.event_date;
deliveredEmail.event_status = "Delivered";
deliveredEmail.event_type = "Delivered";
deliveredEmail.id = sentEmail.id;
deliveredEmail.number = sentEmail.number;
deliveredEmail.laststoretransaction = sentEmail.laststoretransaction;
delivered.Add(deliveredEmail); //add the new delivered
deliveredEmail = new event_data();
//remove bounce, it only applies to one send!
bounced.Remove(bounce);
}
if (bounced.Count() == 0)
{
break; //no more bounces to match!
}
}
So I did some testing and it's processing about 12 sent records per second. At 1M+ records, it will take 25+ hours to process!
Two questions:
How can I find the exact line that is taking the most time?
I am assuming it's the lambda expression finding the bounce that is taking the longest since this was much faster before I put that in there. How can I speed this up?
Thanks!
Edit
---Ideas---
One idea I just had is to sort the sends by date like I did the bounces so that the search through the bounces will be more efficient, since an early send would be likely to hit an early bounce as well.
Another idea I just had is to run a couple of these processes in parallel, although I would hate to multi-thread this simple application.

I would be reasonably confident in saying that yes it is your find that is taking the time.
It looks like you are certain that the find method will return 0 or 1 records only (not a list) in which case the way to speed this up would be to create a lookup (a dictionary) instead of creating a List<event_data> for your bounced var, create a Dictionary<key, event_data> instead, then you can just look-up the value by key instead of doing a find.
The trick is in creating your key (I don't know enough about your app to help with that) but essentially the same criteria that is in your find.
EDIT. (adding some pseudo code)
void Main()
{
var hugeListOfEmails = GetHugeListOfEmails();
var allBouncedEmails = GetAllBouncedEmails();
IDictionary<string, EmailInfo> CreateLookupOfBouncedEmails = CreateLookupOfBouncedEmails(allBouncedEmails);
foreach(var info in hugeListOfEmails)
{
if(CreateLookupOfBouncedEmails.ContainsKey(info.emailAddress))
{
// Email is bounced;
}
else
{
// Email is not bounced
}
}
}
public IEnumerable<EmailInfo> GetHugeListOfEmails()
{
yield break;
}
public IEnumerable<EmailInfo> GetAllBouncedEmails()
{
yield break;
}
public IDictionary<string, EmailInfo> CreateLookupOfBouncedEmails(IEnumerable<EmailInfo> emailList)
{
var result = new Dictionary<string, EmailInfo>();
foreach(var e in emailList)
{
if(!result.ContainsKey(e.emailAddress))
{
if(//satisfies the date conditions)
{
result.Add(e.emailAddress, e);
}
}
}
return result;
}
public class EmailInfo
{
public string emailAddress { get; set; }
public DateTime DateSent { get; set; }
}

You should improve by using ToLookup method to create lookup table for email address
var bouncedLookup = bounced.ToLookup(k => k.email.ToLower());
and use this in the loop to lookup by the email first
var filteredBounced = bouncedLookup[sent_email.email.ToLower()];
// mini optimisation here
var endDate = sentEmail.event_date.AddDays(deliveredCalcDelayDays);
event_data bounce = filteredBounced.Find(item => item.event_date > sentEmail.event_date && item.event_date < endDate));
I could not compile it but I think that should do. Please try it.

You are finding items in a list. That means it has to traverse the whole list so it is an order (n) operation. Could you not store those sent emails in a Dictionary with the key being the email address you are searching on. The go through the bounces linking back to the emails in the dictionary. The lookup will be constant time and the you will go through the bounces so it will be order (n) overall. You current method is order (n squared)

Converting bounced to sortedlist might be a good solution
SortedList<string,data> sl = new SortedList<string,event_data>(bounced.ToDictionary(s=>s.email,s=>s));
and to find a bounce use
sl.Select(c=>c.Key.Equals(item => item.email,StringComparison.OrdinalIgnoreCase) && ...).FirstOrDefault();

There's another concern about your code, that I want to point out.
Memory consumption. I don't know your machine configuration, but here are some thoughts about the code:
Initially you are allocating space for 1,2M+ objects of event_data
type. I can't see event_data full type definition, but assuming
that emails are all unique and seeing that the type has quite many
properties, I can assume that such a collection is rather heavy
(hundreds of Meg possibly).
Next you are allocating another bunch of event_data objects
(almost 1M if I've counted it right). It's getting even more heavy
in terms of memory consumption
I don't know about other objects, that are present in data-model of your application, but considering all things I've mentioned, you can easily get close to memory limit
for 32-bit process and thus force GC to work very often. In fact
you can easily have a GC collecting after each call
bounced.Remove(bounce); And it's really would significantly slow down your app.
So, even if you are having a plenty of memory left and/or your app is 64-bit, I would try to minimize memory consumption. Pretty sure it would get your code run faster. For example, you can do complete processing of deliveredEmail, without storing it, or load your initial event_data in chunks etc.

On Consideration, the number of bounces is relatively small, so,
Why not pre omptimise the bounce lookup as much as possible, this code makes a delegate for each possible bounce and groups them into a dictionary for access by the e-mail key.
private static DateInRange(
DateTime sendDate,
DateTime bouncedDate,
int deliveredCalcDelayDays)
{
if (sentDate < bouncedDate)
{
return false;
}
return sentDate < bouncedDate.AddDays(deliveredCalcDelayDays);
}
static IEnumerable<event_data> GetDeliveredMails(
IEnumerable<event_data> sent,
IEnumerable<event_data> bounced,
int siteId,
int mlId,
int mId,
int deliveredCalcDelayDays)
{
var grouped = bounced.GroupBy(
b => b.email.ToLowerInvariant());
var lookup = grouped.ToDictionary(
g => g.Key,
g => g.OrderBy(e => e.event_date).Select(
e => new Func<DateTime, bool>(
s => DateInRange(s, e.event_date, deliveredCalcDelayDays))).ToList());
foreach (var s in sent)
{
var key = s.email.ToLowerInvariant();
List<Func<DateTime, nool>> checks;
if (lookup.TryGetValue(key, out checks))
{
var match = checks.FirstOrDefault(c => c(s.event_date));
if (match != null)
{
checks.Remove(match);
continue;
}
}
yield return new event_data
{
.sid = siteid;
.mlid = mlid;
.mid = mid;
.email = s.email;
.event_date = s.event_date;
.event_status = "Delivered";
.event_type = "Delivered";
.id = s.id;
.number = s.number;
.laststoretransaction = s.laststoretransaction
};
}
}
You could try pre-compiling the delegates in lookup if this is not fast enough.

Ok the final solution I found was a Dictionary for the bounces.
The sent LinkedList was sorted by sent_date so it would loop through in chronological order. That's important because I have to match the right send to the right bounce.
I made a Dictionary<string,<List<event_data>>, so the key was email, and the value was a List of all <event_data> bounces for the email address. The List was sorted by event_date since I wanted to make sure the first bounce was matched to the send.
Final result...it went from processing 700 records/minute to 500k+ records/second.
Here is the final code:
LinkedList sent = GetMyHugeListOfSends();
IEnumerable sentOrdered = sent.OrderBy(send => send.event_date);
Dictionary> bounced = GetMyListOfBouncesAsDictionary();
List delivered = new List();
event_data deliveredEmail = new event_data();
List bounces = null;
bool matchedBounce = false;
foreach (event_data sentEmail in sentOrdered)
{
matchedBounce = false;
//create delivered records
if (bounced.TryGetValue(sentEmail.email, out bounces))
{
//there was a bounce! find out if it was within 4 days after the send!
foreach (event_data bounce in bounces)
{
if (bounce.event_date > sentEmail.event_date &&
bounce.event_date <= sentEmail.event_date.AddDays(4))
{
matchedBounce = true;
//remove the record because a bounce can only match once back to a send
bounces.Remove(bounce);
if(bounces.Count == 0) //no more bounces for this email
{
bounced.Remove(sentEmail.email);
}
break;
}
}
if (matchedBounce == false) //no matching bounces in the list!
{
//if sent is not bounced, it's delivered
deliveredEmail.sid = siteid;
deliveredEmail.mlid = mlid;
deliveredEmail.mid = mid;
deliveredEmail.email = sentEmail.email;
deliveredEmail.event_date = sentEmail.event_date;
deliveredEmail.event_status = "Delivered";
deliveredEmail.event_type = "Delivered";
deliveredEmail.id = sentEmail.id;
deliveredEmail.number = sentEmail.number;
deliveredEmail.laststoretransaction = sentEmail.laststoretransaction;
delivered.Add(deliveredEmail); //add the new delivered
deliveredEmail = new event_data();
}
}
else
{
//if sent is not bounced, it's delivered
deliveredEmail.sid = siteid;
deliveredEmail.mlid = mlid;
deliveredEmail.mid = mid;
deliveredEmail.email = sentEmail.email;
deliveredEmail.event_date = sentEmail.event_date;
deliveredEmail.event_status = "Delivered";
deliveredEmail.event_type = "Delivered";
deliveredEmail.id = sentEmail.id;
deliveredEmail.number = sentEmail.number;
deliveredEmail.laststoretransaction = sentEmail.laststoretransaction;
delivered.Add(deliveredEmail); //add the new delivered
deliveredEmail = new event_data();
}
if (bounced.Count() == 0)
{
break; //no more bounces to match!
}
}

C#: recursive search in two different data structures

I need to perform a search in two different data structures in C#, and here's the deal:
I have one name (which is a string) and I want to perform a search. I have a function called Exists which will return a bool indicating whether it exists or not.
In case it exists, I increase the name (simply adding a 1 at the end of the string), and then I need to perform the search again (via method exists) to see if an object with the new name exists.
This would go on until there's an unused name, which I could use, BUT, in case it doesn't exist, now I should perform a search another data structure which contains the objects that were deleted, and if the string is found there, then I'd have to increase the name again, and start searching since the beginning.
This would all end in case there's no object with such name neither using Exists method nor in the data structure where all the deleted objects are.
How could I approach this problem?
I hope I expressed myself clearly :-)
Thanks a lot in advance!

string BuildNextName(string originalName)
{
string name = originalName;
while( Exists(name) || deletedNames.Contains(name))
{
name = Increment(name);
}
return name;
}
Or did I miss something?
Using a for loop:
string BuildNextName(string originalName)
{
for (string name=originalName;
Exists(name) || deletedNames.Contains(name);
name = Increment(name));
return name;
}
BTW, I guess your name incrementation algorithm is more complex than simply adding 1: name, name1, name2,... Basically, if the name doesn't end in a number, you append "1". If it does, you increment that number. right?

a non recursive and simple solution could be something like this ( I don't see any need of recursion in this case)
//pseudocode
String name;
bool condition = true;
while(condition)
{
if(ExistInFirstDataStructure(name))
{
//increment name
}
else
{
if(ExistInDeletedDataStructure(String name))
{
//increment name
}
else
{
condition = false;
}
}
}
bool ExistInFirstDataStructure(String name)
{
}
bool ExistInDeletedDataStructure(String name)
{
}

Why use a loop at all?? (I know LINQ will under the hood)
var LastUsedObjectName =
MyObjects.Select(mo => mo.Name)
.Union( MyDeletedObjects.Select(mo => mo.Name))
.OrderByDescending(name => /*Function to order by integer part of name*/).First();
// Now add 1 to LastUseObjectName and use that.

How about this one:
var listOfExistingNames = new List<string> { "MyName", "MyName1", "MyName3" };
var listOfDeletedNames = new List<string> { "MyName2", "MyName5" };
int counter = 0;
string baseToFindFreePlace = "MyName";
string newName = baseToFindFreePlace;
var allNames = listOfExistingNames.Concat(listOfDeletedNames);
while (allNames.Contains(newName))
{
counter++;
newName = baseToFindFreePlace + counter;
}
listOfExistingNames.Add(newName);

if you create Exists methods for both data structures, you can search with recursion like this:
pseudo code:
string resultName;
void Search(string name)
{
if(ExistsInFirstStructure(name)) //name is in first data structure
Search(name + "1"); //add 1 and try again
else
if(ExistsInSecondStructure(name)) //name exists in second data structure
Search(name + "1"); //perform search again
else
resultName = name; //current name wasn't found in first and second data structures - we have result
}