I'm posting data to a service that requires that I submit duplicate query string keys (ugly and not specified in any standards).
I'm using WebClient object to build the request. I'd like to keep using it since it is used frequently elsewhere in the project.
When I do this
foreach(var f in formats)
client.QueryString.Add("formats", f);
I get a list &formats=format_1,format_2,format_3 which the service does not support.
Is there a better alternative than this old-school ugliness:
var extraQueryString = string.Empty;
extraQueryString += "?apiKey=" + TRANSCODE_KEY;
extraQueryString += "&fileKey=" + fileKey;
foreach (var f in formats)
extraQueryString += "&formats=" + f;
var response = client.UploadData(TRANSCODE_URI + "task" + extraQueryString , new byte[] { });
The reason for this is because the NameValueCollection separates duplicate keys with commas. You could extend the NameValueCollection and override the Get method and have it return the format you want.
public class DupeNVC: NameValueCollection
{
private string _duplicateKey;
public DupeNVC(string duplicateKey = null)
{
_duplicateKey = duplicateKey;
}
public override string Get(int index)
{
//check if duplicate key has been specified
//if not, then call the default Get implementation
if (!String.IsNullOrEmpty(_duplicateKey))
{
ArrayList list = (ArrayList)base.BaseGet(index);
int num = (list != null) ? list.Count : 0;
if (num == 1)
{
return (string)list[0];
}
if (num > 1)
{
StringBuilder stringBuilder = new StringBuilder((string)list[0]);
for (int i = 1; i < num; i++)
{
//format our string and append the duplicate key specified
stringBuilder.AppendFormat("&{0}=", _duplicateKey);
stringBuilder.Append((string)list[i]);
}
return stringBuilder.ToString();
}
return null;
}
else
return base.Get(index);
}
}
You can use it like a normal NameValueCollection but if you pass in a duplicate strning in the constructor, it will look for that duplicate key and run the modified code above (otherwise it will just use the default base.Get method.
DupeNVC dnvc = new DupeNVC("formats");
foreach(var f in formats)
dnvc.Add("formats", f);
webClient.QueryString = dnvc;
This hasn't been fully tested but it should output the querystring format you want. Of course, this could be extended further by taking a collection of duplicate keys but this was just to give you an idea for your current problem.
Here's my take on this. WebClient essentially works like the ToString method of this class; it gets all the keys and then retrieves the values one at a time, doing a concatenate. So I override the AllKeys to return an array with repeated elements..
For example if a particular key has multiple values:
nvc["hello"] = { "a", "b", "c" }
Then my AllKeys will return an array with "hello" 3 times. WebClient will naively request it 3 times. A Dictionary tracks how many times a "hello" has been requested, and returns a different one each time (pseudo enumerator)
public class ParrotingNameValueCollection : NameValueCollection
{
Dictionary<string, int> _indexTracker = new Dictionary<string, int>();
public override string[] AllKeys
{
get
{
var l = new List<string>();
foreach (var k in base.AllKeys)
{
foreach (var x in (ArrayList)base.BaseGet(k))
l.Add(k);
_indexTracker[k] = 0;
}
return l.ToArray();
}
}
public override string Get(string name)
{
var list = (ArrayList)base.BaseGet(name);
var toReturn = (string)list[_indexTracker[name]];
_indexTracker[name]++;
return toReturn;
}
public override string ToString()
{
string delimiter = String.Empty;
StringBuilder values = new StringBuilder();
foreach (string name in this.AllKeys)
{
values.Append(delimiter);
values.Append((name));
values.Append("=");
values.Append((this[name]));
delimiter = "&";
}
return values.ToString();
}
}
Related
Question is how can I get only the values from my dictionary that contains one key but has a list of values assigned to it. I want to be able to grab all of the values and convert them into a tuple. Any help would be greatly appreciated.
Main code
private static Dictionary<string, List<string>> v_dict_info = new Dictionary<string, List<string>>();
public static Dictionary<string, List<string>> V_dict_info
{
get => v_dict_info;
set => v_dict_info = value;
}
public static string Vista_con_s
{
get => _vista_con_s;
set => _vista_con_s = value;
}
public void get_emp_info_addr()
{
string info_addr = "select " +
"c.empno " +
"from " +
"[V4].[dbo].[person] c ";
try
{
vista_conn = new SqlConnection();
vista_conn.ConnectionString = _vista_con_s;
vista_command = new SqlCommand(info_addr, vista_conn);
vista_conn.Open();
vista_reader = vista_command.ExecuteReader();
if (vista_reader.HasRows)
{
while (vista_reader.Read())
{
v_dict_info.Vista_addr_list("empno", vista_reader.GetValue(0).ToString());
}
}
}
catch (SqlException sq_x)
{
System.Diagnostics.Debug.WriteLine("Error connecting to the database! -Error Msg: {0}", sq_x.ToString());
}
var count = v_dict_info["empno"].Count;
for (int i = 0; i < count; i++)
{
var empno = v_dict_info["empno"][i];
Console.WriteLine("{0}", empno);
}
// how can I get only the values associated to "empno" key
var xx = empno //this only return the first element :( I want all of them
Console.WriteLine(xx.ToString());
}
Here is my method for adding multiple values to one key in a dictionary.
public static class Multi_dict
{
public static void Vista_addr_list<T, U>(this IDictionary<T, List<U>> dict, T key, U elementToList)
{
List<U> list;
bool exists = dict.TryGetValue(key, out list);
if (exists)
{
dict[key].Add(elementToList);
}
else
{
dict[key] = new List<U>();
dict[key].Add(elementToList);
}
}
Presuming this is really necessary (and I agree with Gusman's comment on the question - I don't think this is really what you need), you're already half-way there.
Access the list via the key, and then select out your tuple:
dict[key].Select(x => new Tuple<string, string>(key, x));
You can find a working fiddle here: https://dotnetfiddle.net/jPX0bU
Major Edit: I am doing a bad job of explaining :(
I have two classes:
public class UserDefinitions// a list of 'Items', each'group of items belong to a user. I handle User logic elsewhere, and it works flawlessly.
{
public List<Item> items { get; set; }
}
public class Item //the User definitions. A user could have 1 or 15 of these. They would all be a single 'line' from the CSV file.
{
public string definitionKey { get; set; }
public string defName { get; set; }
public string defValue { get; set; }
}
Which I wanna build with a CSV File. I build this CSV File, so I make it using the same parameters every time.
I run SQL on my company's DB to generate results like so: http://i.imgur.com/gS1UJot.png
Then I read the file like so:
class Program
{
static void Main(string[] args)
{
var userData = new UserDefinitions();
var csvList = new List<Item>();
string json = "";
string fPath = #"C:\test\csvTest.csv";
var lines = File.ReadAllLines(fPath);
Console.WriteLine(lines);
List<string> udata = new List<string>(lines);
foreach (var line in udata)
{
string[] userDataComplete = line.Split(new string[] { "," }, StringSplitOptions.RemoveEmptyEntries);// this cleans any empty cells from the CSV
csvList.Add(new Item { definitionKey = userDataComplete[1], defName = userDataComplete[2], defValue = userDataComplete[3] });
}
json = JsonConvert.SerializeObject(csvList); //everything below is for debugging/tracking progress
Console.WriteLine(json);
Console.ReadKey();
StreamWriter sw = new StreamWriter("C:\\test\\testjson.txt");
sw.WriteLine(json);
sw.Close();
}
}
This ALMOST does what I want. The output json is from the first 'column' of the csv data
[{"definitionKey":"uuid1","defName":"HairColor","defValue":"Brown"},{"definitionKey":"uuid1","defName":"HairColor","defValue":"Blonde"},{"definitionKey":"uuid1","defName":"HairColor","defValue":"Blue"}]
When using the screen shot as an example, the wanted output should be
[{"attributeDefinitionKey":"uuid1","name":"HairColor","value":"Brown"},{"definitionKey":"uuid2","defName":"FreckleAmount","defValue":"50"}]
[{"attributeDefinitionKey":"uuid1","name":"HairColor","value":"Blonde"},{"definitionKey":"uuid2","defName":"FreckleAmount","defValue":"null"}]
[{"attributeDefinitionKey":"uuid1","name":"HairColor","value":"Blue"},{"definitionKey":"uuid3","defName":"Tattoos","defValue":"5"}]
I can't pick out certain aspects at will, or apply them to Items. For example there maybe 10 users or 5000 users, but the definitionKey will always be the [1], and adding '3' will get every subsequent defintionKey. Just like the defName will always be in the [2] spot and adding 3 will get every subsequent defName if there are any, this is all per line.
I know I have to add some +3 logic, but not quite sure how to incorporate that. Maybe a for loop? a nested for loop after a foreach loop? I feel I am missing something obvious!
Thanks again for any help
This reads the csv line for line and converts each row to json, while adapting to the change in the amount of columns.
This only works if the CSV follows your rules:
one userId and
x amount of "Things" with 3 columns per "Thing".
private static void Main(string[] args)
{
var file = new StreamReader(#"C:\test\csvTest.csv");
string line;
var itemsJson = new List<string>();
file.ReadLine();
while ((line = file.ReadLine()) != null)
{
var sb = new StringBuilder();
var fields = line.Split(',');
sb.Append(GetKeyValueJson("UserId", fields[0]));
for (var i = 1; i < fields.Length; i += 3)
{
var x = (i + 3) / 3;
sb.Append(GetKeyValueJson($"Thing {i + x} ID", fields[i]));
sb.Append(GetKeyValueJson($"Thing {i + x} ID", fields[i + 1]));
sb.Append(i + 3 == fields.Length
? GetKeyValueJson($"Thing {i + x} ID", fields[i + 2], true)
: GetKeyValueJson($"Thing {i + x} ID", fields[i + 2]));
}
itemsJson.Add(WrapJson(sb.ToString()));
}
var json = WrapItems(itemsJson);
Console.ReadLine();
}
private static string GetKeyValueJson(string id, string value, bool lastPair = false)
{
var sb = new StringBuilder();
sb.Append('"');
sb.Append(id);
sb.Append('"');
sb.Append(':');
sb.Append('"');
sb.Append(value);
sb.Append('"');
if (!lastPair)
sb.Append(',');
return sb.ToString();
}
private static string WrapJson(string s)
{
var sb = new StringBuilder();
sb.Append('{');
sb.Append(s);
sb.Append('}');
return sb.ToString();
}
private static string WrapItems(List<string> jsonList)
{
var sb = new StringBuilder();
sb.Append('"');
sb.Append("Items");
sb.Append('"');
sb.Append(':');
sb.Append('[');
sb.Append(jsonList.Aggregate((current, next) => current + "," + next));
sb.Append(']');
return WrapJson(sb.ToString());
}
}
It's not pretty and sorting would be tough, but it should adapt to the column amount as long as it is in 3's.
I have a massive json file that is very nested. I need to write the multiple csv files depending on the name of a certain field, if it exists then add the values to the headers I've created if it does not then create a new one. This is working just fine. However I have ran into a problem where the headers do not match because this particular header doesn't exist for that record. Example:
Header: Dog Cat Mouse Horse
Record1: yes yes yes yes
// above is an example of a file with all values
Adding record Two where a header value is not listed at all
Header: Dog Cat Mouse Horse
Record1: yes yes yes yes
Record2: yes yes yes ***
Record2 above does not have a mouse on the record but because it doesn't line up the yes shifted left. I need to write a Null under than header before spitting out the values to the file. Below is my code if you could help that would be great as I'm lost at this point:
static List<string> headList = new List<string>();
static List<string> newHeaderList = new List<string>();
static List<string> valueList = new List<string>();
static List<string> oldHeadList = new List<string>();
static void Main()
{
var data = JsonConvert.DeserializeObject<dynamic>(File.ReadAllText(
#"C:\Users\nphillips\workspace\2016R23\UITestAutomation\SeedDataGenerator\src\staticresources\seeddata.resource"));
string fileName = "";
var bundles = data.RecordSetBundles;
foreach (var bundle in bundles)
{
var records = bundle.Records;
foreach (var record in records)
{
var test = record.attributes;
foreach (var testagain in test)
{
// Getting the object Name Ex. Location, Item, etc.
var jprop = testagain as JProperty;
if (jprop != null)
{
fileName = jprop.First.ToString().Split('_')[2] + ".csv";
}
break;
}
string header = "";
string value = "";
foreach (var child in record)
{
var theChild = child as JProperty;
if (theChild != null && !theChild.Name.Equals("attributes"))
{
// adding the name and values to list
headList.Add(child.Name);
valueList.Add(child.Value.ToString());
}
}
// calling method to write columns and values
writeCSV(headList, valueList, fileName);
valueList.Clear();
headList.Clear();
}
}
}
public static void writeCSV(List<string> headList, List<string> valList, string fileName)
{
string headerString = "";
string value = "";
if (!File.Exists(fileName))
{
foreach (var header in headList)
{
foreach (var val in valList)
{
value += val + ",";
}
oldHeadList.Add(header);
headerString += header + ',';
}
headerString += "+" + Environment.NewLine;
File.WriteAllText(fileName, headerString);
}
else
{
foreach (var header in headList)
{
foreach (var oldHeader in oldHeadList)
{
foreach (var val in valList)
{
if (header != oldHeader)
{
value += "null,";
}
else
{
value += val + ",";
}
}
}
}
}
File.AppendAllText(fileName, value);
value += Environment.NewLine;
}
}
My horrific json file that I cannot change as its used by my company: https://codeshare.io/rGL6K5
Is there some kind of pattern?
Reason I am asking is, maybe you could create a service that serialize the JSON to a complex object(s). Once that is done have a service that serializes that object to csv. The service would know to write multiple csv files as needed.
I'd stay away from using {dynamic}, if there is a reliable pattern to the JSON. I'd get a sample JSON file, copy it into the clipboard and using the Paste JSON to Classes feature in Visual Studio.
https://blogs.msdn.microsoft.com/webdev/2012/12/18/paste-json-as-classes-in-asp-net-and-web-tools-2012-2-rc/
After that, deserialize it with Newtonsoft.JSON into a nice reliable object from which to build your CSV.
I've stored all URLs in my application with "http://" - I now need to go through and replace all of them with "https:". Right now I have:
foreach (var link in links)
{
if (link.Contains("http:"))
{
/// do something, slice or replace or what?
}
}
I'm just not sure what the best way to update the string would be. How can this be done?
If you're dealing with uris, you probably want to use UriBuilder since doing a string replace on structured data like URIs is not a good idea.
var builder = new UriBuilder(link);
builder.Scheme = "https";
Uri modified = builder.Uri;
It's not clear what the type of links is, but you can create a new collection with the modified uris using linq:
IEnumerable<string> updated = links.Select(link => {
var builder = new UriBuilder(link);
builder.Scheme = "https";
return builder.ToString();
});
The problem is your strings are in a collection, and since strings are immutable you can't change them directly. Since you didn't specify the type of links (List? Array?) the right answer will change slightly. The easiest way is to create a new list:
links = links.Select(link => link.Replace("http://","https://")).ToList();
However if you want to minimize the number of changes and can access the string by index you can just loop through the collection:
for(int i = 0; i < links.Length; i++ )
{
links[i] = links[i].Replace("http://","https://");
}
based on your current code, link will not be replace to anything you want because it is read only (see here: Why can't I modify the loop variable in a foreach?). instead use for
for(int a = 0; a < links.Length; a++ )
{
links[a] = links[a].Replace("http:/","https:/")
}
http://myserver.xom/login.aspx?returnurl=http%3a%2f%2fmyserver.xom%2fmyaccount.aspx&q1=a%20b%20c&q2=c%2b%2b
What about the urls having also url in the querystring part? I think we should also replace them. And because of the url encoding-escaping this is the hard part of the job.
private void BlaBla()
{
// call the replacing function
Uri myNewUrl = ConvertHttpToHttps(myOriginalUrl);
}
private Uri ConvertHttpToHttps(Uri originalUri)
{
Uri result = null;
int httpsPort = 443;// if needed assign your own value or implement it as parametric
string resultQuery = string.Empty;
NameValueCollection urlParameters = HttpUtility.ParseQueryString(originalUri.Query);
if (urlParameters != null && urlParameters.Count > 0)
{
StringBuilder sb = new StringBuilder();
foreach (string key in urlParameters)
{
if (sb.Length > 0)
sb.Append("&");
string value = urlParameters[key].Replace("http://", "https://");
string valuEscaped = Uri.EscapeDataString(value);// this is important
sb.Append(string.Concat(key, "=", valuEscaped));
}
resultQuery = sb.ToString();
}
UriBuilder resultBuilder = new UriBuilder("https", originalUri.Host, httpsPort, originalUri.AbsolutePath);
resultBuilder.Query = resultQuery;
result = resultBuilder.Uri;
return result;
}
Use string.Replace and some LINQ:
var httpsLinks = links.Select(l=>l.Replace("http://", "https://");
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Remove duplicates from a List<T> in C#
i have a List like below (so big email list):
source list :
item 0 : jumper#yahoo.com|32432
item 1 : goodzila#yahoo.com|32432|test23
item 2 : alibaba#yahoo.com|32432|test65
item 3 : blabla#yahoo.com|32432|test32
the important part of each item is email address and the other parts(separated with pipes are not important) but i want to keep them in final list.
as i said my list is to big and i think it's not recommended to use another list.
how can i remove duplicate emails (entire item) form that list without using LINQ ?
my codes are like below :
private void WorkOnFile(UploadedFile file, string filePath)
{
File.SetAttributes(filePath, FileAttributes.Archive);
FileSecurity fSecurity = File.GetAccessControl(filePath);
fSecurity.AddAccessRule(new FileSystemAccessRule(#"Everyone",
FileSystemRights.FullControl,
AccessControlType.Allow));
File.SetAccessControl(filePath, fSecurity);
string[] lines = File.ReadAllLines(filePath);
List<string> list_lines = new List<string>(lines);
var new_lines = list_lines.Select(line => string.Join("|", line.Split(new string[] { " " }, StringSplitOptions.RemoveEmptyEntries)));
List<string> new_list_lines = new List<string>(new_lines);
int Duplicate_Count = 0;
RemoveDuplicates(ref new_list_lines, ref Duplicate_Count);
File.WriteAllLines(filePath, new_list_lines.ToArray());
}
private void RemoveDuplicates(ref List<string> list_lines, ref int Duplicate_Count)
{
char[] splitter = { '|' };
list_lines.ForEach(delegate(string line)
{
// ??
});
}
EDIT :
some duplicate email addrresses in that list have different parts ->
what can i do about them :
mean
goodzila#yahoo.com|32432|test23
and
goodzila#yahoo.com|asdsa|324234
Thanks in advance.
say you have a list of possible duplicates:
List<string> emailList ....
Then the unique list is the set of that list:
HashSet<string> unique = new HashSet<string>( emailList )
private void RemoveDuplicates(ref List<string> list_lines, ref int Duplicate_Count)
{
Duplicate_Count = 0;
List<string> list_lines2 = new List<string>();
HashSet<string> hash = new HashSet<string>();
foreach (string line in list_lines)
{
string[] split = line.Split('|');
string firstPart = split.Length > 0 ? split[0] : string.Empty;
if (hash.Add(firstPart))
{
list_lines2.Add(line);
}
else
{
Duplicate_Count++;
}
}
list_lines = list_lines2;
}
The easiest thing to do is to iterate through the lines in the file and add them to a HashSet. HashSets won't insert the duplicate entries and it won't generate an exception either. At the end you'll have a unique list of items and no exceptions will be generated for any duplicates.
1 - Get rid of your pipe separated string (create an dto class corresponding to the data it's representing)
2 - which rule do you want to apply to select two object with the same id ?
Or maybe this code can be useful for you :)
It's using the same method as the one in #xanatos answer
string[] lines= File.ReadAllLines(filePath);
Dictionary<string, string> items;
foreach (var line in lines )
{
var key = line.Split('|').ElementAt(0);
if (!items.ContainsKey(key))
items.Add(key, line);
}
List<string> list_lines = items.Values.ToList();
First, I suggest to you load the file via stream.
Then, create a type that represent your rows and load them into a HashSet(for
performance considerations).
Look (Ive removed some of your code to make it simple):
public struct LineType
{
public string Email { get; set; }
public string Others { get; set; }
public override bool Equals(object obj)
{
return this.Email.Equals(((LineType)obj).Email);
}
}
private static void WorkOnFile(string filePath)
{
StreamReader stream = File.OpenText(filePath);
HashSet<LineType> hashSet = new HashSet<LineType>();
while (true)
{
string line = stream.ReadLine();
if (line == null)
break;
string new_line = string.Join("|", line.Split(new string[] { " " }, StringSplitOptions.RemoveEmptyEntries));
LineType lineType = new LineType()
{
Email = new_line.Split('|')[3],
Others = new_line
};
if (!hashSet.Contains(lineType))
hashSet.Add(lineType);
}
}