Develop Reference

Read more than 1000 rows from Azure Table Storage OData filter query?

How do we read more than 1000 rows from Azure Table Storage? Here is the code I'm using for reading data from Table Storage and this only retrieves 1000 rows:
readData()
{
var s = #$"https://{storageAccountName}.table.core.windows.net/{tableName}()";
var baseurl = $"{s}<sas-token>&$filter=Name%20eq%20'XYZ'";
var data = GetForOData(baseurl);
var responseData = data.Data.Replace(".", "_");
var odata = JsonConvert.DeserializeObject<ODataResponse>(responseData);
}
GetForOData(string url) {return InvokeForOData<Object>("GET", url, null, null);}
private static HttpResponseData InvokeForOData<T>(string method, string url, Object id, T data)
{
var Response = new HttpResponseData()
{
Code = HttpStatusCode.RequestTimeout, Data = string.Empty, Message = string.Empty
};
var PostParam = string.Empty;
if (data != null) { PostParam = data.ToString(); }
var postData = Encoding.UTF8.GetBytes(PostParam);
var request = (HttpWebRequest)WebRequest.Create(new Uri(url + (id == null ? "" : '/' + id.ToString())));
request.Method = method;
// add headers
if (postData.Length > 0)
{
using (Stream requestStream = request.GetRequestStream())
{ requestStream.Write(postData, 0, postData.Length); }
}
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
Response.Code = response.StatusCode;
using (var stream = new StreamReader(response.GetResponseStream(), Encoding.UTF8))
{ Response.Data = stream.ReadToEnd(); }
}
return Response;
}
Where do I check for x-ms-continuation-NextPartitionKey and x-ms-continuation-NextRowKey and use them in the next request?
Update: I was able to find nextPartitionKey and nextRowKey header values. How do I pass these values in the next request?
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
Response.Code = response.StatusCode;
var nextPartitionKey = response.Headers["x-ms-continuation-NextPartitionKey"];
var nextRowKey = response.Headers["x-ms-continuation-NextRowKey"];
using (StreamReader stream = new StreamReader(response.GetResponseStream(), Encoding.UTF8))
{
Response.Data = stream.ReadToEnd();
}
}

A single call to Azure Table Storage will return a maximum of 1000 entities. If there are more entities matching a query, you will get a continuation token back and you will need to use that to get the next set of entities.
So in your case, in order to read more than 1000 entities you will have to send the request, get the data and check for continuation token in the response (x-ms-continuation-NextPartitionKey and x-ms-continuation-NextRowKey) and use them in the next request.
You can learn more about pagination in Azure Table Storage here: https://learn.microsoft.com/en-us/rest/api/storageservices/query-timeout-and-pagination.
UPDATE
Please see the code below (untested though):
private static HttpResponseData InvokeForOData<T>(string method, string url, Object id, T data)
{
var Response = new HttpResponseData()
{
Code = HttpStatusCode.RequestTimeout,
Data = string.Empty,
Message = string.Empty,
NextPartitionKey = string.Empty,
NextRowKey = string.Empty
};
var PostParam = string.Empty;
if (data != null) { PostParam = data.ToString(); }
var postData = Encoding.UTF8.GetBytes(PostParam);
var request = (HttpWebRequest)WebRequest.Create(new Uri(url + (id == null ? "" : '/' + id.ToString())));
request.Method = method;
// add headers
if (postData.Length > 0)
{
using (Stream requestStream = request.GetRequestStream())
{ requestStream.Write(postData, 0, postData.Length); }
}
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
Response.Code = response.StatusCode;
var nextPartitionKey = response.Headers["x-ms-continuation-NextPartitionKey"];
var nextRowKey = response.Headers["x-ms-continuation-NextRowKey"];
using (StreamReader stream = new StreamReader(response.GetResponseStream(), Encoding.UTF8))
{
Response.Data = stream.ReadToEnd();
Response.NextPartitionKey = nextPartitionKey;
Response.NextRowKey = nextRowKey;
}
}
return Response;
}
Basically what I have done is added two properties (NextPartitionKey and NextRowKey) in your HttpResponseData object and populate that with the nextPartitionKey and nextRowKey respectively you get from response header.
What you have to do in your code (where you're processing the response) is check if either of these two values are not null or empty. A non null/empty value would indicate that more entities are present.
If that's the case, then what you have to do is modify the URL by appending NextPartitionKey and NextRowKey values as query string parameters (please see the link above for details) and send the request again. You will need to do this till the time you get both of these values as either null or empty strings.

Async/Await not working when calling the web API

Hi I have this Asynchronous Web API, I think when I execute this in a button click, the rest of the processes won't continue until the API returns the results, Below is my API,
btnSubmit.Click += async (sender, e) =>
{
var jobId = txtJobID.Text;
if (txtJobID.Text.Length <= 0)
{
txtJobID.RequestFocus();
txtJobID.SetError("Job ID required", iconError);
}
else
{
var request = HttpWebRequest.Create(string.Format("http://192.168.79.174:90/api/test/" + jobId));
request.ContentType = "application/json";
request.Method = "GET";
using (HttpWebResponse response = await request.GetResponseAsync() as HttpWebResponse)
{
if (response.StatusCode != HttpStatusCode.OK)
Console.Out.WriteLine("Error fetching data. Server returned status code: {0}", response.StatusCode);
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
var content = reader.ReadToEnd();
dynamic arr = JsonConvert.DeserializeObject(content);
foreach (dynamic obj in arr)
{
tvJobID.Text = obj.JobID;
tvJobType.Text = obj.JobType;
tvDueDate.Text = obj.DueDate;
tvVisitTime.Text = obj.Time;
tvVisitStatus.Text = obj.VisitStatus;
tvAddress1.Text = obj.Address1;
tvAddress2.Text = obj.Address2;
tvPostCode.Text = obj.PostCode;
tvAuthority.Text = obj.Authority;
}
if (content == null || content == "" || content == "[]")
{
//Console.Out.WriteLine("Response contained empty body...");
Toast.MakeText(ApplicationContext, "Invalid Job ID or no visits for this ID. Please try again", ToastLength.Long).Show();
}
else
{
layoutController.Visibility = ViewStates.Visible;
}
}
}
}
};
When I debug the code, Specially where the foreach statement is, it takes quite a bit of time for API to return results. What am I doing wrong? Thanks a lot for any help!

How to add a specific user to a segment in OneSignal

I'm developing a C#.NET MVC Web Api for an Android application. At the moment I am using OneSignal to send push notifications to the users by calling the OneSignal Api and passing the notification content. I need to know how to add a user to a specific segment so that i can send notifications to individual users as well as users of that segment collectively. I have searched in on their documentation but I didn't understand how to do it using OneSignal.SendTag method. So basically how to do it in Visual Studio? So far i have done this:
string api_key = "dsabjd";
var request = WebRequest.Create("https://onesignal.com/api/v1/notifications") as HttpWebRequest;
if (user != null)
{
string message = "This job is posted by: \n" + user.Name + "\n" + user.Contact + "\n" +user.City;
if (request != null)
{
request.KeepAlive = true;
request.Method = "POST";
request.ContentType = "application/json";
request.Headers.Add("authorization", "Basic "+api_key);
var serializer = new JavaScriptSerializer();
var obj = new
{
app_id = "1651",
contents = new { en = message },
//data = new { image = "http://dsadasdasd.png" },
data = new { image = imageUrl },
included_segments = new string[] { "All" }
};
var param = serializer.Serialize(obj);
byte[] byteArray = Encoding.UTF8.GetBytes(param);
try
{
using (var writer = request.GetRequestStream())
{
writer.Write(byteArray, 0, byteArray.Length);
}
string responseContent=null;
using (var response = request.GetResponse() as HttpWebResponse)
{
using (var reader = new StreamReader(response.GetResponseStream()))
{
responseContent = reader.ReadToEnd();
}
}
if (responseContent != null)
{
// parsing the json returned by OneSignal Push API
dynamic json = JObject.Parse(responseContent);
int noOfRecipients = json.recipients;
if (noOfRecipients > 0)
{
flag = true;
}
}
}
catch (WebException ex)
{
flag = false;
}
}
}

To set tags it is recommend you use sendTags from the OneSignal Android SDK in your app it's supported offline and handles retries for you.
If you need to target individual users it is recommend to call idsAvailable in your app and send this to your server. You can later use the include_player_ids field on the create notification REST API POST call to send a notification to a list of users.

StreamWritter UTF8 Encoding Error

We are developing an android,wp &iOS application using File Linking strategy of Xamarin.
We have written WCF REST Service to input/output as JSON format data.
In database class we have used ASCII format and it works fine, but we tried to change UTF8 format to support for multiple languages. While changing we are getting error as "The remote server returned an error: (400) Bad Request" in HttpWebResponse line.
Please find the sample database function for sending data to WCF REST Service with UTF-8. Kindly suggest to resolve.
public async Task<String> GetString<T>(T item, String ServiceAddress, String FunctionName, String[] Parameters, object[] ParameterValues) where T : BusinessLayer.Contracts.IBusinessEntity, new()
{
string ParameterString = string.Empty;
try
{
var request = HttpWebRequest.Create(ServiceAddress);
request.ContentType = "application/json";
request.Method = "POST";
request.Headers["Authorization"] = GlobalVariables.glbtoken;
if (Parameters.Length > 0)
ParameterString = "{";
for (int i = 0; i < Parameters.Length; i++)
{
ParameterString = ParameterString + "\"" + Parameters[i] + "\":\"" + ParameterValues[i] + "\"" + ",";
}
if (Parameters.Length > 0)
{
ParameterString = ParameterString.Remove(ParameterString.Length - 1, 1);
ParameterString += "}";
}
// var sw = new StreamWriter(request.GetRequestStream(), System.Text.Encoding.ASCII);
var sw = new StreamWriter(request.GetRequestStream(), Encoding.UTF8);
sw.Write(ParameterString); **//This string is not passed to server and shows null**
sw.Close();
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse) // Here the line breaks and returned to catch with no parameters passed.
{
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
var content = reader.ReadToEnd();
if (content != "")
OutputString = (string)Newtonsoft.Json.JsonConvert.DeserializeObject(content);
}
}
}
catch (Exception e) { }
return OutputString;
}

Google Translate V2 cannot hanlde large text translations from C#

I've implemented C# code using the Google Translation V2 api with the GET Method.
It successfully translates small texts but when increasing the text length and it takes 1,800 characters long ( including URI parameters ) I'm getting the "URI too large" error.
Ok, I burned down all the paths and investigated the issue across multiple pages posted on Internet. All of them clearly says the GET method should be overriden to simulate a POST method ( which is meant to provide support to 5,000 character URIs ) but there is no way to find out a code example to of it.
Does anyone has any example or can provide some information?
[EDIT] Here is the code I'm using:
String apiUrl = "https://www.googleapis.com/language/translate/v2?key={0}&source={1}&target={2}&q={3}";
String url = String.Format(apiUrl, Constants.apiKey, sourceLanguage, targetLanguage, text);
Stream outputStream = null;
byte[] bytes = Encoding.ASCII.GetBytes(url);
// create the http web request
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(url);
webRequest.KeepAlive = true;
webRequest.Method = "POST";
// Overrride the GET method as documented on Google's docu.
webRequest.Headers.Add("X-HTTP-Method-Override: GET");
webRequest.ContentType = "application/x-www-form-urlencoded";
// send POST
try
{
webRequest.ContentLength = bytes.Length;
outputStream = webRequest.GetRequestStream();
outputStream.Write(bytes, 0, bytes.Length);
outputStream.Close();
}
catch (HttpException e)
{
/*...*/
}
try
{
// get the response
HttpWebResponse webResponse = (HttpWebResponse)webRequest.GetResponse();
if (webResponse.StatusCode == HttpStatusCode.OK && webRequest != null)
{
// read response stream
using (StreamReader sr = new StreamReader(webResponse.GetResponseStream(), Encoding.UTF8))
{
string lista = sr.ReadToEnd();
DataContractJsonSerializer serializer = new DataContractJsonSerializer(typeof(TranslationRootObject));
MemoryStream stream = new MemoryStream(Encoding.UTF8.GetBytes(lista));
TranslationRootObject tRootObject = (TranslationRootObject)serializer.ReadObject(stream);
string previousTranslation = string.Empty;
//deserialize
for (int i = 0; i < tRootObject.Data.Detections.Count; i++)
{
string translatedText = tRootObject.Data.Detections[i].TranslatedText.ToString();
if (i == 0)
{
text = translatedText;
}
else
{
if (!text.Contains(translatedText))
{
text = text + " " + translatedText;
}
}
}
return text;
}
}
}
catch (HttpException e)
{
/*...*/
}
return text;
}

Apparently using WebClient won't work as you cannot alter the headers as needed, per the documentation:
Note: You can also use POST to invoke the API if you want to send more data in a single request. The q parameter in the POST body must be less than 5K characters. To use POST, you must use the X-HTTP-Method-Override header to tell the Translate API to treat the request as a GET (use X-HTTP-Method-Override: GET).
You can use WebRequest, but you'll need to add the X-HTTP-Method-Override header:
var request = WebRequest.Create (uri);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.Headers.Add("X-HTTP-Method-Override", "GET");
var body = new StringBuilder();
body.Append("key=SECRET");
body.AppendFormat("&source={0}", HttpUtility.UrlEncode(source));
body.AppendFormat("&target={0}", HttpUtility.UrlEncode(target));
//--
body.AppendFormat("&q={0}", HttpUtility.UrlEncode(text));
var bytes = Encoding.ASCII.GetBytes(body.ToString());
if (bytes.Length > 5120) throw new ArgumentOutOfRangeException("text");
request.ContentLength = bytes.Length;
using (var output = request.GetRequestStream())
{
output.Write(bytes, 0, bytes.Length);
}

The accepted answer appears to be out of date. You can now use the WebClient (.net 4.5) successfully to POST to the google translate API making sure to set the X-HTTP-Method-Override header.
Here is some code to show you how.
using (var webClient = new WebClient())
{
webClient.Headers.Add("X-HTTP-Method-Override", "GET");
var data = new NameValueCollection()
{
{ "key", GoogleTranslateApiKey },
{ "source", "en" },
{ "target", "fr"},
{ "q", "<p>Hello World</p>" }
};
try
{
var responseBytes = webClient.UploadValues(GoogleTranslateApiUrl, "POST", data);
var json = Encoding.UTF8.GetString(responseBytes);
var result = JsonConvert.DeserializeObject<dynamic>(json);
var translation = result.data.translations[0].translatedText;
}
catch (Exception ex)
{
loggingService.Error(ex.Message);
}
}

? What? it is trivial to post using C# - it is right there in the documentation.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
{
// Set type to POST
request.Method = "POST";
From there on you bascially put the data into fom fields into the content stream.
This is not "simulate a post meethod", it is fully doing a post request as per specifications.
Btw. hwhere does json enter here? You say "in C#". There is no need to use json?