I need some help with WebTestRequest on visual studio
I have 2 requests that the 2nd request need some data from the 1st request response
how do i use the specific data that i need from the 1st response to the 2nd request ? (in my case its Sessionid)
Test requests:
{
using System;
using System.Collections.Generic;
using System.Text;
using Microsoft.VisualStudio.TestTools.WebTesting;
public class WebTest1Coded : WebTest
{
public WebTest1Coded()
{
this.PreAuthenticate = true;
this.Proxy = "default";
}
public override IEnumerator<WebTestRequest> GetRequestEnumerator()
{
WebTestRequest request1 = new WebTestRequest("https://xxx.xxx.com/api/xxx");
request1.Method = "POST";
request1.QueryStringParameters.Add("format", "json", false, false);
StringHttpBody request1Body = new StringHttpBody();
request1Body.ContentType = "application/json";
request1Body.InsertByteOrderMark = false;
request1Body.BodyString = "{\"UserName\":\"xxx\",\"Password\":\"xxx\"}";
request1.Body = request1Body;
yield return request1;
request1 = null;
WebTestRequest request2 = new WebTestRequest("https://xxx.xxx.com/api/xxx");
request2.Method = "POST";
request2.QueryStringParameters.Add("format", "json", false, false);
StringHttpBody request2Body = new StringHttpBody();
request2Body.ContentType = "application/json";
request2Body.InsertByteOrderMark = false;
request2Body.BodyString = #"
RequestHeader"":
""SessionId"""": ""xxx"",
""ApiKey"""": ""xxx""
request2.Body = request2Body;
yield return request2;
request2 = null;
}
}
It is easier to do in the original Web Test file than in a Coded web Test. Just add an extraction rule to the first request. It saves the value into a context parameter which can then be used in the second request.
If you really want to use a Coded Web Test then you might create a sandbox web test with a suitable extraction rule and context parameter usage. Then convert that to Coded and copy the interesting bits into your test.
An ExtractText rule to extract text between abc and def and store the results in the acbdef context parameter generated the code below.
WebTestRequest request3 = new WebTestRequest("http://localhost/");
request3.ThinkTime = 1;
request3.ParseDependentRequests = false;
ExtractText extractionRule1 = new ExtractText();
extractionRule1.StartsWith = "abc";
extractionRule1.EndsWith = "def";
extractionRule1.IgnoreCase = false;
extractionRule1.UseRegularExpression = false;
extractionRule1.Required = true;
extractionRule1.ExtractRandomMatch = false;
extractionRule1.Index = 0;
extractionRule1.HtmlDecode = true;
extractionRule1.SearchInHeaders = false;
extractionRule1.ContextParameterName = "abcdef";
request3.ExtractValues += new EventHandler<ExtractionEventArgs>(extractionRule1.Extract);
yield return request3;
request3 = null;
Other extraction rules have different properties that will need to be given values. I recommend putting the main details of what you want into the web test and then doing a generate code to find out exactly what C# code is necessary.
The extracted value is used in three places in the next request:
WebTestRequest request4 = new WebTestRequest(
("http://localhost/" + (this.Context["abcdef"].ToString() + "/more")));
request4.ThinkTime = 1;
request4.ParseDependentRequests = false;
request4.Headers.Add(new WebTestRequestHeader("Use-abcdef-again",
("pqr" + (this.Context["abcdef"].ToString() + "stu"))));
request4.Headers.Add(new WebTestRequestHeader("Use-abcdef",
this.Context["abcdef"].ToString()));
yield return request4;
request4 = null;
Related
So, i'm calling the method to update the primary "sendAs" object of a google account, without results. The documentation from google at users.settings.sendAs/update indicates all i need and did:
i've set the domain wide account and scopes
i'm generating a token and accessing it no problem
i'm calling the "list" method first (with that token) as shown in users.settings.sendAs/list documentation, and finding the one that is the primary (the "isPrimary" attribute is true)
After that, changing the "signature" value to "Its a Test Signature", and sending the PUT request with it doesn't do anything.
The JSON sent to the update API (via PUT method) is the exact one i collected from the list (the primary one), but with the signature changed.
There is no error at all, and i receive an "sendAs" object back as a response (as the documentation says i should in case of sucess), but the signature is unchanged.
What can i be?
EDIT (adding the code section for the call, again - no errors)
public bool Update()
{
string json = null;
using (WebClient wc = new WebClient())
{
wc.Headers[HttpRequestHeader.ContentType] = "application/json";
wc.Headers["Authorization"] = "Bearer " + GenerateServerToServerToken(this.OwnerMail, this.scope);
json = wc.DownloadString("https://gmail.googleapis.com/gmail/v1/users/" + this.OwnerMail + "/settings/sendAs");
JSon.Query response = JSon.Parse(ref json);
json = null;
response = response["sendAs"];
List<JSon.Query> mailAs = null;
if (response.TryParseList(out mailAs))
{
JSon.Query main = null;
bool prim = false;
foreach (JSon.Query q in mailAs)
{
if (q["isPrimary"].TryParseBoolean(out prim) && prim)
{
if (q["sendAsEmail"].TryParseString(out json)) { main = q; }
break;
} else { json = null; }
}
if (main != null)
{
JSon.ObjectValue mainO = (JSon.ObjectValue)main.Value;
if (mainO.ContainsKey("signature"))
{
((JSon.StringValue)mainO["signature"]).Data = this.HtmlSignature.Replace("<", ("\\" + "u003c")).Replace(">", ("\\" + "u003e"));
mainO["verificationStatus"] = new MdIO.JSon.StringValue("accepted");
json = wc.UploadString("https://gmail.googleapis.com/gmail/v1/users/" + this.OwnerMail + "/settings/sendAs/" + json, "PUT", main.Value.ToJSON());
response = JSon.Parse(ref json);
if (response["sendAsEmail"].TryParseString(out json) && !string.IsNullOrEmpty(json)) { return true; }
}
}
}
}
return false;
}
I used svcUtil.exe to create classes from https://gw.sam.gov/SAMWS/1.0/Entity?wsdl
but I cannot for the life of me find what to deserialize the result into? When I created my own classes the root is envelope but it isn't even in the new classes.
I can paste the classes but it is really long? Is there a general answer to this?
I will paste the classes upon request.
Thanks in advance...
The classes are over 10x too long to post.
Adding code for the pull:
static void Main(string[] args)
{
ServicePointManager.Expect100Continue = true;
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
var _url = "https://gw.sam.gov/SAMWS/1.0/Entity";
//Run date is a specific date if provided otherwise use yesterday
DateTime startDateTime = DateTime.Now.AddDays(-1).Date;
for (int hr = 0; hr < 24; hr++)
{
XMLclassRequest xmlSoap = new XMLclassRequest();
string soap = xmlSoap.BuildSOAPrequest(startDateTime.AddHours(hr));
//string soap2 = xmlSoap.BuildSOAPrequest2(startDateTime.AddHours(hr));
string response = null; //This is the original pull with FAR and DFAR Responses
//string response2 = null; //This is FAR and DFAR
using (MyWebClient client = new MyWebClient())
{
client.Headers.Add("Content-Type", "text/xml;charset=utf-8");
client.Headers.Add("SOAPAction", "\"http://tempuri.org/IMyService/MyOperation\"");
try
{
response = client.UploadString(_url, soap);
//response2 = client.UploadString(_url, soap2);
}
catch
{ } //This will skip the hour attempted and move to next. The error I have been receiving is no data which is differently formatted XML that causes the error
}
//File.WriteAllText(#"D:\temp\bigpull.xml", response);
MemoryStream stream = null;
if (response != null)
{
byte[] byteArray = Encoding.Unicode.GetBytes(response);
stream = new MemoryStream(byteArray);
}
getEntities results;
XmlSerializer serializer = new XmlSerializer(typeof(getEntities));
try
{ results = (getEntities)serializer.Deserialize(stream); }
catch
{ } //This will skip the hour attempted and move to next. The error I have been receiving is no data which is differently formatted XML that causes the error
stream.Close();
string str;
}
The response is coming back. I just can't get it into an object to use.
I couldn't get this loaded into an object but ended up loading it to an XDocument and parsed that:
var _url = "https://gw.sam.gov/SAMWS/1.0/Entity";
//Run date is a specific date if provided otherwise use yesterday
DateTime startDateTime = DateTime.Now.AddDays(-1).Date;
for (int hr = 0; hr < 24; hr++)
{
XMLclassRequest xmlSoap = new XMLclassRequest();
string soap = xmlSoap.BuildSOAPrequest(startDateTime.AddHours(hr));
//string soap2 = xmlSoap.BuildSOAPrequest2(startDateTime.AddHours(hr));
string response = null; //This is the original pull with FAR and DFAR Responses
//string response2 = null; //This is FAR and DFAR
using (MyWebClient client = new MyWebClient())
{
client.Headers.Add("Content-Type", "text/xml;charset=utf-8");
client.Headers.Add("SOAPAction", "\"http://tempuri.org/IMyService/MyOperation\"");
try
{
response = client.UploadString(_url, soap);
//response2 = client.UploadString(_url, soap2);
}
catch
{ } //This will skip the hour attempted and move to next. The error I have been receiving is no data which is differently formatted XML that causes the error
}
//File.WriteAllText(#"D:\temp\far20190604.xml", response);
XDocument xdoc = XDocument.Parse(response);
var entities = from e in xdoc.Descendants("entity") select e;
foreach (var e in entities)
{
string DUNS = e.Descendants("DUNS").FirstOrDefault().Value;
var provisions = from p in e.Descendants("provision") select p;
foreach (var p in provisions)
{
string ParentID = p.Descendants("id").FirstOrDefault().Value;
var answers = from a in p.Descendants("answer") select a;
foreach (var a in answers)
{
var section = a.Descendants("section").Count()>0? a.Descendants("section").FirstOrDefault().Value : "";
var answerText = a.Descendants("answerText").Count() > 0 ? a.Descendants("answerText").FirstOrDefault().Value : "";
Console.WriteLine(DUNS + " " + ParentID + " " + section + " " + answerText);
}
}
}
Here's my code and I have doubt on thread safe implementation. My questions are below
The return value from GetHtmlPageAsync is object. Is it thread safe? I will use this object and add into the collection and finally upload into database.
The main method logic is below (implementation in-progress). I have set of domains, I have list of 10000 domains in the collection, the idea is, I will put it in the queue and call the GetHtmlPageAsync to get the HTML of the page. Based on the HTML, I will get the necessary hyperlinks. Once I get the hyper links, I will check certain word is available in the link. If the word is available in the link, I will call the same method GetHTMLPageAsync to get the HTML of that page. So the same thread may call the GetHtmlPageAsync to process another link. I am trying to reuse the same method for multiple calls in thread safe way. Please help.
#edit1 . I have added the main method. Instead of Queue. I have used ForEach
public static async Task<int> ProcessDomainAsync(List<string> domains)
{
Parallel.ForEach(domains, async (currentDomain) =>
{
var domainBody = await GetHtmlPageAsync(currentDomain);
var language = string.Empty;
var country = string.Empty;
var createdOn = DateTime.SpecifyKind(DateTime.Now, DateTimeKind.Local);
var updatedOn = DateTime.SpecifyKind(DateTime.Now, DateTimeKind.Local);
var machine = Environment.MachineName;
var message = "[" + domainBody.ErrorCode + "] - " + domainBody.ErrorMessage;
var active = false;
var stage = "End";
var url = currentDomain;
if (domainBody.ErrorCode == 0)
{
var html = domainBody.Body;
language = Common.GetLanguageIdentification(html);
country = Common.GetCountryIdentification(currentDomain);
message = string.Empty;
active = true;
stage = "Stage1";
var hyperLinks = Common.GetAllAHrefTags(html);
//Process Hyper Links
}
_domainList.Add(new Domain
{
Url = url,
Language = language,
Country = country,
MachineName = machine,
Message = message,
Active = active,
Stage = stage,
CreatedOn = createdOn,
UpdatedOn = updatedOn
});
domainCount++;
});
return domainCount;
}
public class DomainBody
{
public string Body;
public string ErrorMessage;
public int ErrorCode;
}
public static class DomainProcessing {
static async Task<DomainBody> GetHtmlPageAsync(string url)
{
#region Initialize Proxy
var sessionId = new Random().Next().ToString();
var proxy = new WebProxy(Constant.ProxyUrl, Constant.ProxyPort);
var login = Constant.ProxyUserName + "-session-" + sessionId;
proxy.Credentials = new NetworkCredential(login,Constant.ProxyPassword);
#endregion
#region Initialize Variables
var user_agent = Common.GenerateRandomUserAgent();
var body = string.Empty;
var errorCode = 0;
var errorMessage = string.Empty;
#endregion
try
{
#region Format URL with Http Protocol
var domainSB = new StringBuilder();
domainSB.Append("http://");
domainSB.Append(url);
#endregion
#region Process Domain
var request = (HttpWebRequest) WebRequest.Create(new Uri(url));
request.Proxy = proxy;
request.UserAgent = user_agent;
request.Timeout = Constant.TimeOut;
using (var response = await request.GetResponseAsync().ConfigureAwait(true))
using (var content = new MemoryStream())
using (var responseStream = response.GetResponseStream())
{
await responseStream.CopyToAsync(content);
var bodyArray = content.ToArray();
body = Encoding.UTF8.GetString(bodyArray, 0, bodyArray.Length);
}
errorCode = 0;
errorMessage = string.Empty;
#endregion
}
catch (HttpRequestException ex)
{
body = string.Empty;
errorCode = ex.InnerException.HResult;
errorMessage = ex.InnerException.Message;
}
catch (Exception ex)
{
body = string.Empty;
errorCode = ex.HResult;
errorMessage = ex.Message;
}
var domainBody = new DomainBody
{
Body = body,
ErrorCode = errorCode,
ErrorMessage = errorMessage
};
return domainBody;
}
}enter code here
Generally speaking, local variables should be thread safe (simply because they have no idea there even is another thread and other threads have no way to access them).
Anything that can be accessed by multiple threads should be looked at. _domainList for example. Make sure the Add method is thread-safe because you are calling it potentially in parallel.
I'm using ScrapySharp to post a form to search cases on the Maryland Case Search web app.
Using Fiddler, I found the form name and form fields; however, every time I post, it always come back the initial search page, not the results.
Not sure what I'm missing, see code below.
Any assistance is truly appreciated.
string url = #"http://casesearch.courts.state.md.us/casesearch/processDisclaimer.jis?disclaimer=Y";
ScrapingBrowser Browser = new ScrapingBrowser();
Browser.AllowAutoRedirect = true;
Browser.AllowMetaRedirect = true;
WebPage PageResult = Browser.NavigateToPage(new Uri(url));
PageWebForm form = PageResult.FindForm("inquiryForm");
form["firstName"] = "";
form["middleName"] = "";
form["partyType"] = "";
form["filingStart"] = "";
form["filingEnd"] = "";
form["action"] = "Search";
form["company"] = "N";
form["countyName"] = "MONTGOMERY COUNTY";
form["courtSystem"] = "B";
form["filingDate"] = "4/4/2016";
form["lastName"] = "A";
form["site"] = "CIVIL";
form.Method = HttpVerb.Post;
WebPage results = form.Submit();
Console.WriteLine(results.ToString());
You need to make async calls.
Ex:
WebPage mainPage = await browser.NavigateToPageAsync(new Uri(url), HttpVerb.Get,"", "text/html; charset=UTF-8");
PageWebForm form = mainPage.FindFormById("some-form_id");
...
WebPage web = await Task.Run(() => form.Submit()); // submit is not async, so let's force it
Try this:
form.FormFields.Where(f => f.Name == "countyName").FirstOrDefault().Value = "MONTGOMERY COUNTY";
For the purposes of user tracking, I have an etagged javascript snippet which i return from a .aspx page.
The relevant code is below:
string eTag = this.Request.Headers["If-None-Match"];
if (eTag != null)
{
Response.StatusCode = 304;
Response.SuppressContent = true;
}
else
{
string data = "asdfasdfs";
string script = "(function(){DoSomething('" + data + "');})();";
Response.ContentType = "application/javascript";
Response.Cache.SetLastModified(DateTime.Now);
Response.Cache.SetCacheability(HttpCacheability.Public);
Response.Cache.SetETag(data);
Response.Write(script);
}
I have converted this project to an MVC WebAPI2 (VS 2013) project, but I'm not sure exactly how to achieve the same functionality.
If this was a full MVC project I could create a custom ActionResult like in this example: http://blogs.microsoft.co.il/alon_nativ/2011/07/10/aspnet-mvc-imagepixel-actionresult/ . But, WebAPI methods do not normally return an ActionResult.
I believe I could use HttpResponseMessage - http://msdn.microsoft.com/en-us/library/system.net.http.httpresponsemessage(v=vs.110).aspx, but this class appears to be missing several things from the code above.
What is the cleanest/best practice way to do this in WebAPI?
I was able to individually track down each item, appears to be working.
Hope this helps someone in the future.
HttpResponseMessage response = null;
var eTag = Request.Headers.IfNoneMatch;
if (eTag != null && eTag.Count > 0)
{
response = Request.CreateResponse(HttpStatusCode.NotModified);
}
else
{
string data = "asdfasdfs";
string script = "(function(){DoSomething('" + data + "');})();";
response = Request.CreateResponse(HttpStatusCode.OK);
response.Content = new StringContent(script, Encoding.UTF8, "application/javascript");
response.Content.Headers.LastModified = DateTime.Now;
response.Headers.CacheControl = new CacheControlHeaderValue() { Public = true };
response.Headers.ETag = new EntityTagHeaderValue("\"" + data + "\"");
}