I am using C# and ASP.NET for this.
We receive a lot of "strange" requests on our IIS 6.0 servers and I want to log and catalog these by domain.
Eg. we get some strange requests like these:
http://www.poker.winner4ever.example.com/
http://www.hotgirls.example.com/
http://santaclaus.example.com/
http://m.example.com/
http://wap.example.com/
http://iphone.example.com/
the latter three are kinda obvious, but I would like to sort them all into one as "example.com" IS hosted on our servers. The rest isn't, sorry :-)
So I am looking for some good ideas for how to retrieve example.com from the above. Secondly I would like to match the m., wap., iphone etc into a group, but that's probably just a quick lookup in a list of mobile shortcuts.I could handcode this list for a start.
But is regexp the answer here or is pure string manipulation the easiest way? I was thinking of "splitting" the URL string by "." and the look for item[0] and item[1]...
Any ideas?
You can use the following nuget Nager.PublicSuffix package. It uses the same data source that browser vendors use.
nuget
PM> Install-Package Nager.PublicSuffix
Example
var domainParser = new DomainParser(new WebTldRuleProvider());
var domainInfo = domainParser.Parse("sub.test.co.uk");
//domainInfo.Domain = "test";
//domainInfo.Hostname = "sub.test.co.uk";
//domainInfo.RegistrableDomain = "test.co.uk";
//domainInfo.SubDomain = "sub";
//domainInfo.TLD = "co.uk";
The following code uses the Uri class to obtain the host name, and then obtains the second level host (examplecompany.com) from Uri.Host by splitting the host name on periods.
var uri = new Uri("http://www.poker.winner4ever.examplecompany.com/");
var splitHostName = uri.Host.Split('.');
if (splitHostName.Length >= 2)
{
var secondLevelHostName = splitHostName[splitHostName.Length - 2] + "." +
splitHostName[splitHostName.Length - 1];
}
There may be some examples where this returns something other than what is desired, but country codes are the only ones that are 2 characters, and they may or may not have a short second level (2 or 3 characters) typically used. Therefore, this will give you what you want in most cases:
string GetRootDomain(string host)
{
string[] domains = host.Split('.');
if (domains.Length >= 3)
{
int c = domains.Length;
// handle international country code TLDs
// www.amazon.co.uk => amazon.co.uk
if (domains[c - 1].Length < 3 && domains[c - 2].Length <= 3)
return string.Join(".", domains, c - 3, 3);
else
return string.Join(".", domains, c - 2, 2);
}
else
return host;
}
This is not possible without a up-to-date database of different domain levels.
Consider:
s1.moh.gov.cn
moh.gov.cn
s1.google.com
google.com
Then at which level you want to get the domain? It's completely depends of the TLD, SLD, ccTLD... because ccTLD in under control of countries they may define very special SLD which is unknown to you.
I've written a library for use in .NET 2+ to help pick out the domain components of a URL.
More details are on github but one benefit over previous options is that it can download the latest data from http://publicsuffix.org automatically (once per month) so the output from the library should be more-or-less on a par with the output used by web browsers to establish domain security boundaries (i.e. pretty good).
It's not perfect yet but suits my needs and shouldn't take much work to adapt to other use cases so please fork and send a pull request if you want.
Use a regular expression:
^https?://([\w./]+[^.])?\.?(\w+\.(com)|(co.uk)|(com.au))$
This will match any URL ending with a TLD in which you are interested. Extend the list for as many as you want. Further, the capturing groups will contain the subdomain, hostname and TLD respectively.
uri.Host.ToLower().Replace("www.","").Substring(uri.Host.ToLower().Replace("www.","").IndexOf('.'))
returns ".com" for
Uri uri = new Uri("http://stackoverflow.com/questions/4643227/top-level-domain-from-url-in-c");
returns ".co.jp" for
Uri uri = new Uri("http://stackoverflow.co.jp");
returns ".s1.moh.gov.cn" for
Uri uri = new Uri("http://stackoverflow.s1.moh.gov.cn");
etc.
Related
I am looking into trying to compare 2 URLs in C# for equal root domains, I.E.
sub.example.com matches example.com and othersub.example.com.
I had a look into using the Uri class, but that will parse the full URL including the subdomains.
I had thought about splitting the string at each . and then comparing the last 2 last elements which would normally be the root domain name, however that causes issues as often the TLD can also have a subdomain I.E. example.com.au would then match other.com.au.
I guess I'm hoping if anyone knows a NuGet library that can take into account common top level domains (including multi part ones) and then compare the actual specified domain
You can use Nager.PublicSuffix package.
Install via nuget:
PM> Install-Package Nager.PublicSuffix
Example:
var domainParser = new DomainParser(new WebTldRuleProvider());
var domainInfo = domainParser.Parse("sub.test.co.uk");
//domainInfo.Domain = "test";
//domainInfo.Hostname = "sub.test.co.uk";
//domainInfo.RegistrableDomain = "test.co.uk";
//domainInfo.SubDomain = "sub";
//domainInfo.TLD = "co.uk";
Then compare via the domainInfo.TLD
My documentation and Google-fu is seriously failing me on this one, so:
how do I use P4API's GetChangelist() function to sync a range of files (i.e. all files from #now to #twoDaysAgo)? I can easily construct the command line to do this like so:
p4 changes -s submitted //...#2016/12/01,2016/12/06
but the API wants me to interface with the server via
GetChangelist(Options options, FileSpec[] files)
It's driving me crazy that I have to construct a combo of Options and Filespecs[] to make the request instead, and (AFAIK) can't just pass the actual command line string. Especially because all documentation seems to be non-existent.
Can somebody enlighten me as to what kind of filespec parameters I have to pass along? (I think that's what I need to use to specify the fact that I want to get a range of all CLs inside a certain time?) Thanks!
(As an aside: I was surprised there isn't a "P4API" tag yet, and I can't create one.)
And here's the non-command line version that you really want to use, from the Perforce documentation (once you find it :))
PathSpec path = new DepotPath("//depot/...");
DateTimeVersion lowerTimeStamp = new DateTimeVersion(new DateTime(2016,12,06));
DateTimeVersion upperTimeStamp = new DateTimeVersion(DateTime.Now);
VersionSpec version = new VersionRange(lowerTimeStamp, upperTimeStamp);
FileSpec[] fileSpecs = { new FileSpec(path, version) };
ChangesCmdOptions changeListOptions = new ChangesCmdOptions(ChangesCmdFlags.FullDescription | ChangesCmdFlags.IncludeTime, null, 0, ChangeListStatus.None, null);
IList<Changelist> changes = m_Repository.GetChangelists(changeListOptions, fileSpecs);
Alright, after a couple more hours of digging, I have found that there is a way to feed the actual command line parameters to the command. You create a DepotSpec, and then something like this is working for me to restrict the time range for CLs retrieved from the server:
ChangesCmdOptions changeListOptions = new ChangesCmdOptions(ChangesCmdFlags.FullDescription|ChangesCmdFlags.IncludeTime, null, 0, ChangeListStatus.None, null);
FileSpec[] fileSpecs = new FileSpec[1] { new FileSpec(new DepotPath("//depot/...#2016/12/05 21:57:30,#now"), null, null, null) };
IList<Changelist> changes = m_Repository.GetChangelists(changeListOptions, fileSpecs);
All this might be "indulgent smile" old news to people who've worked with the API for a while. It's just all a bit confusing to newcomers when documentation like the two pages mentioned in this post ("FileSpec object docs", "SyncFiles method docs") are offline now: Perforce Api - How to command "get revision [changelist number]"
First of, I am completely new to octopus client, used it for the first time just before posting this.
So, I've been landed with this project to update the version number on a webpage monitoring some of our octopus deployed projects. I have been looking around the octopus client and not really gotten anywhere. The best I have so far is:
OctopusServerEndpoint endPoint = new OctopusServerEndpoint(server, apiKey);
OctopusRepository repo = new OctopusRepository(endPoint);
var releases = repo.Releases.FindAll();
From these releases I can get the ProjectId and even the Version, the issue is that releases is 600 strong and I am only looking for 15 of them.
The existing code I have to work from used to parse the version from local files so that is all out the window. Also, the existing code only deals with the actual names of the projects, like "AWOBridge", not their ProjectId, which is "Projects-27".
Right now my only option is to manually write up a keyList or map to correlate the names I have with the IDs in the octopus client, which I of course rather not since it is not very extendable or good code practice in my opinion.
So if anyone has any idea on how to use the names directly with octopus client and get the version number from that I would very much appriciate it.
I'll be getting down into octopus client while waiting. Let's see if I beat you to it!
Guess I beat you to it!
I'll just leave an answer here if anyone ever has the same problem.
I ended up using the dashboardto get what I needed:
OctopusServerEndpoint endPoint = new OctopusServerEndpoint(server, apiKey);
OctopusRepository repo = new OctopusRepository(endPoint);
DashboardResource dash = repo.Dashboards.GetDashboard();
List<DashboardItemResource> items = dash.Items;
DashboardItemResource item = new DashboardItemResource();
List<DashboardProjectResource> projs = dash.Projects;
var projID = projs.Find(x => x.Name == projectName).Id;
item = items.Find(x => x.ProjectId == projID && x.IsCurrent == true);
The dashboard is great since it contains all the info that the web dashboard shows. So you can use Project, Release, Deployment and Environment with all the information they contain.
Hope this helps someone in the future!
I'm using LINQPad to run C# snippets for Octopus automation using the Octopus Client library and I have come up with following to get any version of a project making use of Regular expression pattern. It works quite well if you use Pre-release semantic versioning.
For example to get latest release for a project:
var project = Repo.Projects.FindByName("MyProjectName");
var release = GetReleaseForProject(project);
To get specific release use that has 'rc1' in the version for example (also useful if you use source code branch name in the version published to Octopus:
var release = GetReleaseForProject(project, "rc1");
public ReleaseResource GetReleaseForProject(ProjectResource project, string versionPattern = "")
{
// create compiled regex expression to use for search
var regex = new Regex(versionPattern, RegexOptions.Compiled | RegexOptions.CultureInvariant | RegexOptions.IgnoreCase);
var releases = Repo.Projects.GetReleases(project);
if (!string.IsNullOrWhiteSpace(versionPattern) && !releases.Items.Any(r => regex.IsMatch(r.Version)))
{
return null;
}
return (!string.IsNullOrWhiteSpace(versionPattern)) ? releases.Items.Where(r => regex.IsMatch(r.Version))?.First() : releases.Items?.First();;
}
I am getting data from a web service end point and place it into a list in a for each loop. The service gets it's data from a Wordpress website.
var list = new ItemList
(
(string)data.id.ToString(),
(string)data.name,
(string)subcategory
);
I then print this on the XAML page. The code works fine in that it successfully gets the data from the service and prints it on the page of my windows 8 app.
However in (string)data.name,, which is the name of the items, if the name contains a "&" it shows up in the app as $#038;. Also if a item name contains a "'", apostrophe s, it shows up as ’.
EG. D & G, shows up as D $#038; G
The "&" and "'" show up as these weird symbols.
How do I get rid of these and fix it so that they render correctly in the app.
I'm going to take the risk of giving you a wrong hint, because I guess you're talking about a Windows 8 Store App (XAML), thus you don't have access to every class on .NET, but...
What about decoding HTML entities?
Check this HttpUtility method: HtmlUtility.HtmlDecode.
Check WebUtility.HtmlDecode, which is on System.dll, thus available for Windows 8 Store Apps.
You'll need to add a reference to System.Web on your Visual Studio project.
It looks like the service is returning XML escaped entities. & means a character with a code of (decimal) 38 (which is &). ’ is similar and means a code of 8217 (which is ’).
You can decode these using System.Web.HttpUtility.HtmlDecode(inputString), but that requires a reference to System.Web. If you don't want to or cannot reference that, you can try something like this:
var xml = new XmlDocument();
xml.LoadXml("<x>" + inputString + "</x>");
var output = xml.InnerText;
Given Testing ’stuff" & things, it will return Testing ’stuff" & things.
I'd go with HtmlDecode() if you can, but absolutely try and avoid rolling your own decoder unless you have no other choice.
You can use WebUtility.HtmlDecode Method (String)
Or you can use if you don't want to add additional libraries.
public string Decode(string text)
{
var replacements = new Dictionary<string, char> {
{ "’", ''' },
// ...etc
}
var sb = new StringBuilder( text );
foreach( var c in replacements.Keys ) {
sb.Replace( c.ToString(), replacements[c] );
}
return sb.ToString();
}
I am working with .net 4.0 c#.
I want to be able to get the url from the current http request, including any virtual directory. So for example (request and sought value):
http://www.website.com/shop/test.aspx -> http://www.website.com/shop/
http://www.website.com/test.aspx -> http://www.website.com/
http://website.com/test.aspx -> http://website.com/
How is it possible to achieve this?
This is what I use
HttpContext.Current.Request.Url.GetLeftPart(UriPartial.Authority) + HttpContext.Current.Request.ApplicationPath;
Request.Url should contain everything you need. At that point it's a matter of checking the string, and what you prefer to grab from it. I've used AbsoluteUri before, and it works.
This example isn't fool proof, but you should be able to figure out what you need from this:
string Uri = Request.Url.AbsoluteUri;
string Output = Uri.Substring(0, Uri.LastIndexOf('/') + 1 );
This solution could work and is shorter:
string url = (new Uri(Request.Url, ".")).OriginalString;
This should work
Request.Url.OriginalString.Substring(0, Request.Url.OriginalString.LastIndexOf(Request.FilePath.Substring(Request.FilePath.LastIndexOf("/")))) + "/"