Develop Reference

Fetch TLD from URI via C# [duplicate]

I am using C# and ASP.NET for this.
We receive a lot of "strange" requests on our IIS 6.0 servers and I want to log and catalog these by domain.
Eg. we get some strange requests like these:
http://www.poker.winner4ever.example.com/
http://www.hotgirls.example.com/
http://santaclaus.example.com/
http://m.example.com/
http://wap.example.com/
http://iphone.example.com/
the latter three are kinda obvious, but I would like to sort them all into one as "example.com" IS hosted on our servers. The rest isn't, sorry :-)
So I am looking for some good ideas for how to retrieve example.com from the above. Secondly I would like to match the m., wap., iphone etc into a group, but that's probably just a quick lookup in a list of mobile shortcuts.I could handcode this list for a start.
But is regexp the answer here or is pure string manipulation the easiest way? I was thinking of "splitting" the URL string by "." and the look for item[0] and item[1]...
Any ideas?

You can use the following nuget Nager.PublicSuffix package. It uses the same data source that browser vendors use.
nuget
PM> Install-Package Nager.PublicSuffix
Example
var domainParser = new DomainParser(new WebTldRuleProvider());
var domainInfo = domainParser.Parse("sub.test.co.uk");
//domainInfo.Domain = "test";
//domainInfo.Hostname = "sub.test.co.uk";
//domainInfo.RegistrableDomain = "test.co.uk";
//domainInfo.SubDomain = "sub";
//domainInfo.TLD = "co.uk";

The following code uses the Uri class to obtain the host name, and then obtains the second level host (examplecompany.com) from Uri.Host by splitting the host name on periods.
var uri = new Uri("http://www.poker.winner4ever.examplecompany.com/");
var splitHostName = uri.Host.Split('.');
if (splitHostName.Length >= 2)
{
var secondLevelHostName = splitHostName[splitHostName.Length - 2] + "." +
splitHostName[splitHostName.Length - 1];
}

There may be some examples where this returns something other than what is desired, but country codes are the only ones that are 2 characters, and they may or may not have a short second level (2 or 3 characters) typically used. Therefore, this will give you what you want in most cases:
string GetRootDomain(string host)
{
string[] domains = host.Split('.');
if (domains.Length >= 3)
{
int c = domains.Length;
// handle international country code TLDs
// www.amazon.co.uk => amazon.co.uk
if (domains[c - 1].Length < 3 && domains[c - 2].Length <= 3)
return string.Join(".", domains, c - 3, 3);
else
return string.Join(".", domains, c - 2, 2);
}
else
return host;
}

This is not possible without a up-to-date database of different domain levels.
Consider:
s1.moh.gov.cn
moh.gov.cn
s1.google.com
google.com
Then at which level you want to get the domain? It's completely depends of the TLD, SLD, ccTLD... because ccTLD in under control of countries they may define very special SLD which is unknown to you.

I've written a library for use in .NET 2+ to help pick out the domain components of a URL.
More details are on github but one benefit over previous options is that it can download the latest data from http://publicsuffix.org automatically (once per month) so the output from the library should be more-or-less on a par with the output used by web browsers to establish domain security boundaries (i.e. pretty good).
It's not perfect yet but suits my needs and shouldn't take much work to adapt to other use cases so please fork and send a pull request if you want.

Use a regular expression:
^https?://([\w./]+[^.])?\.?(\w+\.(com)|(co.uk)|(com.au))$
This will match any URL ending with a TLD in which you are interested. Extend the list for as many as you want. Further, the capturing groups will contain the subdomain, hostname and TLD respectively.

uri.Host.ToLower().Replace("www.","").Substring(uri.Host.ToLower().Replace("www.","").IndexOf('.'))
returns ".com" for
Uri uri = new Uri("http://stackoverflow.com/questions/4643227/top-level-domain-from-url-in-c");
returns ".co.jp" for
Uri uri = new Uri("http://stackoverflow.co.jp");
returns ".s1.moh.gov.cn" for
Uri uri = new Uri("http://stackoverflow.s1.moh.gov.cn");
etc.

Octopus client, getting version from project name in C#

First of, I am completely new to octopus client, used it for the first time just before posting this.
So, I've been landed with this project to update the version number on a webpage monitoring some of our octopus deployed projects. I have been looking around the octopus client and not really gotten anywhere. The best I have so far is:
OctopusServerEndpoint endPoint = new OctopusServerEndpoint(server, apiKey);
OctopusRepository repo = new OctopusRepository(endPoint);
var releases = repo.Releases.FindAll();
From these releases I can get the ProjectId and even the Version, the issue is that releases is 600 strong and I am only looking for 15 of them.
The existing code I have to work from used to parse the version from local files so that is all out the window. Also, the existing code only deals with the actual names of the projects, like "AWOBridge", not their ProjectId, which is "Projects-27".
Right now my only option is to manually write up a keyList or map to correlate the names I have with the IDs in the octopus client, which I of course rather not since it is not very extendable or good code practice in my opinion.
So if anyone has any idea on how to use the names directly with octopus client and get the version number from that I would very much appriciate it.
I'll be getting down into octopus client while waiting. Let's see if I beat you to it!

Guess I beat you to it!
I'll just leave an answer here if anyone ever has the same problem.
I ended up using the dashboardto get what I needed:
OctopusServerEndpoint endPoint = new OctopusServerEndpoint(server, apiKey);
OctopusRepository repo = new OctopusRepository(endPoint);
DashboardResource dash = repo.Dashboards.GetDashboard();
List<DashboardItemResource> items = dash.Items;
DashboardItemResource item = new DashboardItemResource();
List<DashboardProjectResource> projs = dash.Projects;
var projID = projs.Find(x => x.Name == projectName).Id;
item = items.Find(x => x.ProjectId == projID && x.IsCurrent == true);
The dashboard is great since it contains all the info that the web dashboard shows. So you can use Project, Release, Deployment and Environment with all the information they contain.
Hope this helps someone in the future!

I'm using LINQPad to run C# snippets for Octopus automation using the Octopus Client library and I have come up with following to get any version of a project making use of Regular expression pattern. It works quite well if you use Pre-release semantic versioning.
For example to get latest release for a project:
var project = Repo.Projects.FindByName("MyProjectName");
var release = GetReleaseForProject(project);
To get specific release use that has 'rc1' in the version for example (also useful if you use source code branch name in the version published to Octopus:
var release = GetReleaseForProject(project, "rc1");
public ReleaseResource GetReleaseForProject(ProjectResource project, string versionPattern = "")
{
// create compiled regex expression to use for search
var regex = new Regex(versionPattern, RegexOptions.Compiled | RegexOptions.CultureInvariant | RegexOptions.IgnoreCase);
var releases = Repo.Projects.GetReleases(project);
if (!string.IsNullOrWhiteSpace(versionPattern) && !releases.Items.Any(r => regex.IsMatch(r.Version)))
{
return null;
}
return (!string.IsNullOrWhiteSpace(versionPattern)) ? releases.Items.Where(r => regex.IsMatch(r.Version))?.First() : releases.Items?.First();;
}

V1APIConnector deprecated

I'm getting a warning while building my asp.net project that V1APIConnector is deprecated and requesting me to use VersionOneAPIConnector.
Using the new VersionOneAPIConnector I had to do the following to get a child projects:
build a string (ex: https://abc.org/V1/rest-1.v1/Data/Scope?where=Scope.Name='FOO'&sel=Children.Name) for the data we require and then pass it on to GetData method.
read the returned stream and then create an xml.
Read the xml to extract the required data.
Retrieving data using V1APIconnector was much simpler.
create a IMetaModel and services instance
query the appropriate assettype (GetAssetType method) and the attributes
(getAttributedefinition)
Is the above approach still valid using VersionOneAPIConnector?
If yes do we have any example on how to get the childprojects of a project?
Thanks

The compilation warnings that you are getting are because VersionOne.SDK.APIClient is currently using internally V1APIconnector, and this warnings are only shown because, I guess, you are including VersionOne.SDK.APIClient inside your project, this approach is, sometimes, not advisable because you may miss updates. The best thing to do should be to use nuget. Add to VS a reference (Tools -> Options -> Packate Manager -> Package Sources) to V1 gallery on myget.org (https://www.myget.org/F/versionone/), in this way you'll have your DLLs updated.
Regarding your questions:
There's no need for you to directly use VersionOneAPIConnector, but here you have two examples of listing child projects: one using Metamodel as you mention that you were using, and another using VersionOneAPIConnector.
//Example 1: Listing all child projects of 'Foo' with MetaModel
var assetType = _context.MetaModel.GetAssetType("Scope");
var query = new Query(assetType);
//Filter Attribute
var parentNameAttribute = assetType.GetAttributeDefinition("Parent.Name");
//Filter
var filter = new FilterTerm(parentNameAttribute);
filter.Equal("Foo");
query.Filter = filter;
var result = _context.Services.Retrieve(query);
Example 2: Listing all child projects of 'Foo' with VersionOneAPIConnector and V1 user and password.
Stream streamResult = new VersionOneAPIConnector("https://abc.org/v1sdktesting/rest-1.v1/Data/Scope?where=Parent.Name=%27Foo%27").WithVersionOneUsernameAndPassword("usr","pwd").GetData();

BizTalk Dynamic Disassembler Problems - The body part is NULL

I started with the solution here http://social.technet.microsoft.com/wiki/contents/articles/20547.biztalk-server-dynamic-schema-resolver-real-scenario.aspx
which matches my scenario perfectly except for the send port, but that isn't necessary. I need the receive port to choose the file and apply a schema to disassemble. From their the orchestration does the mapping, some of it custom, etc.
I've done everything in the tutorial but I keep getting the following error.
"There was a failure executing the receive pipeline... The body part is NULL"
The things I don't get from the tutorial but don't believe they should be an issue are:
I created a new solution and project to make the custompipeline component (reference figure 19) and thus the dll file. Meaning it is on it's own namespace. However, it looks like from the tutorial they created the project within the main biztalk solution (ie the one with the pipeline and the orchestration) and thus the namespace has "TechNetWiki.SchemaResolver." in it. Should I make the custompipeline component have the namespace of my main solution? I'm assuming this shouldn't matter because I should be able to use this component in other solutions as it is meant to be generic to the business rules that are associated with the biztalk application.
The other piece I don't have is Figure 15 under the "THEN Action" they have it equal the destination schema they would like to disassemble to but then they put #Src1 at the end of "http://TechNetWiki.SchemaResolver.Schemas.SRC1_FF#Src1". What is the #Src1 for?

In the sample you've linked to, the probe method of the pipeline component is pushing the first 4 characters from the filename into a typed message that is then passed into the rules engine. Its those 4 characters that match the "SRC1" in the example.
string srcFileName = pInMsg.Context.Read("ReceivedFileName", "http://schemas.microsoft.com/BizTalk/2003/file-properties This link is external to TechNet Wiki. It will open in a new window. ").ToString();
srcFileName = Path.GetFileName(srcFileName);
//Substring the first four digits to take source code to use to call BRE API
string customerCode = srcFileName.Substring(0, 4);
//create an instance of the XML object
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.LoadXml(string.Format(#"<ns0:Root xmlns:ns0='http://TechNetWiki.SchemaResolver.Schemas.SchemaResolverBRE This link is external to TechNet Wiki. It will open in a new window. '>
<SrcCode>{0}</SrcCode>
<MessageType></MessageType>
</ns0:Root>", customerCode));
//retreive source code in case in our cache dictionary
if (cachedSources.ContainsKey(customerCode))
{
messageType = cachedSources[customerCode];
}
else
{
TypedXmlDocument typedXmlDocument = new TypedXmlDocument("TechNetWiki.SchemaResolver.Schemas.SchemaResolverBRE", xmlDoc);
Microsoft.RuleEngine.Policy policy = new Microsoft.RuleEngine.Policy("SchemaResolverPolicy");
policy.Execute(typedXmlDocument);
So the matching rule is based on the 1st 4 characters of the filename. If one isn't matched, the probe returns a false - i.e. unrecognised.
The final part is that the message type is pushed into the returned message - this is made up of the namespace and the root schema node with a # separator - so your #src1 is the root node.

You need to implement IProbeMessage near to class
I forgot to add IProbeMessage in the code of article. It is updated now.
but it is there in sample source code
Src1 is the the root node name of schema. I mentioned that in article that message type is TargetNamespace#Root
I recommend to download the sample code
I hope this will help you

SQLite invalid URI after updating System.Data.SQLite.dll

I first had this SQLite version: 1.0.77.0 (sqlite-netFx40-static-binary-bundle-Win32-2010-1.0.77.0)
and everything was working fine.
After updating my System.Data.SQLite.dll to version 1.0.82.0 (sqlite-netFx40-static-binary-bundle-Win32-2010-1.0.82.0) I now receive this:
Unable to open database. Connection string: 'datetimeformat=ISO8601;synchronous=Off;uri="file://C:/Users/username/Documents/data/test.db";version=3;journal mode=Truncate;default isolationlevel=ReadCommitted;pooling=True'; Error: 'System.InvalidOperationException: Invalid connection string: invalid URI
I've also tried file:/// instead of file:// without any luck.
Could anybody tell me what is wrong with my URI, why it doesn't work anymore in v1.0.82.0 but worked in v1.0.77.0?
http://system.data.sqlite.org/index.html/doc/trunk/www/downloads.wiki
(1.0.82.0 == 3.7.14)

The best way to get correct connection string is to always use the SQLiteConnectionStringBuilder class to generate them.
SQLiteConnectionStringBuilder conn_builder = new SQLiteConnectionStringBuilder
{
DateTimeFormat = SQLiteDateFormats.ISO8601,
SyncMode = SynchronizationModes.Off,
Version = 3,
JournalMode = SQLiteJournalModeEnum.Truncate,
DefaultIsolationLevel = System.Data.IsolationLevel.ReadCommitted,
Pooling = true,
Uri = new Uri(#"c:\tmp\db.sqlite").AbsoluteUri
};
string conn_str = conn_builder.ConnectionString;
if Uri is not working, try setting DataSource instead (if you are using a local file).
A quick check shows that DataSource is evaluated before Uri in SQLiteConnection.cs and takes precedence over it, so if there is a bug in the Uri handling, using DataSource may help bypassing it.

According to the docs starting from the version 1.0.82.0 it was changed:
Add support for URI file names via the new FullUri connection string
property.
Example:
var connection = new SQLiteConnection("FullUri=file:mydb.sqlite?cache=shared");

following works for me, for following example you can't not use Data Source
FullUri=file::memory:?cache=shared

See these URI filename examples, but your URI (with three slashes) looks correct.

I think you should try to strip down your connection string to the basics and then add the options.
Take a look at this site for example SQLite connection string options.
http://www.connectionstrings.com/sqlite
The 3 /// is only valid when you are trying to escape out the spaces in the uri, I would also say that you should try moving the db out of the users folder to the root on c:\ in case the access permissions are not valid for it to access the DB and it also means a simpler connection string for you to try.
Good luck