I am building a web service that serves geographic boundary data in JSON format.
The geographic data is stored in an SQL Server 2008 R2 database using the geography type in a table. I use [ColumnName].ToString() method to return the polygon data as text.
Example output:
POLYGON ((-6.1646509904325884 56.435153006374627, ... -6.1606079906751 56.4338050060666))
MULTIPOLYGON (((-6.1646509904325884 56.435153006374627 0 0, ... -6.1606079906751 56.4338050060666 0 0)))
Geographic definitions can take the form of either an array of lat/long pairs defining a polygon or in the case of multiple definitions, an array or polygons (multipolygon).
I have the following regex that converts the output to JSON objects contained in multi-dimensional arrays depending on the output.
Regex latlngMatch = new Regex(#"(-?[0-9]{1}\.\d*)\s(\d{2}.\d*)(?:\s0\s0,?)?", RegexOptions.Compiled);
private string ConvertPolysToJson(string polysIn)
{
return this.latlngMatch.Replace(polysIn.Remove(0, polysIn.IndexOf("(")) // remove POLYGON or MULTIPOLYGON
.Replace("(", "[") // convert to JSON array syntax
.Replace(")", "]"), // same as above
"{lng:$1,lat:$2},"); // reformat lat/lng pairs to JSON objects
}
This is actually working pretty well and converts the DB output to JSON on the fly in response to an operation call.
However I am no regex master and the calls to String.Replace() also seem inefficient to me.
Does anyone have any suggestions/comments about performance of this?
Again just to just to close this off I will answer my own question with the solution im using.
This method takes the output from a ToString() call on an a MS SQL Geography Type.
If the string returned contains polygon data contructed form GPS points, this method will parse and reformatted it to a JSON sting.
public static class PolyConverter
{
static Regex latlngMatch = new Regex(#"(-?\d{1,2}\.\dE-\d+|-?\d{1,2}\.?\d*)\s(-?\d{1,2}\.\dE-\d+|-?\d{1,2}\.?\d*)\s?0?\s?0?,?", RegexOptions.Compiled);
static Regex reformat = new Regex(#"\[,", RegexOptions.Compiled);
public static string ConvertPolysToJson(string polysIn)
{
var formatted = reformat.Replace(
latlngMatch.Replace(
polysIn.Remove(0, polysIn.IndexOf("(")), ",{lng:$1,lat:$2}")
.Replace("(", "[")
.Replace(")", "]"), "[");
if (polysIn.Contains("MULTIPOLYGON"))
{
formatted = formatted.Replace("[[", "[")
.Replace("]]", "]")
.Replace("[[[", "[[")
.Replace("]]]", "]]");
}
return formatted;
}
}
This is specific to my apllication, but maybe useful to somebody and maybe even create a better implementation.
To convert from WKT to GeoJson you can use NetTopologySuite from nuget. Add NetTopologySuite and NetTopologySuite.IO.GeoJSON
var wkt = "POLYGON ((10 20, 30 40, 50 60, 10 20))";
var wktReader = new NetTopologySuite.IO.WKTReader();
var geom = wktReader.Read(wkt);
var feature = new NetTopologySuite.Features.Feature(geom, new NetTopologySuite.Features.AttributesTable());
var featureCollection = new NetTopologySuite.Features.FeatureCollection();
featureCollection.Add(feature);
var sb = new StringBuilder();
var serializer = new NetTopologySuite.IO.GeoJsonSerializer();
serializer.Formatting = Newtonsoft.Json.Formatting.Indented;
using (var sw = new StringWriter(sb))
{
serializer.Serialize(sw, featureCollection);
}
var result = sb.ToString();
Output:
{
"features": [
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
10.0,
20.0
],
[
30.0,
40.0
],
[
50.0,
60.0
],
[
10.0,
20.0
]
]
]
},
"properties": {}
}
],
"type": "FeatureCollection"
}
To answer your question about efficiency, For this particular case, I don't think that Replace vs RegEx is going to be that big of a difference. All we are really changing is some parenthesis and commas. Personally, I prefer to do things in TSQL for web applications because I can offload the computational work onto the SQL Server instead of the Web Server. In my case I have a lot of data that I am generating for a map and therefore don't want to bog down the webserver with lots of conversions of data. Additionally, for performance, I usually put more horsepower on the SQL server than I do a webserver, so even if there is some difference between the two functions, if Replace is less efficient it is at least being handled by a server with lots more resources. In general, I want my webserver handling connections to clients and my SQL server handling data computations. This also keeps my web server scripts clean and efficient. So my suggestion is as follows:
Write a Scalar TSQL function in your database. This uses the SQL REPLACE function and is somewhat brute force, but it performs really well. This function can be used directly on a SELECT statement or to create calculated columns in a table if you really want to simplify your web server code. Currently this example only supports POINT, POLYGON and MULTIPOLYGON and provides the "geometry" JSON element for the geoJSON format.
GetGeoJSON Scalar Function
CREATE FUNCTION GetGeoJSON (#geo geography) /*this is your geography shape*/
RETURNS varchar(max)
WITH SCHEMABINDING /*this tells SQL SERVER that it is deterministic (helpful if you use it in a calculated column)*/
AS
BEGIN
/* Declare the return variable here*/
DECLARE #Result varchar(max)
/*Build JSON "geometry" element for geoJSON*/
SELECT #Result = '"geometry":{' +
CASE #geo.STGeometryType()
WHEN 'POINT' THEN
'"type": "Point","coordinates":' +
REPLACE(REPLACE(REPLACE(REPLACE(#geo.ToString(),'POINT ',''),'(','['),')',']'),' ',',')
WHEN 'POLYGON' THEN
'"type": "Polygon","coordinates":' +
'[' + REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(#geo.ToString(),'POLYGON ',''),'(','['),')',']'),'], ',']],['),', ','],['),' ',',') + ']'
WHEN 'MULTIPOLYGON' THEN
'"type": "MultiPolygon","coordinates":' +
'[' + REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(#geo.ToString(),'MULTIPOLYGON ',''),'(','['),')',']'),'], ',']],['),', ','],['),' ',',') + ']'
ELSE NULL
END
+'}'
/* Return the result of the function*/
RETURN #Result
END
Next, use your GetGeoJSON function in your SELECT statement, for example:
SELECT dbo.GetGeoJSON([COLUMN]) as Geometry From [TABLE]
I hope this provides some insight and helps others looking for a methodology, good luck!
The method outlined in James's answer works great. But I recently found an error when converting WKT where the Longitude had a value over 99.
I changed the regular expression:
#"(-?\d{1,2}\.\dE-\d+|-?\d{1,3}\.?\d*)\s(-?\d{1,2}\.\dE-\d+|-?\d{1,2}\.?\d*)\s?0?\s?0?,?"
Notice the second "2" has been changed to a "3" to allow longitude to go up to 180.
Strings are immutable in .net, so when you replacing some, you creating an edited copy of previous string. This is not so critical for performance, as for memory usage.
Look at JSON.net
Or use StringBuilder to generate it properly.
StringBuilder sb = new StringBuilder();
sb.AppendFormat();
Utility function that is used for formatting spatial cells as GeoJSON is shown below.
DROP FUNCTION IF EXISTS dbo.geometry2json
GO
CREATE FUNCTION dbo.geometry2json( #geo geometry)
RETURNS nvarchar(MAX) AS
BEGIN
RETURN (
'{' +
(CASE #geo.STGeometryType()
WHEN 'POINT' THEN
'"type": "Point","coordinates":' +
REPLACE(REPLACE(REPLACE(REPLACE(#geo.ToString(),'POINT ',''),'(','['),')',']'),' ',',')
WHEN 'POLYGON' THEN
'"type": "Polygon","coordinates":' +
'[' + REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(#geo.ToString(),'POLYGON ',''),'(','['),')',']'),'], ',']],['),', ','],['),' ',',') + ']'
WHEN 'MULTIPOLYGON' THEN
'"type": "MultiPolygon","coordinates":' +
'[' + REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(#geo.ToString(),'MULTIPOLYGON ',''),'(','['),')',']'),'], ',']],['),', ','],['),' ',',') + ']'
WHEN 'MULTIPOINT' THEN
'"type": "MultiPoint","coordinates":' +
'[' + REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(#geo.ToString(),'MULTIPOINT ',''),'(','['),')',']'),'], ',']],['),', ','],['),' ',',') + ']'
WHEN 'LINESTRING' THEN
'"type": "LineString","coordinates":' +
'[' + REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(#geo.ToString(),'LINESTRING ',''),'(','['),')',']'),'], ',']],['),', ','],['),' ',',') + ']'
ELSE NULL
END)
+'}')
END
Related
I have a Table in my database where on of the columns is of type "geometry".
I want to take the data contained in this column (polygons and circles) and convert them into something easily parse-able and rendered in HTML (preferably as SVG), but i cannot seem to be able to do it.
I have been able to extract GML and WKT from the database, but neither of these seem to get me closer to my goal without additional parsing on the front-end that will be prone to error. Are there any built-ins or third party librarys that are capable of doing this? Is there a simple conversion I can make that I am overlooking? I am at a loss to why this cant be done, as i would have thought this would be a useful tool others would need.
It is relatively easy to convert to GeoJson from GEOMETRY and GEOGRAPHY SQL types.
From there it is easier to convert to SVG.
Edit:
NPM package for GeoJson to SVG converter
GitHub repo for another GeoJson to SVG converter
SQL: Returning Spatial Data in GeoJson Format (part 1)
SQL: Returning Spatial Data in GeoJson Format (part 1)
And the relevant SQL function from the second tutorial:
CREATE FUNCTION dbo.geometry2json ( #geo GEOMETRY )
RETURNS NVARCHAR(MAX)
AS
BEGIN
RETURN (
'{' +
(CASE #geo.STGeometryType()
WHEN 'POINT' THEN
'"type": "Point","coordinates":' +
REPLACE(REPLACE(REPLACE(REPLACE(#geo.ToString(),'POINT ',''),'(','['),')',']'),' ',',')
WHEN 'POLYGON' THEN
'"type": "Polygon","coordinates":' +
'[' + REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(#geo.ToString(),'POLYGON ',''),'(','['),')',']'),'], ',']],['),', ','],['),' ',',') + ']'
WHEN 'MULTIPOLYGON' THEN
'"type": "MultiPolygon","coordinates":' +
'[' + REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(#geo.ToString(),'MULTIPOLYGON ',''),'(','['),')',']'),'], ',']],['),', ','],['),' ',',') + ']'
WHEN 'MULTIPOINT' THEN
'"type": "MultiPoint","coordinates":' +
'[' + REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(#geo.ToString(),'MULTIPOINT ',''),'(','['),')',']'),'], ',']],['),', ','],['),' ',',') + ']'
WHEN 'LINESTRING' THEN
'"type": "LineString","coordinates":' +
'[' + REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(#geo.ToString(),'LINESTRING ',''),'(','['),')',']'),'], ',']],['),', ','],['),' ',',') + ']'
ELSE NULL
END)
+'}');
END;
The Data:
{
\"value\": 17.11, \"year\": 2015, \"sub\": [ {\"x\": 0, \"y\": 0.94 }, {\"x\": 1, \"y\": 1.08 }]
}
,
{
\"value\": 17.23, \"year\": 2015, \"sub\": [ {\"x\": 0, \"y\": 0.23 }, {\"x\": 1, \"y\": 1.22 }]
}
I've got a list of JSON objects in a format you see above and I need to split the objects at the ',' between the two objects. The problem is that there are other commas present in the file. Apart from actually serializing the JSON into a List, is there any other way to get this done?
I'm trying to get the data into a string array like:
string[] split = json.split(',');
Note that the data above is actually all coming on one line, there aren't any line breaks, tabs, or anything. I organized it above to make it more readable.
Writing your own parser would be tricky. It would be much easier if you used a JSON parser. Your format is not valid JSON, but it would be with surrounding []. So if you insert those characters, you should be able to use a real parser:
using Newtonsoft.Json
// ...
var objects = JsonConvert.DeserializeObject<List<SomeClass>>("[" + json + "]");
Guys I have a big headaches trying to serialize as3 file to json with C#.
Right now i stumbled on with this =>
"licvarreelVideosConfig":[{
url: "ChoiceSlot2/GEOLJSlot/videos/00.flv",
width: 224,
height: 224,
onWholeReel: false,
transparent: true
}, {
url:"ChoiceSlot2/GEOLJSlot/videos/01.flv",
width: 224,
height: 224,
onWholeReel: false,
transparent: true
}]
Lets say I generate the json keys based on what is given from the as3 file.
But in some of the classes there are missing double quotes in the keys.
Any easy way to properly add them ?
Thanks in advance
If the properties are not quoted, then you can't really call this JSON.
According to this site, in all the standards, except for RFC 7159, the whole content has to be wrapped in { }
Putting aside these, a quick solution that comes to my mind involves using a regular expression to replace the unquoted property names with quoted ones.
Example
var unquotedJson = "\"licvarreelVideosConfig\":[{" +
"url: \"ChoiceSlot2/GEOLJSlot/videos/00.flv\"," +
"width: 224," +
"height: 224," +
"onWholeReel: false," +
"transparent: true" +
"}, {" +
"url:\"ChoiceSlot2/GEOLJSlot/videos/01.flv\"," +
"width: 224," +
"height: 224," +
"onWholeReel: false," +
"transparent: true" +
"}]";
var quotedJson = new Regex("([a-zA-Z0-9_$]+?):(.*?)[,]{0,1}").Replace(unquotedJson, "\"$1\":$2");
// if the serializer needs nested { ... }
// var nestedQuotedJson = string.Format("{{{0}}}", quotedJson);
// do the serialization
Note, this is really not comprehensive, it only supports property names with a-z, A-Z, 0-9, $ and _ characters in them.
I am using Irony to create a parser for a scripting language, but I've come across a little problem: how do I translate an EBNF expression like this in Irony?
'(' [ Ident { ',' Ident } ] ')'
I already tried some tricks like
Chunk.Rule = (Ident | Ident + "," + Chunk);
CallArgs.Rule = '(' + Chunk + ')' | '(' + ')';
But it's ugly and I'm not even sure if that works the way it should (haven't tried it yet...). Has anyone any suggestions?
EDIT:
I found out these helper methods (MakeStarList, MakePlusList) but couldn't find out how to use them, because of the complete lack of documentation of Irony... Has anyone any clue?
// Declare the non-terminals
var Ident = new NonTerminal("Ident");
var IdentList = new NonTerminal("Term");
// Rules
IdentList.Rule = ToTerm("(") + MakePlusRule(IdentList, ",", Ident) + ")";
Ident.Rule = // specify whatever Ident is (I assume you mean an identifier of some kind).
You can use the MakePlusRule helper method to define a one-or-many occurrence of some terminal. The MakePlusRule is basically just present your terminals as standard recursive list-idiom:
Ident | IdentList + "," + Ident
It also marks the terminal as representing a list, which will tell the parser to unfold the list-tree as a convenient list of child nodes.
Given the following scenario, I am wondering if a better solution could be written with Regular Expressions for which I am not very familiar with yet. I am seeing holes in my basic c# string manipulation even though it somewhat works. Your thoughts and ideas are most appreciated.
Thanks much,
Craig
Given the string "story" below, write a script to do the following:
Variable text is enclosed by { }.
If the variable text is blank, remove any other text enclosed in [ ].
Text to be removed can be nested deep with [ ].
Format:
XYZ Company [- Phone: [({404}) ]{321-4321} [Ext: {6789}]]
Examples:
All variable text filled in.
XYZ Company - Phone: (404) 321-4321 Ext: 6789
No Extension entered, remove "Ext:".
XYZ Company - Phone: (404) 321-4321
No Extension and no area code entered, remove "Ext:" and "( ) ".
XYZ Company - Phone: 321-4321
No extension, no phone number, and no area code, remove "Ext:" and "( ) " and "- Phone: ".
XYZ Company
Here is my solution with plain string manipulation.
private string StoryManipulation(string theStory)
{
// Loop through story while there are still curly brackets
while (theStory.IndexOf("{") > 0)
{
// Extract the first curly text area
string lcCurlyText = StringUtils.ExtractString(theStory, "{", "}");
// Look for surrounding brackets and blank all text between
if (String.IsNullOrWhiteSpace(lcCurlyText))
{
for (int lnCounter = theStory.IndexOf("{"); lnCounter >= 0; lnCounter--)
{
if (theStory.Substring(lnCounter - 1, 1) == "[")
{
string lcSquareText = StringUtils.ExtractString(theStory.Substring(lnCounter - 1), "[", "]");
theStory = StringUtils.ReplaceString(theStory, ("[" + lcSquareText + "]"), "", false);
break;
}
}
}
else
{
// Replace current curly brackets surrounding the text
theStory = StringUtils.ReplaceString(theStory, ("{" + lcCurlyText + "}"), lcCurlyText, false);
}
}
// Replace all brackets with blank (-1 all instances)
theStory = StringUtils.ReplaceStringInstance(theStory, "[", "", -1, false);
theStory = StringUtils.ReplaceStringInstance(theStory, "]", "", -1, false);
return theStory.Trim();
}
Dealing with nested structures is generally beyond the scope of regular expressions. But I think there is a solution, if you run the regex replacement in a loop, starting from the inside out. You will need a callback-function though (a MatchEvaluator):
string ReplaceCallback(Match match)
{
if(String.IsNullOrWhiteSpace(match.Groups[2])
return "";
else
return match.Groups[1]+match.Groups[2]+match.Groups[3];
}
Then you can create the evaluator:
MatchEvaluator evaluator = new MatchEvaluator(ReplaceCallback);
And then you can call this in a loop until the replacement does not change anything any more:
newString = Regex.Replace(
oldString,
#"
\[ # a literal [
( # start a capturing group. this is what we access with "match.Groups[1]"
[^{}[\]]
# a negated character class, that matches anything except {, }, [ and ]
* # arbitrarily many of those
) # end of the capturing group
\{ # a literal {
([^{}[\]]*)
# the same thing as before, we will access this with "match.Groups[2]"
} # a literal }
([^{}[\]]*)
# "match.Groups[3]"
] # a literal ]
",
evaluator,
RegexOptions.IgnorePatternWhitespace
);
Here is the whitespace-free version of the regex:
\[([^{}[\]]*)\{([^{}[\]]*)}([^{}[\]]*)]