I'm having difficulties with the speed of svg2png and wondering if there was any way to improve it. I'm using D3 to create a Radar Chart. This is all rendered by Node.js in a jsdom. The resulting SVG is converted to a PNG using svg2png. The purpose of doing this is to then insert the image into reports which are provided to the end user. The same javascript which renders the radar chart is used within the app (without node.js) and works very quickly.
Using .NET core 2.1 and Node.js v8.11.2
Time taken to convert the svg to png is approximately 2-3 seconds.
Invocation of node services;
public async Task<string> GetRadarChartAsync(dynamic options)
{
return await _nodeServices.InvokeAsync<string>("./wwwroot/js/node-radar-chart.js", options);
}
Which is being called like so and the base64 image extracted.
Task<string> result = (Task<string>)mapped_function.DynamicInvoke(objects.ToArray<object>());
string img_base64 = result.Result;
The javascript wrapper looks like this;
module.exports = function(callback, options, data) {
const dom = new JSDOM(`<!DOCTYPE html><div id="body" class="radar-chart"></div>`);
var options1 = {
window: dom.window,
selector: '.radar-chart',
data: JSON.parse(data.radar)
}
var options_combined = Object.assign(options1, options);
var chart1 = new RadarChart(options_combined);
// Convert SVG to PNG and return it to controller
var svgText = chart1.html();
svg2png(Buffer.from(svgText))
.then(buffer => buffer.toString('base64'))
.then(buffer => callback(null, buffer));
}
UPDATE 14/8/18
Previously incorrectly labelled this issue as an invocation issue between node and .NET Core 2.1. Further investigation revealed that svg2png is causing the problem.
FURTHER UPDATE
Issue is likely due to svg2png utilising PhantomJS. Proposed idea was to allow svg2png to use the same PhantomJS instance over multiple calls, but there is no development on this issue as yet. See svg2png github.
I'll have to deal with my speed issues until a better solution presents itself.
When I tried out the NodeServices samples on GitHub (especially the server-side rendering one) then the invocation time was 1 second minimum when using .NET 2.0. When I tried it with .NET 2.1 I think it decreased to like 100ms or less if I remember correctly. So it's possible that the NodeServices invocation time got a huge improvement from 2.0 to 2.1.
If it is not possible to upgrade then maybe you could refactor so that you need to invoke NodeServices only once.
I have never used svg2png but I see that you do buffer.toString('base64') which probably is not too efficient. I would imagine this happens:
The whole buffer is read to the end and converted at the same time to Base64, and Base64 in itself is not very efficient when it comes to size on disk (as far as I know).
Then this whole conversion is moved from Node to ASP.NET and I guess there could be some serialization going on here which will be slow with huge Base64 content(?).
I would look into using streams on both sides as that's what is usually used when it comes to media files.
Related
abstract
In the process of writing an unpacker for a IPK-archive (NDS Homebrew Image Viewer) I got stuck at the point where I had to decode single MCUs. The original Sourcecode indicates the usage of DCT but the reference implementation is all in ARM assembly...
I'm currently writing a rudimentary unpacker for the files utilized by NDS Homebrew "Image Viewer" (http://gamebrew.org/wiki/Image_Viewer).
The original site seems down but I found the source code (including a file format description) inside the archive hosted on gamebrew.
The unpacker is written in C# due to convenience (BinaryReader) and also because there is no need for it to be fast.
I've already managed to extract the thumbnails. Those are stored as deflated (zlib) BGR-Bitmaps.
The full-size images are split up into MCU's (8x8 samples). Each MCU is also stored in deflated form.
As of now I got the decompressed data with what seems like DCTed data. It also looks like they've "multiplexed" the individual channels (is this the 4:1:1?).
In the NDS version the byte data then gets passed together with a quantization-table (array of 64 signed 16bit-integers) into a function which looks like this:
void customjpeg_DecodeYUV411(s32 *pQuantizeTable,u8 *pData,u16 *pBuf)
{
pDCs=(s16*)pData;
pACs=(s8*)&pDCs[4*4*6];
static __attribute__ ((section (".dtcm"))) DCTELEM _y0[DCTSIZE2],_y1[DCTSIZE2],_y2[DCTSIZE2],_y3[DCTSIZE2],_cb[DCTSIZE2],_cr[DCTSIZE2];
TYUV411toRGB15_Data YUV411toRGB15_Data[4]={
{ NULL, _y0, &_cb[((4*0)*DCTSIZE)+(4*0)], &_cr[((4*0)*DCTSIZE)+(4*0)] },
{ NULL, _y1, &_cb[((4*0)*DCTSIZE)+(4*1)], &_cr[((4*0)*DCTSIZE)+(4*1)] },
{ NULL, _y2, &_cb[((4*1)*DCTSIZE)+(4*0)], &_cr[((4*1)*DCTSIZE)+(4*0)] },
{ NULL, _y3, &_cb[((4*1)*DCTSIZE)+(4*1)], &_cr[((4*1)*DCTSIZE)+(4*1)] },
};
// PrfStart();
// 3.759ms
for(u32 y=0;y<4;y++){
YUV411toRGB15_Data[0]._pBuf=&pBuf[((8*0)*64)+(8*0)];
YUV411toRGB15_Data[1]._pBuf=&pBuf[((8*0)*64)+(8*1)];
YUV411toRGB15_Data[2]._pBuf=&pBuf[((8*1)*64)+(8*0)];
YUV411toRGB15_Data[3]._pBuf=&pBuf[((8*1)*64)+(8*1)];
for(u32 x=0;x<4;x++){
DCT13bit_asm(pQuantizeTable,_y0);
DCT13bit_asm(pQuantizeTable,_y1);
DCT13bit_asm(pQuantizeTable,_y2);
DCT13bit_asm(pQuantizeTable,_y3);
DCT13bit_asm(pQuantizeTable,_cb);
DCT13bit_asm(pQuantizeTable,_cr);
YUV411_13bit_toRGB15_asm(YUV411toRGB15_Data);
YUV411toRGB15_Data[0]._pBuf+=DCTSIZE*2;
YUV411toRGB15_Data[1]._pBuf+=DCTSIZE*2;
YUV411toRGB15_Data[2]._pBuf+=DCTSIZE*2;
YUV411toRGB15_Data[3]._pBuf+=DCTSIZE*2;
}
pBuf+=(DCTSIZE*2)*64;
}
// PrfEnd(0); ShowLogHalt();
}
(taken from: customjpeg.cpp)
why are the calls run 4x4 times per mcu?
is it pure DCT? because i don't see where the "dequantization" is done and i've seen (afaik) some references to the zigzag-pattern(?) in the assembly..
am I missing something?
I've checked with some online resources, but they only seem to focus on the encoding process.
Unfortunately I lack the mathematical background, therefore I haven't grasped on to the full concept of DCT and thus the reversal process isn't as straight forward to me..
I've spent a whole day to get this running by utilizing libraries and C#-DCT implementations but I was unsuccessful.
I am creating an application that interfaces with Google's Maps API v3. My current approach is using a WebBrowser control by WebBrowser.Navigate("Map.html"). This is working correctly at the moment; however, I am also aware of WebBrowser.InvokeScript(). I have seen this used to execute a javascript function, but I would like to have something like the following structure:
APICalls.js - Contains different functions that can be called, or even separated out into a file for each function if necessary.
MapInterface.cs
WebBrowser.InvokeScript("APICalls.js", args) - Or control the javascript variables directly.
I have seen the InvokeScript method used, but none of the examples gave any detail to the source of the function, so I'm not sure if it was calling it from an html file or js file. Is it possible to have a structure like this, or a similarly organized structure, rather than creating an html file with javascript in each one and using Navigate()?
Additionally, are there any easier ways to use Google Maps with WPF. I checked around, but all of the resources I found were at least 2-3 years old, which I believe is older than the newest version of the maps API.
I can't suggest a better way of using Google Maps API with WPF (although I'm sure it exists), but I can try to answer the rest of the question.
First, make sure to enable FEATURE_BROWSER_EMULATION for your WebBrowser app, so Google Maps API recognizes is it as modern HTML5-capable browser.
Then, navigate to your "Map.html" page and let it finish loading. Here's how it can be done using async/await (the code is for the WinForms version of WebBrowser control, but the concept remains the same).
You can have your APICalls.js as a separate local file, but you'd need to create and populate a <script> element for it from C#. You do it once for the session.
Example:
var scriptText = File.ReadAllText("APICalls.js");
dynamic htmlDocument = webBrowser.Document;
var script = htmlDocument.createElement("script");
script.type = "text/javascript";
script.appendChild(htmlDocument.createTextNode(scriptText));
htmlDocument.body.appendChild(script);
Then you can call functions from this script in a few different ways.
For example, your JavaScript entry point function in APICalls.js may look like this:
(function() {
window.callMeFromCsharp = function(arg1, arg2) {
window.alert(arg1 + ", " +arg2);
}
})();
Which you could call from C# like this:
webBrowser.InvokeScript("callMeFromCsharp", "Hello", "World!");
[UPDATE] If you're looking for a bit more modular or object-oriented approach, you can utilize the dynamic feature of C#. Example:
JavaScript:
(function() {
window.apiObject = function() {
return {
property: "I'm a property",
Method1: function(arg) { alert("I'm method 1, " + arg); },
Method2: function() { return "I'm method 2"; }
};
}
})();
C#:
dynamic apiObject = webBrowser.InvokeScript("apiObject");
string property = apiObject.property;
MessageBox.Show(property);
apiObject.Method1("Hello!");
MessageBox.Show(apiObject.Method2());
I just found out about NRefactory 5 and I would guess, that it is the most suitable solution for my current problem. At the moment I'm developing a little C# scripting application for which I would like to provide code completion. Until recently I've done this using the "Roslyn" project from Microsoft. But as the latest update of this project requires .Net Framework 4.5 I can't use this any more as I would like the app to run under Win XP as well. So I have to switch to another technology here.
My problem is not the compilation stuff. This can be done, with some more effort, by .Net CodeDomProvider as well. The problem ist the code completion stuff. As far as I know, NRefactory 5 provides everything that is required to provide code completion (parser, type system etc.) but I just can't figure out how to use it. I took a look at SharpDevelop source code but they don't use NRefactory 5 for code completion there, they only use it as decompiler. As I couldn't find an example on how to use it for code completion in the net as well I thought that I might find some help here.
The situation is as follows. I have one single file containing the script code. Actually it is not even a file but a string which I get from the editor control (by the way: I'm using AvalonEdit for this. Great editor!) and some assemblies that needs to get referenced. So, no solution files, no project files etc. just one string of source code and the assemblies.
I've taken a look at the Demo that comes with NRefactory 5 and the article on code project and got up with something like this:
var unresolvedTypeSystem = syntaxTree.ToTypeSystem();
IProjectContent pc = new CSharpProjectContent();
// Add parsed files to the type system
pc = pc.AddOrUpdateFiles(unresolvedTypeSystem);
// Add referenced assemblies:
pc = pc.AddAssemblyReferences(new CecilLoader().LoadAssemblyFile(
System.Reflection.Assembly.GetAssembly(typeof(Object)).Location));
My problem is that I have no clue on how to go on. I'm not even sure if it is the right approach to accomplish my goal. How to use the CSharpCompletionEngine? What else is required? etc. You see there are many things that are very unclear at the moment and I hope you can bring some light into this.
Thank you all very much in advance!
I've just compiled and example project that does C# code completion with AvalonEdit and NRefactory.
It can be found on Github here.
Take a look at method ICSharpCode.NRefactory.CSharp.CodeCompletion.CreateEngine. You need to create an instance of CSharpCompletionEngine and pass in the correct document and the resolvers. I managed to get it working for CTRL+Space compltition scenario. However I am having troubles with references to types that are in other namespaces. It looks like CSharpTypeResolveContext does not take into account the using namespace statements - If I resolve the references with CSharpAstResolver, they are resolved OK, but I am unable to correctly use this resolver in code completition scenario...
UPDATE #1:
I've just managed to get the working by obtaining resolver from unresolved fail.
Here is the snippet:
var mb = new DefaultCompletionContextProvider(doc, unresolvedFile);
var resolver3 = unresolvedFile.GetResolver(cmp, loc); // get the resolver from unresolvedFile
var engine = new CSharpCompletionEngine(doc, mb, new CodeCompletionBugTests.TestFactory(resolver3), pctx, resolver3.CurrentTypeResolveContext );
Update #2:
Here is the complete method. It references classes from unit test projects, sou you would need to reference/copy them into your project:
public static IEnumerable<ICompletionData> DoCodeComplete(string editorText, int offset) // not the best way to put in the whole string every time
{
var doc = new ReadOnlyDocument(editorText);
var location = doc.GetLocation(offset);
string parsedText = editorText; // TODO: Why there are different values in test cases?
var syntaxTree = new CSharpParser().Parse(parsedText, "program.cs");
syntaxTree.Freeze();
var unresolvedFile = syntaxTree.ToTypeSystem();
var mb = new DefaultCompletionContextProvider(doc, unresolvedFile);
IProjectContent pctx = new CSharpProjectContent();
var refs = new List<IUnresolvedAssembly> { mscorlib.Value, systemCore.Value, systemAssembly.Value};
pctx = pctx.AddAssemblyReferences(refs);
pctx = pctx.AddOrUpdateFiles(unresolvedFile);
var cmp = pctx.CreateCompilation();
var resolver3 = unresolvedFile.GetResolver(cmp, location);
var engine = new CSharpCompletionEngine(doc, mb, new CodeCompletionBugTests.TestFactory(resolver3), pctx, resolver3.CurrentTypeResolveContext );
engine.EolMarker = Environment.NewLine;
engine.FormattingPolicy = FormattingOptionsFactory.CreateMono();
var data = engine.GetCompletionData(offset, controlSpace: false);
return data;
}
}
Hope it helps,
Matra
NRefactory 5 is being used in SharpDevelop 5. The source code for SharpDevelop 5 is currently available in the newNR branch on github. I would take a look at the CSharpCompletionBinding class which has code to display a completion list window using information from NRefactory's CSharpCompletionEngine.
My question is related with the C# implementation of the google protocol buffers (protobuf-csharp-port, by jon skeet, great job!)
I am experiencing troubles with the extensions: let's say I wrote:
"transport_file.proto" with a "transport message" and some code to
deal with it "code_old".
and I wrote an extension of the transport message on
"Mytransport.proto" file, and new code to read it "code_new".
I'm trying to read a new message (from MyTransport.proto) with the code_old expecting to ignore the extension, but I get an exception in the merge method from TextFormat: "transport" has no field named "whatever_new_field"
Transport.Builder myAppConfigB = new Transport.Builder();
System.IO.StreamReader fich = System.IO.File.OpenText("protocolBus.App.cfg");
TextFormat.Merge(fich.ReadToEnd(),myAppConfigB);
fich.Close();
new extended file looks like:
...
Transport
{
TransportName: "K6Server_0"
DllImport: "protocolBus.Transports.CentralServer"
TransportClass: "K6Server"
K6ServerParams
{
K6Server { host: "85.51.11.23" port: 40069 }
Service: "TZinTalk"
...
}
}
...
while the old one, not extended:
...
Transport
{
TransportName: "K6Server_0"
DllImport: "Default"
TransportClass: "Multicast"
}
...
The whole idea is to use the text based protocol buffer as a config file in which I write some params, and based on one of those I load and assembly (which will read the whole message with the new extension (params to initialize the object).
Any idea? (it is a desperate question :D )
I'm using MSVC# 2008Express edition, protobuf-csharp-port version 0.9.1 (someday I'll upgrade everything).
THANKS in advance.
I'm working on a non centrilized Publish-Subscribe framework of messages (for any written message in a proto file I auto create a Publish and a Subscriber class) with different transports. By the default I use multicast, but broadcast and a "UDP star" are also included. I let the extension mechanism to let people add new transports with its owm config params that should be read by my main code_old (just to load the assembly) and let the new transport (.dll) read it again (fully).
Curious? the previous, almost functional, version is in http://protocolbus.casessite.org
Update 1
Extended types in text format are enclosed in brackets (good to know, I was not aware of it :D ) so I should have written:
[K6ServerParams]
{
K6Server { host: "85.51.11.23" port: 40069 }
Service: "TZinTalk"
...
}
Protocol buffers are designed to be backwards and forwards compatible when using their binary format, but certainly the current code doesn't expect to parse the text format with unknown fields. It could potentially be changed to do that, but I'd want to check with the Java code to try to retain parity with that.
Is there any reason you're not using the binary representation to start with? That's the normal intended usage, and the one where the vast majority of the work has gone in. (Having said which, it all seems a bit of a blur after this long away from the code...)
My plan:
I'm trying to setup my C# project to communicate with Nodebox to call a certain function which populates a graph and draws it in a new window.
Current situation: [fixed... see Update2]
I have already included all python-modules needed, but im still getting a
Library 'GL' not found
it seems that the pyglet module needs a reference to GL/gl.h, but can't find it due to IronPython behaviour.
Requirement:
The project needs to stay as small as possible without installing new packages. Thats why i have copied all my modules into the project-folder and would like to keep it that or a similar way.
My question:
Is there a certain workaround for my problem or a fix for the library-folder missmatch.
Have read some articles about Tao-Opengl and OpenTK but can't find a good solution.
Update1:
Updated my sourcecode with a small pyglet window-rendering example. Problem is in pyglet and referenced c-Objects. How do i include them in my c# project to be called? No idea so far... experimenting alittle now. Keeping you updated.
SampleCode C#:
ScriptRuntimeSetup setup = Python.CreateRuntimeSetup(null);
ScriptRuntime runtime = new ScriptRuntime(setup);
ScriptEngine engine = Python.GetEngine(runtime);
ScriptSource source = engine.CreateScriptSourceFromFile("test.py");
ScriptScope scope = engine.CreateScope();
source.Execute(scope);
SampleCode Python (test.py):
from nodebox.graphics import *
from nodebox.graphics.physics import Vector, Boid, Flock, Obstacle
flock = Flock(50, x=-50, y=-50, width=700, height=400)
flock.sight(80)
def draw(canvas):
canvas.clear()
flock.update(separation=0.4, cohesion=0.6, alignment=0.1, teleport=True)
for boid in flock:
push()
translate(boid.x, boid.y)
scale(0.5 + boid.depth)
rotate(boid.heading)
arrow(0, 0, 15)
pop()
canvas.size = 600, 300
def main(canvas):
canvas.run(draw)
Update2:
Line 139 [pyglet/lib.py] sys.platform is not win32... there was the error. Fixed it by just using the line:
from pyglet.gl.lib_wgl import link_GL, link_GLU, link_WGL
Now the following Error:
'module' object has no attribute '_getframe'
Kind of a pain to fix it. Updating with results...
Update3:
Fixed by adding following line right after first line in C#-Code:
setup.Options["Frames"] = true;
Current Problem:
No module named unicodedata, but in Python26/DLLs is only a *.pyd file`. So.. how do i implement it now?!
Update4:
Fixed by surfing: link text and adding unicodedata.py and '.pyd to C# Projectfolder.
Current Problem:
'libGL.so not found'... guys.. im almost giving up on nodebox for C#.. to be continued
Update5:
i gave up :/ workaround: c# communicating with nodebox over xml and filesystemwatchers. Not optimal, but case solved.
-X:Frames enables the frames option as runtime (it slows code down a little to have access to the Python frames all the time).
To enable frames when hosting you just need to do:
ScriptRuntimeSetup setup = Python.CreateRuntimeSetup(new Dictionary<string, object>() {
{ "Frames", true }
});
Instead of the null that you're passing now. That's just creating a new dictionary for the options dictionary w/ the contents "Frames" set to true. You can set other options in there as well and in general the -X:Name option is the same here as it is for the command line.