Fabric Message is too large - c#

I'm trying to pass 5MB~ data from a service to an actor, and I'm getting the error:
Fabric Message is too large
How can I increase the maximum size that can be transferred between micro-services?
I looked at the following page to see my options.
I tried setting:
<Section Name="ServiceReplicatorConfig">
...
<Parameter Name="MaxReplicationMessageSize" Value="1073741824" />
</Section>
Please help.

Somebody helped me on GitHub with the following:
In order to set maximum size for the remoting transport, you can use the following attribute and place it on your actor interface assembly or configure the maximum message size in the ServiceProxyFactory and ServiceRemotingListener creation.
[assembly: FabricTransportActorRemotingProvider(MaxMessageSize = 1073741824)]
https://msdn.microsoft.com/en-us/library/azure/microsoft.servicefabric.actors.remoting.fabrictransport.fabrictransportactorremotingproviderattribute.aspx
The discussion in GitHub.

Can you just pass a reference to that data like a URL to a blob/document storage? Passing 5MB of data within the SF service/actors is quite big and SF is not designed to store that big of data or state.

Related

Custom Column(s) with MsSqlServer Sink and AppSettings

I've got an application that runs without problem with the File and Console Sinks and now I'm trying to add the MSSqlServer Sink.
Looking at the documentation on Github I've got my application to write to the SQL Database as well as the other sinks.
<add key="serilog:write-to:MSSqlServer.connectionString" value="Server=servername;Database=databasename;User Id=userid;Password=password;"/>
<add key="serilog:write-to:MSSqlServer.tableName" value="Logs"/>
<add key="serilog:write-to:MSSqlServer.autoCreateSqlTable" value="true"/>
One improvement is I'd like to add a custom column to the Logs table that Serilog uses to store a number indicating a unique RunId for my application. The idea being that a simple query would allow grouping by RunId to see all messages for that one run.
So I added the following, based on the documentation (and I haven't been able to find any other examples) as it seemed logical:
<add key="serilog:write-to:MSSqlServer.columnOptions.ColumnName" value="RunId"/>
<add key="serilog:write-to:MSSqlServer.columnOptions.PropertyName" value="RunId"/>
<add key="serilog:write-to:MSSqlServer.columnOptions.DataType" value="SqlDbType.Int"/>
<add key="serilog:write-to:MSSqlServer.columnOptions.DataLength" value="32"/>
and then in my code all I need to do is:
Log.Information("{RunId}{Message}", RunId, Message);
to see a new entry with {RunId} in the RunId column and {Message} in the Message column... however everytime I do this nothing is written to the RunId column, it remains as NULL whereas every log message the console/file has is also duplicated in the Table.
So it seems logging is working, it must be the keys wrong and I'm really not sure what should be used.
Would anyone be able to point me in the direction I need to be going or where I've gone wrong?
Thank you.
Logging definitely was working but some digging and finally I found out I had two issues:
Permissions on database where insuffident for the user serilog was using, and
AppSettings settings I was using were wrong
#1 was easy enough to fix, I created a dedicated Serilog account for this database and fixed the permissions as per documentation.
#2 however was very frustrating but eventually I was able to get the following to work:
<configSections>
<section name="MSSqlServerSettingsSection" type="Serilog.Configuration.MSSqlServerConfigurationSection, Serilog.Sinks.MSSqlServer"/>
</configSections>
<MSSqlServerSettingsSection DisableTriggers="false" ClusteredColumnstoreIndex="false" PrimaryKeyColumnName="Id">
<!-- SinkOptions parameters -->
<TableName Value="Logs"/>
<Columns>
<add ColumnName="RunId" DataType="int"/>
</Columns>
</MSSqlServerSettingsSection>
On compile and execution my program will now create the Logs table in the database and then create a RunId column which I have been able to populate with a {RunId} expression in my Log.Debug() calls.
FWIW: I hope this will be helpful and if I can work out how I'll see if I can add this to documentation as an example of using AppSettings for this use case. Searching this question most people seem to be using JSON and not AppSettings.

How to create and setup ONVIF PresetTour using onvif ver20 ptz.wsdl

I would like to know how to create and setup a PresetTour using the ONVIF standard.
Using the the ONVIF ptz wsdl that has been added to my c# project's Connected Services, I can see that I can create a PresetTour for my PTZClient.
_ptzClient.CreatePresetTour(_profile.token);
After using this command, I can get the infos of this newly created preset tour using:
PresetTour[] tours = _ptzClient.GetPresetTours(_profile.token);
I would like to know how to create a new TourSpot or use an already existing Preset as a TourSpot to make my camera go to this spot when launching/starting the preset tour/patrol on my camera.
How do I set the position of a newly created tour spot?
How am I supposed to configure an entire PresetTour?
If there is no solution to my problem, I think I'll keep a list of preset on the side of my application and do everything manually.
You need to invoke ModifyPresetTour. It requires a tt:PresetTour struct, which has, among other fields, a TourSpot element of type tt:PTZPresetTourSpot. Beware that TourSpot is defined in the XML schema as minOccurs="0" maxOccurs="unbounded", thus you can specify any number of tour spots.
The PTZPresetTourSpot struct has a PresetDetail filed of type tt: PTZPresetTourPresetDetail.
Finally PTZPresetTourPresetDetail has a PresetToken field, where you can specify the preset.
I agree it is not very programmer-friendly.

Configure SAP .NET Connector for SNC using config file in C#

We have a number of applications developed in C# that interface with SAP using the SAP .NET Connector 3.0. I'm very familiar with this, and have recently been asked to look at securing this interface by using the SNC (Secure Network Communications) options, which I've also been able to configure and get working.
However, I want to configure my SNC destinations entirely via config file and not programmatically. To specify an unsecure connection, I can specify the following destination in the config file:
<SAP.Middleware.Connector>
<ClientSettings>
<DestinationConfiguration>
<destinations>
<add NAME="MySAPName" USER="MyUser" PASSWD="orly" CLIENT="100" LANG="EN" ASHOST="mysapname.mydomain.com" SYSNR="70" MAX_POOL_SIZE="20" IDLE_TIMEOUT="10"/>
</destinations>
</DestinationConfiguration>
</ClientSettings>
</SAP.Middleware.Connector>
But, to create a secure SNC connection, so far I've only figured out how to do it by configuring the destination programatically, e.g.:
Params.Add(RfcConfigParameters.AppServerHost, "mysapname.mydomain.com");
Params.Add(RfcConfigParameters.Name, "MySAPName");
Params.Add(RfcConfigParameters.SystemNumber, "70");
Params.Add(RfcConfigParameters.Language, "EN");
Params.Add(RfcConfigParameters.Client, "100");
Params.Add(RfcConfigParameters.User,"MyUser");
Params.Add(RfcConfigParameters.Password, "orly");
// Additional Params for SNC, not settable in config?
Params.Add(RfcConfigParameters.SncMode, "8");
Params.Add(RfcConfigParameters.SncPartnerName, "p:CN=RemovedForConfidentiality, OU=, O=, L=,C=GB");
Params.Add(RfcConfigParameters.SncMyName, "p:CN=MyRemovedPartnerName, C=GB, O=, OU=");
Params.Add(RfcConfigParameters.SncQOP, "8");
Params.Add(RfcConfigParameters.Trace, "2");
So, given this context my question is: Can I configure an SNC based SAP Destination using only the config file? If so, how?
I realise I could store SncPartnerName etc. in AppSettings but it would be much nicer if it were specifiable in the DestinationConfiguration section. I can't seem to find any documentation on this, however. I should note I'm aware of the the SAP SCN website and have had an unfruitful look on there, although I do not have access to the SAP Service Marketplace.
A colleague of mine has managed to discover the solution. Sample config file with the parameters required for SNC is as follows:
<SAP.Middleware.Connector>
<ClientSettings>
<DestinationConfiguration>
<destinations>
<add NAME="MySAPName" USER="MyUser" PASSWD="orly" CLIENT="100" LANG="EN"
ASHOST="mysapname.mydomain.com" SYSNR="70" MAX_POOL_SIZE="20" IDLE_TIMEOUT="10"
SNC_PARTNERNAME="p:CN=mycn.com, OU=A, O=B, L=C, C=GB"
SNC_MYNAME="p:CN=myname.com, C=GB, O=A, OU=B"
SNC_QOP="8" SNC_MODE="8" TRACE="2"/>
</destinations>
</DestinationConfiguration>
</ClientSettings>
</SAP.Middleware.Connector>

Best way to store associative array in app.config

I've been working on my software lately and I have been wondering what the best way is to store an associative array.
The only thing I could come up with out of the blue is to do something like this:
<add key="disks" value="C|1|10,D|2|20,E|1|5,Z|1|3"/>
But this doesn't offer a lot of readability in my config file and I want my config file to be readable as it is a console application.
The reason for this because I've written a program that checks the diskspace of the disks specified in the app.config file but I want different thresholds for different disks.
How would you solve it?
Here's a part of my current config file.
<!-- DISK FEATURE SETTINGS -->
<!-- Type 1 is threshold by percentage and type 2 is threshold by a certain limit -->
<add key="threshold_type" value="1" />
<add key="threshold_limit" value="0,1" />
<!-- Space_type defines if you want to limit using kilobytes (1), megabytes (2) or gigabytes (3) if using threshold_type 2 -->
<add key="space_type" value="3" />
<!-- Put the disks here delimited by a comma like this: C,D,E -->
<add key="disks" value="C,D,E,Z"/>
<!-- SERVICE FEATURE SETTINGS -->
<!-- Put the services here delimited by a comma like this: C,D,E -->
<add key="services" value="spooler,ekrn,RadeonPro Support Service,TeamViewer6"/>
<!-- Put this on 1 if you want to log your output to a text file -->
<add key="logging" value="1"/>
I want to use the same principle for my performancecounter program that uses the perfmon counters to get some data and store it in a text file.
I hope people can help me for a bit here :)
I suggesst you to create your own configuration section. Custom configuration gives more readability and type safety. Here are links to create custom configuration
http://msdn.microsoft.com/en-us/library/2tw134k3.aspx and
http://haacked.com/archive/2007/03/11/custom-configuration-sections-in-3-easy-steps.aspx (old one but easy to follow).
As far as standard configuration mechanism works with XML serialization, the best (and, IMHO, the wise) way to store dictionaries in App.config is a List<KeyValuePair<K,V>>.
you may want to use Hashtable from system.collection or List<>
below are few pointers for hashtable,
http://www.dotnetperls.com/hashtable
http://www.tutorialspoint.com/csharp/csharp_hashtable.htm
i hope this helps!! thanks :)

How to read an extension .proto file (textformat.merge) with old code?

My question is related with the C# implementation of the google protocol buffers (protobuf-csharp-port, by jon skeet, great job!)
I am experiencing troubles with the extensions: let's say I wrote:
"transport_file.proto" with a "transport message" and some code to
deal with it "code_old".
and I wrote an extension of the transport message on
"Mytransport.proto" file, and new code to read it "code_new".
I'm trying to read a new message (from MyTransport.proto) with the code_old expecting to ignore the extension, but I get an exception in the merge method from TextFormat: "transport" has no field named "whatever_new_field"
Transport.Builder myAppConfigB = new Transport.Builder();
System.IO.StreamReader fich = System.IO.File.OpenText("protocolBus.App.cfg");
TextFormat.Merge(fich.ReadToEnd(),myAppConfigB);
fich.Close();
new extended file looks like:
...
Transport
{
TransportName: "K6Server_0"
DllImport: "protocolBus.Transports.CentralServer"
TransportClass: "K6Server"
K6ServerParams
{
K6Server { host: "85.51.11.23" port: 40069 }
Service: "TZinTalk"
...
}
}
...
while the old one, not extended:
...
Transport
{
TransportName: "K6Server_0"
DllImport: "Default"
TransportClass: "Multicast"
}
...
The whole idea is to use the text based protocol buffer as a config file in which I write some params, and based on one of those I load and assembly (which will read the whole message with the new extension (params to initialize the object).
Any idea? (it is a desperate question :D )
I'm using MSVC# 2008Express edition, protobuf-csharp-port version 0.9.1 (someday I'll upgrade everything).
THANKS in advance.
I'm working on a non centrilized Publish-Subscribe framework of messages (for any written message in a proto file I auto create a Publish and a Subscriber class) with different transports. By the default I use multicast, but broadcast and a "UDP star" are also included. I let the extension mechanism to let people add new transports with its owm config params that should be read by my main code_old (just to load the assembly) and let the new transport (.dll) read it again (fully).
Curious? the previous, almost functional, version is in http://protocolbus.casessite.org
Update 1
Extended types in text format are enclosed in brackets (good to know, I was not aware of it :D ) so I should have written:
[K6ServerParams]
{
K6Server { host: "85.51.11.23" port: 40069 }
Service: "TZinTalk"
...
}
Protocol buffers are designed to be backwards and forwards compatible when using their binary format, but certainly the current code doesn't expect to parse the text format with unknown fields. It could potentially be changed to do that, but I'd want to check with the Java code to try to retain parity with that.
Is there any reason you're not using the binary representation to start with? That's the normal intended usage, and the one where the vast majority of the work has gone in. (Having said which, it all seems a bit of a blur after this long away from the code...)

Categories

Resources