can we use Log4net to generate json format log with custom layout/fields?
I'm using log4net to log some information. Now, because of this and that, we need to log it as json format.
I'm using log4net.Ext.Json for this, it logs the information like this:
{"date":"2018-10-29T15:18:26.7785983-07:00","level":"INFO","logger":"Service.Services.LogService","message":"data_length: 10"}
{"date":"2018-10-29T15:18:26.7796462-07:00","level":"INFO","logger":"Service.Services.LogService","message":"max_parallelism: 1"}
However, since we will log lots of information, and we will feed this log to another program to analyze. So, we want to output it like:
{
"_index": "error_201843",
"_type": "error_web",
"_id": "AWaytV_hi121qded",
"_version": 1,
"_source": {
"ApplicationSource": "Data Feed",
"ErrorType": "RequestTimeout",
"XStackTrace": "",
"ErrorMessageText": ""
}
}
_index, _typem _id, _version are constant. _source data comes from the actual log fields.
How can we do this? Any ideas? I'm thinking of have a method to build the entire string, then output the string. But it should have a better way to do this, I think.
Thanks
We have been doing this for some time and can be done using a dynamic object. We have different log event types which are centralized and this allows the flexibility to add whatever information required. The loggingEvent event object is from log4net and an Append override in a custom appender.
JsonSerializerSettings _jsonSerializerSettings = new JsonSerializerSettings() { NullValueHandling = NullValueHandling.Ignore };
dynamic m = new System.Dynamic.ExpandoObject();
m.SessionId = _sessionId;
m.ProcessId = _processId.ToString();
m.ProcessName = _processName;
m.MachineName = _machineName;
m.UserName = _userName;
m.Level = loggingEvent.Level.DisplayName;
m.Domain = loggingEvent.Domain;
m.TimeStamp = loggingEvent.TimeStamp.ToString("yyyyMMddHHmmssfff");
m.MessageObject = loggingEvent.MessageObject;
if (loggingEvent.ExceptionObject != null)
{
m.Exception = loggingEvent.ExceptionObject.ToString();
m.StackTrace = loggingEvent.ExceptionObject.StackTrace;
}
//Convert the object to a json string
string msg = JsonConvert.SerializeObject(m, Formatting.None, _jsonSerializerSettings);
Related
I'm looking to change a specific property for each json record in my json file. I'd like to change the "Completed" property to "true" when a method finishes executing.
My json file looks like:
{
"LoanRecords": [
{
"LoanGUID": "{70dbec7e-5e94-460d-831c-0a5dc2d085e2}",
"RecordDT": "2020-11-10T14:44:34.378Z",
"Completed": "false",
"Environment": "TEBE",
"ProcessType": "RateLock"
},
{
"LoanGUID": "{70dbec7e-5e94-460d-831c-0a5dc2d085e2}",
"RecordDT": "2020-11-10T14:53:12.187Z",
"Completed": "false",
"Environment": "TEBE",
"ProcessType": "RateLock"
}
]
}
My C# code is the following:
private void ExecuteEvent(object sender, ElapsedEventArgs e)
{
string fileRecord = File.ReadAllText(jsonfile);
LoanRecordRoot LoanRecord = JsonConvert.DeserializeObject<LoanRecordRoot>(fileRecord);
foreach (var rec in LoanRecord.LoanRecords)
{
if (rec.Completed == "false")
{
bool recordModified = ManipulateEncompass(rec.LoanGUID, rec.ProcessType);
if (recordModified)
{
// What should I do here to update "rec.Completed" to "true"
// for this particular record and write it back to the json file
// for that specific entry?
}
}
}
Console.WriteLine("Successfully manipulated records!");
}
Is there a way to flip the "Completed" property to "true" for the specific record in my "foreach" iteration, and update the json file accordingly for that specific record? I am hoping to avoid reading the entire file, deserializing, changing the property then writing the entire content back to the json file, I'm looking to just flip that specific property for each record in my "foreach" loop. -- I hope that makes sense.
I've looked at similar questions, which seem close to what I'm looking for, but the examples I've seen don't reflect writing back to the json file specifically without overwriting the file contents -- unless this specific action isn't possible, or I'm failing to understand the entire process (highly possible.)
Ex of a solution that's close to what I'm looking for: How to update a property of a JSON object using NewtonSoft -- but doesn't seem to quite fit the bill for what I'm wanting to do.
Thank you in advance for any helpful leads!
you need to save the complete JSON when you update a property of an element of the array
static void Main(string[] args)
{
const string jsonPath = #"C:\Logs\recordRoot.json";
var loanRecordRoot = JsonConvert.DeserializeObject<LoanRecordRoot>(File.ReadAllText(jsonPath));
foreach (var record in loanRecordRoot.LoanRecords)
{
if (record.Completed == "false")
{
if (ManipulateEncompass(rec.LoanGUID, rec.ProcessType))
{
record.Completed = "true";
}
}
}
//Save Json
var json = JsonConvert.SerializeObject(loanRecordRoot, Formatting.Indented);
File.WriteAllText(jsonPath, json);
}
Looking at your JSON, it appears the "Completed" property is being serialized as of type string
Therefore, all you need to do is set it to "Completed": "true" within your condition in your snippet.
if (recordModified)
{
rec.Completed = "true";
}
At the end of your processing, simply serialize your LoanRecord object and write it back to your file.
using Kitchen_Mini_Project.Constants;
using Kitchen_Mini_Project.Moduls;
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace Kitchen_Mini_Project.Services
{
public class Update
{
public static void UpdateAnyProduct()
{
string readdedFile = File.ReadAllText(PathConst.ProductDBPath);
IList<Restaurant> products = JsonConvert.DeserializeObject<IList<Restaurant>>(readdedFile);
foreach (var product in products[0].FoodItems)
{
if (product.foodName == "Chicken Burrito")
{
product.foodName = "Chicken Burrito is Update ha ha";
}
}
var json = JsonConvert.SerializeObject(products, Formatting.Indented);
File.WriteAllText(PathConst.ProductDBPath, json);
}
}
}
1-install package Newtonsoft.Json
https://learn.microsoft.com/en-us/nuget/consume-packages/install-use-packages-visual-studio
2-use
string json = File.ReadAllText("server_client _input.json");
dynamic jsonObj = Newtonsoft.Json.JsonConvert.DeserializeObject(json);
jsonObj["Bots"][0]["Password"] = "new password";
string output = Newtonsoft.Json.JsonConvert.SerializeObject(jsonObj, Newtonsoft.Json.Formatting.Indented);
File.WriteAllText("settings.json", output);
(For Install package use of this page :https://www.newtonsoft.com/json)
So I have downloaded an asset for Unity called "JSON .NET For Unity", and I got it working, but I have a problem.
I have different classes and variables and I want to store them, the problem its that I dont know how to save them in the same file. I have a different method for each thing that I want to save but I dont know how to do that in the same method or making it write in the same file.
This is one example: In this method I save the class named World and I get it from the file. I have other methods like this one that asks for different things (a list, a variable...)
public void SaveWorld(World worldToSave)
{
SaveSystem.Init();
string json = JsonConvert.SerializeObject(worldToSave, Formatting.Indented, new JsonSerializerSettings
{
TypeNameHandling = TypeNameHandling.Auto,
PreserveReferencesHandling = PreserveReferencesHandling.Objects
});
File.WriteAllText(SaveSystem.SAVE_FOLDER + "/Save.json", json);
}
public World LoadWorld()
{
World saveWorld = null;
if (File.Exists(SaveSystem.SAVE_FOLDER + "/Save.json"))
{
string saveString = File.ReadAllText(SaveSystem.SAVE_FOLDER + "/Save.json");
saveWorld = JsonConvert.DeserializeObject<World>(saveString, new JsonSerializerSettings
{
TypeNameHandling = TypeNameHandling.Auto,
ReferenceLoopHandling = ReferenceLoopHandling.Serialize
});
}
return saveWorld;
}
Then for saving I would call this:
void Save()
{
saveLoadSystem.SaveWorld(worldHandler.World);
saveLoadSystem.SaveInstalledObjects(worldHandler.installedObjectList);
saveLoadSystem.SaveUnits(unitHandler.unitList);
}
Where saveLoadSystem is the script that has all the methods.
Thank you
Edit: Using Hacettepe Hesabı answer.
I can save correctly like he said and saving a class that only contains an int called numData and another script with a List of ints I get this:
{
"$id": "1",
"numData": 2
}[
0,
1
]
The problem is that when I load I get this error:
JsonReaderException: Additional text encountered after finished reading JSON content: [. Path '', line 4, position 1.
For loading Im using the next method:
public DATA LoadData()
{
DATA saveData = null;
if (File.Exists(SaveSystem.SAVE_FOLDER + "/Save.json"))
{
string saveString = File.ReadAllText(SaveSystem.SAVE_FOLDER + "/Save.json");
saveData = JsonConvert.DeserializeObject<DATA>(saveString, new JsonSerializerSettings
{
TypeNameHandling = TypeNameHandling.Auto,
ReferenceLoopHandling = ReferenceLoopHandling.Serialize
});
}
return saveData;
}
To save them all to the same file you could have the three instances in the same class and then save that class.
I'm creating a software on which I added a profiles feature where the user can create profile to load his informations faster. To store these informations, I'm using a JSON file, which contains as much objects as there are profiles.
Here is the format of the JSON file when a profile is contained (not the actual one, an example) :
{
"Profile-name": {
"form_email": "example#example.com",
//Many other informations...
}
}
Here is the code I'm using to write the JSON and its content :
string json = File.ReadAllText("profiles.json");
dynamic profiles = JsonConvert.DeserializeObject(json);
if (profiles == null)
{
File.WriteAllText(jsonFilePath, "{}");
json = File.ReadAllText(jsonFilePath);
profiles = JsonConvert.DeserializeObject<Dictionary<string, Profile_Name>>(json);
}
profiles.Add(profile_name.Text, new Profile_Name { form_email = form_email.Text });
var newJson = JsonConvert.SerializeObject(profiles, Formatting.Indented);
File.WriteAllText(jsonFilePath, newJson);
profile_tr.Nodes.Add(profile_name.Text, profile_name.Text);
debug_tb.Text += newJson;
But when the profiles.json file is completely empty, the profile is successfully written, but when I'm trying to ADD a profile when another one already exists, I get this error :
The best overloaded method match for 'Newtonsoft.Json.Linq.JObject.Add(string, Newtonsoft.Json.Linq.JToken)' has some invalid arguments on the profiles.Add(); line.
By the way, you can notice that I need to add {} by a non-trivial way in the file if it's empty, maybe it has something to do with the error ?
The expected output would be this JSON file :
{
"Profile-name": {
"form_email": "example#example.com",
//Many other informations...
},
"Second-profile": {
"form_email": "anotherexample#example.com"
//Some other informations...
}
}
Okay so I found by reading my code again, so I just replaced dynamic profiles = JsonConvert.DeserializeObject(json); to dynamic profiles = JsonConvert.DeserializeObject<Dictionary<string, Profile_Name>>(json);.
But it still don't fix the non-trivial way I use to add the {} to my file...
The object the first DeserializeObject method returns is actually a JObject, but below you deserialize it as a Dictionary. You shouldn't be mixing the types, choose either one.
If you use the JObject then to add objects you need to convert them to JObjects:
profiles.Add(profile_name.Text, JObject.FromObject(new Profile_Name { form_email = form_email.Text }));
In both cases, when the profile is null you just need to initialize it:
if (profiles == null)
{
profiles = new JObject(); // or new Dictionary<string, Profile_Name>();
}
I'm currently looking for a way to dynamically create a FormDialog from values predefined in the database. In other words, my field types, prompts and settings are all stored in a database, and what I'm trying to achieve is reading those settings and building the appropriate form dynamically.
What I tried so far is something similar to the following. Suppose I have a form with a Name (string) and an Age (int) field (FieldDefinition is a class I created to store the parameters of a field, assuming they are fetched from the database) (The code is stripped just to illustrate the idea):
public static IForm<dynamic> BuildForm()
{
string FormMessage = "Welcome to demo contact form!";
string CompletionMessage = "Thank your for your info. Our team will contact you as soon as possible.";
var fields = new List<FieldDefinition>()
{
new FieldDefinition()
{
Name = "Name",
FieldType = typeof(string),
Prompts = new string[] { "What's your name?", "Please input your name" }
},
new FieldDefinition()
{
Name = "Age",
FieldType = typeof(int),
Prompts = new string[] { "What's your age?", "How old are you?" }
}
};
var builder = new FormBuilder<dynamic>();
builder.Message(FormMessage);
foreach (var f in fields)
{
builder.Field(
new FieldReflector<dynamic>(f.Name)
.SetType(f.FieldType)
);
}
builder.AddRemainingFields()
.OnCompletion(async (context, order) => {
var message = context.MakeMessage();
message.Text = CompletionMessage;
await context.PostAsync(message);
});
return builder.Build();
}
So here's the problems:
I thought I could use a dynamic type. But a method cannot return a dynamic object as it is determined at run-time. Therefore, I got an error when I tried building the form using the following:
dynamic values; var form = new FormDialog<dynamic>(values, ContactForm.BuildForm, FormOptions.PromptInStart, null);`
I need to create the properties of the object dynamically, therefore I looked for a way to create a Type on runtime. I ended up with something called TypeBuilder but I was a bit skeptical if it could solve my problem or not.
Therefore, I guess the ultimate start is by using the FieldReflector but I have no idea how to achieve this. I'm looking for something similar to the above but that does actually work.
Have you looked at FormBuilderJson? You could dynamically construct the .json string, and build the form at runtime:
public static IForm<JObject> BuildJsonForm()
{
string fromFlowJson = GetFormFlowJson();
return new FormBuilderJson(schema)
.AddRemainingFields()
.Build();
}
See here for more information: https://learn.microsoft.com/en-us/azure/bot-service/dotnet/bot-builder-dotnet-formflow-json-schema?view=azure-bot-service-3.0
I have a project similar(Almost identical) to Conference API project which is taking similar approach to the noted project for returning CollectionJson content. I am having difficulty Setting the Collection property of the ReadDocument (Line 30) as it does not have any setter. I could bypass this problem by doing the following change
public CollectionJsonContent(Collection collection)
{
var serializerSettings = new JsonSerializerSettings
{
NullValueHandling = NullValueHandling.Ignore,
Formatting = Newtonsoft.Json.Formatting.Indented,
ContractResolver = new CamelCasePropertyNamesContractResolver()
};
collection.Version = "1.0";
Headers.ContentType = new MediaTypeHeaderValue("application/vnd.collection+json");
using (var writer = new JsonTextWriter(new StreamWriter(_memoryStream)){CloseOutput = false})
{
//var readDocument = new ReadDocument(); {IReadDocument.Collection = collection};
var serializer = JsonSerializer.Create(serializerSettings);
serializer.Serialize(writer,collection);
writer.Flush();
}
_memoryStream.Position = 0;
}
Although above code compiles and to some extent sorts out the problem but then again I will have another problem of not being able to consume the JsonCollection content in my controller unit tests. Consider the following unit test code snippet:
using (var request = CreateRequest())
{
var controller = new TestController(DataService) {Request = request};
var temp = await controller.ListAsync(gridSearchData, sampleSearchData);
if ((temp is NotFoundResult) && (sampleCollection.Any()))
{
Assert.Fail("Controller did not return any result but query did");
}
var json = await temp.ExecuteAsync(cancellationTokenSource);
var readDocument = json.Content.ReadAsAsync<ReadDocument>(new[] {new CollectionJsonFormatter()}, cancellationTokenSource).Result;
}
Since I did not set the collection property of ReadDocument readDocument is always empty and I cant read its content.
How do you asynchronously read the contents of JsonCollection on the client side in WEB API projects?
To get a Clear picture of the approach look at the Conference Web Api
and the authors blog
OK all, this has been fixed. The Collection property is now settable again.
I have just pushed release 0.7.0 with this fix, a major naming refactoring as well as a nice improvement to serialization to not write out empty collections.
Please see the release notes for the changes (especially the naming as the package names and namespaces have changed)
As far as I see from your code, you do not serialize a ReadDocument object, but only a property of it (Collection), and then you try to deserialize that value into a new ReadDocument object.
A sample ReadDocument should serialize like this
"{"Collection": [1,2,3,4,5] }"
But you serialize collection, so you get
"[1,2,3,4,5]"
I recommend a surrogate class for serialization like this
class SerializableReadDocument
{
public Collection Collection { get; set; }
}
and update your serialization code like this
using (var writer = new JsonTextWriter(new StreamWriter(_memoryStream)){CloseOutput = false})
{
var readDocument = new SerializableReadDocument() { Collection = collection };
var serializer = JsonSerializer.Create(serializerSettings);
serializer.Serialize(writer, readDocument);
writer.Flush();
}
But, this will not resolve your problem when you try to deserialize your output since ReadDocument does not have a settable Collection property, deserialization will either fail, or return a ReadDocument object with an empty Collection.
You can use SerializableReadDocument if you like in your unit tests.
I am looking into this and will come up with a solution hopeful this weekend, which will either be to make it a public setter, or make the setter internal and have public ctor that accepts a collection.
Sorry for the difficulty.