Hy,
I am pretty rookie to OpenNLP.Net and am a little bit lost on basic step.
I looked at some java code and try to convert it in C# but I think I am quite wrong given that I don't find any C# code
Right now I am trying to run this code located in the main
using System;
using System.IO;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Newtonsoft.Json;
using opennlp.tools.doccat;
using opennlp.tools.tokenize;
using opennlp.tools.util;
public class Account
{
public string Name { get; set; }
public string Email { get; set; }
public DateTime DOB { get; set; }
}
namespace Loading_OpenNLP
{
class Program
{
static void Main(string[] args)
{
Account account = new Account
{
Name = "John Doe",
Email = "john#microsoft.com",
DOB = new DateTime(1980, 2, 20, 0, 0, 0, DateTimeKind.Utc),
};
string json = JsonConvert.SerializeObject(account, Formatting.Indented);
Console.WriteLine(json);
getNLPModel();
string pause = Console.ReadLine();
}
static void getNLPModel()//java.io.File openNLPTraining)
{
InputStreamFactory inputStreamFactory = new MarkableFileInputStreamFactory(new java.io.File("D:\\text.txt"));
ObjectStream lineStream = new PlainTextByLineStream(inputStreamFactory, "UTF-8");
ObjectStream sampleStream = new DocumentSampleStream(lineStream);
}
}
}
It compiles but the File is not Found ... What's wrong?
You can implement InputStreamFactory by your own.
Here is F# sample that train custom NER model, InputStreamFactory is implemented using F# object expressions
open java.nio.charset
open java.io
#I "../packages/OpenNLP.NET/lib/"
#r "opennlp-tools-1.8.4.dll"
#r "opennlp-uima-1.8.4.dll"
open opennlp.tools.util
open opennlp.tools.namefind
let train (inputFile:string) =
let factory =
{ new InputStreamFactory with
member __.createInputStream () =
new FileInputStream(inputFile) :> InputStream }
let lineStream = new PlainTextByLineStream(factory, StandardCharsets.UTF_8)
use sampleStream = new NameSampleDataStream(lineStream)
let nameFinderFactory = new TokenNameFinderFactory()
let trainingParameters = new TrainingParameters();
//trainingParameters.put(TrainingParameters.ITERATIONS_PARAM, "5");
//trainingParameters.put(TrainingParameters.CUTOFF_PARAM, "200");
NameFinderME.train ("en", "person", sampleStream, trainingParameters, nameFinderFactory)
in C# the same code may look like this
using java.nio.charset;
using java.io;
using opennlp.tools.util;
using opennlp.tools.namefind;
namespace OpenNLP.Train
{
class MyStreamFactory: InputStreamFactory
{
public Factory(string fileName) => _filename = fileName;
private readonly string _filename;
public InputStream createInputStream()
=> new FileInputStream(_filename);
}
class Program
{
static void Main(string[] args)
{
var factory = new MyStreamFactory("D:\\text.txt");
var lineStream = new PlainTextByLineStream(factory, StandardCharsets.UTF_8);
var sampleStream = new NameSampleDataStream(lineStream);
var nameFinderFactory = new TokenNameFinderFactory();
var trainingParameters = new TrainingParameters();
var model = NameFinderME.train("en", "person", sampleStream, trainingParameters, nameFinderFactory);
}
}
}
Related
I'm having a problem with my Discord Bot where the only method of interfacing with the bot is by tagging it in a message. I've setup the code for how I thought the prefix functionality was supposed to work, but I've done something wrong. I don't get any error messages when in use and the bot functions perfectly fine if its tagged, but doesn't react at all if you try to use the prefix instead. Can someone explain please? (I'm using DSharp for the discord api)
Here's the exact part of the code I think there's a problem with:
var commandsConfig = new CommandsNextConfiguration
{
StringPrefixes = new List<string>() { configJson.Prefix },
EnableDms = false,
EnableMentionPrefix = true,
DmHelp = false,
EnableDefaultHelp = true,
UseDefaultCommandHandler = true
};
Console.WriteLine($"HELLO! {configJson.Prefix}"); //Testing it could read the desired prefix (it can)
Commands = Client.UseCommandsNext(commandsConfig);
The configJson.cs looks like this:
public struct ConfigJson
{
[JsonProperty("token")]
public string Token { get; private set; }
[JsonProperty("prefix")]
public string Prefix { get; private set; }
}
The config.Json file looks like this (The token is removed for security)
{
"token": "#######",
"prefix": "!"
}
I'll also post the entire .cs if you want to recreate the conditions
using DSharpPlus;
using DSharpPlus.CommandsNext;
using DSharpPlus.EventArgs;
using DSharpPlus.Interactivity;
using DSharpPlus.Interactivity.Extensions;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.IO;
using System.Text;
using System.Threading.Tasks;
using VLB.Commands.VLBCommands;
namespace VLB
{
public class Bot
{
public DiscordClient Client { get; private set; }
public InteractivityExtension Interactivity { get; private set; }
public CommandsNextExtension Commands { get; private set; }
public async Task RunAsync()
{
var json = string.Empty;
using (var fs = File.OpenRead("config.json"))
using (var sr = new StreamReader(fs, new UTF8Encoding(false)))
json = await sr.ReadToEndAsync().ConfigureAwait(false);
var configJson = JsonConvert.DeserializeObject<ConfigJson>(json);
var config = new DiscordConfiguration
{
Token = configJson.Token,
TokenType = TokenType.Bot,
AutoReconnect = true,
MinimumLogLevel = LogLevel.Debug,
};
Client = new DiscordClient(config);
Client.Ready += OnClientReady;
Client.UseInteractivity(new InteractivityConfiguration
{
Timeout = TimeSpan.FromSeconds(30)
});
var commandsConfig = new CommandsNextConfiguration
{
StringPrefixes = new List<string>() { configJson.Prefix },
EnableDms = false,
EnableMentionPrefix = true,
DmHelp = false,
EnableDefaultHelp = true,
UseDefaultCommandHandler = true
};
Console.WriteLine($"HELLO! {configJson.Prefix}"); //Testing it could read the desired prefix (it can)
Commands = Client.UseCommandsNext(commandsConfig);
Commands.RegisterCommands<VLBCommands>();
await Client.ConnectAsync();
await Task.Delay(-1);
}
private Task OnClientReady(DiscordClient client, ReadyEventArgs e)
{
return Task.CompletedTask;
}
}
}
I've tried changing the prefix to various conventional characters.
I've also tried changing the assignment to "StringPrefixes" to a string array instead of a list.
I'm only trying to use one prefix, but after scouring DSharp documentation I'm not sure CommandsNextConfiguration has a way to indicate only one prefix.
Thanks for reading :)
Got some strange behaviour with a script, the constructor of my FileHandler class appears to be calling the class and running the script.
The class itself is only being referenced in VS15 once and that is by its constructor, the main method has not yet even have an object of FileHandler, which is not mentioned anywhere else in the code.
Surely this code should not be running?
Edit: I placed a breakpoint at the start of the Program.cs and began stepping through, but when I did this I noticed that the public class FileHandler becomes class Program and my constructor is 'replaced' by a Main method.
Is this something that C# does by design?
Programs.cs
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using QuantConnect;
using QuantConnect.Securities;
using QuantConnect.Securities.Forex;
namespace TradingDaysFileChecker
{
class Program
{
static void Main(string[] args)
{
var securityType = SecurityType.Forex;
var ticker = TickType.Trade;
var marketHoursDatabase = MarketHoursDatabase.FromDataFolder();
var market = Market.FXCM;
var symbol = Symbol.Create(ticker.ToString(), securityType, market);
var marketHoursDbEntry = marketHoursDatabase.GetEntry(symbol.ID.Market, symbol.Value, symbol.ID.SecurityType);
var exchange = new ForexExchange(marketHoursDbEntry.ExchangeHours);
Console.ReadLine();
}
}
}
FileHandler.cs
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using QuantConnect.Securities.Forex;
namespace TradingDaysFileChecker
{
public class FileHandler
{
private readonly StreamWriter _writeToFile;
private readonly List<Tuple<string, string>> _missingDays;
private readonly string _dataFilePath;
private readonly DateTime _startDate;
private readonly DateTime _endDate;
private readonly ForexExchange _exchange;
private readonly IEnumerable<DateTime> _validTradingDays;
private readonly string[] _forexSecuritiesFolders;
public FileHandler(ForexExchange exchange)
{
_startDate = new DateTime(2007, 04, 01);
_endDate = new DateTime(2016, 07, 25);
_exchange = exchange;
_writeToFile = new StreamWriter(#"C:\Users\RichardsPC\Documents");
_dataFilePath = #"C:\Users\RichardsPC\Desktop\export\exporter\forex\fxcm\minute\";
_forexSecuritiesFolders = Directory.GetDirectories(_dataFilePath);
_missingDays = new List<Tuple<string, string>>();
_validTradingDays = IterateOverDateRange(_exchange, _startDate, _endDate);
}
public void CheckForMissingFiles()
{
foreach (var validDay in _validTradingDays)
{
foreach (var forexSecurity in _forexSecuritiesFolders)
{
var fxPair = new DirectoryInfo(forexSecurity).Name;
var formattedDate = FormatDate(validDay);
var path = SetPath(_dataFilePath, fxPair, formattedDate);
if (!File.Exists(path))
{
_missingDays.Add(Tuple.Create(fxPair, formattedDate));
}
}
}
Results();
}
public void Results()
{
if (_missingDays.Count > 0)
{
foreach (var missingDay in _missingDays.OrderBy(md => md.Item1))
{
var formattedTupleOutput = missingDay.ToString().TrimStart('(').TrimEnd(')');
Console.WriteLine(formattedTupleOutput);
WriteResultsToFile(formattedTupleOutput);
}
}
else
{
var noFilesMissing = "No results missing";
Console.WriteLine(noFilesMissing);
WriteResultsToFile(noFilesMissing);
}
Console.WriteLine("Records missing: " + _missingDays.Count);
}
public void WriteResultsToFile(string result)
{
_writeToFile.WriteLine(result);
}
public static string FormattedFileName(string tradingDay)
{
return tradingDay + "_quote.zip";
}
public string FormatDate(DateTime validDay)
{
return validDay.ToString("yyyyMMdd");
}
public static string SetPath(string dataFilePath, string fxPair, string formattedDate)
{
return dataFilePath + fxPair + #"\" + FormattedFileName(formattedDate);
}
public IEnumerable<DateTime> IterateOverDateRange(ForexExchange exchange, DateTime start, DateTime end)
{
for (var day = start.Date; day.Date <= end.Date; day = day.AddDays(1))
if (exchange.IsOpenDuringBar(day.Date, day.Date.AddDays(1), false))
{
yield return day;
}
}
}
}
I figured out what was happening.
I had an old version of TradingDaysFileChecker.cs in the Documents folder of my system that I had backed up for some reason.
In that version all the file handling logic was inside Program.cs.
I refactored and extracted out the file handling to the new class.
For some reason when I was running the script, it was still using that old copy, even though it was not in the solution folder.
That's why the change in the class name and appeared to happen, it was jumping into the Program.cs and Main method of that other file, pulling it up from my Documents folder.
How that happened, I do not know.
I deleted the file from my Documents folder and now it is behaving correctly.
How does one use a MongoDB ConventionPack in C# I have the following code:
MongoDatabase Repository = Server.GetDatabase(RepoName);
this.Collection = Repository.GetCollection<T>(CollectionName);
var myConventions = new ConventionPack();
myConventions.Add(new CamelCaseElementNameConvention());
Does the convention pack automatically attach to this.Collection? When I load in a new object will it automatically persist it as this case? Do I have to add tags in my class declaration (like a data contract)?
You need to register the pack in the ConventionRegistry:
var pack = new ConventionPack();
pack.Add(new CamelCaseElementNameConvention());
ConventionRegistry.Register("camel case",
pack,
t => t.FullName.StartsWith("Your.Name.Space."));
If you want to apply this globally, you can replace the last param with something simpler like t => true.
Working sample code that serializes and de-serializes (driver 1.8.20, mongodb 2.5.0):
using System;
using System.Linq;
using MongoDB.Bson;
using MongoDB.Bson.Serialization.Conventions;
using MongoDB.Driver;
namespace playground
{
class Simple
{
public ObjectId Id { get; set; }
public String Name { get; set; }
public int Counter { get; set; }
}
class Program
{
static void Main(string[] args)
{
MongoClient client = new MongoClient("mongodb://localhost/test");
var db = client.GetServer().GetDatabase("test");
var collection = db.GetCollection<Simple>("Simple");
var pack = new ConventionPack();
pack.Add(new CamelCaseElementNameConvention());
ConventionRegistry.Register("camel case", pack, t => true);
collection.Insert(new Simple { Counter = 1234, Name = "John" });
var all = collection.FindAll().ToList();
Console.WriteLine("Name: " + all[0].Name);
}
}
}
Does anyone know how to convert a Hashtable to an XML String then back to a HashTable without using the .NET based XMLSerializer. The XMLSerializer poses some security concerns when code runs inside of IE and the browser's protected mode is turned on -
So basically I am looking for an easy way to convert that Hashtable to string and back to a Hashtable.
Any sample code would be greatly appreciated.
Thanks
You could use the DataContractSerializer class:
using System;
using System.Collections;
using System.IO;
using System.Runtime.Serialization;
using System.Text;
using System.Xml;
public class MyClass
{
public string Foo { get; set; }
public string Bar { get; set; }
}
class Program
{
static void Main()
{
var table = new Hashtable
{
{ "obj1", new MyClass { Foo = "foo", Bar = "bar" } },
{ "obj2", new MyClass { Foo = "baz" } },
};
var sb = new StringBuilder();
var serializer = new DataContractSerializer(typeof(Hashtable), new[] { typeof(MyClass) });
using (var writer = new StringWriter(sb))
using (var xmlWriter = XmlWriter.Create(writer))
{
serializer.WriteObject(xmlWriter, table);
}
Console.WriteLine(sb);
using (var reader = new StringReader(sb.ToString()))
using (var xmlReader = XmlReader.Create(reader))
{
table = (Hashtable)serializer.ReadObject(xmlReader);
}
}
}
I don't have time to test this, but try:
XDocument doc = new XDocument("HashTable",
from de in hashTable
select new XElement("Item",
new XAttribute("key", de.Key),
new XAttribute("value", de.Value)));
If I create a class in C#, how can I serialize/deserialize it to a file? Is this somethat that can be done using built in functionality or is it custom code?
XmlSerializer; note that the exact xml names can be controlled through various attributes, but all you really need is:
a public type
with a default constructor
and public read/write members (ideally properties)
Example:
using System;
using System.Xml;
using System.Xml.Serialization;
public class Person {
public string Name { get; set; }
}
static class Program {
static void Main() {
Person person = new Person { Name = "Fred"};
XmlSerializer ser = new XmlSerializer(typeof(Person));
// write
using (XmlWriter xw = XmlWriter.Create("file.xml")) {
ser.Serialize(xw, person);
}
// read
using (XmlReader xr = XmlReader.Create("file.xml")) {
Person clone = (Person) ser.Deserialize(xr);
Console.WriteLine(clone.Name);
}
}
}
You need to use class XmlSerializer. Main methods are Serialize and Deserialize. They accept streams, text readers\writers and other classes.
Code sample:
public class Program
{
public class MyClass
{
public string Name { get; set; }
}
static void Main(string[] args)
{
var myObj = new MyClass { Name = "My name" };
var fileName = "data.xml";
var serializer = new XmlSerializer(typeof(MyClass));
using (var output = new XmlTextWriter(fileName, Encoding.UTF8))
serializer.Serialize(output, myObj);
using (var input = new StreamReader(fileName))
{
var deserialized = (MyClass)serializer.Deserialize(input);
Console.WriteLine(deserialized.Name);
}
Console.WriteLine("Press ENTER to finish");
Console.ReadLine();
}
}