it is possible using OpenPop.dll.
Pop3Client objPOP3Client = new Pop3Client();
int intTotalEmail = 0;
DataTable dtEmail = new DataTable();
object[] objMessageParts;
try
{
dtEmail = GetAllEmailStructure();
if (objPOP3Client.Connected)
objPOP3Client.Disconnect();
objPOP3Client.Connect(strHostName, intPort, bulUseSSL);
try
{
objPOP3Client.Authenticate(strUserName, new Common()._Decode(strPassword));
intTotalEmail = objPOP3Client.GetMessageCount();
AddMapping();
for (int i = 1; i <= intTotalEmail; i++)
{
objMessageParts = GetMessageContent(i, ref objPOP3Client, dtExistMailList);
if (objMessageParts != null && objMessageParts[0].ToString() == "0")
{
AddToDtEmail(objMessageParts, i, dtEmail, dtUserList, dtTicketIDList, dtBlacklistEmails, dtBlacklistSubject, dtBlacklistDomains);
}
}
}
catch (Exception ex)
{
}
}
catch (Exception ex)
{
ParserLogError(ex, "GetAllEmail()");
}
finally
{
if (objPOP3Client.Connected)
objPOP3Client.Disconnect();
}
// function
public object[] GetMessageContent(int intMessageNumber, ref Pop3Client objPOP3Client, DataTable dtExistingMails)
{
object[] strArrMessage = new object[10];
Message objMessage;
MessagePart plainTextPart = null, HTMLTextPart = null;
string strMessageId = "";
try
{
strArrMessage[0] = "";
strArrMessage[1] = "";
strArrMessage[2] = "";
strArrMessage[3] = "";
strArrMessage[4] = "";
strArrMessage[5] = "";
strArrMessage[6] = "";
strArrMessage[7] = null;
strArrMessage[8] = null;
strArrMessage[7] = "";
strArrMessage[8] = "";
objMessage = objPOP3Client.GetMessage(intMessageNumber);
strMessageId = (objMessage.Headers.MessageId == null ? "" : objMessage.Headers.MessageId.Trim());
if (!IsExistMessageID(dtExistingMails, strMessageId)) //check in data base message id is exists or not
{
strArrMessage[0] = "0";
strArrMessage[1] = objMessage.Headers.From.Address.Trim(); // From EMail Address
strArrMessage[2] = objMessage.Headers.From.DisplayName.Trim(); // From EMail Name
strArrMessage[3] = objMessage.Headers.Subject.Trim();// Mail Subject
plainTextPart = objMessage.FindFirstPlainTextVersion();
strArrMessage[4] = (plainTextPart == null ? "" : plainTextPart.GetBodyAsText().Trim());
HTMLTextPart = objMessage.FindFirstHtmlVersion();
strArrMessage[5] = (HTMLTextPart == null ? "" : HTMLTextPart.GetBodyAsText().Trim());
strArrMessage[6] = strMessageId;
List<MessagePart> attachment = objMessage.FindAllAttachments();
strArrMessage[7] = null;
strArrMessage[8] = null;
if (attachment.Count > 0)
{
if (attachment[0] != null && attachment[0].IsAttachment)
{
strArrMessage[7] = attachment[0].FileName.Trim();
strArrMessage[8] = attachment[0];
}
}
}
else
{
strArrMessage[0] = "1";
}
}
catch (Exception ex)
{
ParserLogError(ex, "GetMessageContent()");
}
return strArrMessage;
}
but, i want to make it faster than above OpenPop.dll. so please let me know if any other technique are there for parsing mails.
please check code and then tell me.
Thanks in advance
but, i want to make it faster than above OpenPop.dll. so please let me
know if any other technique are there for parsing mails.
In your GetMessageContent() method, the 1 place that consumes the vast amount of time is:
objMessage = objPOP3Client.GetMessage(intMessageNumber);
The network I/O part of downloading a message cannot really be optimized, but OpenPop.NET's parser is slow (based on my own performance tests).
MimeKit is 25x faster than OpenPop.NET at parsing email messages.
One of the main performance problems in OpenPop.NET's MIME parser is the fact that it uses a StreamReader for parsing (which is slow due to unnecessary charset conversion, reading 1 line at a time, etc - I have an analysis of another email library that uses StreamReader for parsing here: https://stackoverflow.com/a/18787176/87117).
Then there's the problem that OpenPop.NET's parser also uses Regex to remove CFWS (Comments and Folding White Space) from a header string before parsing/decoding it. This is expensive. It's far better to write a good tokenizer that can deal with CFWS.
If you are interested in some of the other techniques I used to optimize MimeKit to be so fast (as fast or faster than highly optimized C implementations), I wrote some blog posts about this:
Optimization Tricks used by MimeKit: Part 1
The summary of the optimization I talk about in part 1 is replacing loops like this that scan for the end of a line:
while (*inptr != (byte) '\n')
inptr++;
with a faster loop, like this:
int* dword = (int*) inptr;
do {
mask = *dword++ ^ 0x0A0A0A0A;
mask = ((mask - 0x01010101) & (~mask & 0x80808080));
} while (mask == 0);
inptr = (byte*) (dword - 1);
while (*inptr != (byte) '\n')
inptr++;
which improved performance by 20% (although on non-x86 architectures, it requires 'dword' to be 4-byte aligned).
Optimization Tricks used by MimeKit: Part 2
In part 2, I talk about writing a more optimized version of System.IO.MemoryStream. The problem with MemoryStream is that it has to keep 1 contiguous block of memory with the content, which means that as you write more data to it and it has to resize its internal byte array, it has to copy the content to the new array (which is expensive, especially once the amount of data in the stream is large).
To work around this performance bottleneck, I wrote a MemoryBlockStream which does not need to use a contiguous block of memory - it uses a linked list of byte arrays. Instead of having to resize the byte array when you overflow the current buffer, it simply allocates another 2048-byte array that the data will overflow into and appends it to the linked list.
Note: MimeKit itself only does email parsing, it doesn't do POP3 or SMTP or IMAP. If you want that kind of functionality, I've also written a library built on MimeKit that does that as well: MailKit
Update:
Sample code using MailKit (as requested) to download/parse all messages:
using System;
using System.Net;
using MailKit.Net.Pop3;
using MailKit;
using MimeKit;
namespace TestClient {
class Program
{
public static void Main (string[] args)
{
using (var client = new Pop3Client ()) {
client.Connect ("pop.gmail.com", 995, true);
// Note: since we don't have an OAuth2 token, disable
// the XOAUTH2 authentication mechanism.
client.AuthenticationMechanisms.Remove ("XOAUTH2");
client.Authenticate ("joey#gmail.com", "password");
int count = client.GetMessageCount ();
for (int i = 0; i < count; i++) {
var message = client.GetMessage (i);
Console.WriteLine ("Subject: {0}", message.Subject);
}
client.Disconnect (true);
}
}
}
}
Related
Hi i'm newbie in RFID reading. So firstly i downloaded pcsc sharp repository from github. Then i tried to read binary from common rfid tag, it works perfect but the next step was to read data from as i think emulated rfid tag. RFID tag controller is pn71501. From this tag using pcsc sharp i can't read any data excluding ATR and uid. I tried to read this tag using my iPhone and it read it. So what i'm doing wrong?
I also tried to use already done software but it couldn't read it also.
Here is what i get using NFC Tools:
PS Smart Card reader i used is ACS ACR1252
Here is my code:
using System;
using System.Text;
using System.Collections.Generic;
using PCSC;
using PCSC.Iso7816;
namespace Transmit {
public class Program {
public static void Main() {
using (var context = ContextFactory.Instance.Establish(SCardScope.System)) {
var readerNames = context.GetReaders();
if (NoReaderFound(readerNames)) {
Console.WriteLine("You need at least one reader in order to run this example.");
Console.ReadKey();
return;
}
var readerName = ChooseRfidReader(readerNames);
if (readerName == null) {
return;
}
String response = "";
using (var rfidReader = context.ConnectReader(readerName, SCardShareMode.Shared, SCardProtocol.Any)) {
// for (byte i = 0x00; i < 0x47; i++) {
var apdu = new CommandApdu(IsoCase.Case3Extended, rfidReader.Protocol) {
CLA = 0xFF,
Instruction = (InstructionCode)0xB0,
P1 = 0x00,
P2 = 0x00,
Le = 0x10
};
using (rfidReader.Transaction(SCardReaderDisposition.Leave)) {
//Console.WriteLine("Retrieving the UID .... ");
var sendPci = SCardPCI.GetPci(rfidReader.Protocol);
var receivePci = new SCardPCI(); // IO returned protocol control information.
var receiveBuffer = new byte[256];
var command = apdu.ToArray();
var bytesReceived = rfidReader.Transmit(
sendPci, // Protocol Control Information (T0, T1 or Raw)
command, // command APDU
command.Length,
receivePci, // returning Protocol Control Information
receiveBuffer,
receiveBuffer.Length); // data buffer
var responseApdu =
new ResponseApdu(receiveBuffer, bytesReceived, IsoCase.Case3Extended, rfidReader.Protocol);
Console.WriteLine(String.Format("SW1: {0:X2} SW2: {1:X2}", responseApdu.SW1, responseApdu.SW2));
//if(responseApdu.DataSize > 0) {
//response += BitConverter.ToString(responseApdu.GetData()).Replace('-', ' ');
response += responseApdu.DataSize;
// }
}
// }
}
/*String[] devidedResponse = response.Split(' ');
String stillResponse = "";
bool notStarted = true;
int skipBytes = 7;
int onByte = 0;
for(int i = 0; i < devidedResponse.Length; i++) {
if (devidedResponse[i] != "D1" && notStarted) {
continue;
} else if (onByte < skipBytes) {
notStarted = false;
onByte += 1;
continue;
} else if (devidedResponse[i] == "FE") {
break;
}
stillResponse += devidedResponse[i] + " ";
}
String res = stillResponse.Trim();
string asciiCharString = "";
var splitResult = res.Split(' ');
foreach (string hexChar in splitResult) {
var byteChar = int.Parse(hexChar, System.Globalization.NumberStyles.HexNumber);
asciiCharString += (char)byteChar;
}*/
Console.WriteLine(response);
}
Console.WriteLine("\nPress any key to exit.");
Console.ReadKey();
}
private static string ChooseRfidReader(IList<string> readerNames) {
// Show available readers.
Console.WriteLine("Available readers: ");
for (var i = 0; i < readerNames.Count; i++) {
Console.WriteLine($"[{i}] {readerNames[i]}");
}
// Ask the user which one to choose.
Console.Write("Which reader is an RFID reader? ");
var line = Console.ReadLine();
if (int.TryParse(line, out var choice) && choice >= 0 && (choice <= readerNames.Count)) {
return readerNames[choice];
}
Console.WriteLine("An invalid number has been entered.");
Console.ReadKey();
return null;
}
private static bool NoReaderFound(ICollection<string> readerNames) =>
readerNames == null || readerNames.Count < 1;
}
}
I know. I looked it up earlier. Can you read the card with the file explorer?
A hardware device like a UART can be read at three different levels
Read/Write UART directly by finding hardware I/O address
Read/Write through driver. In c# Use Open Serial Port. The driver gets the hardware I/O
Read/Write through an application. The application does 1 and/or 2 above.
You have a working application (number 3) and I do not know if it is using method 1 or 2.
With the card reader I'm trying to make your programming as simple as possible. The easiest method is 3 if you have an API. Next easiest is method 2 which you should be able to use if you installed the vendor driver. You should see device in device manager.
To unlock the card (encryption) you also need to install the certificate than came with card. The file explorer should be able to read/write card.
I am trying to learn C# through trials and errors. I have a server in Python, and the client in C#.
The client is supposed to get data from the server, save it to disk ((which it doesn't)), then continue using that data to communicate until the user decides to exit it.
Unfortunately, it has been raising an Exception Access Violation for quite some time, and provides no helpful information as to why.
Code:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.Scripting.Hosting;
using System.Threading;
using Rage;
using System.Net.Sockets;
using System.IO;
[assembly: Rage.Attributes.Plugin("LSPDFROnlineClient", Description = "LSPDFR Online Client. Used to connect to LSPDFR Online.", Author = "Thecheater887")]
namespace LSPDFROnlineClient
{
public class EntryPoint
{
private static int IsDead;
public static void Main()
{
IsDead = 0;
Thread mt = new Thread(new ThreadStart(Run));
mt.Start();
while (true)
{
if (IsDead == 1)
{
break;
}
GameFiber.Yield();
}
return;
}
public static void Run()
{
File.WriteAllText("C:/ProgramData/dbg.log","1");
try
{
Byte[] header;
Byte[] packet;
Byte[] data;
Byte[] kad;
Byte[] td;
int paylen;
String rd;
String nd;
String msgid;
File.WriteAllText("C:/ProgramData/dbg.log", "2");
TcpClient connector = new TcpClient();
connector.Connect("127.0.0.1", 5773);
NetworkStream conn = connector.GetStream();
try {
File.WriteAllText("C:/ProgramData/dbg.log", "3");
FileStream savedat = File.OpenRead("C:/Users/Public/Documents/save.dat");
BinaryReader savdat = new BinaryReader(savedat);
nd = savdat.ReadString();
savdat.Close();
if (nd.Length == 16)
{
} else {
File.WriteAllText("C:/ProgramData/save.dat", "user000000000000");
nd = "user000000000000";
}
}
catch
{
File.WriteAllText("C:/ProgramData/save.dat", "user000000000000");
nd = "user000000000000";
}
File.WriteAllText("C:/ProgramData/dbg.log", "4");
data = new Byte[26];
data = Encoding.ASCII.GetBytes("clogr00000" + nd);
conn.Write(data, 0, data.Length);
while (true)
{
try
{
// Get header of packet
header = new Byte[26];
Int32 rcvhead = conn.Read(header, 0, header.Length);
String hd = Encoding.ASCII.GetString(header, 0, rcvhead);
//Deal with it 8-)
msgid = hd.Substring(0, 5);
paylen = Convert.ToInt32(hd.Substring(5, 5));
string servkey = hd.Substring(10, 16);
}
catch
{
continue;
}
File.WriteAllText("C:/ProgramData/dbg.log", "5");
try
{
//Receive packet data
if (paylen > 0)
{
packet = new Byte[paylen];
Int32 newdata = conn.Read(packet, 0, packet.Length);
rd = Encoding.ASCII.GetString(packet, 0, newdata);
}
else
{
rd = null;
}
File.WriteAllText("C:/ProgramData/dbg.log", "6");
if (msgid == "ConOK")
{
File.WriteAllText("C:/ProgramData/dbg.log", "7");
string userid = rd.Substring(0, 16);
Game.DisplayHelp(rd.Substring(16, (rd.Length - 16)));
File.WriteAllText("C:/ProgramData/save.dat", userid);
File.WriteAllText("C:/ProgramData/dbg.log", "8");
}
else if (msgid == "savdt")
{
File.WriteAllText("C:/ProgramData/dbg.log", "9");
string[] tnd = rd.Split(',');
var nud = new List<string>();
nud.Add("Player1");
nud.AddRange(tnd);
File.WriteAllText("C:/ProgramData/dbg.log", "A");
string name = nud[0];
string streetname = nud[1];
int money = Convert.ToInt32(nud[2]);
int bounty = Convert.ToInt32(nud[3]);
int playerrep = Convert.ToInt32(nud[4]);
File.WriteAllText("C:/ProgramData/dbg.log", "B");
int rep = Convert.ToInt32(nud[5]);
string pclass = nud[6];
bool canbecop = Convert.ToBoolean(nud[7]);
int rank = Convert.ToInt32(nud[8]);
int stars = Convert.ToInt32(nud[9]);
int cites = Convert.ToInt32(nud[10]);
File.WriteAllText("C:/ProgramData/dbg.log", "C");
int citesgiven = Convert.ToInt32(nud[11]);
int citesdismissed = Convert.ToInt32(nud[12]);
int arrestsmade = Convert.ToInt32(nud[13]);
int arrested = Convert.ToInt32(nud[14]);
int convictionsmade = Convert.ToInt32(nud[15]);
int convitced = Convert.ToInt32(nud[16]);
string warrant = nud[17];
File.WriteAllText("C:/ProgramData/dbg.log", "D");
int prisontimeremaining = Convert.ToInt32(nud[18]);
int copskilled = Convert.ToInt32(nud[19]);
int crimskilled = Convert.ToInt32(nud[20]);
int civskilled = Convert.ToInt32(nud[21]);
int bountyclaimed = Convert.ToInt32(nud[22]);
int overflowprep = Convert.ToInt32(nud[23]);
string title = nud[24];
bool banned = Convert.ToBoolean(nud[25]);
bool vip = Convert.ToBoolean(nud[26]);
int viprank = Convert.ToInt32(nud[27]);
File.WriteAllText("C:/ProgramData/dbg.log", "E");
var v3 = new Vector3();
float posx = Convert.ToSingle(nud[29]);
float posy = Convert.ToSingle(nud[30]);
float posz = Convert.ToSingle(nud[31]);
v3.X = posx;
v3.Y = posy;
v3.Z = posz;
File.WriteAllText("C:/ProgramData/dbg.log", "EE");
int rot = Convert.ToInt32(nud[32]);
File.WriteAllText("C:/ProgramData/dbg.log", "FF");
World.TeleportLocalPlayer(v3, false);
File.WriteAllText("C:/ProgramData/dbg.log", "F");
string custommessage = nud[28];
if (custommessage == "null")
{
} else {
Game.DisplayNotification(custommessage);
}
}
else if (msgid == "isalv")
{
kad = new Byte[26];
kad = Encoding.ASCII.GetBytes("yesil00000" + nd);
conn.Write(kad, 0, kad.Length);
}
else if (msgid == "pospk")
{
}
else
{
Game.DisplayNotification("Unknown packet recieved! ID: " + msgid);
}
//send end client turn
td = new Byte[26];
td = Encoding.ASCII.GetBytes("endmt00000" + nd);
conn.Write(td, 0,td.Length);
File.WriteAllText("C:/ProgramData/dbg.log", "0");
//
}
catch (Exception e)
{
Game.DisplayHelp(Convert.ToString(e));
Game.DisplayNotification("LSPDFR Online has crashed. Try reloading it maybe..?");
}
}
} catch (Exception e) {
Game.DisplayHelp(Convert.ToString(e));
Game.DisplayNotification("Connection interrupted! Reconnecting....");
IsDead = 1;
return;
}
}
}
}
The protocol goes as such;
Client -> Server: LoginRequest
Server -> Client: LoginOkay
Client -> Server: EndTurnMessage
Server -> Client: SaveDataMessage
Client -> Server: EndTurnMessage
Server -> Client: PositionUpdatePacket
Client -> Server: EndTurnMessage
Then continue routine, however, the server only receives one of these EndTurnMessage packets, which means it is choking on the save data portion, right?
Possibly, but that was working at an earlier time without flaw, and hasn't been modified since.
It is a class file, so it can't be debugged, and I've been tearing my hair out as to what is causing it.
Yes, it is crap-code, and needs rewritten at some point, I am aware, but I'd like it to work before I rewrite it entirely.
TL;DR: Why is this code raising an Access Violation? It's around the savdt sector or after.
UPDATE: I fixed the issue posted in the first answer, however, that didn't do much, so, as posted in both the answer and comments, it's rather difficult to debug with a program, so I'll try the old fashioned route of logging info every so many lines of code. I'll keep this updated.
UPDATE 2: I have figured out from the log debugging, that the line causing the error is World.TeleportLocalPlayer(v3, false);. Unfortunatley, World can't be inherited, and the documentation claims that Vector3 requires you to set it's internal values using get and set. I saw that on MSDN previously, but have no clue about how to search it, and there is nor get or set methods available within that Vector3 object.
You might have a stream that remained open, which prevents a new one to be created. If the msgid is "ConOK" you are creating a new instance without closing it after the write operation is done.
if (msgid == "ConOK"){
string userid = rd.Substring(0, 16);
Game.DisplayHelp(rd.Substring(16, (rd.Length - 16)));
FileStream savedat = File.OpenWrite(("C:/ProgramData/save.dat"));
BinaryWriter savdat = new BinaryWriter(savedat);
savdat.Write(userid);
// Close file stream here
}
But that's just a first guess. You can help us and yourself by making use of the debugger. The fact that your code is contained by a "class file" is no problem but a requirement.
Hava a look at this article for more information about debugging in the world of C#:
http://www.dotnetperls.com/debugging
At a first glance, you can have different class of bugs. Disregarding the logic flows and intended behaviour of the program, let's start with basic debugging.
at this step, don't use threads and fibers, from Main just call Run
there isn't strong input validation
use a lot more of try catch, isolating small pieces of code
in the catch, print ex.Message and ex.StackTrace
read carefully the docs about the methods you call and their possible exceptions
it is weird you write a file inside a exception (catch)
possible race conditions on global variables?
inside Run to set IsDead use Interlocked.Increment
https://msdn.microsoft.com/en-us/library/dd78zt0c(v=vs.110).aspx
...
remove the unused ( I think like Microsoft.Scripting.Hosting ), it only confuse-a-cat
I do not want to read the whole file at any point, I know there are answers on that question, I want t
o read the First or Last line.
I know that my code locks the file that it's reading for two reasons 1) The application that writes to the file crashes intermittently when I run my little app with this code but it never crashes when I am not running this code! 2) There are a few articles that will tell you that File.ReadLines locks the file.
There are some similar questions but that answer seems to involve reading the whole file which is slow for large files and therefore not what I want to do. My requirement to only read the last line most of the time is also unique from what I have read about.
I nead to know how to read the first line (Header row) and the last line (latest row). I do not want to read all lines at any point in my code because this file can become huge and reading the entire file will become slow.
I know that
line = File.ReadLines(fullFilename).First().Replace("\"", "");
... is the same as ...
FileStream fs = new FileStream(#fullFilename, FileMode.Open, FileAccess.Read, FileShare.Read);
My question is, how can I repeatedly read the first and last lines of a file which may be being written to by another application without locking it in any way. I have no control over the application that is writting to the file. It is a data log which can be appended to at any time. The reason I am listening in this way is that this log can be appended to for days on end. I want to see the latest data in this log in my own c# programme without waiting for the log to finish being written to.
My code to call the reading / listening function ...
//Start Listening to the "data log"
private void btnDeconstructCSVFile_Click(object sender, EventArgs e)
{
MySandbox.CopyCSVDataFromLogFile copyCSVDataFromLogFile = new MySandbox.CopyCSVDataFromLogFile();
copyCSVDataFromLogFile.checkForLogData();
}
My class which does the listening. For now it simply adds the data to 2 generics lists ...
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using MySandbox.Classes;
using System.IO;
namespace MySandbox
{
public class CopyCSVDataFromLogFile
{
static private List<LogRowData> listMSDataRows = new List<LogRowData>();
static String fullFilename = string.Empty;
static LogRowData previousLineLogRowList = new LogRowData();
static LogRowData logRowList = new LogRowData();
static LogRowData logHeaderRowList = new LogRowData();
static Boolean checking = false;
public void checkForLogData()
{
//Initialise
string[] logHeaderArray = new string[] { };
string[] badDataRowsArray = new string[] { };
//Get the latest full filename (file with new data)
//Assumption: only 1 file is written to at a time in this directory.
String directory = "C:\\TestDir\\";
string pattern = "*.csv";
var dirInfo = new DirectoryInfo(directory);
var file = (from f in dirInfo.GetFiles(pattern) orderby f.LastWriteTime descending select f).First();
fullFilename = directory + file.ToString(); //This is the full filepath and name of the latest file in the directory!
if (logHeaderArray.Length == 0)
{
//Populate the Header Row
logHeaderRowList = getRow(fullFilename, true);
}
LogRowData tempLogRowList = new LogRowData();
if (!checking)
{
//Read the latest data in an asynchronous loop
callDataProcess();
}
}
private async void callDataProcess()
{
checking = true; //Begin checking
await checkForNewDataAndSaveIfFound();
}
private static Task checkForNewDataAndSaveIfFound()
{
return Task.Run(() => //Call the async "Task"
{
while (checking) //Loop (asynchronously)
{
LogRowData tempLogRowList = new LogRowData();
if (logHeaderRowList.ValueList.Count == 0)
{
//Populate the Header row
logHeaderRowList = getRow(fullFilename, true);
}
else
{
//Populate Data row
tempLogRowList = getRow(fullFilename, false);
if ((!Enumerable.SequenceEqual(tempLogRowList.ValueList, previousLineLogRowList.ValueList)) &&
(!Enumerable.SequenceEqual(tempLogRowList.ValueList, logHeaderRowList.ValueList)))
{
logRowList = getRow(fullFilename, false);
listMSDataRows.Add(logRowList);
previousLineLogRowList = logRowList;
}
}
//System.Threading.Thread.Sleep(10); //Wait for next row.
}
});
}
private static LogRowData getRow(string fullFilename, bool isHeader)
{
string line;
string[] logDataArray = new string[] { };
LogRowData logRowListResult = new LogRowData();
try
{
if (isHeader)
{
//Asign first (header) row data.
//Works but seems to block writting to the file!!!!!!!!!!!!!!!!!!!!!!!!!!!
line = File.ReadLines(fullFilename).First().Replace("\"", "");
}
else
{
//Assign data as last row (default behaviour).
line = File.ReadLines(fullFilename).Last().Replace("\"", "");
}
logDataArray = line.Split(',');
//Copy Array to Generics List and remove last value if it's empty.
for (int i = 0; i < logDataArray.Length; i++)
{
if (i < logDataArray.Length)
{
if (i < logDataArray.Length - 1)
{
//Value is not at the end, from observation, these always have a value (even if it's zero) and so we'll store the value.
logRowListResult.ValueList.Add(logDataArray[i]);
}
else
{
//This is the last value
if (logDataArray[i].Replace("\"", "").Trim().Length > 0)
{
//In this case, the last value is not empty, store it as normal.
logRowListResult.ValueList.Add(logDataArray[i]);
}
else { /*The last value is empty, e.g. "123,456,"; the final comma denotes another field but this field is empty so we will ignore it now. */ }
}
}
}
}
catch (Exception ex)
{
if (ex.Message == "Sequence contains no elements")
{ /*Empty file, no problem. The code will safely loop and then will pick up the header when it appears.*/ }
else
{
//TODO: catch this error properly
Int32 problemID = 10; //Unknown ERROR.
}
}
return logRowListResult;
}
}
}
I found the answer in a combination of other questions. One answer explaining how to read from the end of a file, which I adapted so that it would read only 1 line from the end of the file. And another explaining how to read the entire file without locking it (I did not want to read the entire file but the not locking part was useful). So now you can read the last line of the file (if it contains end of line characters) without locking it. For other end of line delimeters, just replace my 10 and 13 with your end of line character bytes...
Add the method below to public class CopyCSVDataFromLogFile
private static string Reverse(string str)
{
char[] arr = new char[str.Length];
for (int i = 0; i < str.Length; i++)
arr[i] = str[str.Length - 1 - i];
return new string(arr);
}
and replace this line ...
line = File.ReadLines(fullFilename).Last().Replace("\"", "");
with this code block ...
Int32 endOfLineCharacterCount = 0;
Int32 previousCharByte = 0;
Int32 currentCharByte = 0;
//Read the file, from the end, for 1 line, allowing other programmes to access it for read and write!
using (FileStream reader = new FileStream(fullFilename, FileMode.Open, FileAccess.Read, FileShare.ReadWrite, 0x1000, FileOptions.SequentialScan))
{
int i = 0;
StringBuilder lineBuffer = new StringBuilder();
int byteRead;
while ((-i < reader.Length) /*Belt and braces: if there were no end of line characters, reading beyond the file would give a catastrophic error here (to be avoided thus).*/
&& (endOfLineCharacterCount < 2)/*Exit Condition*/)
{
reader.Seek(--i, SeekOrigin.End);
byteRead = reader.ReadByte();
currentCharByte = byteRead;
//Exit condition: the first 2 characters we read (reading backwards remember) were end of line ().
//So when we read the second end of line, we have read 1 whole line (the last line in the file)
//and we must exit now.
if (currentCharByte == 13 && previousCharByte == 10)
{
endOfLineCharacterCount++;
}
if (byteRead == 10 && lineBuffer.Length > 0)
{
line += Reverse(lineBuffer.ToString());
lineBuffer.Remove(0, lineBuffer.Length);
}
lineBuffer.Append((char)byteRead);
previousCharByte = byteRead;
}
reader.Close();
}
I have to implement a tcp connection where raw xml data is passed.
Unfortunately there is no message framing, I now this is realy bad, but I have to deal with this...
The Message would look like this:
<?xml version="1.0" encoding="utf-8"?>
<DATA></DATA>
or this
<?xml version="1.0" encoding="utf-8"?>
<DATA />
Now I have to receive messages that could have self closed tags. The message is always the same, it is always like xml description and a data tag with inner xml that is the message content.
So if it would be without self closed tags, this would be easy, but how can I read both?
By the way I am using the TcpListener.
Edit :
Everything is fine if there is no self closed tag.
if (_clientSocket != null)
{
NetworkStream networkStream = _clientSocket.GetStream();
_clientSocket.ReceiveTimeout = 100; // 1000 miliseconds
while (_continueProcess)
{
if (networkStream.DataAvailable)
{
bool isMessageComplete = false;
String messageString = String.Empty;
while (!isMessageComplete)
{
var bytes = new byte[_clientSocket.ReceiveBufferSize];
try
{
int bytesReaded = networkStream.Read(bytes, 0, (int) _clientSocket.ReceiveBufferSize);
if (bytesReaded > 0)
{
var data = Encoding.UTF8.GetString(bytes, 0, bytesReaded);
messageString += data;
if (messageString.IndexOf("<DATA", StringComparison.OrdinalIgnoreCase) > 0 &&
messageString.IndexOf("</DATA", StringComparison.OrdinalIgnoreCase) > 0)
{
isMessageComplete = true;
}
}
}
catch (IOException)
{
// Timeout
}
catch (SocketException)
{
Console.WriteLine("Conection is broken!");
break;
}
}
}
Thread.Sleep(200);
} // while ( _continueProcess )
networkStream.Close();
_clientSocket.Close();
}
Edit 2 (30.03.2015 12:00)
Unfortunately it is not possible to use some kind of message frame.
So I ended up to use this part of code (DATA is my root node):
if (_clientSocket != null)
{
NetworkStream networkStream = _clientSocket.GetStream();
_clientSocket.ReceiveTimeout = 100;
string data = string.Empty;
while (_continueProcess)
{
try
{
if (networkStream.DataAvailable)
{
Stopwatch sw = new Stopwatch();
sw.Start();
var bytes = new byte[_clientSocket.ReceiveBufferSize];
int completeXmlLength = 0;
int bytesReaded = networkStream.Read(bytes, 0, (int) _clientSocket.ReceiveBufferSize);
if (bytesReaded > 0)
{
message.AddRange(bytes);
data += Encoding.UTF8.GetString(bytes, 0, bytesReaded);
if (data.IndexOf("<?", StringComparison.Ordinal) == 0)
{
if (data.IndexOf("<DATA", StringComparison.Ordinal) > 0)
{
Int32 rootStartPos = data.IndexOf("<DATA", StringComparison.Ordinal);
completeXmlLength += rootStartPos;
var root = data.Substring(rootStartPos);
int rootCloseTagPos = root.IndexOf(">", StringComparison.Ordinal);
Int32 rootSelfClosedTagPos = root.IndexOf("/>", StringComparison.Ordinal);
// If there is an empty tag that is self closed.
if (rootSelfClosedTagPos > 0)
{
string rootTag = root.Substring(0, rootSelfClosedTagPos +1);
// If there is no '>' between the self closed tag and the start of '<DATA'
// the root element is empty.
if (rootTag.IndexOf(">", StringComparison.Ordinal) <= 0)
{
completeXmlLength += rootSelfClosedTagPos;
string messageXmlString = data.Substring(0, completeXmlLength + 1);
data = data.Substring(messageXmlString.Length);
try
{
// parse complete xml.
XDocument xmlDocument = XDocument.Parse(messageXmlString);
}
catch(Exception)
{
// Invalid Xml.
}
continue;
}
}
if (rootCloseTagPos > 0)
{
Int32 rootEndTagStartPos = root.IndexOf("</DATA", StringComparison.Ordinal);
if (rootEndTagStartPos > 0)
{
var endTagString = root.Substring(rootEndTagStartPos);
completeXmlLength += rootEndTagStartPos;
Int32 completeEndPos = endTagString.IndexOf(">", StringComparison.Ordinal);
if (completeEndPos > 0)
{
completeXmlLength += completeEndPos;
string messageXmlString = data.Substring(0, completeXmlLength + 1);
data = data.Substring(messageXmlString.Length);
try
{
// parse complete xml.
XDocument xmlDocument = XDocument.Parse(messageXmlString);
}
catch(Exception)
{
// Invalid Xml.
}
}
}
}
}
}
}
sw.Stop();
string timeElapsed = sw.Elapsed.ToString();
}
}
catch (IOException)
{
data = String.Empty;
}
catch (SocketException)
{
Console.WriteLine("Conection is broken!");
break;
}
}
This code I had use if ther were some kind of message framing, in this case 4 bytes of message length:
if (_clientSocket != null)
{
NetworkStream networkStream = _clientSocket.GetStream();
_clientSocket.ReceiveTimeout = 100;
string data = string.Empty;
while (_continueProcess)
{
try
{
if (networkStream.DataAvailable)
{
Stopwatch sw = new Stopwatch();
sw.Start();
var lengthBytes = new byte[sizeof (Int32)];
int bytesReaded = networkStream.Read(lengthBytes, 0, sizeof (Int32) - offset);
if (bytesReaded > 0)
{
offset += bytesReaded;
message.AddRange(lengthBytes.Take(bytesReaded));
}
if (offset < sizeof (Int32))
{
continue;
}
Int32 length = BitConverter.ToInt32(message.Take(sizeof(Int32)).ToArray(), 0);
message.Clear();
while (length > 0)
{
Int32 bytesToRead = length < _clientSocket.ReceiveBufferSize ? length : _clientSocket.ReceiveBufferSize;
byte[] messageBytes = new byte[bytesToRead];
bytesReaded = networkStream.Read(messageBytes, 0, bytesToRead);
length = length - bytesReaded;
message.AddRange(messageBytes);
}
try
{
string xml = Encoding.UTF8.GetString(message.ToArray());
XDocument xDocument = XDocument.Parse(xml);
}
catch (Exception ex)
{
// Invalid Xml.
}
sw.Stop();
string timeElapsed = sw.Elapsed.ToString();
}
}
catch (IOException)
{
data = String.Empty;
}
catch (SocketException)
{
Console.WriteLine("Conection is broken!");
break;
}
}
Like you can see I wanted to measure the elapsed time, to see witch methode has a better performance. The strange thing is that the methode whith no message framing has an average time of 0,2290 ms, the other methode has an average time of 1,2253 ms.
Can someone explain me why? I thought the one without message framing would be slower...
Hand the NetworkStream to the .NET XML infrastructure. For example create an XmlReader from the NetworkStream.
Unfortunately I did not find a built-in way to easily create an XmlDocument from an XmlReader that has multiple documents in it. It complains about multiple root elements (which is correct). You would need to wrap the XmlReader and make it stop returning nodes when the first document is done. You can do that by keeping track of some state and by looking at the nesting level. When the nesting level is zero again the first document is done.
This is just a raw sketch. I'm pretty sure this will work and it handles all possible XML documents.
No need for this horrible string processing code that you have there. The existing code looks quite slow as well but since this approach is much better it serves no purpose to comment on the perf issues. You need to throw this away.
I had the same problem - 3rd party system sends messages in XML format via TCP but my TCP client application may receive message partially or several messages at once. One of my colleagues proposed very simple and quite generic solution.
The idea is to have a string buffer which should be populated char by char from TCP stream, after each char try to parse buffer content with regular .Net XML parser. If parser throws an exception - continue adding chars to the buffer. Otherwise - message is ready and can be processed by application.
Here is the code:
private object _dataReceiverLock = new object();
private string _messageBuffer;
private Stopwatch _timeSinceLastMessage = new Stopwatch();
private List<string> NormalizeMessage(string rawMsg)
{
lock (_dataReceiverLock)
{
List<string> result = new List<string>();
//following code prevents buffer to store too old information
if (_timeSinceLastMessage.ElapsedMilliseconds > _settings.ResponseTimeout)
{
_messageBuffer = string.Empty;
}
_timeSinceLastMessage.Restart();
foreach (var ch in rawMsg)
{
_messageBuffer += ch;
if (ch == '>')//to avoid extra checks
{
if (IsValidXml(_messageBuffer))
{
result.Add(_messageBuffer);
_messageBuffer = string.Empty;
}
}
}
return result;
}
}
private bool IsValidXml(string xml)
{
try
{
//fastest way to validate XML format correctness
using (XmlTextReader reader = new XmlTextReader(new StringReader(xml)))
{
while (reader.Read()) { }
}
return true;
}
catch
{
return false;
}
}
Few comments:
Need to control lifetime of string buffer, otherwise in case network disconnection old information may stay in the buffer forever
There major problem here is the performance - parsing after every new character is quite slow. So need to add some optimizations, such as parse only after '>' character.
Make sure this method is thread safe, otherwise several threads may flood string buffer with different XML pieces.
The usage is simple:
private void _tcpClient_DataReceived(byte[] data)
{
var rawMsg = Encoding.Unicode.GetString(data);
var normalizedMessages = NormalizeMessage(rawMsg);
foreach (var normalizedMessage in normalizedMessages)
{
//TODO: your logic
}
}
Im trying to send some object from a server to the client.
My problem is that when im sending only 1 object, everything works correctly. But at the moment i add another object an exception is thrown - "binary stream does not contain a valid binaryheader" or "No map for object (random number)".
My thoughts are that the deserialization does not understand where the stream starts / ends and i hoped that you guys can help me out here.
heres my deserialization code:
public void Listen()
{
try
{
bool offline = true;
Dispatcher.Invoke(System.Windows.Threading.DispatcherPriority.Normal,
new Action(() => offline = Offline));
while (!offline)
{
TcpObject tcpObject = new TcpObject();
IFormatter formatter = new BinaryFormatter();
tcpObject = (TcpObject)formatter.Deserialize(serverStream);
if (tcpObject.Command == Command.Transfer)
{
SentAntenna sentAntenna = (SentAntenna)tcpObject.Object;
int idx = 0;
foreach (string name in SharedProperties.AntennaNames)
{
if (name == sentAntenna.Name)
break;
idx++;
}
if (idx < 9)
{
PointCollection pointCollection = new PointCollection();
foreach (Frequency f in sentAntenna.Frequencies)
pointCollection.Add(new Point(f.Channel, f.Intensity));
SharedProperties.AntennaPoints[idx] = pointCollection;
}
}
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message); // raise an event
}
}
serialization code:
case Command.Transfer:
Console.WriteLine("Transfering");
Thread transfer = new Thread(new ThreadStart(delegate
{
try
{
string aName = tcpObject.Object.ToString();
int indx = 0;
foreach (string name in names)
{
if (name == aName)
break;
indx++;
}
if (indx < 9)
{
while (true) // need to kill when the father thread terminates
{
if (antennas[indx].Frequencies != null)
{
lock (antennas[indx].Frequencies)
{
TcpObject sendTcpObject = new TcpObject();
sendTcpObject.Command = Command.Transfer;
SentAntenna sa = new SentAntenna(antennas[indx].Frequencies, aName);
sendTcpObject.Object = sa;
formatter.Serialize(networkStream, sendTcpObject);
}
}
}
}
}
catch (Exception ex) { Console.WriteLine(ex); }
}));
transfer.Start();
break;
Interesting. There's nothing particularly odd in your serialization code, and I've seen people use vanilla concatenation for multiple objects in the past, although I've actually always advised against it as BinaryFormatter does not explicitly claim this scenario is OK. But: if it isn't, the only thing I can suggest is to implement your own framing; so your write code becomes:
serialize to an empty MemoryStream
note the length and write the length to the NetworkStream, for example as a simple fixed-width 32-bit network-byte-order integer
write the payload from the MemoryStream to the NetworkStream
rinse, repeat
And the read code becomes:
read exactly 4 bytes and compute the length
buffer that many bytes into a MemoryStream
deserialize from the NetworkStream
(Noting in both cases to set the MemoryStream's position back to 0 between write and read)
You can also implement a Stream-subclass that caps the length if you want to avoid a buffer when reading, bit that is more complex.
apperantly i came up with a really simple solution. I just made sure only 1 thread is allowed to transfer data at the same time so i changed this line of code:
formatter.Serialize(networkStream, sendTcpObject);
to these lines of code:
if (!transfering) // making sure only 1 thread is transfering data
{
transfering = true;
formatter.Serialize(networkStream, sendTcpObject);
transfering = false;
}