Introduction to the Task at hand: can be skipped if impatient
The company I work for is not a software company, but focus on mechanical and thermodynamic engineering problems.
To help solve their system design challenges, they have developed a software for calculating the system impact of replacing individual components.
The software is quite old, written in FORTRAN and has evolved over a period of 30 years, which means that we cannot quickly re-write it or update it.
As you may imagine the way this software is installed has also evolved, but significantly slower than the rest of the system, meaning that packaging is done by a batch script that gathers files from different places, and puts them in a folder, which is then compiled into an iso, burned to a cd, and shipped with mail.
You young programmers (I am 30), may expect a program to load dll's, but otherwise be fairly self-contained after linking. Even if the code is made up of several classes, from different namespaces etc..
In FORTRAN 70 however.. Not so much. Which means that the software it self consists of an alarming number of calls to prebuilt modules (read: seperate programs)..
We need to be able to distribute via the internet, as any other modern company have been able to for a while. To do this we could just make the *.iso downloadable right?
Well, unfortunately no, the iso contains several files which are user specific.
As you may imagine with thousands of users, that would be thousands of isos, that are nearly identical.
Also we wan't to convert the old FORTRAN based installation software, into a real installation package, and all our other (and more modern) programs are C# programs packaged as MSI's..
But the compile time for a single msi with this old software on our server, is close to 10 seconds, so it is simply not an option for us to build the msi, when requested by the user. (if multiple users requests at the same time, the server won't be able to complete before requests timeout..)
Nor can we prebuild the user specific msi's and cache them, as we would run out of memory on the server.. (total at ~15 giga Byte per released version)
Task Description tl:dr;
Here is what I though I would do: (inspired by comments from Christopher Painter)
Create a base MSI, with dummy files instead of the the user specific files
Create cab file for each user, with the user specific files
At request time inject the userspecific cab file into a temporary copy of the base msi using the "_Stream" table.
Insert a reference into the Media table with a new 'DiskID' and a 'LastSequence' corresponding to the extra files, and the name of the injected cabfile.
Update the Filetable with the name of the user specific file in the new cab file, a new Sequence number (in the range of the new cab files sequence range), and the file size.
Question
My code fails to do the task just described. I can read from the msi just fine, but the cabinet file is never inserted.
Also:
If I open the msi with DIRECT mode, it corrupts the media table, and if I open it in TRANSACTION mode, it fails to change anything at all..
In direct mode the existing line in the Media table is replaced with:
DiskId: 1
LastSequence: -2145157118
Cabinet: "Name of action to invoke, either in the engine or the handler DLL."
What Am I doing wrong ?
Below I have provided the snippets involved with injecting the new cab file.
snippet 1
public string createCabinetFileForMSI(string workdir, List<string> filesToArchive)
{
//create temporary cabinet file at this path:
string GUID = Guid.NewGuid().ToString();
string cabFile = GUID + ".cab";
string cabFilePath = Path.Combine(workdir, cabFile);
//create a instance of Microsoft.Deployment.Compression.Cab.CabInfo
//which provides file-based operations on the cabinet file
CabInfo cab = new CabInfo(cabFilePath);
//create a list with files and add them to a cab file
//now an argument, but previously this was used as test:
//List<string> filesToArchive = new List<string>() { #"C:\file1", #"C:\file2" };
cab.PackFiles(workdir, filesToArchive, filesToArchive);
//we will ned the path for this file, when adding it to an msi..
return cabFile;
}
snippet 2
public int insertCabFileAsNewMediaInMSI(string cabFilePath, string pathToMSIFile, int numberOfFilesInCabinet = -1)
{
//open the MSI package for editing
pkg = new InstallPackage(pathToMSIFile, DatabaseOpenMode.Direct); //have also tried direct, while database was corrupted when writing.
return insertCabFileAsNewMediaInMSI(cabFilePath, numberOfFilesInCabinet);
}
snippet 3
public int insertCabFileAsNewMediaInMSI(string cabFilePath, int numberOfFilesInCabinet = -1)
{
if (pkg == null)
{
throw new Exception("Cannot insert cabinet file into non-existing MSI package. Please Supply a path to the MSI package");
}
int numberOfFilesToAdd = numberOfFilesInCabinet;
if (numberOfFilesInCabinet < 0)
{
CabInfo cab = new CabInfo(cabFilePath);
numberOfFilesToAdd = cab.GetFiles().Count;
}
//create a cab file record as a stream (embeddable into an MSI)
Record cabRec = new Record(1);
cabRec.SetStream(1, cabFilePath);
/*The Media table describes the set of disks that make up the source media for the installation.
we want to add one, after all the others
DiskId - Determines the sort order for the table. This number must be equal to or greater than 1,
for out new cab file, it must be > than the existing ones...
*/
//the baby SQL service in the MSI does not support "ORDER BY `` DESC" but does support order by..
IList<int> mediaIDs = pkg.ExecuteIntegerQuery("SELECT `DiskId` FROM `Media` ORDER BY `DiskId`");
int lastIndex = mediaIDs.Count - 1;
int DiskId = mediaIDs.ElementAt(lastIndex) + 1;
//wix name conventions of embedded cab files is "#cab" + DiskId + ".cab"
string mediaCabinet = "cab" + DiskId.ToString() + ".cab";
//The _Streams table lists embedded OLE data streams.
//This is a temporary table, created only when referenced by a SQL statement.
string query = "INSERT INTO `_Streams` (`Name`, `Data`) VALUES ('" + mediaCabinet + "', ?)";
pkg.Execute(query, cabRec);
Console.WriteLine(query);
/*LastSequence - File sequence number for the last file for this new media.
The numbers in the LastSequence column specify which of the files in the File table
are found on a particular source disk.
Each source disk contains all files with sequence numbers (as shown in the Sequence column of the File table)
less than or equal to the value in the LastSequence column, and greater than the LastSequence value of the previous disk
(or greater than 0, for the first entry in the Media table).
This number must be non-negative; the maximum limit is 32767 files.
/MSDN
*/
IList<int> sequences = pkg.ExecuteIntegerQuery("SELECT `LastSequence` FROM `Media` ORDER BY `LastSequence`");
lastIndex = sequences.Count - 1;
int LastSequence = sequences.ElementAt(lastIndex) + numberOfFilesToAdd;
query = "INSERT INTO `Media` (`DiskId`, `LastSequence`, `Cabinet`) VALUES (" + DiskId.ToString() + "," + LastSequence.ToString() + ",'#" + mediaCabinet + "')";
Console.WriteLine(query);
pkg.Execute(query);
return DiskId;
}
update: stupid me, forgot about "committing" in transaction mode - but now it does the same as in direct mode, so no real changes to the question.
I will answer this my self, since I just learned something about DIRECT mode that I didn't know before, and wan't to keep it here to allow for the eventual re-google..
Apparently we only succesfully updates the MSI, if we closed the database handle before the program eventually chrashed.
for the purpose of answering the question, this destructor should do it.
~className()
{
if (pkg != null)
{
try
{
pkg.Close();
}
catch (Exception ex)
{
//rollback not included as we edit directly?
//do nothing..
//atm. we just don't want to break anything if database was already closed, without dereferencing
}
}
}
after adding the correct close statement, the MSI grew in size
(and a media row was added to the media table :) )
I will post the entire class for solving this task, when its done and tested,
but I'll do it in the related question on SO.
the related question on SO
Related
I am using a WPF TextBoxes inside my WinForm application for spell checking. Each time I create one, I load the same file in as a CustomDictionary. All has been fine until recently. Now, they take a long time to load, up to a second. Some forms have 30 or more, meaning delays of nearly half a minute. This seems to be the case Windows 10 (not Windows 8 as I originally posted). The application is running under DotNet 4.0, I have tried 4.5 and 4.6 (not 4.61) and all versions are slow.
I have seen sfaust’s question Spell check textbox in Win10 - Slow and am7zd’s answer. Thanks to these, I looked at the GLOBAL registry key in HKEY_CURRENT_USER\Software\Microsoft\Spelling\Dictionaries. I have 580 entries (after pruning out entries without matching files) and still things are slow.
At present, every time I create a TextBox and add a custom dictionary to it, a new entry seems to be generated in _GLOBAL_
Is there a better way of doing things than loading the custom dictionary in from file every time?
Is there a way of re-using the same entry in _GLOBAL_ every time instead of creating a new one?
Is there a clean way of clearing previous entries in GLOBAL created by my application and their matching .dic files when closing the application (or on restarting it)?
I could clear _GLOBAL_ completely each time I start my application. This brings back the speed I want, but what is the downside?
Any advice gratefully received.
No answers from anyone else, so this is what I have done:
I made sure I use CustomDictionaries.Remove on all textboxes with custom dictionaries before closing the form they are on. This gets rid of new entries in _GLOBAL_ and the related files in AppData\Local\Temp.
But there will be times when things go wrong or the user just ends the task, leaving _GLOBAL_ entries and .dic files in place, so:
I decided to take things a stage further. When I start my application, I will not only clean entries in _GLOBAL_ that don't have matching files (as suggested in the previous post referenced above), but also to remove all entries referring to .dic files in AppData\Local\Temp. My theory being that anyone who has left entries there didn't mean to, otherwise they would probably have saved the .dic file in a different folder (as Microsoft Office does).
try
{
string[] allDictionaries = (string[])Registry.GetValue(#"HKEY_CURRENT_USER\Software\Microsoft\Spelling\Dictionaries", "_Global_", new string[0]);
if (allDictionaries.Count() > 0)
{
List<string> realDictionaries = new List<string>();
bool changedSomething = false;
foreach (string thisD in allDictionaries)
{
if (File.Exists(thisD))
{
if (thisD.Contains(#"\AppData\Local\Temp\"))
{
// Assuming that anyone who wants to keep a permanent .dic file will not store it in \AppData\Local\Temp
// So delete the file and don't copy the name of the dictionary into the list of good dictionaries.
File.Delete(thisD);
changedSomething = true;
}
else
{
realDictionaries.Add(thisD);
}
}
else
{
// File does not exist, so don't copy the name of the dictionary into the list of good dictionaries.
changedSomething = true;
}
}
if (changedSomething)
{
Registry.SetValue(#"HKEY_CURRENT_USER\Software\Microsoft\Spelling\Dictionaries", "_Global_", realDictionaries.ToArray());
}
}
}
catch (Exception ex)
{
MessageBox.Show(this, "Error clearing up old dictionary files.\n\nFull message:\n\n" + ex.Message, "Unable to delete file", MessageBoxButtons.OK, MessageBoxIcon.Warning);
}
I am still wondering if it is totally safe to clear entries in _GLOBAL_ that refer to files in AppData\Local\Temp. Surely people shouldn't be leaving important stuff in a temp folder... should they?
What would be really nice would be an overload to CustomDictionaries.Add that allows us to set the name and folder of the .dic file, allowing all the textboxes in the same application to share the same .dic file and making sure we don't leave a load of redundant entries and files with seemingly random names hanging around in the first place..... please Microsoft.
I am working on a program that will read in a text file and then insert areas of the text file into different columns on a database. The text file is generally set up like this:
"Intro information"
"more Intro information"
srvrmgr> "information about system"
srbrmgr> list parameters for component *ADMBatchProc*
"Headers"
*Name of record* *alias of record* *value of record*
The columns create a table containing all of the setting information for this component. One all of the settings are listed, the file moves to another component and returns all the information for that component in a new table. I need to read in the component and the information on the tables without the headers or the other information. I will then need to be able to transfer that data into a database. The columns are fixed width on each table within the file.
Any recommendations about how to approach this are welcome. I have never read in a file this complex so I dont really know how to approach ignoring alot of information while trying to get other information ready for a database. Also the component value I am trying to gather always follows the word component on a line that starts with "srvrmgr".
The '*' represents areas that will be put into datbase.
Siebel Enterprise Applications Siebel Server Manager, Version 8.1.1.11 [23030] LANG_INDEPENDENT
Copyright (c) 1994-2012, Oracle. All rights reserved.
The Programs (which include both the software and documentation) contain
proprietary information; they are provided under a license agreement containing
restrictions on use and disclosure and are also protected by copyright, patent,
and other intellectual and industrial property laws. Reverse engineering,
disassembly, or decompilation of the Programs, except to the extent required to
obtain interoperability with other independently created software or as specified
by law, is prohibited.
Oracle, JD Edwards, PeopleSoft, and Siebel are registered trademarks of
Oracle Corporation and/or its affiliates. Other names may be trademarks
of their respective owners.
If you have received this software in error, please notify Oracle Corporation
immediately at 1.800.ORACLE1.
Type "help" for list of commands, "help <topic>" for detailed help
Connected to 1 server(s) out of a total of 1 server(s) in the enterprise
srvrmgr> configure list parameters show PA_NAME,PA_ALIAS,PA_VALUE
srvrmgr>
srvrmgr> list parameters for component ADMBatchProc
PA_NAME PA_ALIAS PA_VALUE
---------------------------------------------------------------------- ------------------------------------- --------------------------------------------------------------------------------------------------------------------
ADM Data Type Name ADMDataType
ADM EAI Method Name ADMEAIMethod Upsert
ADM Deployment Filter ADMFilter
213 rows returned.
srvrmgr> list parameters for component ADMObjMgr_enu
PA_NAME PA_ALIAS PA_VALUE
---------------------------------------------------------------------- ------------------------------------- --------------------------------------------------------------------------------------------------------------------
AccessibleEnhanced AccessibleEnhanced False
This is the beginning of the text file. It a produced in a system called Siebel to show all of the settings for this environment. I need to pull the component name (there are multiple on the actual file but the ones shown here are 'ADMBatchProc' and 'ADMObjMgr_enu'), and then the data shown on the table below it that was created by Siebel. The rest of the information is irrelevant for the purpose of the task I need.
I would recommend using Test-Driven Development techniques in this case. I'm guessing that your possible variations of input format are near infinite.
Try this:
1) Create an interface that will represent the data operations or parsing logic you expect the application to perform. For example:
public interface IParserBehaviors {
void StartNextComponent();
void SetTableName(string tableName);
void DefineColumns(IEnumerable<string> columnNames);
void LoadNewDataRow(IEnumerable<object> rowValues);
DataTable ProduceTableForCurrentComponent();
// etc.
}
2) Gather as many small examples of discrete inputs that have well-defined behaviors as possible.
3) Inject a behaviors handler into your parser. For example:
public class Parser {
private const string COMPONENT_MARKER = "srvrmgr";
private readonly IParserBehaviors _behaviors;
public Parser(IParserBehaviors behaviors) {
_behaviors = behaviors;
}
public void ReadFile(string filename) {
// bla bla
foreach (string line in linesOfFile) {
// maintain some state
if (line.StartsWith(COMPONENT_MARKER)) {
DataTable table = _behaviors.ProduceTableForCurrentComponent();
// save table to the database
_behaviors.StartNextComponent();
}
else if (/* condition */) {
// parse some text
_behaviors.LoadNewDataRow(values);
}
}
}
}
4) Create tests around the expected behaviors, using your preferred mocking framework. For example:
public void FileWithTwoComponents_StartsTwoNewComponents() {
string filename = "twocomponents.log";
Mock<IParserBehaviors> mockBehaviors = new Mock<IParserBehaviors>();
Parser parser = new Parser(mockBehaviors.Object);
parser.ReadFile(filename);
mockBehaviors.Verify(mock => mock.StartNextComponent(), Times.Exactly(2));
}
This way, you will be able to integrate under controlled tests. When (not if) someone runs into a problem, you can distill what case wasn't covered, and add a test surrounding that behavior, after extracting the case from the log being used. Separating concerns this way also allows your parsing logic to be independent from your data operation logic. The needs of parsing specific behaviors seems to be central to your application, so it seems like a perfect fit for creating some domain-specific interfaces.
You'll want to read the text file using StreamReader:
using (FileStream fileStream = File.OpenRead(path))
{
byte[] data = new byte[fileStream.Length];
for (int index = 0; index < fileStream.Length; index++)
{
data[index] = (byte)fileStream.ReadByte();
}
Console.WriteLine(Encoding.UTF8.GetString(data)); // Displays: your file - now you can decide how to manipulate it.
}
Perhaps then you'll use Regex to capture the date you'd like to insert:
You might insert into the db like this:
using (TransactionScope transactionScope = new TransactionScope())
{
using (SqlConnection connection = new SqlConnection(connectionString))
{
connection.Open();
SqlCommand command1 = new SqlCommand(
“INSERT INTO People ([FirstName], [LastName], [MiddleInitial])
VALUES(‘John’, ‘Doe’, null)”,
connection);
SqlCommand command2 = new SqlCommand(
“INSERT INTO People ([FirstName], [LastName], [MiddleInitial])
VALUES(‘Jane’, ‘Doe’, null)”,
connection);
command1.ExecuteNonQuery();
command2.ExecuteNonQuery();
}
transactionScope.Complete();
}
Examples adapted from Wouter de Kort's C# 70-483.
I have a particular situation where my client require to import (periodically) an ms-access database into his mysql website database (so it's a remote database).
Because the hosting plan is a shared hosting (not a vps), the only way to do it is through PHP through an SQL query, because I don't have ODBC support on hosting.
My current idea is this one (obviusly the client has a MS-Windows O.S.):
Create a small C# application that convert MS-Access database into a big SQL query written on a file
The application will then use FTP info to send the file into a specified directory on the website
A PHP script will then run periodically (like every 30 minutes) and check if file exists, eventually importing it into the database
I know it's not the best approach so I'm proposing a question to create a different workaround for this problem. The client already said that he wants keep using his ms-access database.
The biggest problem I have is that scripts can last only 30 seconds, which is obviusly a problem to import data.
To work around the 30-second limit, call your script repeatedly, and keep track of your progress. Here's one rough idea:
if(!file_exists('upload.sql')) exit();
$max = 2000; // the maximum number you want to execute.
if(file_exists('progress.txt')) {
$progress = file_get_contents('progress.txt');
} else {
$progress = 0;
}
// load the file into an array, expecting one query per line
$file = file('upload.sql');
foreach($file as $current => $query) {
if($current < $progress) continue; // skip the ones we've done
if($current - $progress >= $max) break; // stop before we hit the max
mysql_query($query);
}
// did we finish the file?
if($current == count($file) - 1) {
unlink('progress.txt');
unlink('upload.sql');
} else {
file_put_contents('progress.txt', $current);
}
I'm building a web application in which I need to scan the user-uploaded files for viruses.
Does anyone with experience in building something like this can provide information on how to get this up and running? I'm guessing antivirus software packages have APIs to access their functionality programatically, but it seems it's not easy to get a hand on the details.
FYI, the application is written in C#.
Important note before use:
Be aware of TOS agreement. You give them full access to everything: "When you upload or otherwise submit content, you give VirusTotal (and those we work with) a worldwide, royalty free, irrevocable and transferable licence to use, edit, host, store, reproduce, modify, create derivative works, communicate, publish, publicly perform, publicly display and distribute such content."
Instead of using a local Antivirus program (and thus binding your program to that particular Antivirus product and requesting your customers to install that Antivirus product) you could use the services of VirusTotal.com
This site provides a free service in which your file is given as input to numerous antivirus products and you receive back a detailed report with the evidences resulting from the scanning process. In this way your solution is no more binded to a particular Antivirus product (albeit you are binded to Internet availability)
The site provides also an Application Programming Interface that allows a programmatically approach to its scanning engine.
Here a VirusTotal.NET a library for this API
Here the comprensive documentation about their API
Here the documentation with examples in Python of their interface
And because no answer is complete without code, this is taken directly from the sample client shipped with the VirusTotal.NET library
static void Main(string[] args)
{
VirusTotal virusTotal = new VirusTotal(ConfigurationManager.AppSettings["ApiKey"]);
//Use HTTPS instead of HTTP
virusTotal.UseTLS = true;
//Create the EICAR test virus. See http://www.eicar.org/86-0-Intended-use.html
FileInfo fileInfo = new FileInfo("EICAR.txt");
File.WriteAllText(fileInfo.FullName, #"X5O!P%#AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*");
//Check if the file has been scanned before.
FileReport fileReport = virusTotal.GetFileReport(fileInfo);
bool hasFileBeenScannedBefore = fileReport.ResponseCode == ReportResponseCode.Present;
Console.WriteLine("File has been scanned before: " + (hasFileBeenScannedBefore ? "Yes" : "No"));
//If the file has been scanned before, the results are embedded inside the report.
if (hasFileBeenScannedBefore)
{
PrintScan(fileReport);
}
else
{
ScanResult fileResult = virusTotal.ScanFile(fileInfo);
PrintScan(fileResult);
}
... continue with testing a web site ....
}
DISCLAIMER
I am in no way involved with them. I am writing this answer just because it seems to be a good update for these 4 years old answers.
You can use IAttachmentExecute API.
Windows OS provide the common API to calling the anti virus software which is installed (Of course, the anti virus software required support the API).
But, the API to calling the anti virus software provide only COM Interface style, not supported IDispatch.
So, calling this API is too difficult from any .NET language and script language.
Download this library from here Anti Virus Scanner for .NET or add reference your VS project from "NuGet" AntiVirusScanner
For example bellow code scan a file :
var scanner = new AntiVirus.Scanner();
var result = scanner.ScanAndClean(#"c:\some\file\path.txt");
Console.WriteLine(result); // console output is "VirusNotFound".
I would probably just make a system call to run an independent process to do the scan. There are a number of command-line AV engines out there from various vendors.
Take a look at the Microsoft Antivirus API. It makes use of COM, which should be easy enough to interface with from .NET. It refers specifically to Internet Explorer and Microsoft Office, but I don't see why you wouldn't be able to use to to on-demand scan any file.
All modern scanners that run on Windows should understand this API.
Various Virus scanners do have API's. One I have integrated with is Sophos. I am pretty sure Norton has an API also while McAfee doesn't (it used to). What virus software do you want to use? You may want to check out Metascan as it will allow integration with many different scanners, but there is an annual license cost. :-P
I also had this requirement. I used clamAv anti virus which provides on-demand scanning by sending the file to their tcp listening port. You can use nClam nuget package to send files to clamav.
var clam = new ClamClient("localhost", 3310);
var scanResult = clam.ScanFileOnServerAsync("C:\\test.txt"); //any file you would like!
switch (scanResult.Result.Result)
{
case ClamScanResults.Clean:
Console.WriteLine("The file is clean!");
break;
case ClamScanResults.VirusDetected:
Console.WriteLine("Virus Found!");
Console.WriteLine("Virus name: {0}", scanResult.Result.InfectedFiles[0].FileName);
break;
case ClamScanResults.Error:
Console.WriteLine("Woah an error occured! Error: {0}", scanResult.Result.RawResult);
break;
}
A simple and detailed example is shown here. Note:- The synchronous scan method is not available in the latest nuget. You have to code like I done above
For testing a virus you can use the below string in a txt file
X5O!P%#AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*
Shameless plug but you might want to check out https://scanii.com, it's basically malware/virus detection as a (REST) service. Oh also, make sure you read and understand virustotal's API terms (https://www.virustotal.com/en/documentation/public-api/) - they are very clear about not allowing commercial usage.
I would recommend using this approach:
using System;
using System.Diagnostics;
using Cloudmersive.APIClient.NET.VirusScan.Api;
using Cloudmersive.APIClient.NET.VirusScan.Client;
using Cloudmersive.APIClient.NET.VirusScan.Model;
namespace Example
{
public class ScanFileAdvancedExample
{
public void main()
{
// Configure API key authorization: Apikey
Configuration.Default.AddApiKey("Apikey", "YOUR_API_KEY");
var apiInstance = new ScanApi();
var inputFile = new System.IO.FileStream("C:\\temp\\inputfile", System.IO.FileMode.Open); // System.IO.Stream | Input file to perform the operation on.
var allowExecutables = true; // bool? | Set to false to block executable files (program code) from being allowed in the input file. Default is false (recommended). (optional)
var allowInvalidFiles = true; // bool? | Set to false to block invalid files, such as a PDF file that is not really a valid PDF file, or a Word Document that is not a valid Word Document. Default is false (recommended). (optional)
var allowScripts = true; // bool? | Set to false to block script files, such as a PHP files, Pythong scripts, and other malicious content or security threats that can be embedded in the file. Set to true to allow these file types. Default is false (recommended). (optional)
var allowPasswordProtectedFiles = true; // bool? | Set to false to block password protected and encrypted files, such as encrypted zip and rar files, and other files that seek to circumvent scanning through passwords. Set to true to allow these file types. Default is false (recommended). (optional)
var restrictFileTypes = restrictFileTypes_example; // string | Specify a restricted set of file formats to allow as clean as a comma-separated list of file formats, such as .pdf,.docx,.png would allow only PDF, PNG and Word document files. All files must pass content verification against this list of file formats, if they do not, then the result will be returned as CleanResult=false. Set restrictFileTypes parameter to null or empty string to disable; default is disabled. (optional)
try
{
// Advanced Scan a file for viruses
VirusScanAdvancedResult result = apiInstance.ScanFileAdvanced(inputFile, allowExecutables, allowInvalidFiles, allowScripts, allowPasswordProtectedFiles, restrictFileTypes);
Debug.WriteLine(result);
}
catch (Exception e)
{
Debug.Print("Exception when calling ScanApi.ScanFileAdvanced: " + e.Message );
}
}
}
}
Note that this way you can even control whether you filter out non-virus threat payloads such as executables, scripts, encrypted/password-protected files, etc.
This approach has a free tier and can also validate the contents of the files that you upload.
We tried two options:
clamav-daemon installed on a tiny linux container + "nClam" .NET library to interact with it. Works fine, but Clam AV misses a lot (a lot!) of viruses, especially dangerous macros hidden in MS Office files. Also ClamAV virus database has to be kept in memory at all times, which uses around 3.5GB of memory, which requires a rather expensive cloud virtual machine.
Ended up using Windows Defender via MpCmdRun.exe CLI api. See answer here
You can try to use DevDragon.io.
It is a web service with an API and .NET client DevDragon.Antivirus.Client you can get from NuGet. Scans are sub 200ms for 1MB file.
More documentation here:
https://github.com/Dev-Dragon/Antivirus-Client
Disclosure: I work for them.
From my experience you can use COM for interfacing with some anti-virus software. But what I would suggest is a bit easier, just parse scan results after scanning. All you need to do is to start the scanner process and point it to file/folder you want to scan, store scan results into file or redirect stdout to your application and parse results.
//Scan
string start = Console.ReadLine();
System.Diagnostics.Process scanprocess = new System.Diagnostics.Process();
sp.StartInfo.WorkingDirectory = #"<location of your antivirus>";
sp.StartInfo.UseShellExecute = false;
sp.StartInfo.FileName = "cmd.exe";
sp.StartInfo.Arguments = #"/c antivirusscanx.exe /scan="+filePath;
sp.StartInfo.CreateNoWindow = true;
sp.StartInfo.RedirectStandardInput = true;
sp.StartInfo.RedirectStandardError = true; sp.Start();
string output = sp.StandardOutput.ReadToEnd();
//Scan results
System.Diagnostics.Process pr = new System.Diagnostics.Process();
pr.StartInfo.FileName = "cmd.exe";
pr.StartInfo.Arguments = #"/c echo %ERRORLEVEL%";
pr.StartInfo.RedirectStandardInput = true;
pr.StartInfo.RedirectStandardError = true; pr.Start();
output = processresult.StandardOutput.ReadToEnd();
pr.Close();
I am working on an application which reads eventlogs(Application) from remote machines. I am making use of EventLog class in .net and then iterating on the Log entries but this is very slow. In some cases, some machines have 40000+ log entries and it takes hours to iterate through the entries.
what is the best way to accomplish this task? Are there any other classes in .net which are faster or in any other technology?
Man, I feel your pain. We had the exact same issue in our app.
Your solution has a branch depending on what server version you're running on and what server version your "target" machine is running on.
If you're both on Vista or Windows Server 2008, you're in luck. You should look at System.Diagnostics.Eventing.Reader.EventLogQuery and System.Diagnostics.Eventing.Reader.EventLogReader. These are new in .net 3.5.
Basically, you can build a query in XML and ship it over to run on the remote computer. Maybe you're just searching for events of a specific type, or maybe just new events from a specific point in time. The search runs on the remote machine, and then you just get back the matching events. The new classes are much faster than the old .net 2.0 way, but again, they are only supported on Vista or Windows Server 2008.
For our app when the target is NOT on Vista/Win2008, we downloaded the raw .evt file from the remote system, and then parsed the file using its binary format. There are several sources of data about the event log format for .evt files (pre-Vista), including link text and an article I recall on codeproject.com that had some c# code.
Vista and Windows Server 2008 machines use a new .evtx format that is a new format, so you can't use the same binary parsing approach across all versions. But the new EventLogQuery and EventLogReader classes are so fast that you won't have to. It's now perfectly speedy to just use the built-in classes.
Event Log Reader is horribly slow... too slow. WTF Microsoft?
Use LogParser 2.2 - Search for C# and LogParser on the Internet (or you can use the log parser commands from the command line). I don't want to duplicate the work already contributed by others.
I pull the log from the remote system by having the log exported as an EVTX file. I then copy the file from the remote system. This process is really quick - even with a network that spans the planet (I had issues with having the log exported to a network resource). Once you have it local, you can do your searches and processing.
There are multiple reasons for having the EVTX - I won't get into the reasons why we do this.
The following is a working example of the code to save a copy of the log as an EVTX:
(Notes: "device" is the network host name or IP. "LogName" is the name of the log desired: "System", "Security", or "Application". outputPathOnRemoteSystem is the path on the remote computer, such as "c:\temp\%hostname%.%LogName%.%YYYYMMDD_HH.MM%.evtx".)
static public bool DumpLog(string device, string LogName, string outputPathOnRemoteSystem, out string errMessage)
{
bool wasExported = false;
string errorMessage = "";
try
{
System.Diagnostics.Eventing.Reader.EventLogSession els = new System.Diagnostics.Eventing.Reader.EventLogSession(device);
els.ExportLogAndMessages(LogName, PathType.LogName, "*", outputPathOnRemoteSystem);
wasExported = true;
}
catch (UnauthorizedAccessException e)
{
errorMessage = "Unauthorized - Access Denied: " + e.Message;
}
catch (EventLogNotFoundException e)
{
errorMessage = "Event Log Not Found: " + e.Message;
}
catch (EventLogException e)
{
errorMessage = "Export Failed: " + e.Message + ", Log: " + LogName + ", Device: " + device;
}
errMessage = errorMessage;
return wasExported;
}
A good Explanation/Example can be found on MSDN.
EventLogSession session = new EventLogSession(Environment.MachineName);
// [System/Level=2] filters out the errors
// Where "Log" is the log you want to get data from.
EventLogQuery query = new EventLogQuery("Log", PathType.LogName, "*[System/Level=2]");
EventLogReader reader = new EventLogReader(query);
for (EventRecord eventInstance = reader.ReadEvent();
null != eventInstance;
eventInstance = reader.ReadEvent())
{
// Output or save your event data here.
}
When waiting 5-20 minutes with the old code this one does it in less than 10 seconds.
Maybe WMI can help you:
WMI with C#
Have you tried using the remoting features in powershell 2.0? They allow you to execute cmdlets (like ones to read event logs) on remote machines and return the results (as objects, of course) to the calling session.
You could place a Program at those machines that save the log to file and sends it to your webapplication i think that would be alot faster as you can do the looping local but im not sure how to do it so i cant ive you any code :(
I recently did such thing via WCF callback interface however my clients interacted with the server through WCF and adding a WCF Callback was easy in my project, full code with examples is available here
Just had the same issue and want to share my solution. It makes a search through application, system and security eventlogs from 260 seconds (using EventLog) about a 100 times faster (using EventLogQuery).
And this in a way where it is possible to check if the event message contains a pattern or any other check without the requirement of FormatDescription().
My trick is to use the same mechanism as PowerShells Get-WinEvent does and then pass it through the result check.
Here is my code to find all events within last 4 days where the event message contains a filter pattern.
string[] eventLogSources = {"Application", "System", "Security"};
var messagePattern = "*Your Message Search Pattern*";
var timeStamp = DateTime.Now.AddDays(-4);
var matchingEvents = new List<EventRecord>();
foreach (var eventLogSource in eventLogSources)
{
var i = 0;
var query = string.Format("*[System[TimeCreated[#SystemTime >= '{0}']]]",
timeStamp.ToUniversalTime().ToString("o"));
var elq = new EventLogQuery(eventLogSource, PathType.LogName, query);
var elr = new EventLogReader(elq);
EventRecord entryEventRecord;
while ((entryEventRecord = elr.ReadEvent()) != null)
{
if ((entryEventRecord.Properties)
.FirstOrDefault(x => (x.Value.ToString()).Contains(messagePattern)) != null)
{
matchingEvents.Add(entryEventRecord);
i++;
}
}
}
Maybe that the remote computers could do a little bit of computing. So this way your server would only deal with relevant information. It would be a kind of cluster using the remote computer to do some light filtering and the server would the the analysis part.