I'm trying to write an application in MVC 5 that will accept a file specified by a user and upload that file information into the database. The file itself has multiple worksheets, which I think FileHelpers handles gracefully, but I can't find any good documentation about working with a byte array. I can get the file just fine, and get to my controller, but don't know where to go from there. I am currently doing this in the controller:
public ActionResult UploadFile(string filepath)
{
//we want to check here that the first file in the request is not null
if (Request.Files[0] != null)
{
var file = Request.Files[0];
byte[] data = new byte[file.ContentLength];
ParseInputFile(data);
//file.InputStream.Read(data, 0, data.Length);
}
ViewBag.Message = "Success!";
return View("Index");
}
private void ParseInputFile(byte[] data)
{
ExcelStorage provider = new ExcelStorage(typeof(OccupationalGroup));
provider.StartRow = 3;
provider.StartColumn = 2;
provider.FileName = "test.xlsx";
}
Am I able to use the Request like that in conjunction with FileHelpers? I just need to read the Excel file into the database. If not, should I be looking into a different way to handle the upload?
So, I decided instead to use ExcelDataReader to do my reading from Excel. It puts the stream (in the below code, test) into a DataSet that I can just manipulate manually. I'm sure it might not be the cleanest way to do it, but it made sense for me, and allows me to work with multiple worksheets fairly easily as well. Here is the snippet of regular code that I ended up using:
//test is a stream here that I get using reflection
IExcelDataReader excelReader = ExcelReaderFactory.CreateOpenXmlReader(test);
DataSet result = excelReader.AsDataSet();
while(excelReader.Read())
{
//process the file
}
excelReader.Close();
Related
I'm trying out a project with ASP.Net MVC and have a large CSV file that I want to save to the LocalDB.
I have been following this tutorial (and the ones before that are about MVC): https://learn.microsoft.com/en-us/aspnet/mvc/overview/getting-started/introduction/creating-a-connection-string
Now I want to add data to this database that I have set up and I would like to read this data from a csv file and then save it to my database.
I have tried this: https://www.aspsnippets.com/Articles/Upload-Read-and-Display-CSV-file-Text-File-data-in-ASPNet-MVC.aspx
but when I try to upload my file I get an error that my file is too large?
I would love it if it could be automated so that when I start my application the database will be populated with the data from my csv file (and if it already is populated it will not do it again) or just some way of coding so that I can add the data from my csv file to the database (LocalDB).
protected override void Seed(ProductsDBContext context)
{
Assembly assembly = Assembly.GetExecutingAssembly();
string resourceName = "WebbApplication.App_Data.SeedData.price_detail.csv";
using (Stream stream = assembly.GetManifestResourceStream(resourceName))
{
using (StreamReader reader = new StreamReader(stream, Encoding.UTF8))
{
CsvReader csvReader = new CsvReader(reader);
var products = csvReader.GetRecords<PriceDetail>().ToArray();
context.PriceDetails.AddOrUpdate(c => c.PriceValueId, products);
}
}
}
Your second link includes the following line:
string csvData = System.IO.File.ReadAllText(filePath);
If you are getting an Out of Memory Exception, then you should not load the entire file into memory at once - i.e. do not read all of the text.
The StreamReader has a built-in function to handle this.
System.IO.StreamReader file = new System.IO.StreamReader("WebbApplication.App_Data.SeedData.price_detail.csv");
while((line = file.ReadLine()) != null)
{
System.Console.WriteLine(line);
//Replace with your operation below
}
Potentially the same problem solved at this question.
With Cinchoo ETL - an open source library, you can bulk load CSV file into sqlserver with few lines of code.
using (var p = new ChoCSVReader(** YOUR CSV FILE **)
.WithFirstLineHeader()
)
{
p.Bcp("** ConnectionString **", "** tablename **");
}
For more information, please visit codeproject article.
Hope it helps.
currently I am trying to create two controller methods reading from file and writing to same file:
public ActionResult UploadText(string name)
{
var path = Path.Combine(Server.MapPath("~/text/"), name);
var fileContents = System.IO.File.ReadAllText(path);
ViewData["text"] = fileContents;
ViewData["textName"] = name;
return View();
}
[HttpPost]
public ActionResult TextPost(string textName)
{
string text = Request["text-content"];
var path = Path.Combine(Server.MapPath("~/text/"), textName);
System.IO.File.WriteAllText(path, text);
return RedirectToAction("Index");
}
Reading file content and writing to it works, but it cannot be read second time, File can't be accessed because it is being used by another process error appears.
What am I doing wrong?
System.IO.File.ReadAllText and System.IO.File.WriteAllText both close the file after they and finished with the file per the documentation. The issue stems from the async nature of the web and well if you have more than one request while the file is open you will get the error you are seeing. Here are some MSDN examples to get you started.
Here are a couple more links for your pleasure
simultaneous read-write a file in C#
How to both read and write a file in C#
---short version:
When I get to the while (!checkReader.EndOfStream) every time after the first, it says EndOfStream = true.
---more detail:
A user will upload a file using an Ajax AsyncFileUpload control. I take that file, ensure it's a very specific format of csv that we use and spit it out into a GridView. This all works great the first time through: I get the file, parse it out, and it displays great.
But, if I call this same code again anytime during the user's session the StreamReader.EndOfStream = true.
For example, a user uploads a file and I spit it out into the GridView. Oops! User realizes there are headers... I have a checkbox available with an event handler that will call the method below to re-read the original file (it's stored in a session variable). User checks the box, event fires, method gets called, but my EndOfStream is now true.
I thought that using () would change that flag and I have tried adding checkReader.DiscardBufferedData just after the while loop below, but neither of those seem to have any affect.
What am I doing wrong?
private void BuildDataFileGridView(bool hasHeaders)
{
//read import file from the session variable
Stream theStream = SessionImportFileUpload.PostedFile.InputStream;
theStream.Position = 0;
StringBuilder sb = new StringBuilder();
using (StreamReader checkReader = new StreamReader(theStream))
{
while (!checkReader.EndOfStream)
{
string line = checkReader.ReadLine();
while (line.EndsWith(","))
{
line = line.Substring(0, line.Length - 1);
}
sb.AppendLine(line);
}
}
using (TextReader reader = new StringReader(sb.ToString()))
{
//read the file in and shove it out for the client
using (CsvReader csv = new CsvReader(reader, hasHeaders, CsvReader.DefaultDelimiter))
{
sDataInputTable = new DataTable();
try
{
//Load the DataTable with csv values
sDataInputTable.Load(csv);
}
catch
{
DisplayPopupMessage("ERROR: A problem was encountered");
}
//Copy only the first 10 rows into a temp table for display.
DataTable displayDataTable = sDataInputTable.Rows.Cast<System.Data.DataRow>().Take(10).CopyToDataTable();
MyDataGridView.DataSource = displayDataTable;
MyDataGridView.DataBind();
}
}
}
Edit:
SessionImportFileUpload is the actual Ajax AsyncFileUpload control being stored as a session variable (this was already the case as a previous person wrote other stuff in that uses it).
You are storing the posted file stream in Session. This is not correct, because the stream is not the data, but rather the mechanism to read the data. The file is uploaded only once, during a single POST request, and you won't be able to read from the same stream again later. Usually you even cannot rewind the stream to re-read it.
That's why I suggest to read the posted file stream only once and put the whole content into Session - this way the content will be reusable, and you'll be able to reprocess it as many times as you need.
I wanna get the name and count of the columns of the user's excel files, but i can't send the whole the file to the server, but just some of its first bytes (enough to read the first rows ). how can i do that in C#? is there any way not to send whole the excel file, and reading first rows? as i know, we need the file path to read an excel file, but i wanna just use a small part of file like a stream :
in code below i use "ExcelDataReader" to read the files:
public List<string> Getheader()
{
List<string> headers = new List<string>();
byte[] hdrToRead = System.Text.Encoding.UTF8.GetBytes(remainedXlsStr);
MemoryStream stream = new MemoryStream(hdrToRead);
IExcelDataReader excelReader = null;
switch (currentFileType)
{
case (XlsType.xlsx):
{
excelReader = ExcelReaderFactory.CreateOpenXmlReader(stream);
break;
}
case (XlsType.xls):
{
excelReader = ExcelReaderFactory.CreateBinaryReader(stream);
break;
}
case (XlsType.notSupported):
{
break;
}
}
if (excelContent.HasHeader)
excelReader.IsFirstRowAsColumnNames = true;
if (excelReader.Read())
{ // first row
for (int i = 0; i < excelReader.FieldCount; i++)
{
// fieldType = excelReader.GetValue(i).GetType();
headers.Add(excelReader.GetValue(i).ToString());
}
}
return headers;
}
...
thanks
I don't know of any way to read a partial compressed file (xlsx), and both COM and ODBC as ways to access xls files require having the entire file. If you really want to give yourself ulcers, you could probably track down the xls file specification and figure out how to parse the initial bytes of the file, but what you are trying to do has a very limited use case so I doubt you'll find anything that'll do it for you or even give you much of a leg forward on the way...
I can successfully inject a piece of VBA code into a generated excel workbook, but what I am trying to do is use the Workbook_Open() event so the VBA code executes when the file opens. I am adding the sub to the "ThisWorkbook" object in my xlsm template file. I then use the openxml productivity tool to reflect the code and get the encoded VBA data.
When the file is generated and I view the VBA, I see "ThisWorkbook" and "ThisWorkbook1" objects. My VBA is in "ThisWorkbook" object but the code never executes on open. If I move my VBA code to "ThisWorkbook1" and re-open the file, it works fine. Why is an extra "ThisWorkbook" created? Is it not possible to inject an excel spreadsheet with a Workbook_Open() sub? Here is a snippet of the C# code I am using:
private string partData = "..."; //base 64 encoded data from reflection code
//open workbook, myWorkbook
VbaProjectPart newPart = myWorkbook.WorkbookPart.AddNewPart<VbaProjectPart>("rId1");
System.IO.Stream data = GetBinaryDataStream(partData);
newPart.FeedData(data);
data.Close();
//save and close workbook
Anyone have ideas?
Based on my research there isn't a way to insert the project part data in a format that you can manipulate in C#. In the OpenXML format, the VBA project is still stored in a binary format. However, copying the VbaProjectPart from one Excel document into another should work. As a result, you'd have to determine what you wanted the project part to say in advance.
If you are OK with this, then you can add the following code to a template Excel file in the 'ThisWorkbook' Microsoft Excel Object, along with the appropriate Macro code:
Private Sub Workbook_Open()
Run "Module1.SomeMacroName()"
End Sub
To copy the VbaProjectPart object from one file to the other, you would use code like this:
public static void InsertVbaPart()
{
using(SpreadsheetDocument ssDoc = SpreadsheetDocument.Open("file1.xlsm", false))
{
WorkbookPart wbPart = ssDoc.WorkbookPart;
MemoryStream ms;
CopyStream(ssDoc.WorkbookPart.VbaProjectPart.GetStream(), ms);
using(SpreadsheetDocument ssDoc2 = SpreadsheetDocument.Open("file2.xlsm", true))
{
Stream stream = ssDoc2.WorkbookPart.VbaProjectPart.GetStream();
ms.WriteTo(stream);
}
}
}
public static void CopyStream(Stream input, Stream output)
{
byte[] buffer = new byte[short.MaxValue + 1];
while (true)
{
int read = input.Read(buffer, 0, buffer.Length);
if (read <= 0)
return;
output.Write(buffer, 0, read);
}
}
Hope that helps.
I found that the other answers still resulted in the duplicate "Worksheet" object. I used a similar solution to what #ZlotaMoneta said, but with a different syntax found here:
List<VbaProjectPart> newParts = new List<VbaProjectPart>();
using (var originalDocument = SpreadsheetDocument.Open("file1.xlsm"), false))
{
newParts = originalDocument.WorkbookPart.GetPartsOfType<VbaProjectPart>().ToList();
using (var document = SpreadsheetDocument.Open("file2.xlsm", true))
{
document.WorkbookPart.DeleteParts(document.WorkbookPart.GetPartsOfType<VbaProjectPart>());
foreach (var part in newParts)
{
VbaProjectPart vbaProjectPart = document.WorkbookPart.AddNewPart<VbaProjectPart>();
using (Stream data = part.GetStream())
{
vbaProjectPart.FeedData(data);
}
}
//Note this prevents the duplicate worksheet issue
spreadsheetDocument.WorkbookPart.Workbook.WorkbookProperties.CodeName = "ThisWorkbook";
}
}
You need to specify "codeName" attribute in the "xl/workbook..xml" object
After feeding the VbaProjectPart with macro. Add this code:
var workbookPr = spreadsheetDocument.WorkbookPart.Workbook.Descendants<WorkbookProperties>().FirstOrDefault();
workbookPr.CodeName = "ThisWorkBook";
After opening the file everything should work now.
So, to add macro you need to:
Change document type to macro enabled
Add VbaProjectPart and feed it with earlier created macro
Add workbookPr codeName attr in xl/workbook..xml with value "ThisWorkBook"
Save as with .xlsm ext.