I have to load huge amount of data, pre-process it, share it among few users and finally gather updates back from users.
This is what I did in my previous project -
Created an excel add-in using C++. Loaded the data in memory using the add-in code and processed it. For each type of data I have sent the processed data to a sheet and saved a new excel file. That way, if I have three types of data, I have created three new excel workbooks. My users then opened those new workbooks, made their changes and dropped a text file that contains their changes (through a button). The main excel keeps polling for those updates (text files) and loads them as soon as they are found. That's the way I get the updates back from my users.
I am not a fan of what I did in my previous project, it produces too many temporary files (of course I can delete those). In my current project I want to use C# VSTO Workbook so I can have more control over excel. I was hoping once I load the data, I will ask my users to open the same excel in Read-Only mode and they will make changes. While testing this, I realized user's excel (opened in read-only) mode does not see the loaded data. And their changes do not update the data held in memory. This probably means I have no idea what I am doing.
Do you guys have any idea how to achieve this? I will really appreciate any help/hint.
Excel supports so-called "co-authoring" mode, when many people can edit the same document at the same time. But there is might be a catch: afaik, you need a Share Point/Office Online server/OneDrive Business to support this scenario (you need a non-free office document server product).
Using VSTO, you can do just the same you have done with C++ add-in, but in C# (means, the set of capabilities is 1:1 - it basically just wraps C++ COM Excel API for .NET)
But for online version of Excel, there may be yet another alternative - javascript addins (now that's called "Office Addins", afaik). But I doubt you'd want to process your "huge amounts of data" with javascript.
So I would say, there is a good rule: Don't fix something that isn't broken :)
If the problem is the number of temporary files, these files is not the only option to transfer data between applications. You know, you can connect two applications directly (so that they can exchange data with messages/updates). Use network, Luke :)
Of course if your 3 users live on 3 deserted islands, totally disconnected from anything, exchanging with text files on USB stick may still be the only viable option...
I think the "web" solution could be: store your file in some "co-authoring"-capable service (sharepoint, google shees, onedrive, officeonline, whatever). Make some web job to update that file in that storage automatically. Just like a "fourth" user would do.
Related
I made an amateur winform, which saves data from textboxes to a shared excel workbook in a first blank row of a specified worksheet. This winform is present on 3 PC's and the workbook is reachable on dropbox. In my situation I cannot make a shared folder instead of dropbox, because the network is a huge one + password protected and I won't get to know it, so the only "shared folder" alternatives are stuff like Dropbox.
Sometimes users need to open the workbook and write&save some data manually. This case gives additional "conflicted copy" files (because dropbox is incompatible of merging information of 2 simultaneously altered files).
If i could somehow make the excel file to be available to only one user at a time - this would make the winform crash on buttonclick if the workbook is open on any other PC.
The excellent goal would be enabling changes the workbook both by the winform and manual input and not getting any conflicting changes choices by excel and conflicted copy files on dropbox.
Any ideas? Thx in advance!
In all honesty, i think you would be better off making your users use a shared google docs spreadsheet, that they can all alter at the same time. You can use many different sheets to prevent confusion or data corruption.
Or if you really need to use a winform, save the data to ( and load from ) a sql express DB, which was made for small multiuser applications. Winforms can link to MS sql express really easily.
I need to read XLSX files and extract a maximum amount of content from it. Which of the API's should I use?
OLE DB, open XML SDK, or Excel Interop?
Which is the easiest to use?
Can you retrieve all the information using one or the other? i.e, date, times, merged cells, tables, pivottables, etc.
You can try all of them and choose the one that fits you most...
Depending on data you want to read, I'd suggest you to use Open XML over Interop or Ole DB.
I don't know an open XML SDK, although I've some experience with EPPlus library which I'm using a lot and can say only good words about it - it's fast, easy to learn, with good examples. The library is basing on Open Office XML format, so I suppose it's pretty much the same as the SDK you've mentioned, and is capable of easy read and write Excel 2007 and 2010 files.
On the linked web, you'll find a library itself, documentation and some example "Hello World" projects to download.
Why that library in the first place? Because with it you will be able to read not only cells values, but also their colors, fonts, widths and heights, merging and all that detailed stuff, that you can not only read, but modify as well. What's more, you don't need the Excel installed to do that.
On the second place - just in case you need to extract tabular data from worksheet - you may play with OLE DB. I'm afraid with that you won't be able to extract any info about formats, colors etc., as well as the data must be in a tabular organized worksheet, so you can treat is as a database's table.
The last one is Interop, because:
- it's a COM library, so you need to be very careful when playing with it via .NET, as it's easy to cause some ugly and hard to find memory leaks (confirmed by myself bad experience) - if you don't dispose their objects properly, it leaves the Excel.exe process opened,
- it's much slower than previous methods,
- basically, it has almost no more added value that one of the previous methods (EPPlus or OleDB) and requires Excel to be installed on client's machine, so why to use it?
Good luck, then.
I am working on a project for visualizing large specialized datasets.
A "Project" consists of two large datasets (that are imported from XLS or CSV) and several other settings and smaller tables. We were planning to store those tables in SQLite (for ease of access).
We would like to allow the users to save a "Project" so they do not need to load the CSV and XLS files again. We would also like to prevent users who get the project file later from tampering with the data. We want managers to be able to create a "Project" file and pass it to other employees (so they can use the visualized data in presentations, but they can not modify it). The program already has the access control built in.
What suggestions do people have about creating this type of "Project" file, preferably based around a SQLite database.
Note: I am asking for advice here because it is impossible to Google for C# and Project. You get Visual Studio how-to results.
Clarification: Imagine this a if you opened Microsoft Word (or OpenOffice Write if your prefer). You can create open and save a "project". You can send a "project" to someone else who has the program. I would like the "project" file not to be usable in a different SQLite client (for data security).
I would not say there is a any kind of template or common structure for those kind of things. Your structure is already good by itself. You may have not only data information, but also meta-information inside database file, specifying (say) user name, data last changed, comments, whatever to make your project/doc file more comprehencive and full of information, keeping constant attention on memory footprint, cause it would be very nice to have a possibility to pass that file to someone by key-drive or send by e-mail.
Good luck.
Could you not create a table that has a series of BLOB data types (for documents) and a read_only boolean field. If you don't know how many docs there will be you need to enter another table called document that links to the project table!
Hello
My application from the stockbroker has this button called "Start excelfeed" which opens an excel file and then updates the incoming prices in realtime. I want these prices extracted into my application (Java), but after several tries with jxl and poi ive found out it only extracts the values which occurred last time i saved the excel files. Are there any possible ways to extract these prices in real time? I'm not sure which method theyre using to feed the excel file, except their app is written in C#.
You can have a look at the VBA code behind the "Start excelfeed" button by hitting Alt+F11 ... Then you can check how the excel macro taps into the data source und try to adapt that in Java
[EDIT]
#Zico Sorry then I got you wrong. In that case I'd try to automate it via the Java Robot Class like user489041 suggested or you could fire up wireshark and try to snoop around the network traffic
They are using the DDE technique and you should use excel automation to pull data from excel into your app.
In the old days, we used DDE links. Chances are your stockbroker addin uses precisely that.
Chances of using DDE from java are pretty slim (close to zero) because it is windows specific. However, possibly there are other methods involved (like COM components or TCP/IP connection). I'd suggest you get the values from teh stockbroker source like the Excelfeed, instead of going through Excel.
Of course, the problem with that is that you won't get help from the vendor of the Excel addin. If things are like they were 8 years ago, they are making tons on your buying the Excel addin and don't really want to help you program against that :)
Worst case, you could use the Java Robot class to create the Excel spreadsheet, save the Excel spreadsheet as a CSV file, then read the CSV file from your Java program.
You can do this with solutions like Obba. Obba allows to access a Java virtual machine running the "Obba Server" directly from Excel.
For your problem, you have to "create" your application from the spreadsheet (load the jar, create an object representing you app - e.g. launching it in a separate thread). Then you can feed the app from Excel... - In this case, Excel is to some extend the "control program" of your app. However, if you start the Obba Server process manually, the process will keep on running if you close and re-open Excel.
I need to be able to extract BLOBs from both Access 2003 and Access 2007. Access 2003 stores BLOBs as "OLE Objects", and Access 2007 gives you another option, "Attachment". The major difference is that multiple attachments can added to a single row, whereas there can be only one BLOB per "OLE Object" data type.
I have to be able to do this without using interop as I cannot force the dependency of having Office installed. That leaves me with either DAO or ADO. So I have written the code to use both technologies to pull the BLOBs out of a test database with loads of different types of file types embedded.
The problem I have is that it seems Access wraps the embedded files in some type of meta data. The net result is that the file, once extracted, is no longer the same and can not be opened by the associated application because its "corrupted". Access stores things like the original file name and so forth in this meta data. I need to be able to strip that meta data off of the files to have the file in its original state.
Is there some dark voodoo magic which can do this? There is very little by way of documentation on this subject. Any help would be appreciated.
Thanks in advance.
This occours because OLE objects are stored like "images" on Access. This leads to performance issues, also problems like yours. To deal with these limitations, Microsoft introduced Attachment fields on Access 2007/2010, wich doesn't need OLE servers to run content. Attachments, wich should be more than one file per record, are automaticaly managed in backstage by MS-Access. Maybe you should use Attachments or change your database for SQL Server, MY-SQL, Firebird.