Right now I have a Microsoft Exchange mail server and I want to extract information from that exchange server directly to my own SQL database so I can analyze them.
Now the problem is right now the way I come up with extract information from this exchange database is to check every user then check every folder they have then check all the content under each folder and if I see any modified or new emails I copy,update that into my own database.
this method does work for small amount of data however if I try to copy large amount of data with this :eg 1000 users the process will take minutes if not longer because I am constantly doing loops over loops to check every single email one by one for difference. I want to have a way to only loop through a folder if it is modified.
I looked at the documentation of Microsoft on Exchange folders I see the function called IsDirty. The description says it returns true or false based on whether the object has been modified. Now I am confused on what is it comparing the object to. Does it compare the object with whether it was modified upon creation? or a specific date I tried to look for information on this and I can't find any. Also is there any other faster way of detecting new emails/modified emails in the exchange without looping through all the folders and all the emails thanks.
Related
I am trying to retrieve the deleted UUIDs from a Openldap server using a .Net Core Console Application.
I was able to see that a Sync Info Message was indeed sent by my Openldap server and that it contained the UUIDS of the present entries by using a Perl script and dumping the whole response.
I set up a Openldap server with the syncprov Overlay (see my previous question Can't get deleted items from OpenLDAP Server using Content Synchronization Operation (syncrepl)).
After re-reading the RFC4533 multiple times and the OpenLdap Syncrepl documentation and analysing the response, with my current configuration (No Accesslog) it is impossible to retrieve deleted entries, only a list of present entries. They are contained in the Sync Info Message. I wish to retrieve the information anyway so I can make a delta between what is sent and what is on my client.
Do you know how to catch the message in C#?
I tried using the DirectoryServices.Protocols and the Novell.Directory.Ldap libraries (separately). I must have missed something but don't know what exactly...
I used the Novell Code sample (the SearchPersist one and adding the corresponding control) available at https://www.microfocus.com/documentation/edirectory-developer-documentation/ldap-libraries-for-c-sharp/.
I can retrieve added/modified entries but not the Sync Info Message containing the present entries.
By digging a bit into the Novell Library, I found some useful classes for decoding ASN1 Objects.
By using the following code I am able to determine the type of the Intermediate Sync Info Message.
var decoder = new LBERDecoder();
(Asn1Tagged)decoder.decode(intermediateResponse.getValue());`
Then, depending on the Tag I am able to decode the message (using the method .decode(valueToDecode))
I have a question concerning redis in a distributed architecture.
Assume I have n clients, either windows desktop applications or asp.net web/web api servers.
One of the clients, lets say client A, hits the cache for a data and has a miss (the data is not in the cache). The client then starts to get the real data (from lets say a database) and then sets it in the cache when it's done.
Client B comes along and wants the same data, does a fetch to the cache and since it's a miss, does the same processing.
Is there a way for Client B to ...(N) not to do the processing (i.e go to the database) until the data is in the cache and fetch the data from the cache instead when it's available?
I understand that on a single app (or web server), using threads it's easy to check that, but in a distributed architecture?
Is this also a correct way of thinking as well? for the wait process that is
If so then could Client A put a flag somewhere stating that he's loading Data X and that all other clients should wait until he's done?
Otherwise, the idea then would be something along the lines of :
Client A requests Data X
Miss in cache
Processes Data X
Looks if Data X is now in cache
If not, add Data X to cache, otherwise, use it and don't store it in cache
Thanks!
As Kevin said, it's called cache stampede.
One of the best documents to do with this problem I have read is Using memcached: How to scale your website easily (comes from Josef Finsel):
What we need in this instance is some way to tell our program that
another program is working on fetching the data. The best way to
handle that is by using another memcached entry as a lock.
When our program queries memcached and fails to find data, the first
thing it attempts to do is to write a value to a specific key. In our
example where we are using the actual SQL request for the key
name we can just append ":lock" to the SQL to create our new key.
What we do next depends on whether the client supports returning
success messages on memcached storage commands. If it does,
then we attempt to ADD the value. If we are the first one to attempt
this then we’ll get a success message back. If the value exists then
we get a failure indication and we know that another process is trying
to update the data and we wait for some predetermined time
before we try to get the data again.
When the process that’s updating
the cache is done, it deletes the lock key.
I send mails from C# code via Lotus Notes and it works really fine - with Win7 and WinXP.
But some users of my program have multiple .nsf files in the directory, like 'user1.nsf' and 'old-user1.nsf'.
How can I retieve which of these files is the active database I have to use?
If your code runs on users machine then you have "MailFile" and "MailServer" variables in notes.ini file.
Where is this directory located?
You don't need to care which mail database is active if you are sending emails. Just create a new NotesDocument object in memory, fill out the appropriate properties (subject, Body, etc), and then call the Send method on that object. Notes takes care of the rest.
If you need to get the mail database information for a user for another reason, there is a GetUserInfo method for that on the NotesRegistration class
Or in formulas, there is the #MailDbName formula.
If you're just using the user's mail file, in LotusScript you can just use
Call notesDatabase.OpenMail
That way, you never need to bother with where the mail file is - you get the handle to it seemlessly.
We're using Exchange 2007 WS to process mail folders and are hitting various problems if we try and forward a message we've already received. Our process is:
Windows Service monitors mailbox folder, on finding a new mail we process the information and move the item to a 'Processed folder' and store the Exchange Message Id.
Users may opt to forward the mail externally. We use the Exchange API to find the item using the Message Id we stored earlier, and then again use the API to forward.
Except finding the mail again is proving rather flaky. We regularly get the following error:
The specified object was not found in the store.
Is there a better/more reliable way we can achieve the same? The documentation for Exchange WS is rather sparse.
This is a bug in microsoft exchange manage API. here is a link for more information
http://maheshde.blogspot.com/2010/09/exchange-web-service-specified-object.html
Are you saving the Message ID of the newly found message or the message once it has been moved to the 'Processed' folder? The id will change when it moves to a new folder.
The method recommended in the book Inside Microsoft Exchange Server 2007 Web Services is to grab the PR_SEARCH_KEY (0x300B, Binary) of the newly discovered item, then move it to the 'Processed' folder. You can then search for it in the new folder based on the PR_SEARCH_KEY and get it's new Message id to forward it.
I have come to the conclusion that this happens to me is because while my app is processing the emails, someone else is fiddling with an email at the same time.
So to cure the problem, I put it the code in a try catch and see if the exception is == the that object not found in store, if so I just skip it and move on to the next item. So for has no issues.
I wrote a program that reads the emails in inbox downloads attachments to the specified folder, wrote the email info and the saved path to the database, and finally deletes the email. I run this program as a windows service. After all tests are finished I run this program to the main server and run it. Program runs successfully but sometimes I got this error. I checked everything and finally I found that I forgot to stop service on my computer. 2 programs that runs on my computer and on real server checking the same mailbox at the same time. If you get this error make sure that only one program can process at the same mailbox.
I need to monitor a certain file on FTP, once it had been updated, I need to fetch it from FTP. but how to identify whether it's updated or not is a problem.
Does Anybody have any experience on this?
You need to send a LIST command. You'll need to parse the results manually using regex, since there is no standard format for the return result.
File modification data and time can be also obtained using a MLST or a MDTM command. Both ones are extensions of FTP protocol (not guaranteed on all servers), but at least some of them is supported by most servers. These commands return standardized format, it has not to be parsed like results of LIST command.
See the more details in this article.