Develop Reference

Microsoft Graph API - Exchange Online Messages call returns ServiceUnavailable

I am fetching messages from Exchange in Office365 using Microsoft Graph API.
However, for some folders I seem to get intermittent exceptions.
What we are using:
Microsoft.Graph Version 3.9.0 - Microsoft Graph Client Library for .Net
Microsoft.Graph.Core Version 1.21.0 - Microsoft Graph Core Client Library for .Net
This is the call being used:
'GET /v1.0/users/{id}/mailFolders/{id}/messages'
And this is the error (ServiceUnavailable with UnknownError as inner exception):
Status Code: ServiceUnavailable Microsoft.Graph.ServiceException:
Code: UnknownError Message: Error while processing response. Inner
error:
AdditionalData:
date: 2020-08-04T13:55:33
request-id: ** ClientRequestId: **
Code: UnknownError Message: Error while processing response. Inner error:
AdditionalData:
date: 2020-08-04T13:55:33
request-id: ** ClientRequestId: **
What I've tried:
Throttling:
These are usually the errors we would see with throttling. However, in this case, there seems to be no indication of throttling being applied. There isn't any 'back-off' time returned in the result. Other requests to different folders returns just fine too. By applying our own 'back-off' time (ranging between 5mins-20mins does not seem so make a difference either).
Beta endpoint:
The call posted above shows /v1.0 used. We've also switched to the /beta endpoint, with no difference.
Amount of mails retrieved:
Graph allows us to retrieve up to 999 mails at a time. We've reduced that all the way down to a mail or 2 at a time, but it still returns with the same error.
Delta token:
We've also tried switching over to using the delta token in order to retrieve the mails. This also returns with the same error.
Graph downgrade:
Hoping that there is some difference in the last few versions, we downgraded Graph. There was no difference.
Check local sync issues:
I've noticed in the past (quite a while back), that when doing this call for a folder that has potential local sync issues, this is the same type of error response returned. In this case, there is no reason to believe that these are local sync issues.
Additional:
When setting up the httpProvider, I've removed the default retry handlers as well. I've seen that using the default retry handler, it would automatically catch the 'ServiceException' and do internal retries (not adhering to back offs (not that there is any)), and would result in a tooManyRetries or a timeout (hiding the actual issue). By removing the default retry handler, we can see the actual 'ServiceException' error returned by the server.
When:
Based on our telemetry, this seems to have started happening a lot more frequently since around the 11-13th of June. Before that we did not experience any issues.
There are days that the requests work, but they are few and far in between.
This is quite a big issue for us, so any suggestions would be greatly appreciated. Any specific Microsoft Support channel that I can log this with would also help.
Thanks in advance.

ActiveMQ access to previously published data on subscription

We're using ActiveMQ locally to transfer data between 5 processes that turn simultaneously.
I have some data I need to send to a process, both at runtime (which works perfectly fine), but also a default value on start. Thing is it is published when the process starts, it just doesn't read because it wasn't subscribed to the topic at the time the data was sent.
I have multiple solutions : I could delay the first publishing for a moment so that the process has time to launch (which doesn't seem very appealing) ; or is there a way to send all stored previously non-treated messages to some process that just subscribed ?
I'm coding in C#.

I don't have any experience with ActiveMQ, but other message system usually have an option which marks the subscription as persistent, which means that; after the first subscription; the message queue itself checks if a certain message is delivered to that system and retries with a timeout. In this scenario you need to start the receiver at least 1 time.
If this is not an option and you want to plug in receiver afterwards, you might want to consider a setup of your messages which allows you to retrieve the full state, i.e. if you send total-messages instead of differential- messages.
After a little google, I came upon this definition durable subscribers, I hope this helps:
See:
http://activemq.apache.org/how-do-durable-queues-and-topics-work.html
and
http://activemq.apache.org/manage-durable-subscribers.html

since you are using C# client i don't konw if this is supported
topic = new ActiveMQTopic("TEST.Topic?consumer.retroactive=true");
http://activemq.apache.org/retroactive-consumer.html
So, another solution is to configure this behavior on the broker side by adding that to the activemq.xml and restart :
The subscription recovery policy allows you to go back in time when
you subscribe to a topic.
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" >
<subscriptionRecoveryPolicy>
<timedSubscriptionRecoveryPolicy recoverDuration="10000" />
<fixedCountSubscriptionRecoveryPolicy maximumSize="10000" />
</subscriptionRecoveryPolicy>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
http://activemq.apache.org/subscription-recovery-policy.html

I went around the issue by sending a message from each process when they're launched back to the main one, and then only sending the info I needed to send.

Dynamically set Endpoint definition in BizTalk Send Port using MQSeries adapter

Introduction
We exchange income data with an external party. Each year income tax regulations change and a new message schema has to be implemented. Altogether we now have 8 different schema versions each of which are deployed in a separate 'year income tax' application and this amount increases by 1 each year.
Because we pay our hosting company per installed application, we want to decrease the amount of applications installed.
All these applications are functionally equal, which means we validate incoming messages, and forward valid messages into a specific MQSeries queue. Each invalid message is routed to a response queue. Each application has it's own 'valid' and 'invalid' message queues.
The plan
One generic application that processes all 8(+) messages. New schemas must be deployable without application changes or downtime for previous, running 'income year tax' flows.
So far...
I can receive multiple messages on the same BizTalk receive port (MessageType XmlDocument) and am able to validate these messages dynamically in an orchestration by calling a custom receive pipeline (XML Disassembler + XML Validator). Exceptions as well as valid messages are processed as prescribed. There are no references between the Schemas and the generic application, so schemas can be deployed without need to stop running processes. So far, so good.
The orchestration has 1 receive shape, and 2 send shapes (valid, invalid).
SSO contains the values for routing the 'valid' and 'invalid' messages to their correct queue. Based on the incoming messagetype SSO is questioned for the correct 'valid' or 'invalid' queuedefinition.
The problem
I have previously dealt with dynamic FTP, FILE, WCF and SMTP ports, which all worked flawlessly after supplying the adapter with the correct Context Properties. Even MSMQ seems to have a fairly straightforward approach on dynamically setting transport properties.
However, I cannot seem to find MQSeries MQMT ContextProperties to set the queuedefinition dynamically.
Microsoft does not provide much information on this, and extensive searches on the internet hasn't provided me with anything useful (examples) either.
I tried matching IBM's docs with Microsoft's, but altogether I am now stuck.

I would suggest to use MQSC adapter for IBM MQ integration. It is part of Host Integration Server MSI. It only requires MQ client to be installed on the server Vs MQ Server for Windows installation required by MQSeries adapter.
Set the OutboundTransportLocation property in following format mqsc://{channelName}/tcp/{server{({port})/{queuemanager}/{queuename}
TransportType = MQSC
Context Properties - Schema can be found within assembly MQSeriesEx.MQSPropertySchemaEx with namespace (http://schemas.microsoft.com/BizTalk/2003/mqs-properties).
There are only few context properties you would need to set if at all required.
Channel_HeartBeat
Channel_MaxMessageLength
Channel_UserId
Channel_Password
ConnectionTimeout
If additional properties are required than use MQSeries.MQSPropertySchema context properties.

Thanks Vikas for your suggestion.
I followed your directions and found it works!
However, I found it a little more complicated than needed as it required me configuring channel names for each flow.
The solution that best suited me was the one I had in mind all along, and it was right before me. My attempts failed because I made a fatal mistake by setting the outgoing message's properties where I should have set the dynamic send port's properties.
SendPort(Microsoft.XLANGs.BaseTypes.Address)="MQS://SERVER/QMANAGER/QUEUENAME";

Let me know when mail was NOT received to the mail (for successful DB backup for example)

I have a site I'm building and have a DB backup running on it several times a day.
Every successful backup is sent to my mail (and directed by a rule to a folder since there are many backups..) and every failure is also sent to me (not directed by a rule).
I'm afraid that the task will stop running from some reason and I will not know (no alerts arriving but how can I notice it in the blur of mails).
Is there a SW or process that alerts me when a mail was not received during a specific time?
The reason I'm asking here is that I want to develop this kind of thing (if does not exist).
Thanks

You cannot know, unless you create something for yourself. E.g. a rule to auto-reply the received message (at receiver's mailbox). Your program (that sent the e-mail) should check his own mailbox within x seconds for the reply after sending.
Normally it is just a send and goodbye system, unless the e-mail box is full, unreachable, etc.

Send thousands of emails at a time, pitfalls/best way?

Im writing a tool c# (and System.Net.Mail) which will be sending around 4000 emails fornigtly.
Can i simply loop through a list of 4000 email addresses and send emails one at a time?
or will this cause any potential problems?
Is there a better way to do this?
Thanks

First of all you must have rDNS record that will map IP address back to domain name. Without it mail services like GMail will hold your e-mails for hours or even days. Furthermore you may become blacklisted after few dozens of letters. rDNS usually can be set using hoster's control panel or by support request.
Second, don't be tempted to use your ISP's or GMail's SMTP servers. You will quickly run out of daily quota. It is usually not allowed to send more than few dozens letters (50 for GMail AFAIK). So either send mail from your own server, or use a 3rd party service like auth smtp.
Such services are useful also if you're unable to set rDNS for some reason.
P.S. If the content of your letters is OK (i.e. they are not spam actually) and rDNS/SMTP are set correctly, you can send at any rate, without pauses.

Provided you don;t break any server limitations, such as memory or disk space, technically there is no issue. However you will run into problems with with ISP's and mail forwarders who may block your domains for spam - rightly or wrongly..

You could send the 4000 emails in one go, but you do run the risk of getting marked very quickly as a spammer.
Your best option is to 'burst send' so send maybe 100 straight away, in 10mins send another 100 etc.
If you do it this way your are less likely to be seen as a spammer.

Check that the machine sending the mails has a valid PTR record - i.e., if you're sending from www.example.com with IP 1.2.3.4, that a PTR query on 1.2.3.4 will return www.example.com; I'd look into SPF and DKIM, too.
Some e-mail hosts have an incoming rate limit (e.g. Hotmail used that in the past) - "if you are sending us more than x emails/minute, you get a higher spam score". I had a script to "stir the queue", so that the per-minute rate to a single server was lowered.
Also, check this - it has plenty of good advice on e-mail content, but that's slightly off-topic.

In addition to what has been mentioned above, be sure to set up adequate logging.
I.e. you should log sent emails so if you need re-send emails you can be sure not to send duplicates.
Also, you can set up an app to log bounced emails and use this to remove addresses etc.
You can of course write your own software or purchase some.
http://www.quiksoft.com/bouncebuster/