Background
I am trying to connect to Office 365, to read the calendars of users that have granted permission. I have tried two options:
Exchange Web Services (EWS)
Microsoft Graph
EWS works but has the downside that I use login/password combinations to connect. Even though I store them encrypted, I'd rather not store them at all.
Microsoft Graph works as well, but has a gigantic downside; any updates I make to an item using the API is sent to all attendees. This behavior can be turned off when using the EWS API, but not (yet?) for Graph.
I'd like to take the OAuth implementation I have for Microsoft Graph, and use the EWS service to connect. No updates to attendees unless users want them, and no stored credentials.
The problem
For my application to work properly, I need to;
Get the timezone of the calendar, which I do by reading the person's work hours;
Read and write calendar items, which is the purpose of the application.
I have already established a connection with OAuth to Office365, using OAuth.
I cannot figure out the smallest subset of permissions I need. I have not found any documentation regarding this. Any subset of rights I tried, I get a 401 when I ask for WorkHours.
Minimal Code sample
This will work when I enable 38 non-admin permissions that the app registration for Exchange Online supports, but will fail for every subset I have tried.
[TestMethod]
public void ConnectUsingEws()
{
var accessToken = "eyJz93a...k4laUWw";
var credentials = new OAuthCredentials(accessToken);
var service = new ExchangeService(TimeZoneInfo.Utc);
service.Url = new Uri("https://outlook.office365.com/EWS/exchange.asmx");
service.TraceEnabled = true;
service.TraceFlags = TraceFlags.All;
service.Credentials = credentials;
// This next line is where the service will always throw a 401.
var workHours = UserConfiguration.Bind(service, "WorkHours",
WellKnownFolderName.Calendar, UserConfigurationProperties.All);
// Do some XML magic on workHours to get the timezone.
}
TLDR
I'm sure it's one permission that needs to be enabled, and I'm also fairly certain it's not one that's very obvious.
EWS doesn't support the same level of Permission scopes that REST does with Oauth (which is a big downside of using EWS for a security perspective).
OAuth authentication for EWS is only available in Exchange as part of
Office 365. EWS applications require the "Full access to user's
mailbox" permission.
ref https://msdn.microsoft.com/en-us/library/office/dn903761(v=exchg.150).aspx
Related
I have been trying to implement a solution for this for days. It's my first experiment with Microsoft Graph. I had our network admin register the app and went through the quick start code in console-app-quickstart.
I looked at active-directory-dotnetcore-daemon-v2 and active-directory-dotnet-iwa-v2.
var App = PublicClientApplicationBuilder
.Create("xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx")
.WithTenantId("xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx")
.Build();
The PublicClientApplication has the AcquireTokenByIntegratedWindowsAuth function. This sounds good because we can launch the console app as whatever user we want to use with a scheduled task. But it errors out with WS-Trust endpoint not found. Where's WS-Trust endpoint defined?
The sample also includes the line var accounts = await App.GetAccountsAsync() but that always returns zero accounts. Some responses to searches for this say that we have to use the global tenant admin. The company doesn't like that idea at all. How can that be safe? Do we create a new user as an admin tenant just for that?
The other option is this
var App = ConfidentialClientApplicationBuilder.Create("xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx")
.WithClientSecret("aeiou~XXXXXXXXXXX")
.WithAuthority(new Uri("https://login.microsoftonline.com/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"))
.Build();
The ConfidentialClientApplication doesn't have the integrated windows auth version. I can get connected and get MailFolders and Messages and process those, but it seems to work only when we use App.AcquireTokenForClient(scopes) and API permissions that allow the app to read everyone's email. Security doesn't like that much either.
I also looked at impersonation-and-ews-in-exchange. I read in some places that ExchangeWebService is deprecated and use MS Graph instead. Is the MS Graph API permissions in the EWS category mean that it's going to be around?
Can anyone out there show me the right combination of pieces needed to do this? (api permissions, client application type, scopes, authority, etc). It needs to be unattended (launched by scheduled task), needs to have permissions to read only one email box, and save the attachments.
(sorry so long)
Thanks, Mike
WS-Trust endpoint not found
The WS-Trust endpoint is your ADFS endpoint, if you have ADFS 2019 then MSAL does support that using WithAdfsAuthority see https://github.com/MicrosoftDocs/azure-docs/blob/main/articles/active-directory/develop/msal-net-initializing-client-applications.md
There are some other restriction around using WIA that are listed at the top of https://github.com/AzureAD/microsoft-authentication-library-for-dotnet/wiki/Integrated-Windows-Authentication-in-MSAL-2.x . If the constraints don't affect you it should work okay.
With the Client Credentials flow which is what your using above you can restrict the scope of the mailboxes it can access see https://learn.microsoft.com/en-us/graph/auth-limit-mailbox-access
I would stick with the Graph rather then EWS as the later is being phased out and requires more permissions as its a legacy API.
The tutorial you shared in the question is an asp.net core console app. Since you want to have a console app and use it to read exchange mails.
Therefore, what we can confirm is that: We need to use MS Graph API to read the exchange mails. Graph API required an Azure AD application with correct API permissions to generate Access token to call the API. API permissions have 2 types, Delegated for Web app because it required users to sign in to obtain the token, Application for daemon app like console application which don't require an user-sign-in.
Since you are using the asp.net core console application, you can only using Application API permission. Using Application permission means the console app has the permission to query messages of any email address in your tenant. You can't control the Graph API itself to query some specific users only. But you can write your own business logic to set authorization.
Then we can make the console application authorized to access the API, we can generate an Access token and use it in the HTTP request header to call the API, we can also use the Graph SDK. Using SDK will help to troubleshoot when met error.
using Microsoft.Graph;
using Azure.Identity;
var scopes = new[] { "https://graph.microsoft.com/.default" };
var tenantId = "tenant_id";
var clientId = "Azure_AD_app_id";
var clientSecret = "Azure_AD_client_secret";
var clientSecretCredential = new ClientSecretCredential(
tenantId, clientId, clientSecret);
var graphClient = new GraphServiceClient(clientSecretCredential, scopes);
var messages = await graphClient.Users["{email_address/user_account/user_id}"].Messages.Request().Select("sender,subject").GetAsync();
I created a service account to impersonate users in my organization in order to make changes to work items in the users’ name. I already added that service account to the group “Project Collection Service Accounts”, which has “Make requests on behalf of others” set to “Allow”. The service account has Visual Studio Subscription.
I then used my code, which is working with our on-premise TFS, to execute the impersonation and I got an error 500 saying that “Access Denied: X needs the following permission(s) to perform this action: Make requests on behalf of others”
What should I do to make it works? Here the code I’m using:
var credential = new VssAadCredential("X#myoganization", "password");
var collection = new TfsTeamProjectCollection(new Uri("my_devops_uri"), credential);
MyTfsConnection.ProjectCollection = collection;
MyTfsConnection.IdentityService = collection.GetService<IIdentityManagementService>();
MyTfsConnection.WIStore = collection.GetService<WorkItemStore>();
var adAccount = "someone#myoganization";
var identity = MyTfsConnection.IdentityService.ReadIdentity(IdentitySearchFactor.AccountName, adAccount, MembershipQuery.None, ReadIdentityOptions.None);
using (var impersonatedCollection = new TfsTeamProjectCollection(new Uri("my_devops_uri"), credential, identity.Descriptor))
{
var impersonatedWIStore = impersonatedCollection.GetService<WorkItemStore>();
}
It's not able to do this, Service account is not intended to connect the client with server either check in code or change work items.
Service account is used to run various services related to TFS. It should have the minimum privileges possible on the machine.
The client should not connect to the server with a service account, they should be using their own account which you grant access to the relevant repositories in TFS. For example, if you connect all clients with the service account, how will you know who checked in each changeset, who should assign work items to?
You will also not able to assign work items to a service account.
I've been attempting to migrate code to DevOps Services that uses impersonation to alter work items in another users name as well. Uncanny how similar it is to yours. It's almost like we all grabbed it from this old post on TFS impersonation. Like yours, this code works with on-premises DevOps Server 2019.1.1 but I ran into the same issue trying to get it working with DevOps Services.
During my search for an answer that works with the TFS / DevOps (SOAP) client API and DevOps Services, I ran across this SO question where the question quotes the following.
"For security reasons (and compliance and a number of other reasons),
the impersonation header isn't supported on Visual Studio Online"
I've yet to find this same information anywhere in the documentation. However, it appears to be true. Impersonation is disabled in Azure DevOps Services. Drat!
Not giving up, my search also turned up the following from the November 2017 release notes, which appears promising.
Grant the bypassrule permission to specific users
Often, when migrating work items from another source, organizations
want to retain all the original properties of the work item. For
example, you may want to create a bug that retains the original
created date and created by values from the system where it
originated.
The API to update a work item has a bypassrule flag to enable that
scenario. Previously the identity who made that API request had to be
member of the Project Collection Administrators group. With this
deployment we have added a permission at the project level to execute
the API with the bypassrule flag.
However, I find no such permission in the collection / organization or project permissions on either DevOps Server or DevOps Services. Drat again! That led me to this SO answer where a work item update is crafted in JSON using the REST API directly. So the bypass rule option must still be valid, just not exposed as a settable permission.
So I started looking at the TFS / DevOps (SOAP) client API again and found the WorkItemStoreFlags.BypassRules flag to pass to the WorkItemStore when it is created. The following should provide the basic mechanics.
// Use a personal access token with Work Items scope
var credentials = new VssBasicCredential(String.Empty, "Your PAT");
// Connect with TFS / DevOps client libs.
// Older SOAP based client but still works with DevOps Services.
var teamProjectCollection = new TfsTeamProjectCollection(new Uri("https://dev.azure.com/your-org"), credentials);
// Don't use teamProjectCollection.GetService<WorkItemStore>().
// New up WorkItemStore using the collection and explicitly specify the BypassRules flag.
// This allows you to set the CreatedBy field later.
var workItemStore = new WorkItemStore(teamProjectCollection, WorkItemStoreFlags.BypassRules);
// Get the project, work item type, and create the new work item.
var project = workItemStore.Projects["YourProject"];
var workItemType = project.WorkItemTypes["Product Backlog Item"];
var workItem = new WorkItem(workItemType);
// Set the work item fields
workItem.Title = "The Title";
// Without the BypassRules flag the CreatedBy value set here will be ignored on
// Save() and replaced with the user account attached to the PAT used to authenticate.
workItem.Fields[CoreField.CreatedBy].Value = "NotYourUser#yourdomain.com";
workItem.Fields[CoreField.AssignedTo].Value = "NotYourUser#yourdomain.com";
workItem.Save();
Scenario
I have an Exchange Online environment and service/daemin (no interactive user) application on the Azure VM. Service uses EWS managed API to work with emails in the mailbox of any tenant user. Now EWS client uses Basic authentication that, according to Microsoft, will become unsupported in EWS to access Exchange Online.
Question/Issue
So, I need to find a way to get valid access token for service/daemon application to use with EWS managed API.
My findings
The following article shows an example of using OAuth 2.0 with EWS managed API. This example works, but it uses interactive method of getting consent (sign-in form appears allowing user authenticate themselves and grant requested permission to application) that is not suitable for service/daemon app scenario, because there is no interactive user.
For service/daemon application I need to use client credential authentication flow.
Registered application
Using admin account on https://aad.portal.azure.com portal I registered application with Azure Active Directory. Added client secret for registered application.
Aforementioned article uses https://outlook.office.com/EWS.AccessAsUser.All as a scope. But I did not find permission with such a URL on the portal. I found only the following permissions under Office 365 Exchange Online > Application permissions > Mail:
https://outlook.office365.com/Mail.Read Allows the app to read mail in all mailboxes without a signed-in user
https://outlook.office365.com/Mail.ReadWrite Allows the app to create, read, update, and delete mail in all mailboxes without a signed-in user.
I added both of them and granted admin consent for all users.
Getting access token
For testing purposes and simplicity I did not use any auth libraries (ADAL, MSAL etc.). I used Postman to get access token, then set token variable in debug (see code snippet later in the post).
I tried different endpoints to get acess token.
OAuth 2.0 token endpoint (v2)
POST: https://login.microsoftonline.com/<TENANT_ID>/oauth2/v2.0/token
grant_type=client_credentials
client_id=***
client_secret=***
scope=https://outlook.office.com/EWS.AccessAsUser.All
Sending this request produces the following error response:
AADSTS70011: The provided request must include a 'scope' input parameter. The provided value for the input parameter 'scope' is not valid. The scope https://outlook.office.com/EWS.AccessAsUser.All is not valid.
I tried changing scope to https://outlook.office.com/.default. Access token was returned, but it appeared to be invalid for EWS. EWS client throws 401 error with the following value of x-ms-diagnostics response header:
2000008;reason="The token contains no permissions, or permissions can not be understood.";error_category="invalid_grant"
OAuth 2.0 token endpoint (v1)
POST: https://login.microsoftonline.com/<TENANT_ID>/oauth2/token
grant_type=client_credentials
client_id=***
client_secret=***
resource=https://outlook.office.com
Access token was returned, but also appeared to be invalid for EWS. EWS client throws 401 error with the same value of x-ms-diagnostics response header as described ealier in #1.
Use aquired access token with EWS managed API
Here is code sample that I used to test EWS client with access token acquired in Postman:
var token = "...";
var client = new ExchangeService
{
Url = new Uri("https://outlook.office365.com/EWS/Exchange.asmx"),
Credentials = new OAuthCredentials(token),
ImpersonatedUserId = new ImpersonatedUserId(ConnectingIdType.SmtpAddress,
"user#domain.onmicrosoft.com"),
};
var folder = Folder.Bind(client, WellKnownFolderName.SentItems);
We had a similar problem: We wanted to use a Service Account to connect to a single mailbox and just doing some stuff with the EWS API (e.g. searching in the GAL) and the full_access_as_app seems like an overkill.
Fortunately it is possible:
Follow the normal "delegate" steps
And use this to get a token via username/password:
...
var cred = new NetworkCredential("UserName", "Password");
var authResult = await pca.AcquireTokenByUsernamePassword(new string[] { "https://outlook.office.com/EWS.AccessAsUser.All" }, cred.UserName, cred.SecurePassword).ExecuteAsync();
...
To make this work you need to enable the "Treat application as public client" under "Authentication" > "Advanced settings" because this uses the "Resource owner password credential flow". (This SO answer helped me alot!)
With that setup we could use a "tradional" username/password way, but using OAuth and the EWS API.
You can protect your client application with either a certificate or a secret. The two permissions that I needed to get this to work were Calendars.ReadWrite.All and full_access_as_app. I never tried acquiring my token via PostMan, but use AcquireTokenAsync in Microsoft.IdentityModel.Clients.ActiveDirectory. In that call, the resource parameter I use is https://outlook.office365.com/. It's pretty simple once you know all the little twists and turns. And full disclosure: I was one lost puppy until MSFT support helped me through this. The doc on the web is often outdated, conflicting, or at best, confusing.
You need to register your app in Azure and use certificate based authentication. https://blogs.msdn.microsoft.com/emeamsgdev/2018/09/11/authenticating-against-exchange-web-services-using-certificate-based-oauth2-tokens/
I run into the same issue while following Microsoft official docs for OAuth 2.0 client credentials flow
According to the Microsoft identity platform and the OAuth 2.0 client credentials flow, the scope "should be the resource identifier (application ID URI) of the resource you want, affixed with the .default suffix" (see default scope doc).
So the question is how to convert https://outlook.office.com/EWS.AccessAsUser.All into the resource identifier.
Experimentally I manage to make it working using scope=https://outlook.office365.com/.default. I granted full_access_as_app (Office 365 Exchange Online / Application permissions) and got administrator consent for it.
I did face this issue while implementing OAuth for EWS. My application is not using EWS Managed API. Here is what all I did to make it working.
Added permission Office 365 Exchange Online > full_access_as_app to application.
Acquired access token for scope https://outlook.office365.com/.default.
POST https://login.microsoftonline.com/{tenant}/oauth2/v2.0/token
form-data = {
client_id,
client_secret,
grant_type: 'client_credentials',
scope: 'https://outlook.office365.com/.default',
};
Added access token as Authorization header and ExchangeImpersonation SOAP header to the request.
<SOAP-ENV:Header>
<t:ExchangeImpersonation>
<t:ConnectingSID>
<t:PrimarySmtpAddress>user#domain.com</t:PrimarySmtpAddress>
</t:ConnectingSID>
</t:ExchangeImpersonation>
</SOAP-ENV:Header>
Late answer, but since this seems to come up, and I was just working with this... why not.
If you use Microsoft's v2.0 URLs for OAUTH2 (https://login.microsoftonline.com/common/oauth2/v2.0/authorize and .../common/oauth2/v2.0/token) then the scope for Office 365 EWS is:
https://outlook.office365.com/EWS.AccessAsUser.All
You'll probably want to combine this scope with "openid" (to get the signed in user's identity) and "offline_access" (to get a refresh token). But then offline_access may not be necessary when using client credentials (because you don't have to prompt a human user for them every time you need an access token).
In other words:
params.add("client_id", "...")
...
params.add("scope", "openid offline_access https://outlook.office365.com/EWS.AccessAsUser.All")
If using v1 OAUTH2 URLs (https://login.microsoftonline.com/common/oauth2/authorize and .../common/oauth2/token) then you can use a "resource" instead of a "scope". The resource for Office 365 is https://outlook.office365.com/.
Or in other words:
params.add("resource", "https://outlook.office365.com/")
Note that in the latter case, you're not asking for any scopes (it's not possible to combine "resource" with scopes). But the token will automatically cover offline_access and openid scopes.
I used this method successfully:
Install Microsoft Authentication Library module ( MSAL.PS)
https://www.powershellgallery.com/packages/MSAL.PS/4.2.1.3
Configure Delegate Access as per MSFT instructions: https://learn.microsoft.com/en-us/exchange/client-developer/exchange-web-services/how-to-authenticate-an-ews-application-by-using-oauth
Configure ApplicationImpersonation for a service account as normal
Grab your token
$cred = Get-Credential
$clientid = ""
$tenantid = ""
$tok = Get-MsalToken -ClientId $clientid -TenantId $tenantid -UserCredential $cred -Scopes "https://outlook.office.com/EWS.AccessAsUser.All"
I have an existing application (a console app that runs as a WebJob) that uses Exchange Web Services to read emails in a shared Outlook 365 mailbox. This works, but it's using basic authentication and I want to use OAuth instead. I'm attempting to do this using Microsoft.Identity.Client.ConfidentialClientApplicationBuilder to get an access token. I've read various articles and posts online which seem to give conflicting advice about what the 'scope' parameter should be when calling AcquireTokenForClient. Some say https://graph.microsoft.com/.default, others say https://outlook.office.com/.default or https://outlook.office365.com/.default. Others seem to suggest that it should be Mail.Read rather than .Default. I've tried all of the above without success. Can anyone tell me what the correct value for 'scope' is?
I assume that you have registered your app for an Office 365 tenant. We are using EWS with modern authentication successfully for some time now. To access the users' mailboxes in your tenant using OAuth authentication you have to grant the registered application the API permission Exchange - full_access_as_app and use https://outlook.office.com/.default as scope.
var clientApp = ConfidentialClientApplicationBuilder
.Create("applicationId")
.WithTenantId("tenantId")
.WithClientSecret("secret")
.Build();
var authenticationResult = await clientApp.AcquireTokenForClient(new[] { "https://outlook.office.com/.default" }).ExecuteAsync();
var accessToken = authenticationResult.AccessToken;
Then add the token to the authorization header of the EWS requests.
We are adding in ability to login to federated accounts to SharePoint online from our non-web application.
Generally have no issues using SAML to call to https://login.microsoftonline.com/extSTS.srf for Office365 accounts.
We have a customer who used federated security, but then also allows access to some other user accounts, specifically to http://outlook.com addresses.
Since this is a federated account, passing credentials to login.microsoftonline.com/extSTS.srf does not work. I was reading some code examples which showed federated calls posting to
login.microsoftonline.com/GetUserRealm.srf
with handler=1&login={0}
When we do that for the http://outlook.com user account, we get back this JSON response:
{ "State":1, "UserState":2, "Login": "name_here#outlook.com", "FederationGlobalVersion": -1, "DomainName": "OUTLOOK.COM", "AuthURL": "https://login.live.com/login.srf?cbcxt=&popupui=&vv=&username=&mkt=&lc=&wfresh=", "NameSpaceType": "Federated", "FederationBrandName": "OUTLOOK.COM", "AuthNForwardType": 0}
The code examples we have see then suggest posting the SAML info to a site constructed from the AuthURL:
String.Format("https://{0}/adfs/services/trust/2005/usernamemixed/", adfsAuthUrl.Host);
That would be: https://login.live.com/adfs/services/trust/2005/usernamemixed/
However, we get a 404 error hitting that. If we were to have called the GetUserRealm using an AD user account, we know we would have gotten a company specific AuthURL back.
Is there a different path that needs to be implemented specifically for a non-domain account (like from http://outlook.com) when trying to authenticate an ADFS-enabled Office365 site?
If you really want your users to authenticate for SharePoint Online, then you should be using either a) Azure AD - get an access token and access the content via the Office 365 REST API or b) a SharePoint App in which you can choose whether to use the user context at all...in which case it still requires an access token, or, with an app-only context. You can see lots of examples of both of these on my blog at SamlMan.wordpress.com.