Develop Reference

Bulk files upload to ADLS gen 1

Is there anyway to upload or transform multiple files to adls in one attempt(Bulk Upload). If that is not possible any parallel processing feature available??.
Am looking for c# example.
Please advise..

If you want to upload multiple files in one directory, we can use the method BulkUpload to implement bulk upload. Regarding how to use the method, please refer to here
For example
Create a service principal and configure permissions
Code
string TENANT = "";
string CLIENTID = "";
Uri ADL_TOKEN_AUDIENCE = new Uri(#"https://datalake.azure.net/");
string secret_key = "";
var adlServiceSettings = ActiveDirectoryServiceSettings.Azure;
adlServiceSettings.TokenAudience = ADL_TOKEN_AUDIENCE;
var adlCred = await ApplicationTokenProvider.LoginSilentAsync(TENANT, CLIENTID, secret_key, adlServiceSettings);
AdlsClient client = AdlsClient.CreateClient("<>.azuredatalakestore.net", adlCred);
But please note that the method has limits. it just can be used to upload files in the same directory one time and it will upload whole files in the directory. If you want to upload some files in one directory or files in different directories, pleas refer to the following code
static async Task Main(string[] args)
{
string TENANT = "";
string CLIENTID = "";
Uri ADL_TOKEN_AUDIENCE = new Uri(#"https://datalake.azure.net/");
string secret_key = "";
var adlServiceSettings = ActiveDirectoryServiceSettings.Azure;
adlServiceSettings.TokenAudience = ADL_TOKEN_AUDIENCE;
var adlCred = await ApplicationTokenProvider.LoginSilentAsync(TENANT, CLIENTID, secret_key, adlServiceSettings);
AdlsClient client = AdlsClient.CreateClient("<>.azuredatalakestore.net", adlCred);
string[] filePaths = { };
int count = 0;
var tasks = new Queue<Task>();
foreach (string filePath in filePaths)
{
var fileName = "";
// Add the upload task to the queue
tasks.Enqueue(uplaodFile(client, filePath, fileName));
count++;
}
// Run all the tasks asynchronously.
await Task.WhenAll(tasks);
}
private static async Task uplaodFile(AdlsClient client, string filePath, string fileName)
{
using (var createStream = client.CreateFile(fileName, IfExists.Overwrite))
{
using (var fileStrem = File.OpenRead(filePath))
{
await fileStrem.CopyToAsync(createStream);
}
}
}

Botframework V4: Video attachment on messenger

Hi i tried all these methods of attaching a video to a bot. All of them are working fine in bot emulator. But when i publish it to messenger it is throwing an exception . (I can't see the exception by the way i just know because of the message. Is there a way to see or log exceptions?). Is video card not supported in messenger? Or is youtube not supported as an url link?
Here are the codes:
AddStep(async (stepContext, cancellationToken) =>
{
var reply = stepContext.Context.Activity.CreateReply();
reply.Attachments = new List<Attachment>();
reply.Attachments.Add(GetVideoCard().ToAttachment());
await stepContext.Context.SendActivityAsync(reply, cancellationToken);
return await stepContext.NextAsync();
});
////////////////
private static VideoCard GetVideoCard()
{
var videoCard = new VideoCard
{
Title = "Budgeting Introduction",
Subtitle = "by Finko",
Media = new List<MediaUrl>
{
new MediaUrl()
{
Url = "https://www.youtube.com/watch?v=XLo1geVokhA",
},
},
Buttons = new List<CardAction>
{
new CardAction()
{
Title = "Learn More at Finko.PH",
Type = ActionTypes.OpenUrl,
Value = "https://m-moreno.wixsite.com/finkoph?fbclid=IwAR1NVtlyKfzZ0mYFIWva8L-d8TUv4KFpt_m1i1ij3raT-pbWr2c3-kqzB2Q",
},
},
};
return videoCard;
}
and
AddStep(async (stepContext, cancellationToken) =>
{
var activity = stepContext.Context.Activity;
await stepContext.Context.SendActivityAsync(CreateResponse(activity, CreateVideoCardAttacment()));
return await stepContext.NextAsync();
});
////////////////////////
private Activity CreateResponse(Activity activity, Attachment attachment)
{
var response = activity.CreateReply();
response.Attachments = new List<Attachment>() { attachment };
return response;
}
private Attachment CreateVideoCardAttacment()
{
return new VideoCard()
{
Title = "Are you a Seafarer? OFW? FREE PERSONAL FINANCIAL ADVICE HERE!!",
Media = new List<MediaUrl>()
{
new MediaUrl("https://www.youtube.com/watch?v=XLo1geVokhA")
},
Buttons = new List<CardAction>()
{
new CardAction()
{
Type = ActionTypes.OpenUrl,
Title = "Learn More at Finko.PH",
Value = "https://m-moreno.wixsite.com/finkoph?fbclid=IwAR1NVtlyKfzZ0mYFIWva8L-d8TUv4KFpt_m1i1ij3raT-pbWr2c3-kqzB2Q"
}
},
Subtitle = "by Finko.Ph",
Text = "Are you tired of getting bogus financial advice? Tired of having 'kape' just to find out it was networking, or a pyramid scheme? Tired of scouring the internet for HOURS but not finding what you're looking for? We're here to help! We give financial advice and will educate you on financial literacy topics, ABSOLUTELY FREE!!"
}.ToAttachment();
}
and
Activity reply = stepContext.Context.Activity.CreateReply();
var card = new VideoCard
{
Title = "Finko.ph",
Media = new List<MediaUrl>()
{
new MediaUrl("https://www.youtube.com/watch?v=XLo1geVokhA")
},
Buttons = new List<CardAction>()
{
new CardAction()
{
Type = ActionTypes.OpenUrl,
Title = "Learn More at Finko.PH",
Value = "https://m-moreno.wixsite.com/finkoph?fbclid=IwAR1NVtlyKfzZ0mYFIWva8L-d8TUv4KFpt_m1i1ij3raT-pbWr2c3-kqzB2Q"
}
},
};
reply.Attachments.Add(card.ToAttachment());
await stepContext.Context.SendActivityAsync(reply);
return await stepContext.NextAsync();
and
var reply1 = stepContext.Context.Activity.CreateReply();
var attachment1 = new Attachment
{
ContentUrl = "https://www.youtube.com/watch?v=XLo1geVokhA",
ContentType = "video/mp4",
Name = "imageName1",
};
reply1.Attachments = new List<Attachment>() { attachment1 };
await stepContext.Context.SendActivityAsync(reply1, cancellationToken);
return await stepContext.NextAsync();
All of these codes are working in bot emulator but not in messenger. Any help would be appreciated thank you.

The BotFramework converts Video Cards into Media Templates for Facebook Messenger, and per Facebook's Developer documentation, media templates do not allow any external URLs, only those on Facebook. You must either upload the video to Facebook or provide a URL directly to the mp4 file which, unfortunately, YouTube doesn't make readily available.
For more details, take a look at Facebooks documentation regarding Media Templates.

CSOM Set Sharing on OneDrive Folder

I'd like to set sharing rights on a folder in OneDrive. I know there is a post out there about ListItems, but I need it at a folder level. First, is this possible or am I wasting my time? I've tried the following:
I'm able to get the site object but I'm not able to get the folder in order to share it. The web object doesn't have the folders available to enumerate through. It says it's not initialized. This code below successfully runs but the folder object is not working:
static void Main(string[] args)
{
var webUrl = "https://tenant-my.sharepoint.com/personal/me_tenant_com";
var userName = "me";
string securePassword = "mypassword";
SecureString sec_pass = new SecureString();
Array.ForEach(securePassword.ToArray(), sec_pass.AppendChar);
using (var ctx = new ClientContext(webUrl))
{
ctx.Credentials = new SharePointOnlineCredentials(userName, sec_pass);
var web = ctx.Web;
ClientResult<Microsoft.SharePoint.Client.Utilities.PrincipalInfo> persons = Microsoft.SharePoint.Client.Utilities.Utility.ResolvePrincipal(ctx, ctx.Web, "dpunchak#AvvenireInc.com", Microsoft.SharePoint.Client.Utilities.PrincipalType.User, Microsoft.SharePoint.Client.Utilities.PrincipalSource.All, null, true);
ctx.ExecuteQuery();
var folder = ctx.Web.GetFolderByServerRelativeUrl("/documents/Test Folder");
Microsoft.SharePoint.Client.Utilities.PrincipalInfo person = persons.Value;
//ShareListItem(folder, person, "Read");
}
}
public static void ShareListItem(ListItem listItem, Principal principal, string permissionLevelName)
{
var ctx = listItem.Context as ClientContext;
var roleDefinition = ctx.Site.RootWeb.RoleDefinitions.GetByName(permissionLevelName);
listItem.BreakRoleInheritance(true, false);
var roleBindings = new RoleDefinitionBindingCollection(ctx) { roleDefinition };
listItem.RoleAssignments.Add(principal, roleBindings);
ctx.ExecuteQuery();
}

I think you have to pass folder.ListItemAllFields property to ShareListItem().
To avoid collection has not been initialized error you can try placing RoleAssignments.Add() inside ctx.ExecuteQuery():
ctx.ExecuteQuery(listItem.RoleAssignments.Add(principal, roleBindings);

Load text from web during xml parsing in Windows 8 Async

I have a unique issue, I want to get the name of an application from it's AppID while I convert an XML file into objects. This is the code I'm using at present:
if (xdoc.Element("Application") != null)
{
var data = from query in xdoc.Descendants("AppID")
select new APP
{
AppID = query.Value,
AppName = GetName(query.Value).ToString(),
};
itemGridView.DataContext = data;
}
This is the code I'm using to convert the GUID into an app name using Microsoft's Store API. I can confirm that it does return the app name. I'm just unsure how I can get this to display.
private async Task<string> GetName(string guid)
{
try
{
string link = "https://services.apps.microsoft.com/browse/6.2.9200-1/615/en-NZ_en-NZ/c/NZ/cp/10005001/Apps/{0}";
string url = string.Format(link, guid);
var httpClient = new HttpClient();
var httpRequestMessage = new HttpRequestMessage(HttpMethod.Get, url);
var response = await httpClient.SendAsync(httpRequestMessage);
var xmlString = await response.Content.ReadAsStringAsync();
XmlDocument NameXML = new XmlDocument();
NameXML = await XmlDocument.LoadFromUriAsync(new Uri(url));
string sAppName = NameXML.GetElementsByTagName("T")[0].ChildNodes[0].NodeValue.ToString();
return sAppName;
}
catch(Exception)
{
return guid;
}
}
I think my problem is with the async / await tasks. I've just been exposed to it now... how would I load up the App Name alongside the AppID when I parse the xml file?
The output that's being displayed when I run the app is "System.Threading.Tasks.Task[System.String]" (The objects load and the links and everything works fine, its just that the above is displayed instead of the app name).
I've been debugging using breakpoints, it appears that the GetName method only seems to be triggered later on, I'm not sure however.

Try to change this line :
AppName = GetName(query.Value).ToString(),
To this :
AppName = await GetName(query.Value),
GetName will return Task<string> instead of string when not awaited. And the method where above code resides required to be async because of using await inside that method :
private async void SomeMethod()
{
....
if (xdoc.Element("Application") != null)
{
var data = from query in xdoc.Descendants("AppID")
select new APP
{
AppID = query.Value,
AppName = await GetName(query.Value),
};
itemGridView.DataContext = data;
}
....
}
UPDATE :
As you already noticed, LINQ has very limited support for async/await currently. So to workaround this limitation, we can use normal for loop to avoid calling async function inside LINQ :
private async void SomeMethod()
{
....
if (xdoc.Element("Application") != null)
{
var query = from query in xdoc.Descendants("AppID")
select query.Value;
var data = new List<App>();
foreach (var q in query)
{
data.Add(new App{ AppId = q, AppName = await GetName(q) });
}
itemGridView.DataContext = data;
}
....
}

C# Google drive sdk. How to get a list of google drive folders?

I'm writing a program to allow a user to upload files to their Google Drive account. I have the upload part working and am using OAuth2. The issue I'm currently having is getting a list of folders from the users Drive account.
I found some code that is supposed to do this using the .setUserCredentials method, but it doesn't work:
DocumentsService service1 = new DocumentsService("project");
service1.setUserCredentials("user","pass");
FolderQuery query1 = new FolderQuery();
// Make a request to the API and get all documents.
DocumentsFeed feed = service1.Query(query1);
// Iterate through all of the documents returned
foreach (DocumentEntry entry in feed.Entries)
{
var blech = entry.Title.Text;
}
Nothing is returned. Ideally, I want to use OAuth2 to do this. I've been trying with the following code, trying to set the authentication token, but I always get denied access:
String CLIENT_ID = "clientid";
String CLIENT_SECRET = "secretid";
var docprovider = new NativeApplicationClient(GoogleAuthenticationServer.Description, CLIENT_ID, CLIENT_SECRET);
var docstate = GetDocAuthentication(docprovider);
DocumentsService service1 = new DocumentsService("project");
service1.SetAuthenticationToken(docstate.RefreshToken);
FolderQuery query1 = new FolderQuery();
DocumentsFeed feed = service1.Query(query1); //get error here
// Iterate through all of the documents returned
foreach (DocumentEntry entry in feed.Entries)
{
// Print the title of this document to the screen
var blech = entry.Title.Text;
}
..
private static IAuthorizationState GetDocAuthentication(NativeApplicationClient client)
{
const string STORAGE = "storagestring";
const string KEY = "keystring";
string scope = "https://docs.google.com/feeds/default/private/full/-/folder";
// Check if there is a cached refresh token available.
IAuthorizationState state = AuthorizationMgr.GetCachedRefreshToken(STORAGE, KEY);
if (state != null)
{
try
{
client.RefreshToken(state);
return state; // Yes - we are done.
}
catch (DotNetOpenAuth.Messaging.ProtocolException ex)
{
}
}
// Retrieve the authorization from the user.
state = AuthorizationMgr.RequestNativeAuthorization(client, scope);
AuthorizationMgr.SetCachedRefreshToken(STORAGE, KEY, state);
return state;
}
Specifically, I get "Execution of request failed: https://docs.google.com/feeds/default/private/full/-/folder - The remote server returned an error: (401) Unauthorized".
I've also tried:
var docauth = new OAuth2Authenticator<NativeApplicationClient>(docprovider, GetDocAuthentication);
DocumentsService service1 = new DocumentsService("project");
service1.SetAuthenticationToken(docauth.State.AccessToken);
but "State" is always null, so I get a null object error. What am I doing wrong and how is this done?

You should use the Drive SDK, not the Documents List API, which allows you to list folders. You can use "root" as a folderId if you want to list the root directory.

I actually implemented the v3 version of the GDrive SDK for .NET and needed to search for folders as well.
I prefer requesting uniquely all folders instead of getting all files and then performing a LinQ query to keep just the folders.
This is my implementation:
private async Task<bool> FolderExistsAsync(string folderName)
{
var response = await GetAllFoldersAsync();
return response.Files
.Where(x => x.Name.ToLower() == folderName.ToLower())
.Any();
}
private async Task<Google.Apis.Drive.v3.Data.FileList> GetAllFoldersAsync()
{
var request = _service.Files.List();
request.Q = "mimeType = 'application/vnd.google-apps.folder'";
var response = await request.ExecuteAsync();
return response;
}
You could request the name on the Q this way as well:
request.Q = $"mimeType = 'application/vnd.google-apps.folder' and name = '{folderName}'";
Which would lead and simplify things to (obviating null checking):
private async Task<bool> FolderExistsAsync(string folderName)
{
var response = await GetDesiredFolder(folderName);
return response.Files.Any();
}
private async Task<FileList> GetDesiredFolder(string folderName)
{
var request = _service.Files.List();
request.Q = $"mimeType = 'application/vnd.google-apps.folder' and name = '{folderName}'";
var response = await request.ExecuteAsync();
return response;
}

private IEnumerable<DocumentEntry> GetFolders(string id) {
if (IsLogged) {
var query = new FolderQuery(id)
{
ShowFolders = true
};
var feed = GoogleDocumentsService.Query(query);
return feed.Entries.Cast<DocumentEntry>().Where(x => x.IsFolder).OrderBy(x => x.Title.Text);
}
return null;
}
...
var rootFolders = GetFolders("root");
if (rootFolders != null){
foreach(var folder in rootFolders){
var subFolders = GetFolders(folder.ResourceId);
...
}
}
where GoogleDocumentsService is a instance of DocumentsService and IsLogged is a success logged flag.

I got this way to get list of folders from google drive
FilesResource.ListRequest filelist= service.Files.List();
filelist.Execute().Items.ToList().Where(x => x.MimeType == "application/vnd.google-apps.folder").ToList()