I am trying to create an azure function that'll clear down files older than a certain age, but when I access the properties of the file they are all null, what am I doing wrong?!
using System;
using System.Collections.Generic;
using Azure.Storage.Files.Shares;
using Azure.Storage.Files.Shares.Models;
using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;
namespace somewhere
{
public static class FileShareCleaner
{
[FunctionName("FileShareCleaner")]
public static void Run([TimerTrigger("*/10 */1 * * * *")]TimerInfo myTimer, ILogger log)
{
string connectionString = Environment.GetEnvironmentVariable("FileShareConnectionString");
string shareName = "files";
ShareServiceClient shareserviceclient = new ShareServiceClient(connectionString);
ShareClient shareclient = shareserviceclient.GetShareClient(shareName);
Queue<ShareDirectoryClient> remaining = new Queue<ShareDirectoryClient>();
remaining.Enqueue(shareclient.GetRootDirectoryClient());
while (remaining.Count > 0)
{
ShareDirectoryClient dir = remaining.Dequeue();
foreach (ShareFileItem item in dir.GetFilesAndDirectories())
{
log.LogInformation(item.Name);
if (item.IsDirectory)
{
remaining.Enqueue(dir.GetSubdirectoryClient(item.Name));
}
else
{
log.LogInformation($"time: {item.Properties.LastModified.ToString()}");
}
}
}
}
}
}
The code finds the files but all the properties are null:
[2021-10-06T10:04:50.048Z] Executing 'FileShareCleaner' (Reason='Timer fired at 2021-10-06T11:04:50.0126493+01:00', Id=af5c7864-4326-4c97-b9d6-82bf98726f4e)
[2021-10-06T10:04:50.341Z] 0304ccf5-4e32-4206-b903-af5acc8652dc.dat
[2021-10-06T10:04:50.344Z] time:
[2021-10-06T10:04:50.347Z] 06716b40-cce4-4ef0-86ec-329dcaeddbf4.dat
[2021-10-06T10:04:50.350Z] time:
[2021-10-06T10:04:50.353Z] 20735b83-d8b2-4110-9ee6-6154b97c154c.dat
[2021-10-06T10:04:50.355Z] time:
[2021-10-06T10:04:50.358Z] 2696a0eb-2aed-4200-b495-0dd2a7152139.dat
[2021-10-06T10:04:50.361Z] time:
You are not doing anything wrong. This is expected behavior.
By default when files and folders are listed in a File Share, only size of the file is returned.
For fetching other properties of the file like last modified or content properties, you will need to get the properties of each file separately.
Update
To get the properties, what you will need to do is create an instance of ShareFileClient using ShareDirectoryClient.GetFileClient and then call GetProperties on that. Your code would look something like below:
while (remaining.Count > 0)
{
ShareDirectoryClient dir = remaining.Dequeue();
foreach (ShareFileItem item in dir.GetFilesAndDirectories())
{
log.LogInformation(item.Name);
if (item.IsDirectory)
{
remaining.Enqueue(dir.GetSubdirectoryClient(item.Name));
}
else
{
var fileClient = dir.GetFileClient(item.Name);
var fileProperties = fileClient.GetProperties();
log.LogInformation($"time: {fileProperties.Value.LastModified.ToString()}");
}
}
}
I have a large directory of folders and files that contain a space at the end of the name, I'm trying to rename the directories with that space to one without, so that another application would be able to access it.
I'm using C# (but if there's a better option that would fix that issue please suggest) and here's my entire code:
using System;
using System.IO;
using System.Text.RegularExpressions;
namespace removing_spaces_in_directories_names
{
class Program
{
public static string path = "../../../old_directory";
static void Main(string[] args)
{
DirectoryInfo di = new DirectoryInfo(path);
WalkDirectoryTree(di);
Console.ReadLine();
}
static void WalkDirectoryTree(System.IO.DirectoryInfo root)
{
if (root.Name != "old_directory")
{ renameDirectory(root); }
DirectoryInfo[] diArr = root.GetDirectories();
foreach(DirectoryInfo di in diArr)
{
WalkDirectoryTree(di);
}
}
static void renameDirectory(System.IO.DirectoryInfo dir)
{
Console.WriteLine("renaming: " + dir.FullName);
string newName = ReplaceLastOccurrence(dir.FullName, " ", "");
if (Directory.Exists(dir.FullName) == false)
{
//dir.MoveTo(newName);
String oldName = #"\\?\"+dir.FullName;
Directory.Move(oldName,newName);
}
}
public static string ReplaceLastOccurrence(string Source, string Find, string Replace)
{
int place = Source.LastIndexOf(Find);
if (place == -1)
return Source;
string result = Source.Remove(place, Find.Length).Insert(place, Replace);
return result;
}
}
}
I have tried adding "\?\" to the beginning of the folder name as suggested here but that's not working, the error I'd get if I add it is: Illeagal characters in path.
On the other hand if I use dir.MoveTo(newName); without the "\?\" characters I'd get the error: Could not find a part of the path 'Volunteer Information '
How can I go through this if at all? would perhaps running this application on linux rather than windows help?
For each directory that you want to rename (remove the trailing space at the end in this case), let's say your DirectoryInfo variable is called di
You want to do this:
string oldName = di.FullName;
string newName = oldName.TrimEnd();
Directory.Move(oldName, newName);
I rewrote this in a PHP application that's sitting on linux and it worked.
I tring to test a new dll that I've build for c#
private void button1_Click(object sender, EventArgs e)
{
String [] first = UserQuery.Get_All_Users();
//MessageBox.Show(first);
}
but I get the following error at String [] first = UserQuery.Get_All_Users();
An unhandled exception of type 'System.NullReferenceException' occurred in User_Query.dll
Additional information: Object reference not set to an instance of an object.
I been tring to figure this one out for hours but can't find any null varibles
I post my dll in case the dll is wrong
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.DirectoryServices;
namespace User_Query
{
public class UserQuery
{
public static string[] Get_All_Users()
{
string[] names = new string[10];
var path = string.Format("WinNT://{0},computer", Environment.MachineName);
using (var computerEntry = new DirectoryEntry(path))
{
var userNames = from DirectoryEntry childEntry in computerEntry.Children
where childEntry.SchemaClassName == "User"
select childEntry.Name;
byte i = 0;
foreach (var name in userNames)
{
Console.WriteLine(name);
names[i] = name;
i++;
}
return names;
}
}
}
}
There is a problem with your. path variable... since there should be \\ instead of //
The problem here turned out not to be the code but be VS2010 not loading the dll. This happen because I decided to change the program from using the dll from the debug to the release version but I did not clean the project after doing it and therefore the program was not correctly loading the dll. All that need to be done was clean the project
Using DISKPART command line utility, I can get something called a "Location path" which appears to give me what I need, you can view this by using the command detail disk after selecting one of your disks in diskpart.
It appears I can get this information programatically via this class: MSFT_Disk
I am unsure about how to get an instance of this class. I have a few examples of using a ManagementObjectSearcher for WMI classes but that method is not working for me, I am also unsure of MSFT_Disk's availability in Windows 7 as the page mentions that this is for Windows 8.
Does anyone know of a good way to get SATA channel information or the "location path" of a disk?
If you want to not require Windows 8, I believe WMI is the way to go:
using System;
using System.Linq;
using System.Management;
namespace DiskScanPOC
{
class Program
{
static void Main()
{
var managementScope = new ManagementScope();
//get disk drives
var query = new ObjectQuery("select * from Win32_DiskDrive");
var searcher = new ManagementObjectSearcher(managementScope, query);
var oReturnCollection = searcher.Get();
//List all properties available, in case the below isn't what you want.
var colList = oReturnCollection.Cast<ManagementObject>().First();
foreach (var property in colList.Properties)
{
Console.WriteLine("Property: {0} = {1}", property.Name, property.Value);
}
//loop through found drives and write out info
foreach (ManagementObject oReturn in oReturnCollection)
{
Console.WriteLine("Name : " + oReturn["Name"]);
Console.WriteLine("Target Id: " + oReturn["SCSITargetId"]);
Console.WriteLine("Port: " + oReturn["SCSIPort"]);
}
Console.Read();
}
}
}
I didn't crack open my case to verify the SATA port numbers, but the above app looks like it gives reasonable results on my machine with 3 SATA hard drives.
If you want to get the location path, SetupDiGetDeviceRegistryProperty is the function you're looking for. Set the property value to SPDRP_LOCATION_INFORMATION.
I'm assuming you already know how to enumerate devices to get the DeviceInfoSet and DeviceInfoData.
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;
using System.Management;
namespace Hard_Disk_Interface
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
private void btnCheck_Click(object sender, EventArgs e)
{
WqlObjectQuery q = new WqlObjectQuery("SELECT * FROM Win32_IDEController");
ManagementObjectSearcher res = new ManagementObjectSearcher(q);
lblHDDChanels.Text = string.Empty;
foreach (ManagementObject o in res.Get())
{
string Caption = o["Caption"].ToString();
lblHDDChanels.Text += Caption + "\n\n";
if (Caption.Contains("Serial"))
{
lblInterface.Text = "S-ATA";
}
}
}
}
}
Note: First Add the reference of System.Management.dll of .net freamwork 4.0
I too have a long running service using plugins and appdomains and am having a memory leak due to using directoryservices. Note that I am using system.directoryservices.accountmanagement but it is my understanding that it uses the same underlying ADSI API's and hence is prone to the same memory leaks.
I've looked at all the CLR memory counters and the memory isn't being leaked there, and is all returned either on a forced GC or when I unload the appdomain. The leak is in private bytes which continually grow. I searched on here and have seen some issues related to a memory leak when using the ADSI API's but they seem to indicate that simply iterating over the directorysearcher fixes the problem. But as you can see in the code below, I am doing that in a foreach block and still the memory is being leaked. Any suggestions? Here is my method:
public override void JustGronkIT()
{
using (log4net.ThreadContext.Stacks["NDC"].Push(GetMyMethodName()))
{
Log.Info("Inside " + GetMyMethodName() + " Method.");
System.Configuration.AppSettingsReader reader = new System.Configuration.AppSettingsReader();
//PrincipalContext AD = null;
using (PrincipalContext AD = new PrincipalContext(ContextType.Domain, (string)reader.GetValue("Domain", typeof(string))))
{
UserPrincipal u = new UserPrincipal(AD);
u.Enabled = true;
//u.Surname = "ju*";
using (PrincipalSearcher ps = new PrincipalSearcher(u))
{
myADUsers = new ADDataSet();
myADUsers.ADUsers.MinimumCapacity = 60000;
myADUsers.ADUsers.CaseSensitive = false;
foreach (UserPrincipal result in ps.FindAll())
{
myADUsers.ADUsers.AddADUsersRow(result.SamAccountName, result.GivenName, result.MiddleName, result.Surname, result.EmailAddress, result.VoiceTelephoneNumber,
result.UserPrincipalName, result.DistinguishedName, result.Description);
}
ps.Dispose();
}
Log.Info("Number of users: " + myADUsers.ADUsers.Count);
AD.Dispose();
u.Dispose();
}//using AD
}//Using log4net
}//JustGronkIT
I made the following changes to the foreach loop and it's better but private bytes still grows and is never reclaimed.
foreach (UserPrincipal result in ps.FindAll())
{
using (result)
{
try
{
myADUsers.ADUsers.AddADUsersRow(result.SamAccountName, result.GivenName, result.MiddleName, result.Surname, result.EmailAddress, result.VoiceTelephoneNumber, result.UserPrincipalName, result.DistinguishedName, result.Description);
result.Dispose();
}
catch
{
result.Dispose();
}
}
}//foreach
I hit a big memory leak because, like you I wrote something like...
foreach (GroupPrincipal result in searcher.FindAll())
{
results.Add(result.Name);
}
But the trick is that FindAll itself returns an object that must be disposed...
using (var searchResults = searcher.FindAll())
{
foreach (GroupPrincipal result in searchResults)
{
results.Add(result.Name);
}
}
I'm fairly sure that this is a known error ( http://social.msdn.microsoft.com/Forums/en-US/netfxbcl/thread/6a09b8ff-2687-40aa-a278-e76576c458e0 ).
The workaround? Use the DirectoryServices library...
I spoke too soon, simply being aggressive with calling Dispose() did NOT solve the problem over the long run. The real solution? Stop using both directoryservices and directoryservices.accountmanagement and use System.DirectoryServices.Protocols instead and do a paged search of my domain because there's no leak on Microsoft's side for that assembly.
As requested, here's some code to illustrate the solution I came up with. Note that I also use a plugin architecture and appDomain's and I unload the appdomain when I am done with it, though I think given that there's no leak in DirectoryServices.Protocols you don't have to do that. I only did it because I thought using appDomains would solve my problem, but since it wasn't a leak in managed code but in un-managed code, it didn't do any good.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.DirectoryServices.Protocols;
using System.Data.SqlClient;
using System.Data;
using System.Data.Linq;
using System.Data.Linq.Mapping;
using System.Text.RegularExpressions;
using log4net;
using log4net.Config;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
using System.Diagnostics;
using System.IO;
namespace ADImportPlugIn {
public class ADImport : PlugIn
{
private ADDataSet myADUsers = null;
LdapConnection _LDAP = null;
MDBDataContext mdb = null;
private Orgs myOrgs = null;
public override void JustGronkIT()
{
string filter = "(&(objectCategory=person)(objectClass=user)(!(userAccountControl:1.2.840.113556.1.4.803:=2)))";
string tartgetOU = #"yourdomain.com";
string[] attrs = {"sAMAccountName","givenName","sn","initials","description","userPrincipalName","distinguishedName",
"extentionAttribute6","departmentNumber","wwwHomePage","manager","extensionName", "mail","telephoneNumber"};
using (_LDAP = new LdapConnection(Properties.Settings.Default.Domain))
{
myADUsers = new ADDataSet();
myADUsers.ADUsers.MinimumCapacity = 60000;
myADUsers.ADUsers.CaseSensitive = false;
try
{
SearchRequest request = new SearchRequest(tartgetOU, filter, System.DirectoryServices.Protocols.SearchScope.Subtree, attrs);
PageResultRequestControl pageRequest = new PageResultRequestControl(5000);
request.Controls.Add(pageRequest);
SearchOptionsControl searchOptions = new SearchOptionsControl(System.DirectoryServices.Protocols.SearchOption.DomainScope);
request.Controls.Add(searchOptions);
while (true)
{
SearchResponse searchResponse = (SearchResponse)_LDAP.SendRequest(request);
PageResultResponseControl pageResponse = (PageResultResponseControl)searchResponse.Controls[0];
foreach (SearchResultEntry entry in searchResponse.Entries)
{
string _myUserid="";
string _myUPN="";
SearchResultAttributeCollection attributes = entry.Attributes;
foreach (DirectoryAttribute attribute in attributes.Values)
{
if (attribute.Name.Equals("sAMAccountName"))
{
_myUserid = (string)attribute[0] ?? "";
_myUserid.Trim();
}
if (attribute.Name.Equals("userPrincipalName"))
{
_myUPN = (string)attribute[0] ?? "";
_myUPN.Trim();
}
//etc with each datum you return from AD
}//foreach DirectoryAttribute
//do something with all the above info, I put it into a dataset
}//foreach SearchResultEntry
if (pageResponse.Cookie.Length == 0)//check and see if there are more pages
break; //There are no more pages
pageRequest.Cookie = pageResponse.Cookie;
}//while loop
}//try
catch{}
}//using _LDAP
}//JustGronkIT method
}//ADImport class
} //namespace
UserPrincipal implements IDisposable. Try calling Dispose on result inside the foreach loop.
I also found this SO question, but there was no agreement on the answer.
After much frustration and some hints gathered here I came up with a solution. I also discovered an interesting thing about a difference in how using a using block with a DirectoryServices resource vs a DataContext as noted in the code snippet below. I probably don't need to use a Finalizer but I did so anyway just to be safe. I have found that by doing what is outlined below, my memory is stable across runs whereas before I would have to kill the application twice a day to free resources.
using System.DirectoryServices;
using System.DirectoryServices.AccountManagement;
namespace myPlugins
{
public class ADImport : Plugin
{
//I defined these outside my method so I can call a Finalizer before unloading the appDomain
private PrincipalContext AD = null;
private PrincipalSearcher ps = null;
private DirectoryEntry _LDAP = null; //used to get underlying LDAP properties for a user
private MDBDataContext _db = null; //used to connect to a SQL server, also uses unmanaged resources
public override GronkIT()
{
using (AD = new PrincipalContext(ContextType.Domain,"my.domain.com"))
{
UserPrincipal u = new UserPrincipal(AD);
u.Enabled=true;
using(ps = new PrincipalSearcher(u))
{
foreach(UserPrincipal result in ps.FindAll())
{
using (result)
{
_LDAP = (DirectoryEntry)result.GetUnderlyingObject();
//do stuff with result
//do stuff with _LDAP
result.Dispose(); //even though I am using a using block, if I do not explicitly call Dispose, it's never disposed of
_LDAP.Dispose(); //even though I am using a using block, if I do not explicitly call Dispose, it's never disposed of
}
}
}
}
}
public override JustGronkIT()
{
using(_db = new MDBDataContext("myconnectstring"))
{
//do stuff with SQL
//Note that I am using a using block and connections to SQL are properly disposed of when the using block ends
}
}
~ADImport()
{
AD.Dispose(); //This works, does not throw an exception
AD = null;
ps.Dispose(); //This works, does not throw an exception
ps = null;
_LDAP.Dispose(); //This works, does not throw an exception
_LDAP = null;
_db.Dispose(); //This throws an exception saying that you can not call Dispose on an already disposed of object
}
}
}
That code works fine for me. I just dispose every instance. In my project i call this method every two minutes. After i call garbage collector outside.
public class AdUser
{
public string SamAccountName { get; set; }
public string DisplayName { get; set; }
public string Mail { get; set; }
}
public List<AdUser> GetAllUsers()
{
List<AdUser> users = new List<AdUser>();
using (PrincipalContext context = new PrincipalContext(ContextType.Domain, Environment.UserDomainName))
{
using PrincipalSearcher searcher = new PrincipalSearcher(new UserPrincipal(context));
using PrincipalSearchResult<Principal> allResults = searcher.FindAll();
foreach (Principal result in allResults)
{
using DirectoryEntry de = result.GetUnderlyingObject() as DirectoryEntry;
AdUser user = new AdUser()
{
SamAccountName = (string)de.Properties["samAccountName"].Value,
DisplayName = (string)de.Properties["displayName"].Value,
Mail = (string)de.Properties["mail"].Value
};
users.Add(user);
result.Dispose();
}
}
return users;
}
First few iterations of calling method above there are seems to be a memory allocation, but after that it's not leaking.
Consider calling memory clean up after each iteration.
GC.Collect();