After scouring the web I am completely stuck on an application I am building to push directories up to Amazon S3 using C# (Targeting .NET 4.5). I am getting NullReferenceExceptions on the line of code that pushes the directory files using the UploadDirectory(TransferUtilityUploadDirectoryRequest) method of the TransferManager class. The problem is I cannot find anything that is null! The debugger doesn't show anything null either, so I'm obviously missing something here.
I read up that if you are uploading to buckets that have periods in them, you need to change the protocol to HTTP otherwise a NullReferenceException might be thrown, however I've done this as well and am continuing to receive the error, even when I created another bucket for testing that has no periods in it.
The portion of my code up to & including the line that causes the exception is below. The class called S3Info is just a helper class that I created that just stores some configuration info such as access/secret keys and other info:
public static void uploadDirectories(S3Info info, List<DirectoryInfo> dirs, Logger logger = null)
{
AmazonS3Config alterConfig = new AmazonS3Config();
alterConfig.CommunicationProtocol = Protocol.HTTP;
AmazonS3Client s3Client = new AmazonS3Client(info.getCredentials(), alterConfig);
TransferUtility directoryTransferUtil = new TransferUtility(s3Client);
TransferUtilityUploadDirectoryRequest uploadDirRequest;
PutObjectRequest completeFileUploadRequest;
uint uploadSuccessCount = 0;
if (dirs == null || dirs.Count == 0)
{
logger.log("Nothing to upload.");
return;
}
//upload directory with PDFs
foreach (DirectoryInfo dir in dirs)
{
try
{
//configure upload request
uploadDirRequest = new TransferUtilityUploadDirectoryRequest();
uploadDirRequest.BucketName = info.selectedBucket.BucketName;
uploadDirRequest.Directory = dir.FullName;
uploadDirRequest.KeyPrefix = dir.Name + #"\";
uploadDirRequest.SearchOption = SearchOption.TopDirectoryOnly;
uploadDirRequest.SearchPattern = "*.pdf";
uploadDirRequest.Timeout = 600000; //10 minutes
//upload directory!
directoryTransferUtil.UploadDirectory(uploadDirRequest); //exception thrown here
I'm a bit stuck at this point so I'm open to any suggestions the community can provide. Thanks.
EDIT: Stack Trace-
Object reference not set to an instance of an object. :
at Amazon.S3.Transfer.Internal.UploadDirectoryCommand.Execute()
at Amazon.S3.Transfer.TransferUtility.UploadDirectory(TransferUtilityUploadDirectoryRequest request)
at S3Delivery.AmazonActions.uploadDirectories(S3Info info, List`1 dirs, Logger logger) in c:\Users\jblacker\Documents\Visual Studio 2012\Projects\S3Delivery\S3Delivery\AmazonActions.cs:line 173
Line 173 is the line referred to above.
A patched version of the SDK (version 1.5.30.1) was released earlier today that fixes this issue.
I posted this same question on the AWS Forums. Apparently the API was broken for the method: UploadDirectory() in version 1.5.30 of the .NET SDK.
Amazon has just posted a patch as version 1.5.30.1
Related
I am trying to test some methods in my library using Google API. More specifically the Cloud Vision API. When I reference the library in LINQPad I get an error
FileNotFoundException: Error loading native library. Not found in any of the possible locations: C:\Users\\AppData\Local\Temp\LINQPad5_dgzgzeqb\shadow_fxuunf\grpc_csharp_ext.x86.dll,C:\Users\\AppData\Local\Temp\LINQPad5_dgzgzeqb\shadow_fxuunf\runtimes/win/native\grpc_csharp_ext.x86.dll,C:\Users\\AppData\Local\Temp\LINQPad5_dgzgzeqb\shadow_fxuunf../..\runtimes/win/native\grpc_csharp_ext.x86.dll
I have tried copying the dll into all of these locations as well as my LINQPad Plugins and the LINQPad folder. I have tried clearing Cancel and Clear query thinking I needed to reset it. I have also closed and reopened LINQPad thinking maybe it re-scans the directory on load. None of this has worked. Has LINQPad changed where to put dlls or am I missing something?
I am using Google.Cloud.Vision.V1
`
var file = new byte[128];
var _settingsCon = new SettingConnector();
var apiKey = Task.Run(() => _settingsCon.Get("Google:Key")).Result.Value;
var credential = Google.Apis.Auth.OAuth2.GoogleCredential.FromJson(apiKey);
var channel = new Grpc.Core.Channel(
ImageAnnotatorClient.DefaultEndpoint.ToString(),
credential.ToChannelCredentials());
var builder = new StringBuilder();
var image = Image.FromBytes(file);
var client = ImageAnnotatorClient.Create(channel);
var response = client.DetectDocumentText(image);
foreach (var page in response.Pages)
{
foreach (var block in page.Blocks)
{
foreach (var paragraph in block.Paragraphs)
{
builder.Append(paragraph);
}
}
}
builder.ToString().Dump();`
This is essentially the function. The file is a dummy file that would be passed in. It shouldn't matter cause it can't make the request any way. The Dump is used instead of return.
I'm using Linq to Excel library for reading excel tables. Until now, it was working good locally, the method ExcelQueryFactory gets the route of the excel by this way:
var book = new ExcelQueryFactory(#"C:\data.xls");
Now, I would like to use it online on a Rest Api, the POST used for uploading the Excel to the web api is the following:
[HttpPost]
[Route("Upload")]
public Task<HttpResponseMessage> UploadFile() {
List<string> savedFilePath = new List<string>();
if (!Request.Content.IsMimeMultipartContent())
{
throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
}
string rootPath = HttpContext.Current.Server.MapPath("~/UploadedFiles");
var provider = new MultipartFileStreamProvider(rootPath);
var task = Request.Content.ReadAsMultipartAsync(provider).
ContinueWith<HttpResponseMessage>(t =>
{
if (t.IsCanceled || t.IsFaulted)
{
Request.CreateErrorResponse(HttpStatusCode.InternalServerError, t.Exception);
}
foreach (MultipartFileData item in provider.FileData)
{
try
{
string name = item.Headers.ContentDisposition.FileName.Replace("\"", "");
string newFileName = Guid.NewGuid() + Path.GetExtension(name);
Debug.WriteLine(item.LocalFileName);
File.Move(item.LocalFileName, Path.Combine(rootPath, newFileName));
Uri baseuri = new Uri(Request.RequestUri.AbsoluteUri.Replace(Request.RequestUri.PathAndQuery, string.Empty));
//RELATIVE PATH
string fileRelativePath = "~/UploadedFiles/" + newFileName;
//LeerExcel(fileRelativePath);
//ABSOLUTE PATH
Uri fileFullPath = new Uri(baseuri, VirtualPathUtility.ToAbsolute(fileRelativePath));
savedFilePath.Add(fileFullPath.ToString());
//LeerExcel(savedFilePath[0]);
}
catch (Exception ex)
{
string message = ex.Message;
}
}
// string rutaFin = "~" + savedFilePath[0];
// string rest = rutaFin.Replace("http://localhost:56618", "");
// LeerExcel(rest);
return Request.CreateResponse(HttpStatusCode.Created, savedFilePath);
});
return task;
}
So, by choosing the excel manually, neither the absolute path or relative path on the server work for the ExcelQueryFactory string route.
The routes get by this method are the following:
ABSOLUTE:
http://localhost:56618/UploadedFiles/9a27e785-e486-4807-8a80-7abb9b940d8b.xls
And the relative:
/UploadedFiles/9a27e785-e486-4807-8a80-7abb9b940d8b.xls
Is possible to use by the way I want to? During the server is online, the obtained absolute path is accesible, so if I access to that URL, the file is downloaded.
the problem is solved:
As I said, I thought only this library worked locally, but it wasn't.
Since Microsoft has launched a new big update, many people have notice some problems when using Database engine, “Unexpected error from external database driver (1). (Microsoft JET Database Engine)” after applying October security updates.
First, I created a Fake.txt file on /UploadedFiles, folder that is located on the project repository and I give it permission to be always copied, as follows:
Fake.txt properties
With this file, I´m achieving that UploadedFiles folder is copied every time I run the server.
Next step:
Due to Microsoft big update, the recomendation is to "download and install the Microsoft Access Database Engine 2010 Redistributable, and then modify the DB connection strings in Microsoft Excel to use ACE as a provider. Example: Change Provider=Microsoft.Jet.OLEDB.4.0 to Provider=Microsoft.ACE.OLEDB.12.0."
I only have downloaded and install that file, but during this day, 3 new windows updates were installed, so, I dont know if this 3 updates are related with the solution of this problem.
The updates are:
Update 1
Update 2
Update 3
After installing the 2010 database engine version file, I changed the excel extension from .xls to .xlsx and now all work.
I am trying to copy a blob from one location to another and it seems like this method is obsolete. Everything I've read says I should use "StartCopy". However, when I try this it doesn't copy the blob. I just get a 404 error at the destination.
I don't seem to be able to find any documentation for this. Can anyone advise me on how to do this in the latest version of the API or point me in the direction of some docs.
Uri uploadUri = new Uri(destinationLocator.Path);
string assetContainerName = uploadUri.Segments[1];
CloudBlobContainer assetContainer =
cloudBlobClient.GetContainerReference(assetContainerName);
string fileName = HttpUtility.UrlDecode(Path.GetFileName(model.BlockBlob.Uri.AbsoluteUri));
var sourceCloudBlob = mediaBlobContainer.GetBlockBlobReference(fileName);
sourceCloudBlob.FetchAttributes();
if (sourceCloudBlob.Properties.Length > 0)
{
IAssetFile assetFile = asset.AssetFiles.Create(fileName);
var destinationBlob = assetContainer.GetBlockBlobReference(fileName);
destinationBlob.DeleteIfExists();
destinationBlob.StartCopyFromBlob(sourceCloudBlob);
destinationBlob.FetchAttributes();
if (sourceCloudBlob.Properties.Length != destinationBlob.Properties.Length)
model.UploadStatusMessage += "Failed to copy as Media Asset!";
}
I'm just posting my comment as the answer to make it easier to see.
It wasn't the access level of the container. It wasn't anything to do with StartCopy either. It turned out to be these lines of code.
var mediaBlobContainer = cloudBlobClient.GetContainerReference(cloudBlobClient.BaseUri + "temporarymedia");
mediaBlobContainer.CreateIfNotExists();
Apparently I shouldn't be supplying the cloudBlobClient.BaseUri, just the name temporarymedia.
var mediaBlobContainer = cloudBlobClient.GetContainerReference("temporarymedia");
There was no relevant error message though. Hopefully it'll save another Azure newbie some time in future.
I hope this isn't too vague. I am writing an app to run at post-commit hook on our SVN server. When I request the repository URI, I sometimes get an answer that doesn't match the path of my commit.
Here's how I get the commit args in general (edited to add more detail):
public Collection<SvnLogEventArgs> GetCommitArgs(string strUri, long lngRevision)
{
try
{
using (SvnClient client = new SvnClient())
{
SvnLogArgs args = new SvnLogArgs();
Collection<SvnLogEventArgs> col;
args.Start = lngRevision;
args.End = lngRevision;
bool bolGotLog = client.GetLog(new Uri(strUri), args, out col);
return col;
}
}
catch
{
return null;
}
}
But here's how I'm getting the repo URI (GetRepository() basically just reformats it to look like a URL):
colCommitArgs = GetCommitArgs(args[0], long.Parse(args[1]));
strRepository = GetRepository(args[0] + "/" + colCommitArgs[0].ChangedPaths[0].RepositoryPath.ToString());
The args[0] is actually referring to the set of args passed from the commit. I found that ChangedPaths is sometimes empty.
For example, if I commit to C:/Repositories/a_repo/somefolder/example.txt, the value I get back from SVN commit args is just C:/Repositories/a_repo/. This may be a problem with our SVN setup (or my lack of understanding thereof). Why are some folders considered repositories and others just considered folders? Is there a way to specify that a folder isn't just a folder? Or is there a different way in SharpSVN to get the committed-to folder? I'm currently concatenating ChangedPaths[0].RepositoryPath at the end of my original URI.
I don't know much about the C# library that you're using but from a generic Subversion standpoint, you generally need to use two different revision numbers for viewing changes. Try using args.Start = lngRevision - 1; and see what happens.
var speechEngine = new SpVoiceClass();
SetVoice(speechEngine, job.Voice);
var fileMode = SpeechStreamFileMode.SSFMCreateForWrite;
var fileStream = new SpFileStream();
try
{
fileStream.Open(filePath, fileMode, false);
speechEngine.AudioOutputStream = fileStream;
speechEngine.Speak(job.Script, SpeechVoiceSpeakFlags.SVSFPurgeBeforeSpeak | SpeechVoiceSpeakFlags.SVSFDefault); //TODO: Change to XML
//Wait for 15 minutes only
speechEngine.WaitUntilDone((uint)new TimeSpan(0, 15, 0).TotalMilliseconds);
}
finally
{
fileStream.Close();
}
This exact code works in a WinForm app, but when I run it inside a webservice I get the following
System.Runtime.InteropServices.COMException was unhandled
Message="Exception from HRESULT: 0x80045003"
Source="Interop.SpeechLib"
ErrorCode=-2147201021
Does anyone have any ideas what might be causing this error? The error code means
SPERR_UNSUPPORTED_FORMAT
For completeness here is the SetVoice method
void SetVoice(SpVoiceClass speechEngine, string voiceName)
{
var voices = speechEngine.GetVoices(null, null);
for (int index = 0; index < voices.Count; index++)
{
var currentToken = (SpObjectToken)voices.Item(index);
if (currentToken.GetDescription(0) == voiceName)
{
speechEngine.SetVoice((ISpObjectToken)currentToken);
return;
}
}
throw new Exception("Voice not found: " + voiceName);
}
I have given full access to USERS on the folder C:\Temp where the file is to be written. Any help would be appreciated!
I don't think the System.Speech works in windows service. It looks like there is a dependency to Shell, which isn't available to services. Try interop with SAPI's C++ interfaces. Some class in System.Runtime.InteropServices may help on that.
Our naming convention requires us to use a non-standard file extension. This works fine in a Winforms app, but failed on our web server. Changing the file extension back to .wav solved this error for us.
Make sure you explicitly set the format on the SPFileStream object. ISpAudio::SetState (which gets called in a lower layer from speechEngine.Speak) will return SPERR_UNSUPPORTED_FORMAT if the format isn't supported.
I just got the webservice to spawn a console app to do the processing. PITA :-)