I am new to Apache spark and trying to run this on Windows machine with Mobius.
but i am getting exception while running it.
first of all i have setup following environment variables
JAVA_HOME C:\Program Files\Java\jdk1.8.0_152
HADOOP_HOME C:\winutils
SPARK_HOME c:\spark [spark-2.0.2-bin-hadoop2.6]
SPARKCLR_HOME E:\SparkClr\spark-clr_2.11-2.0.200\runtime
In code provided correct file to read data from as file:///E:/Mobius/Mobius/SampleText.txt
I tried to run it in debug mode
E:\SparkClr\spark-clr_2.11-2.0.200\runtime\scripts>sparkclr-submit.cmd debug
[sparkclr-submit.cmd] SPARKCLR_JAR=spark-clr_2.11-2.0.200.jar
[sparkclr-submit.cmd] LAUNCH_CLASSPATH="E:\SparkClr\spark-clr_2.11-2.0.200\runtime\lib\spark-clr_2.11-2.0.200.jar;c:\spark\jars\*"
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
17/12/19 11:34:23 INFO CSharpRunner: Starting CSharpBackend!
17/12/19 11:34:25 INFO CSharpRunner: Port number used by CSharpBackend is 5567
***********************************************************************
* [CSharpRunner.main] Backend running debug mode. Press enter to exit *
***********************************************************************
17/12/19 11:35:09 INFO SparkContext: Running Spark version 2.0.2
In my App.config Added following Entry [Port name with i got from above]
<add key="CSharpBackendPortNumber" value="5567"/>
I am getting below Exception message
JVM method execution failed: Nonstatic method partitions failed for class 12 when called with no parameters
Stack Trace
at Microsoft.Spark.CSharp.Interop.Ipc.JvmBridge.CallJavaMethod(Boolean isStatic, Object classNameOrJvmObjectReference, String methodName, Object[] parameters)
at Microsoft.Spark.CSharp.Interop.Ipc.JvmBridge.CallNonStaticJavaMethod(JvmObjectReference objectId, String methodName)
at Microsoft.Spark.CSharp.Proxy.Ipc.RDDIpcProxy.GetNumPartitions()
at Microsoft.Spark.CSharp.Core.RDD`1.GetDefaultPartitionNum()
at Microsoft.Spark.CSharp.Core.PairRDDFunctions.PartitionBy[K,V](RDD`1 self, Int32 numPartitions, Func`2 partitionFunc)
at Microsoft.Spark.CSharp.Core.PairRDDFunctions.ReduceByKey[K,V](RDD`1 self, Func`3 reduceFunc, Int32 numPartitions)
at Microsoft.Spark.CSharp.Examples.WordCountExample.Main(String[] args) in E:\Mobius\Mobius\examples\Batch\WordCount\Program.cs:line 35
I am not sure about CSharpWorkerPath.
Please help if anyone face this kind of issue earlier.
Related
We have the IBM.Data.DB2.Core (3.1.0.300) nuget package installed in our .netstandard2.1 application library, to connect to an external DB2 instance on prem.
On running the following code from a .net core azure function v3 Windows 64bit (from a netcoreapp3.1 project)
var connectionString= "Server=DB2_IP:DB2_Port;Database=DB2_DBNAME;UID=DB2_UserId;PWD=DB2_Password;"
var connection = new DB2Connection(connectionString);
(note the values in the connection string are placeholders for anonymity)
We are receiving the following error
IBM.Data.DB2.Core.DB2Exception (0x80004005): ERROR [58005] [IBM][DB2.NET] SQL0902 An unexpected
exception has occurred in
Process: 21308 Thread 33 AppDomain: Name:Microsoft.Azure.WebJobs.Script.WebHost
There are no context policies. Function: AESEncryptADONET (Encryption Info)
CallStack: at System.Environment.get_StackTrace()
at IBM.Data.DB2.Core.DB2ConnPool.HandleUnknownErrors(String strFncMsg, Exception exception, Boolean bThrow)
at IBM.Data.DB2.Core.DB2ConnPool.EncryptString(String value)
at IBM.Data.DB2.Core.DB2ConnPool.ReplaceConnectionStringParms(DB2Connection connection, String szValue, DB2ConnSettings& pSettings, DB2ConnSettingsInternal& pSettingsInternal, Boolean bAttach, Boolean pushDownStrAppended)
at IBM.Data.DB2.Core.DB2Connection.set_ConnectionString(String value)
at IBM.Data.DB2.Core.DB2Connection..ctor(String connectionString)
Does anyone have any idea where we're going wrong, or alternatively have a coherent end-to-end guide on the setup that may give me a clue.
All documentations I have found thus far has been rather sparse and a little inconsistent.
This is my first stackoverflow question. I already searched for some answers on this site, but all the solutions I found didn't worked for me, or they were meant for Java or javascript.
The situation
As a junior automation tester I created a sort of Automation Test Framework in C# with the Selenium Webdriver and Protractor.Net.
The Framework consists of 2 parts.
The page objects
The tests
There is also a page object base class. In this class I defined methods that are used by every page object, such as a method for opening a specific program module.
In the tests, there is also a base class. In this class the Test attributes (C# UnitTest) are defined, such as TestInitialize, TestCleanup, AssemblyCleanup.
In the Test Base class I also defined the NgWebDriver, ChromeDriver:
driver = new NgWebDriver(new OpenQA.Selenium.Chrome.ChromeDriver());
driver.Manage().Window.Maximize();
In the Page Object Base class I defined a method to open a specific part of the
website:
public void OpenSPA(string spa)
{
driver.Navigate().GoToUrl(spa, true);
driver.Navigate().Refresh();
}
The pararmeter spa contains the url to navigate to.
The Testcase
The testcase is just simple:
open module x of the website
module x is just an url
The complete code of the testcase:
[TestMethod]
[TestCategory(cat), TestCategory(regression)]
public void Organisatie_CheckActief()
{
bool is_actief;
var organisatie = new CrmSPA(driver);
organisatie.VulVeld("Vrij zoeken", "XXXX");
is_actief = organisatie.IsRelatieActief();
//Assertion
Assert.IsTrue(is_actief == true, "Relatie is niet actief.");
The problem:
Since a couple of weeks ago I get the following error when running the testcase:
Test Name: Organisatie_CheckActief
Test FullName: FirstTest.Webdrivers.WijzigenOrganisatie_Basis.Organisatie_CheckActief
Test Source: C:\Users\XXXX\TestCases\\Organisatie\WijzigenOrganisatie_basis.cs : line 168
Test Outcome: Failed
Test Duration: 0:00:41,1353784
Result StackTrace:
bij OpenQA.Selenium.Remote.RemoteWebDriver.UnpackAndThrowOnError(Response errorResponse)
bij OpenQA.Selenium.Remote.RemoteWebDriver.Execute(String driverCommandToExecute, Dictionary`2 parameters)
bij OpenQA.Selenium.Remote.RemoteWebDriver.ExecuteScriptCommand(String script, String commandName, Object[] args)
bij OpenQA.Selenium.Remote.RemoteWebDriver.ExecuteAsyncScript(String script, Object[] args)
bij Protractor.NgWebDriver.set_Url(String value) in c:\Users\Bruno\Projets\GitHub\bbaia\protractor-net\src\Protractor\NgWebDriver.cs:regel 163
--- End of inner exception stack trace ---
bij Protractor.NgWebDriver.set_Url(String value) in c:\Users\Bruno\Projets\GitHub\bbaia\protractor-net\src\Protractor\NgWebDriver.cs:regel 193
bij Protractor.NgNavigation.GoToUrl(String url, Boolean ensureAngularApp) in c:\Users\Bruno\Projets\GitHub\bbaia\protractor-net\src\Protractor\NgNavigation.cs:regel 112
bij FirstTest.PageObjects.BaseSPA.OpenSPA(String spa) in C:\Users\\Documents\Visual Studio 2015\Projects\\\FirstTest\PageObjects\BaseSPA.cs:regel 55
bij FirstTest.PageObjects.CrmSPA.CrmSPA..ctor(NgWebDriver driver) in C:\Users\\Documents\Visual Studio 2015\Projects\\\FirstTest\PageObjects\CRM\CrmSPA\CrmSPA.cs:regel 37
bij FirstTest.Webdrivers.WijzigenOrganisatie_Basis.Organisatie_CheckActief() in C:\Users\\Documents\Visual Studio 2015\Projects\\\FirstTest\TestCases\RelatieSPA\Organisatie\WijzigenOrganisatie_basis.cs:regel 174
Result Message:
Test method FirstTest.Webdrivers.WijzigenOrganisatie_Basis.Organisatie_CheckActief threw exception:
System.InvalidOperationException: **Angular could not be found on the page** 'https://XXXXX/CRM/Relaties/ ---> OpenQA.Selenium.WebDriverTimeoutException: asynchronous script timeout: result was not received in 30 seconds
(Session info: chrome=66.0.3359.139)
(Driver info: chromedriver=2.38.552522 (437e6fbedfa8762dec75e2c5b3ddb86763dc9dcb),platform=Windows NT 6.1.7601 SP1 x86_64)
The Result on screen/browser
When running the test the following happens:
the chrome browser starts (OK)
the chrome browser opens maximized (ok)
the chrome browser gets the url (ok)
the browser is empty shows blank page (NOK)
somehow i found the following error:
angular never provided resumeBootstrap
I searched on google and stackoverflow and i tried some answers like the synchronize option:
driver.IgnoreSynchronization
the problem does not always shows up. sometimes the page is loaded and the test continues.
I already tried to reinstall protractor-net, a newer chromedriver and seleniumdriver. The problem still exist.
If anyone can help me with this.
This error message...
OpenQA.Selenium.Remote.RemoteWebDriver.UnpackAndThrowOnError(Response errorResponse)
...implies that the RemoteWebDriver incurred an error while executing a command.
Selenium being an open source framework, you can have a look at the source code here.
GoToUrl() is defined in RemoteNavigator.cs and accepts only one argument string url as follows :
/// Navigate to a url for your test
/// </summary>
/// <param name="url">String of where you want the browser to go to</param>
public void GoToUrl(string url)
{
this.driver.Url = url;
}
But in your code block while invoking GoToUrl() method along with the string spa you have also passed true. This possibly raises the error.
Solution
Within GoToUrl() method removing the argument true will solve your issue.
I am currently experiencing an issues where a handful of queries to stored procedures hooked up via Telerik Open Access (v2015.1.87.3777) are returning the following error:
Exception: System.ArgumentNullException: Value cannot be null. Parameter name: converterName.
The error occurs at the point when the stored procedure is executed in the dynamically generated embedded .cs file of the .rlinq designer. E.g.,
IEnumerable<TestResult> queryResult = this.ExecuteQuery<TestResult>("[TestSchema].[spS_GetTestResult]", CommandType.StoredProcedure, parameterContextID, parameterReturnValue);
However, this is only happening on certain environments and not others, with seemingly the exact same config. This only started happening last night with no code changes. Prior to yesterday everything was working fine across the board.
I can reproduce the issue locally, and if I run the solution locally and point to a database server that I know works everything is fine. But if I change the connection string and point to a different server that I know doesn’t work I get the error. So the underlying code is exactly the same (given I am running it locally) with the only thing changing is the connection string. And it’s calling identical stored procedures – in fact the entire database is a clone of another that works.
I tried rolling back to previous versions of Telerik Open Access, which has had no effect. I have absolutely no idea what could’ve caused this, and no explanation why this is has surfaced, and in particular why it works on some environments, not others.
Here is the exception details.
System.ArgumentNullException was unhandled by user code
HResult=-2147467261
Message=Value cannot be null. Parameter name: converterName
Source=Telerik.OpenAccess.Runtime
ParamName=converterName
StackTrace:
at OpenAccessRuntime.Relational.metadata.RelationalMappingResolver.ResolveTypeConverterName(TypeConverterName
converterName)
at OpenAccessRuntime.Relational.metadata.RelationalMappingResolver.CreateConverter(TypeConverterName
converterName)
at OpenAccessRuntime.DataReaderTypeConverterImpl.Setup(Int32 offset, Type targetType)
at OpenAccessRuntime.DataReaderTypeConverterImpl.Convert(Type targetType, Int32 offset)
at Telerik.OpenAccess.Data.Common.OADataReader.GetConvertedValue[T](Int32
columnIndex)
at Telerik.OpenAccess.Data.Common.PropertyAccessor.Impl`2.Set(Object
inst, OADataReader reader, Int32 columnIndex)
at Telerik.OpenAccess.Data.Common.NonPersistentCapableMaterializer 1.d__0.MoveNext()
at Telerik.OpenAccess.RT.TypedEnumerator 1.MoveNext()
at System.Collections.Generic.List 1..ctor(IEnumerable 1 collection)
at Telerik.OpenAccess.OpenAccessContextBase.ExecuteQuery[T](String
commandText, CommandType commandType, DbParameter[] parameters)
at TestProject.ORM.PresentationData.GetTestResult(Nullable 1 contextID, Int32& returnValue) in
c:\Projects...\DataModels\PresentationData1.cs:line 1789
at TestProject.ORM.PresentationData.GetTestResult(Nullable 1 contextID) in c:\Projects...\DataModels\PresentationData1.cs:line 1768
at TestProject.ORM.DataHelpers.Presentation.GetTestResult(Int32 contextId) in c:\Projects...\DataHelpers\Presentation.cs:line 229
at TestProject.DataServices.PresentationService.GetTestResult(Int32
contextId) in c:\Projects...\PresentationService.svc.cs:line 404
at SyncInvokeGetTestResult(Object , Object[] , Object[] )
at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object
instance, Object[] inputs, Object[]& outputs)
at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc&
rpc)
InnerException:
This was due to security updates that were rolled out on said servers. Rolling these patches back resolved the issue: The two security updates were:
KB3194719 (SQL)
KB3197873 (Windows)
I am trying to clone a git repository from the local file system:
using System;
using LibGit2Sharp;
class Program
{
static void Main()
{
var sourceUrl = #"file:///c:/work/libgit2sharp";
using (Repository.Clone(sourceUrl, "targetDir", bare: true))
{
Console.WriteLine("repository successfully cloned");
}
}
}
and I get an exception:
Unhandled Exception: LibGit2Sharp.LibGit2SharpException: An error was raised by libgit2. Category = Odb (Error).
Failed to find the memory window file to deregister
at LibGit2Sharp.Core.Ensure.Success(Int32 result, Boolean allowPositiveResult) in c:\work\libgit2sharp\LibGit2Sharp\Core\Ensure.cs:line 85
at LibGit2Sharp.Core.Proxy.git_clone_bare(String url, String workdir, git_transfer_progress_callback transfer_cb) in c:\work\libgit2sharp\LibGit2Sharp\Core\Proxy.cs:line 219
at LibGit2Sharp.Repository.Clone(String sourceUrl, String workdirPath, Boolean bare, Boolean checkout, TransferProgressHandler onTransferProgress, CheckoutProgressHandler onCheckoutProgress) in c:\work\libgit2sharp\LibGit2Sharp\Repository.cs:line 431
at Program.Main() in c:\work\ConsoleApplication1\Program.cs:line 10
I've also tried the following source url:
var sourceUrl = #"c:\work\libgit2sharp\.git\";
and got another exception:
Unhandled Exception: LibGit2Sharp.LibGit2SharpException: An error was raised by libgit2. Category = Config (Error).
Failed to parse config file: Unexpected end of file while parsing multine var (in c:/work/ConsoleApplication1/bin/Debug/targetDir/config:23, column 0)
at LibGit2Sharp.Core.Ensure.Success(Int32 result, Boolean allowPositiveResult) in c:\work\libgit2sharp\LibGit2Sharp\Core\Ensure.cs:line 85
at LibGit2Sharp.Core.Proxy.git_clone_bare(String url, String workdir, git_transfer_progress_callback transfer_cb) in c:\work\libgit2sharp\LibGit2Sharp\Core\Proxy.cs:line 219
at LibGit2Sharp.Repository.Clone(String sourceUrl, String workdirPath, Boolean bare, Boolean checkout, TransferProgressHandler onTransferProgress, CheckoutProgressHandler onCheckoutProgress) in c:\work\libgit2sharp\LibGit2Sharp\Repository.cs:line 431
at Program.Main() in c:\work\ConsoleApplication1\Program.cs:line 12
targetDir is never created.
If on the other hand I use HTTP transport, the Repository.Clone method works fine:
var sourceUrl = "https://github.com/libgit2/libgit2sharp";
So my question is if I am doing something wrong or if this is unsupported feature or a bug in the native git2.dll?
So my question is if I am doing something wrong or if this is unsupported feature or a bug in the native git2.dll?
A bit a both, actually.
The first exception is clearly a bug. This should not happen and will be troubleshot.
The second one requires a deeper analysis. Would you be so kind as to open a issue in the LibGit2Sharp project?
Good news are that a pull request from BenStraub was recently merged. This pull request implements the local fetch transport which should pretty solve the issue.
LibGit2Sharp will be updated in the following days with a new a new version of libgit2 binaries which should allow you perform a local clone/fetch. I'll update this answer as soon as it's been done.
Update
This test shows how do to a Clone and a Push over against a local repository.
I built a WPF application on my m/c and passed on the executable to a client. The application works perfectly well on my machine but throws an error on the client machine. My Logger picked up the following.
Message :Object reference not set to an instance of an object.
Source :App.Feeds
Stack Trace : at App.Feeds.Port
<AdjustWeights>b__1(KeyValuePair`2 item) in
C:\App\1.0.1\App.Feeds\Port\IPort.cs:line 42
at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
at System.Linq.Enumerable.Sum(IEnumerable`1 source)
at System.Linq.Enumerable.Sum[TSource](IEnumerable`1 source, Func`2 selector)
at App.Feeds.Port.Port.AdjustWeights() in
C:\App\1.0.1\App.Feeds\Port\IPort.cs:line 42
at App.Feeds.Port.Port.BuildPort(String
fileName) in C:\App\1.0.1\App.Feeds\Port\IPort.cs:line 153
The strange thing I noticed is that the logger refers to a path C:\App\1.0.1\App.Feeds\Port\IPort.cs I use on the development machine. Is there something wrong with that?
What could be going wrong?
I'm guessing the Where clause is returning null and when you do a sum on that, it breaks. May be you need to check if the result of the where clause has any items before you do the sum.