The Environment:
Nodes: 5
Replication: 5
Consistency: 1
The code:
using System;
using Cassandra;
namespace CassandraSelectTest
{
class Program
{
static void Main(string[] args)
{
var cluster = Cluster.Builder()
.AddContactPoints("192.168.0.18","192.168.0.21","192.168.0.22","192.168.0.23","192.168.0.24")
.WithPort(9042)
.Build();
var session = cluster.Connect("test_keyspace");
var results = session.Execute("SELECT * FROM test_table");
foreach(var result in results)
{
Console.WriteLine(result.GetValue<string>("col1"));
}
Console.WriteLine($"Finished");
Console.ReadKey();
}
}
}
The problem:
Some of the nodes ideally need to be highly portable, which results in the IP address of the node changing when it is in a different location, and then changing back to its normal IP address when back to its original location. This happens a few times a week.
The question:
Is it possible to configure a single node to have multiple IP addresses, or dynamic addresses which change automatically?
I think that in that scenario the driver will receive a protocol event which will make the driver refresh the cluster's topology. As long as the node's IP is up to date in system.peers table, i.e., you update broadcast_rpc_address / rpc_address / listen_address on the node's cassandra.yml the driver should be able to notice that the old IP is no longer in that table (which will remove the old host) and that there's a new IP (which will add a new host).
If the control connection is not able to reconnect to any of the IPs on the local metadata cache (e.g. if all the nodes go down and change their IP addresses at the same time) then the driver will not be able to reconnect. There's this open ticket which will resolve this issue as long as the user provides hostnames as the contact points (and DNS resolution returns the updated IPs).
Related
I'm building a Kafka consumer for my .net proyect. I'm using kafka-net (A native c# client fro Apache Kafka mady by James Roland).
The problem I have is that this code (based on the documentation) fetches all messages from the beginning by default:
private void StartKafkaConsumer(string ipKafka, string portKafka, string topicKafka)
{
string topic = topicKafka;
Uri uri = new Uri($"http://{ipKafka}:{portKafka}");
var options = new KafkaOptions(uri);
using (var router = new BrokerRouter(options))
{
using (var consumer = new Consumer(new ConsumerOptions(topic, router)))
{
foreach (var message in consumer.Consume())
{
Console.WriteLine(Encoding.UTF8.GetString(message.Value));
}
}
}
}
...
StartKafkaConsumer("localhost", "9092", "test"); //this fetches messages sent weeks ago, since the creation of the 'test' topic
Basically, this code does the same than this command:
.\bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic test --from-beginning
All I want to do is to fetch messages from the time the client gets connected to the Apache server, not from the beginning. I know it's possible since I tried the last command without the "--from-beginning" part and it worked.
Any suggestion would be appreciated.
Look at the ConsumerOptions methods. There should be an option to set a property/config value:
auto.offset.reset
Set the above property/config to latest. At that point, anytime you connect with an unknown/new consumer group id, the consumer will default to start at the latest offset.
If, however, the consumer group id was already known (i.e., had already consumed at least once from that topic/partition), it will attempt to get the last committed offset + 1. If that offset is unavailable, because maybe it has gone past the retention threshold, then it will default to latest.
Here's more detailed documentation:
https://kafka.apache.org/documentation/#newconsumerconfigs
Im playing with using akka.Net to develop a plugin architecture whereby each dll that contains one or more plugins is loaded into its own AppDomain and a new actor system is initialized ready to recieve messages from the "Host".
I become unstuck trying to get this to work with multiple plugins.
So the Host config looks like:
akka {
actor {
provider = ""Akka.Remote.RemoteActorRefProvider, Akka.Remote""
}
remote {
helios.tcp {
transport-class = ""Akka.Remote.Transport.Helios.HeliosTcpTransport, Akka.Remote""
applied-adapters = []
transport-protocol = tcp
port = 50003
hostname = localhost
}
}
}
And the plugin config looks like:
akka {
actor {
provider = ""Akka.Remote.RemoteActorRefProvider, Akka.Remote""
}
remote {
helios.tcp {
transport-class = ""Akka.Remote.Transport.Helios.HeliosTcpTransport, Akka.Remote""
applied-adapters = []
transport-protocol = tcp
port = 50004
hostname = localhost
}
}
(there are many of these)
My question is how do i get messages from the Host to all of the plugins?
The best recommendation is to use Akka.Cluster. Here's a well-documented example: https://github.com/petabridge/akkadotnet-code-samples/tree/master/Cluster.WebCrawler
Edit - removed suggestion to use dynamic port. Much better off using static ones so node reboots can be handled correctly.
Have each plugin config use a plugin-specific port (akka.remote.helios.tcp.port = 1231) and then define a clustered router that talks to actor systems fulfilling specific roles.
/api/broadcaster {
router = broadcast-group
routees.paths = ["user/api"]
cluster {
enabled = on
max-nr-of-instances-per-node = 1
allow-local-routees = on
use-role = crawler
}
}
This router, deployed at the path user/api/broadcaster on some node can communicate (via the Broadcast routing strategy) with any actor deployed at path user/api on any node in the cluster with role crawler without needing to look up IP addresses, ports, or any of that crap.
You configure a node's clustering information via the following section in Akka.NET's config:
cluster {
#manually populate other seed nodes here, i.e. "akka.tcp://lighthouse#127.0.0.1:4053"
seed-nodes = ["akka.tcp://webcrawler#127.0.0.1:4053"]
roles = [crawler]
}
Seed nodes - has to be a well-known, statically-defined port and IP address. Read the article for an explanation on why this is important.
Roles - comma-delimited strings that define what this particular nodes' capabilities are. They're more like tags. You can use them inside clustered routers (like the one I showed earlier) to articulate which types of nodes you want to communicate with.
I know that I can change the computers global proxy setting, Software\\Microsoft\\Windows\\CurrentVersion\\Internet Settings, to affect IE instances created using Watin.
But is there any way to intercept requests made by the IE browsers and run them through a proxy? My goal is to run multiple instances of IE, each with its own proxy, which isn't possible with my current solution above.
WatiN IE creates multiple ProcessIDs (single instance IE creates multiple process ids). in order to overwrite proxy settings just for WatiN by using Fiddler Core we need to get all child process ids which are created by WatiN IE. Helper class can be found here PInvoke: Getting all child handles of window – Svett Ralchev class. And then we check all process ids inside the BeforeRequest event and wait for watin process id to overwrite proxy settings.
private void FiddlerApplication_BeforeRequest(Session sess)
{
//Debug.WriteLine("FiddlerApplication_BeforeRequest: " + sess.LocalProcessID.ToString());
if (WatinIEprocessHolder.ContainsKey(sess.LocalProcessID))
{
//see http://stackoverflow.com/questions/14284256/how-to-manually-set-upstream-proxy-for-fiddler-core
sess["X-OverrideGateway"] = WatinIEprocessHolder[sess.LocalProcessID];
}
}
Working Test Application can be downloaded here http://www.rentanadviser.com/downloads/WatiN-2.1.0.1196.zip
Test results with different anonymous proxy below. (ipaddress=browser.Text)
Process Ids:3852,7852,, Your IP address: 119.46.110.17, Proxy:119.46.110.17:8080
Process Ids:2508,6948,, Your IP address: 178.21.112.27, Proxy:178.21.112.27:3128
Process Ids:1348,1368,, Your IP address: 122.96.59.107, Proxy:122.96.59.107:83
Process Ids:7152,5104,, Your IP address: 136.0.16.217, Proxy:136.0.16.217:3127
Process Ids:4128,3480,, Your IP address: 198.52.199.152, Proxy:198.52.199.152:7808
Process Ids:2036,7844,, Your IP address: 122.96.59.107, Proxy:122.96.59.107:82
Sample code:
private void this_FormClosing(object sender, FormClosingEventArgs e)
{
StopFiddler();
}
private void Form1_Load(object sender, EventArgs e)
{
this.FormClosing += this_FormClosing;
ProxyHolder = new List<string>();
ProxyHolder.Add("119.46.110.17:8080");
ProxyHolder.Add("178.21.112.27:3128");
ProxyHolder.Add("122.96.59.107:83");
ProxyHolder.Add("136.0.16.217:3127");
ProxyHolder.Add("198.52.199.152:7808");
ProxyHolder.Add("122.96.59.107:82");
StartFiddler();
System.Threading.Thread.Sleep(500);
for (var i = 0; i < ProxyHolder.Count; i++)
{
WhatIsMyIpThroughProxy(ProxyHolder[i]);
Application.DoEvents();
System.Threading.Thread.Sleep(500);
}
//WhatIsMyIpThroughProxy();
}
private Dictionary<int, string> WatinIEprocessHolder = new Dictionary<int, string>();
private List<string> ProxyHolder = null;
public void WhatIsMyIpThroughProxy(string ProxyIPandPort)
{
using (var browser = new IE(true))// we should not navigate now. Because we need process ids.
{
WindowHandleInfo ChildHandles = new WindowHandleInfo(browser.hWnd);
foreach (var cHandle in ChildHandles.GetAllChildHandles())
{
int pid = new WatiN.Core.Native.Windows.Window(cHandle).ProcessID;
if (WatinIEprocessHolder.ContainsKey(pid) == false)
WatinIEprocessHolder.Add(pid, ProxyIPandPort);
}
System.Text.StringBuilder processIDs = new System.Text.StringBuilder();
foreach (var k in WatinIEprocessHolder.Keys)
{
processIDs.Append(k.ToString() + ",");
//Debug.WriteLine(string.Format("{0}:{1}", k, WatinIEprocessHolder[k]));
}
//we got the process ids above. Navigate now.
browser.GoTo("http://www.rentanadviser.com/en/common/tools.ashx?action=whatismyip");
browser.WaitForComplete();
WatinIEprocessHolder.Clear();
System.Net.IPAddress ip;
if (System.Net.IPAddress.TryParse(browser.Text, out ip))
{
Debug.WriteLine(string.Format("Process Ids:{0}, Your IP address: {1}, Proxy:{2}", processIDs.ToString(), browser.Text, ProxyIPandPort));
}
else
{
Debug.WriteLine(string.Format("Process Ids:{0}, Your IP address: {1}, Proxy:{2}", processIDs.ToString(), "Failed", ProxyIPandPort));
}
}
}
private void StartFiddler()
{
FiddlerApplication.BeforeRequest += FiddlerApplication_BeforeRequest;
FiddlerApplication.Startup(8888, true, true, true);
}
private void StopFiddler()
{
FiddlerApplication.BeforeRequest -= FiddlerApplication_BeforeRequest;
if (FiddlerApplication.IsStarted())
{
FiddlerApplication.Shutdown();
}
}
private void FiddlerApplication_BeforeRequest(Session sess)
{
//Debug.WriteLine("FiddlerApplication_BeforeRequest: " + sess.LocalProcessID.ToString());
if (WatinIEprocessHolder.ContainsKey(sess.LocalProcessID))
{
//see http://stackoverflow.com/questions/14284256/how-to-manually-set-upstream-proxy-for-fiddler-core
sess["X-OverrideGateway"] = WatinIEprocessHolder[sess.LocalProcessID];
}
}
I've created an app called Process Proxifier which uses FiddlerCore to add proxy settings to the Windows applications dynamically. You can find its full source code here: https://processproxifier.codeplex.com/
Also I should mention that this solution is limited to target processes with system's default "CERN" proxy setting (which is pointed at Fiddler/FiddlerCore).
It's not possible to do that with IE or even with WebBrowser (it's just an instance of IE).
But you can manipute WebBrowser behavior to achieve your desired feature.
It's possible to write your custom WebBrowser which fetched data by sending your custom WebRequest that contain your different proxy.
How to load web browser with web response
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create("http://example.com");
webRequest.Proxy = new WebProxy(host, port);
HttpWebResponse response = (HttpWebResponse)webRequest.GetResponse();
Stream receiveStream = response.GetResponseStream();
WebBrowser webBrowser = new WebBrowser();
webBrowser.DocumentStream = receiveStream;
WebRequest.Proxy
I know you are looking for an alternative solution without using the computers global proxy setting but I thought of adding this here so others who don't have this constraint know about it.
The solution was on your question - The Windows Registry.
It is simple to change the proxy settings globally at runtime, you need to change the registry keys you are interested in using the Microsoft.Win32.Registry class in the Microsoft.Win32 namespace.
You can find MSDN documentation for this here: http://msdn.microsoft.com/en-us/library/microsoft.win32.registry(v=vs.110).aspx
See below an example of how to do this.
RegistryKey myKey = Registry.LocalMachine.OpenSubKey("SOFTWARE\\Your key", true);
myKey.SetValue("My String Value", "Test Value", RegistryValueKind.String);
Now to change proxy settings on the box you need to change or create the right proxy registry keys you can find all the available keys at:
MSDN Documentation - 2.2.1.10 Internet Settings.
Below is a few of the keys you need to set. Each version of IE has their own keys but the ones below are identical to all browsers.
UseProxyServer
REG_DWORD
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ProxyEnable
ProxyServerAndPort
REG_DWORD
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ProxyServer
ProxyOverride
REG_SZ
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ProxyOverride
HTTP1_1ThroughProxy
REG_DWORD
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings\HTTP1_1ThroughProxy
User Specific
Please bear in mind these are Current user registry entries so you may need to set them in the context of the windows identity. Also the simplest way to see what should be the values for these keys is to apply the proxy changes on the Internet Settings dialog and check them on RegEdit.exe.
Create user automatically
This is your saving grace here because you can run your process for watin on a local windows account with the settings that way you dont need to change your own proxy settings.
You can then have one user called WatinUser that the proxy settings are set against you can automate the creation of this user using the System.DirectoryServices.AccountManagement Namespace classses.
See examples here at SO: create local user account
There are products like Proxifier that let you setup rules to route traffic to different proxies based on application names, IP addresses, hostnames and port numbers. This would not let you use different proxies for multiple IE processes, but if those processes were accessing different URLs you could route the traffic through separate proxy servers. Proxifier works using the WinSocks stack, similar to what many antivirus use, and it is transparent to the application layer.
Another suggestion is to write your own web request interceptor/proxy server that will grab proxy server info from requested url and forward normalized url to the real proxy server.
for e.g. from watin you launch url "someurl?ProxyServer=10.10.10.12" now this will be intercepted by your own proxy server and it will use the proxy server param to redirect requested url i.e. "someurl" to 10.10.10.12 your proxy server implementation can set proxy details at run time and fetch the results from your server using dynamic proxy.
I hope it makes some sense.
I think I may be in a situation where the answer is this is not possible but in case not here goes...
I have written an ASP .NET MVC 3 application and I am using the Request.UserHostName property and then passing the value that that returns into Dns.GetHostEntry to find out all the possible IPs and the host name for the currently connected client, for example:
var clientAddress = Request.UserHostName;
var entry = Dns.GetHostEntry(clientAddress);
Generally that is fine except I have a case where I get a "host not found" SocketException from the Dns.GetHostEntry call.
The really odd thing is that the address that is returned from the Request.UserHostName property is not the public address or any of the private addresses. To prove this I ran this bit of code on the client machine in question...
var host = Dns.GetHostEntry(Dns.GetHostName());
foreach (var a in host.Aliases)
{
Console.WriteLine("alias '{0}'", a);
}
foreach (var a in host.AddressList)
{
Console.WriteLine("ip address '{0}'", a);
}
// ...from http://stackoverflow.com/a/2353177/1039947...
String direction = "";
WebRequest request = WebRequest.Create("http://checkip.dyndns.org/");
using (WebResponse response = request.GetResponse())
{
using (var stream = new StreamReader(response.GetResponseStream()))
{
direction = stream.ReadToEnd();
}
}
int first = direction.IndexOf("Address: ") + 9;
int last = direction.LastIndexOf("</body>");
direction = direction.Substring(first, last - first);
Console.WriteLine("Public IP: '{0}'", direction);
It prints three IP addresses (::1, one private and one public) but none of them are the address that is returned from Request.UserHostName.
If I pass in any of the addresses printed from the above test application into the Dns.GetHostEntry method I get a sensible value back.
So, is there any way that I could get from this strange IP address that is not the public nor any of the privates, to one where I could get the host entry for it without an exception (and what is this address)?
By the way, there is no X_FORWARD_FOR header or anything else that I may be able to identify the client with, as far as I can tell, in the HTTP message?
Background to the Question
So it was pointed out (thanks Damien) that if I explained why I am asking this perhaps someone can provide an alternative approach so here is some background...
I have a requirement that the administrator of the application should be allowed to specify in the configuration a single machine that is allowed to view the page - IP address or machine name - I can probably get the machine name requirement removed but even if they specify the IP address in the configuration it will still not match the IP address that is returned from the UserHostName property since they will use the IP address that is returned when they ping the machine name.
My thinking, therefore, was that if I take whatever is sent in the HTTP header and pass that into GetHostEntry then take all the possible results from that (all the IPs and the host name) and see if any of them match the configured value I could say "allow" otherwise "disallow" (I was going to remove the part of the host name before the first dot too, to cover that eventuality). That scheme has been blown out of the water by this situation I have where the IP address is not at all what I would expect.
The host name of the client is not normally known because it is not transmitted at the HTTP level. The server cannot know it. Look at the HTTP requests with Fiddler to see for yourself that there is not a lot of information available to the server (and the client can forge all request contents of course).
Use the UserHostAddress property to get the IP address. That is the most you can reliably find out. Once you have that you can try to reverse the IP to a host name but that is not always possible.
I have a more specific answer to your question. By examining the source code for HttpRequest.UserHostName here, I found that it maps to a IIS server variable named REMOTE_HOST which is described here. The property will return the IP adddress of the client, unless you have configured IIS in the way described, in which case IIS will do a reverse DNS lookup to attempt to return the name associated with the IP.
Make sure you read the Remarks section at Dns.GetHostEntry on the many cases it can (partially) fail:
Remarks
The GetHostEntry method queries a DNS server for the IP
address that is associated with a host name or IP address.
If an empty string is passed as the hostNameOrAddress argument, then this method
returns the IPv4 and IPv6 addresses of the local host.
If the host name could not be found, the SocketException exception is returned
with a value of 11001 (Windows Sockets error WSAHOST_NOT_FOUND). This
exception can be returned if the DNS server does not respond. This
exception can also be returned if the name is not an official host
name or alias, or it cannot be found in the database(s) being queried.
The ArgumentException exception is also returned if the
hostNameOrAddress parameter contains Any or IPv6Any.
The GetHostEntry method assumes that if an IP literal string is passed in the
hostNameOrAddress parameter that the application wants an IPHostEntry
instance returned with all of the properties set. These properties
include the AddressList, Aliases, and HostName. As a result, the
implementation of the GetHostEntry method exhibits the following
behavior when an IP string literal is passed:
The method tries to parse the address. If the hostNameOrAddress parameter contains a legal IP string literal, then the first phase succeeds.
A reverse lookup using the IP address of the IP string literal is attempted to obtain the host name. This result is set as the HostName property.
The host name from this reverse lookup is used again to obtain all the possible
IP addresses associated with the name and set as the AddressList
property.
For an IPv4 string literal, all three steps above may
succeed. But it is possible for a stale DNS record for an IPv4 address
that actually belongs to a different host to be returned. This may
cause step #3 to fail and throw an exception (there is a DNS PTR
record for the IPv4 address, but no DNS A record for the IPv4
address).
For IPv6, step #2 above may fail, since most IPv6
deployments do not register the reverse (PTR) record for an IPv6
address. So this method may return the string IPv6 literal as the
fully-qualified domain (FQDN) host name in the HostName property.
The GetHostAddresses method has different behavior with respect to IP
literals. If step #1 above succeeds (it successfully parses as an IP
address), that address is immediately returned as the result. There is
no attempt at a reverse lookup.
IPv6 addresses are filtered from the results of the GetHostEntry method if the local computer does not have IPv6 installed. As a result, it is possible to get back an empty
IPHostEntry instance if only IPv6 results where available for the
hostNameOrAddress.parameter.
The Aliases property of the IPHostEntry instance returned is not populated by this method and will always be empty.
how can I get the client's computer name in a web application. The user in a network.
Regards
// Already tryed this option
string IP = System.Web.HttpContext.Current.Request.UserHostAddress;
string compName = DetermineCompName(IP);
System.Net.IPHostEntry teste = System.Net.Dns.GetHostEntry(IP);
ssresult = IP + " - " + teste.HostName;
// TODO: Write implementation for action
private static string DetermineCompName(string IP)
{
IPAddress myIP = IPAddress.Parse(IP);
IPHostEntry GetIPHost = Dns.GetHostEntry(myIP);
string[] compName = GetIPHost.HostName.ToString().Split('.');
return compName[0];
}
All of that, gives me only the IP :/
You can't do this in a way that is guaranteed to work.
The closest you will be able to get is to go down the route of doing a reverse dns lookup using System.Net.Dns.GetHostEntry which is what you have already tried.
The problem is that your machine has no way of knowing the hostname of a remote web client via its IP address alone (unless it is on the same subnet, in which case you may be able to retrieve it).
You have to fall back on your DNS infrastructure being able to map the IP back into a hostname [this is what nslookup does when you type in a hostname], and plenty of places just won't bother setting up reverse IP records.
Plus, often if they do, they won't match the hostname. It is quite common to see a reverse lookup for "1.2.3.4" come back as something line "machine-1.2.3.4", instead of the actual hostname.
This problem is exacerbated further if the clients are behind any sort of Network Address Translation so that many client computers have a single IP from an external perspective. This is probably not the case for you since you state "The user in a network".
As an aside, if you go to the server and type in, at a command prompt,
nslookup <some ip>
(where is an example of one of these client machines), do you get a hostname back?
If you do, then System.Net.Dns.GetHostEntry should be able to as well, if not then it probably can't for the reasons mentioned above.
you can get the windows machine name with - System.Environment.MachineName
Assuming your clients are Windows based running IE you could use this client side code to get the names and pass them back to the server:
<script type="text/javascript">
function create()
{
var net = new ActiveXObject("wscript.network");
document.write(net.ComputerName);
}
</script>
Yeah would require you to keep requesting and caching the computers terribly inefficient, was a bad idea.
I would go with running nslookup in the background I haven't tested this code and neither is it handling errors for failures, but basically, you can do:
System.Diagnostics.Process process = new System.Diagnostics.Process();
System.Diagnostics.ProcessStartInfo sInfo = new System.Diagnostics.ProcessStartInfo("nslookup.exe", "192.168.1.100");
string result = process.StandardOutput.ReadToEnd();
Then just parse the result value.