Develop Reference

Unable to deserialize the body bytes into object after upgrade the RabbitMQ Client from 5.x.x to 6.x.x

I am using RabbitMQ.Client for messaging and using dead letter pattern to implement retry mechanism, everything works fine with RabbitMQ.Client with version :5.1.1 .
The pseudo-code is below：
public async Task Handle(BasicDeliverEventArgs basicDeliverEventArgs)
{
try
{
var bodyBytes = basicDeliverEventArgs.Body.ToArray();
var bodyString = Encoding.UTF8.GetString(bodyBytes);
// extension method with newtonsoft.json behind
bodyString.TryDeserializeObject<T>(out var message);
// do somesting
}
catch (Exception ex)
{
var properties = basicDeliverEventArgs.BasicProperties;
if (properties.Headers == null)
{
properties.Headers = new Dictionary<string, object>();
}
var bodyBytes = basicDeliverEventArgs.Body.ToArray();
var bodyString = Encoding.UTF8.GetString(bodyBytes);
// retrive the x-death-count by header
var retryTimes = GetRetryTimes(properties.Headers);
if (retryTimes <= _retryConfiguration.MaxCount)
{
using var channel = _conn.CreateModel();
var delay = _retryConfiguration.GetNextDelay(retryTimes);
properties.Expiration = delay.TotalMilliseconds.ToString();
string deadLetterExchangeName = $"ErrorExchange_{_rabbitSubscriptionConfiguration.Queue}";
string deadLetterQueueName = $"{_rabbitSubscriptionConfiguration.Queue}.Deadletter";
string deadLetterRoutingKey = $"{_rabbitSubscriptionConfiguration.RoutingKey}.Deadletter";
channel.ExchangeDeclare(deadLetterExchangeName, "direct", true, true);
channel.QueueDeclare(deadLetterQueueName, true, false, false, new Dictionary<string, object>()
{
{ "x-dead-letter-exchange", deadLetterExchangeName },
{ "x-dead-letter-routing-key",_rabbitSubscriptionConfiguration.RoutingKey}
});
channel.QueueBind(_rabbitSubscriptionConfiguration.Queue, deadLetterExchangeName, _rabbitSubscriptionConfiguration.RoutingKey);
channel.QueueBind(deadLetterQueueName, deadLetterExchangeName, deadLetterRoutingKey);
channel.BasicPublish(deadLetterExchangeName, deadLetterRoutingKey, false, properties, bodyBytes);
}
else
{
var messageStorage = _serviceProvider.GetService<IMessageStorage>();
await messageStorage.StoreDeadLetter(bodyString, ex);
}
}
}
However, when I upgrade the RabbitMQ.Client to 6.2.4,there is some probability that the body bytes of message unable deserialize to origin object when I receiving the message again from dead letter queue.
When the retry ended,I inspect the dead letter in the database,some of them failed to convert to a normal string
By the way,the RabbitMQ Server is 3.8.5

Consume EventArgs from CouchDB

i am struggling with consuming the data i get from my CouchDB database.
I am trying to consume new data that comes to the specific view.
CouchDB offers an option for feed=continous, but i tested it and dont get any data, same in postman.
But if i change it to feed=eventsource i can see the changes in the console. But i dont know how to handle the events.
I opened a method with the right connection, but im stuck now, any help would be great.
public async Task ObserveDbAndTrigger()
{
var url = "http://localhost:5984/MyDB/_changes?feed=eventsource&filter=_view&view=MyView&include_docs=true&attachments=true&heartbeat=1000&since=0";
using (var client = new HttpClient())
{
client.Timeout = TimeSpan.FromMilliseconds(Timeout.Infinite);
client.DefaultRequestHeaders.Add("Accept", "application/json");
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", Convert.ToBase64String(System.Text.ASCIIEncoding.ASCII.GetBytes($"user:password" + $"")));
var request = new HttpRequestMessage(HttpMethod.Get, url);
// handle the incoming events and work with the incoming data
}
}
Any suggestions ?

Clearly there's work to be done. Normally I shy away from answering such questions as posed because it seems like a code service request, but I believe this answer may benefit others beyond the OP.
Here is an extremely naïve bit of code meant to illustrate event delegation and the simplicity of communicating with CouchDB over TCP.
Ultimately this demonstrates the publish/subscribe pattern, which is a reasonable fit. I tested this against CouchDB 2.3 on Windows. The code is hardwired to localhost:5984 because whatever.
class NaiveChangeWatcher
{
static void Main(string[] args)
{
if (args.Length >= 4)
{
// set up server info.
string db = args[0];
string auth = "Basic " + Convert.ToBase64String(ASCIIEncoding.ASCII.GetBytes(String.Join(":", args[1], args[2])));
string query = db + "/_changes?feed=continuous&since=0&heartbeat=" + args[3];
// init the publisher
ChangesPublisher pub = new ChangesPublisher();
// let's subscribe to the OnChange event which writes event data to the console.
pub.OnChange += (sender, e) => Console.WriteLine(e.Value);
pub.OnException += (sender, e) => Console.WriteLine(e.Value.ToString() + "\r\n\r\nPress a key to exit.");
//// start publishing.
Task.Run(async () =>
{
await pub.Begin("localhost", 5984, query, auth, int.Parse(args[3]));
});
// Press a key when bored of it all
Console.ReadKey();
// stop the publisher gracefully
pub.Stop = true;
}
else
{
Console.WriteLine("usage: NaiveChangeWatcher db_name username password timeout_millis");
}
}
//
// The ChangesPublisher notifies subscribers of new data from the changes feed
// via the ChangeEvent. The publisher will trigger an OnException event in the
// event of an exception prior to ending its task.
//
public class ChangesPublisher
{
// Set to true to stop publishing. This causes the Begin method to complete.
public bool Stop { get; set; }
// The event posted when data from the server arrived
public class ChangeEvent : EventArgs
{
public string Value { get; set; }
public ChangeEvent(string value)
{
Value = value;
}
}
// Event triggered when the subscriber croaks by exception
public class ExceptionEvent : EventArgs
{
public Exception Value { get; set; }
public ExceptionEvent(Exception value)
{
Value = value;
}
}
// Subscription to changes from the _changes endpoint
public event EventHandler<ChangeEvent> OnChange = delegate { };
// Subscription to publisher exit on error
public event EventHandler<ExceptionEvent> OnException = delegate { };
public async Task Begin(string serverAddr, int port, string query, string auth, int timeout)
{
using (var client = new TcpClient())
{
string request = String.Join("\r\n", new List<string> {
String.Format("GET /{0} HTTP/1.1",query),
"Authorization: " + auth,
"Accept: application/json",
"Host: " + serverAddr,
"Connection: keep-alive",
"\r\n"
});
try
{
await client.ConnectAsync(serverAddr, port);
using (NetworkStream stream = client.GetStream())
{
StreamWriter writer = new StreamWriter(stream);
await writer.WriteAsync(request);
await writer.FlushAsync();
// read lines from the server, ad nauseum.
StreamReader reader = new StreamReader(stream);
while (!Stop)
{
string data = await reader.ReadLineAsync();
// emit a change event
OnChange(this, new ChangeEvent(data));
}
}
}
catch (Exception e)
{
OnException(this, new ExceptionEvent(e));
}
}
}
}
}

gmail api returning null for a specific history id. History id received from Cloud Pub/Sub push subscription

From Cloud pub/sub push service i got a history id. Using that history id i am trying to read the recent mail's but It returns null.
I have configured cloud pub/sub push subscription and add a watch to "Unread" label.
Scenario 1:
I have received a push notification. From that push notification i have taken history id to get the recent messages. it's returning me null value.
Scenario 2:
I have logged into that configured mail id and then the message loaded in inbox. After that if i try to read i am getting the history list.
static string[] Scopes = { GmailService.Scope.MailGoogleCom };
static void Main(string[] args)
{
string UserId = "####.gmail.com";
UserCredential credential;
using (var stream =
new FileStream("client_secret_#####.json", FileMode.Open, FileAccess.Read))
{
string credPath = "token.json";
credential = GoogleWebAuthorizationBroker.AuthorizeAsync(
GoogleClientSecrets.Load(stream).Secrets,
Scopes,
UserId,
CancellationToken.None,
new FileDataStore(credPath, true)).Result;
Console.WriteLine("Credential file saved to: " + credPath);
}
var service = new GmailService(new BaseClientService.Initializer()
{
HttpClientInitializer = credential,
ApplicationName = ApplicationName,
});
List<History> result = new List<History>();
UsersResource.HistoryResource.ListRequest request = service.Users.History.List(UserId);
//history id received from cloud pub/sub push subscription.
request.StartHistoryId = Convert.ToUInt64("269871");
do
{
try
{
ListHistoryResponse response = request.Execute();
if (response.History != null)
{
result.AddRange(response.History);
}
request.PageToken = response.NextPageToken;
}
catch (Exception e)
{
Console.WriteLine("An error occurred: " + e.Message);
}
} while (!String.IsNullOrEmpty(request.PageToken));
foreach (var vHistory in result)
{
foreach (var vMsg in vHistory.Messages)
{
string date = string.Empty;
string from = string.Empty;
string subject = string.Empty;
string body = string.Empty;
var emailInfoRequest = service.Users.Messages.Get(UserId, vMsg.Id);
var emailInfoResponse = emailInfoRequest.Execute();
if(emailInfoResponse!= null)
{
foreach (var mParts in emailInfoResponse.Payload.Headers)
{
if (mParts.Name == "Date")
{
date = mParts.Value;
}
else if (mParts.Name == "From")
{
from = mParts.Value;
}
else if (mParts.Name == "Subject")
{
subject = mParts.Value;
}
if (date != "" && from != "")
{
if (emailInfoResponse.Payload.Parts != null)
{
foreach (MessagePart p in emailInfoResponse.Payload.Parts)
{
if (p.MimeType == "text/html")
{
byte[] data = FromBase64ForUrlString(p.Body.Data);
body = Encoding.UTF8.GetString(data);
}
else if(p.Filename!=null && p.Filename.Length>0)
{
string attId = p.Body.AttachmentId;
string outputDir = #"D:\#####\";
MessagePartBody attachPart = service.Users.Messages.Attachments.Get(UserId, vMsg.Id, attId).Execute();
String attachData = attachPart.Data.Replace('-', '+');
attachData = attachData.Replace('_', '/');
byte[] data = Convert.FromBase64String(attachData);
File.WriteAllBytes(Path.Combine(outputDir, p.Filename), data);
}
}
}
else
{
byte[] data = FromBase64ForUrlString(emailInfoResponse.Payload.Body.Data);
body = Encoding.UTF8.GetString(data);
}
}
}
}
}
}
public static byte[] FromBase64ForUrlString(string base64ForUrlInput)
{
int padChars = (base64ForUrlInput.Length % 4) == 0 ? 0 : (4 - (base64ForUrlInput.Length % 4));
StringBuilder result = new StringBuilder(base64ForUrlInput, base64ForUrlInput.Length + padChars);
result.Append(String.Empty.PadRight(padChars, '='));
result.Replace('-', '+');
result.Replace('_', '/');
return Convert.FromBase64String(result.ToString());
}
}
Please let me know how to read the full message using history id. when i receive push notification.

The Gmail Api documentation states that the Users.history:list method requires startHistoryId as a parameter to be executed, rather than giving you this parameter as a response. This is confusing, since it states as an optional parameter, but it is also specifies that it is required. The documentation also specifies:
The supplied startHistoryId should be obtained from the historyId of a
message, thread, or previous list response.
I suggest you to test the methods you use first with "Try this API" and OAuth 2.0 Playground. This makes it easier to understand which parameters you need to supply and which responses you can obtain from each method.

I have dealt with this. The point is that the history_id you are receiving is to be interpreted like the "latest moment when something happened". So, in order to make this work, you MUST use a history_id coming from a previous execution (that, don't forget, in GMail Push API means that you have to implement the initial full sync, or at the very least you should be executing a second run of your partial sync), which will return the events that span from the previous history_id to the one you just received.
I have just published an article on medium, since the detail of the history_id, in my opinion, can be a little sneaky. Article is here.

.NET Core 2.0 Is the below code thread safe

Here's my code and I have doubt on thread safe implementation. My questions are below
The return value from GetHtmlPageAsync is object. Is it thread safe? I will use this object and add into the collection and finally upload into database.
The main method logic is below (implementation in-progress). I have set of domains, I have list of 10000 domains in the collection, the idea is, I will put it in the queue and call the GetHtmlPageAsync to get the HTML of the page. Based on the HTML, I will get the necessary hyperlinks. Once I get the hyper links, I will check certain word is available in the link. If the word is available in the link, I will call the same method GetHTMLPageAsync to get the HTML of that page. So the same thread may call the GetHtmlPageAsync to process another link. I am trying to reuse the same method for multiple calls in thread safe way. Please help.
#edit1 . I have added the main method. Instead of Queue. I have used ForEach
public static async Task<int> ProcessDomainAsync(List<string> domains)
{
Parallel.ForEach(domains, async (currentDomain) =>
{
var domainBody = await GetHtmlPageAsync(currentDomain);
var language = string.Empty;
var country = string.Empty;
var createdOn = DateTime.SpecifyKind(DateTime.Now, DateTimeKind.Local);
var updatedOn = DateTime.SpecifyKind(DateTime.Now, DateTimeKind.Local);
var machine = Environment.MachineName;
var message = "[" + domainBody.ErrorCode + "] - " + domainBody.ErrorMessage;
var active = false;
var stage = "End";
var url = currentDomain;
if (domainBody.ErrorCode == 0)
{
var html = domainBody.Body;
language = Common.GetLanguageIdentification(html);
country = Common.GetCountryIdentification(currentDomain);
message = string.Empty;
active = true;
stage = "Stage1";
var hyperLinks = Common.GetAllAHrefTags(html);
//Process Hyper Links
}
_domainList.Add(new Domain
{
Url = url,
Language = language,
Country = country,
MachineName = machine,
Message = message,
Active = active,
Stage = stage,
CreatedOn = createdOn,
UpdatedOn = updatedOn
});
domainCount++;
});
return domainCount;
}
public class DomainBody
{
public string Body;
public string ErrorMessage;
public int ErrorCode;
}
public static class DomainProcessing {
static async Task<DomainBody> GetHtmlPageAsync(string url)
{
#region Initialize Proxy
var sessionId = new Random().Next().ToString();
var proxy = new WebProxy(Constant.ProxyUrl, Constant.ProxyPort);
var login = Constant.ProxyUserName + "-session-" + sessionId;
proxy.Credentials = new NetworkCredential(login,Constant.ProxyPassword);
#endregion
#region Initialize Variables
var user_agent = Common.GenerateRandomUserAgent();
var body = string.Empty;
var errorCode = 0;
var errorMessage = string.Empty;
#endregion
try
{
#region Format URL with Http Protocol
var domainSB = new StringBuilder();
domainSB.Append("http://");
domainSB.Append(url);
#endregion
#region Process Domain
var request = (HttpWebRequest) WebRequest.Create(new Uri(url));
request.Proxy = proxy;
request.UserAgent = user_agent;
request.Timeout = Constant.TimeOut;
using (var response = await request.GetResponseAsync().ConfigureAwait(true))
using (var content = new MemoryStream())
using (var responseStream = response.GetResponseStream())
{
await responseStream.CopyToAsync(content);
var bodyArray = content.ToArray();
body = Encoding.UTF8.GetString(bodyArray, 0, bodyArray.Length);
}
errorCode = 0;
errorMessage = string.Empty;
#endregion
}
catch (HttpRequestException ex)
{
body = string.Empty;
errorCode = ex.InnerException.HResult;
errorMessage = ex.InnerException.Message;
}
catch (Exception ex)
{
body = string.Empty;
errorCode = ex.HResult;
errorMessage = ex.Message;
}
var domainBody = new DomainBody
{
Body = body,
ErrorCode = errorCode,
ErrorMessage = errorMessage
};
return domainBody;
}
}enter code here

Generally speaking, local variables should be thread safe (simply because they have no idea there even is another thread and other threads have no way to access them).
Anything that can be accessed by multiple threads should be looked at. _domainList for example. Make sure the Add method is thread-safe because you are calling it potentially in parallel.

MailChimp Integaration with MVC 5

I am working on an MVC5 project, the client is interested in using MailChimp for sending emails. I have explored the MailChimp and wrappers ( MailChimp.NET ) and tried in my project as well. I tested the REST API as well and it seems to work , for example; I was able to grab lists and templates using REST API. But, still I am having issues with sending email through MailChimp.
So far, I have tried the following code and its working. Now I want to send an email to a newly registered user. Kindly give me detailed code example that How can I achieve this, because I am totally struck here..
var apiKey = "myapikey-us11";
var listId = "mylistid";
var subscribeRequest = new
{
apikey = apiKey,
id = listId,
email = new
{
email = "muhammad.waqas#seventechnology.co.uk"
},
double_optin = true,
};
var requestJson = JsonConvert.SerializeObject(subscribeRequest);
var reqresult = CallMailChimpApi("lists/", requestJson);
CallMailChimApi
private static string CallMailChimpApi(string method, string requestJson)
{
var endpoint = String.Format("https://{0}.api.mailchimp.com/3.0/{1}", "us11", method);
var wc = new WebClient();
try
{
return wc.UploadString(endpoint, requestJson);
}
catch (WebException we)
{
using (var sr = new StreamReader(we.Response.GetResponseStream()))
{
return sr.ReadToEnd();
}
}
}

I Use this function and it work successfully
public void SendEmailByApiMailChimp ()
{
try
{
string UserEmail = " Exemple#gmail.com ";
MailChimpManager mc = new MailChimpManager("16d***********-us14");
EmailParameter email = new EmailParameter()
{
Email = UserEmail
};
EmailParameter resulte = mc.Subscribe("yourlistnumber", email);
var test = resulte;
}
catch (Exception ex)
{
var ters = ex;
}
}