Develop Reference

Is there a way to know the number of key from the root on Firebase RealtimeDatabase?

I'm using firebase RealtimeDatabase to manage information using Json. I can get the snapshot and all the information inside with the key identifier:
FirebaseDatabase.DefaultInstance.GetReference("myKey").GetValueAsync().ContinueWith(task =>
{
// my code
});
And I don't know how I can get the childrenCount from the root "Key", I already tried using "default-rtdb" but it didn't work, nether while navigating the FirebaseDatabase.DefaultInstance.Root etc...

There is no way to get just the count of nodes from the Firebase Realtime Database. If you want that, you will actually have to store such a value in the database, and keep it up to date yourself.
But you can load all the data from any node in the database. If you want to load all data from the root, call GetReference without passing in a path. So: FirebaseDatabase.DefaultInstance.GetReference().GetValueAsync()

Trying to sync data from third party api

This question has probably been asked correctly before, and I'll gladly accept an answer pointing me to the right spot. The problem is I don't know how to ask the question correctly to get anything returned in a search.
I'm trying to pull data from a 3rd party api (ADP) and store data in my database using asp.net core.
I am wanting to take the users returned from the API and store them in my database, where I have an ADP ancillary table seeded with the majority of the data from the api.
I would then like to update or add any missing or altered records in my database FROM the API.
I'm thinking that about using an ajax call to the api to retrieve the records, then either storing the data to another table and using sql to look for records that are changed between the two tables and making any necessary changes(this would be manually activated via a button), or some kind of scheduled background task to perform this through methods in my c# code instead of ajax.
The question I have is:
Is it a better fit to do this as a stored procedure in sql or rather have a method in my web app perform the data transformation.
I'm looking for any examples of iterating through the returned data and updating/creating records in my database.
I've only seen vague not quite what I'm looking for examples and nothing definitive on the best way to accomplish this. If I can find any reference material or examples, I'll gladly research but I don't even know where to start, or the correct terms to search for. I've looked into model binding, ajax calls, json serialization & deserialization. I'm probably overthinking this.
Any suggestions or tech I should look at would be appreciated. Thanks for you time in advance.
My app is written in asp.net core 2.2 using EF Core
* EDIT *
For anyone looking - https://learn.microsoft.com/en-us/dotnet/csharp/tutorials/console-webapiclient
This with John Wu's Answer helped me achieve what I was looking for.

If this were my project this is how I would break down the tasks, in this order.
First, start an empty console application.
Next, write a method that gets the list of users from the API. You didn't tell us anything at all about the API, so here is a dummy example that uses an HTTP client.
public async Task<List<User>> GetUsers()
{
var client = new HttpClient();
var response = await client.GetAsync("https://SomeApi.com/Users");
var users = await ParseResponse(response);
return users.ToList();
}
Test the above (e.g. write a little shoestring code to run it and dump the results, or something) to ensure that it works independently. You want to make sure it is solid before moving on.
Next, create a temporary table (or tables) that matches the schema of the data objects that are returned from the API. For now you will just want to store it exactly the way you retrieve it.
Next, write some code to insert records into the table(s). Again, test this independently, and review the data in the table to make sure it all worked correctly. It might look a little like this:
public async Task InsertUser(User user)
{
using (var conn = new SqlConnection(Configuration.ConnectionString))
{
var cmd = new SqlCommand();
//etc.
await cmd.ExecuteNonQueryAsync();
}
}
Once you know how to pull the data and store it, you can finish the code to extract the data from the API and insert it. It might look a little like this:
public async Task DoTheMigration()
{
var users = await GetUsers();
var tasks = users.Select
(
u => InsertUser(u)
);
await Task.WhenAll(tasks.ToArray());
}
As a final step, write a series of stored procedures or a DTS package to move the data from the temp tables to their final resting place. If you are using MS Access, you can write a series of queries and execute them in order with some VBA. At a high level it would:
Check for any records that exist in the temp table but not in the final table and insert them into the final table.
Check for any records that exist in the final table but not the temp table and remove them or mark them as deleted.
Check for any records in common that have different column values and update the final table.
Each of these development activities raises it own set of questions, of course, which you can post back to StackOverflow with details. As it is your question doesn't have enough specificity for a more in-depth answer.

Prevent insertion of duplicate documents into Lotus notes database

I have a c# web api hosted in iis which has a post method that takes a list of document ids to insert into a lotus notes database.
The post method can be called multiple times and I want to prevent insertion of duplicate documents.
This is the code(in a static class) that is called from the post:
lock (thisLock)
{
var id = "some unique id";
doc = vw.GetDocumentByKey(id, false);
if (doc == null)
{
NotesDocument docNew = db.CreateDocument();
//some more processing
docNew.Save(true, false, false);
}
}
Even with the lock in place, I am running into scenarios where duplicate documents are inserted. Is it because a request can be execute on a new process? What is the best way to prevent it from happening?

Your problem is: getdocumentbykey depends on the view index being up to date. On a busy server there is no guarantee that this is true. You can TRY to call a vw.Update, but unfortunately this does not trigger an update of the view index, so it might be without any effect (it just updates the vw object to represent what has changed in the backend, if the backend did not update, then it does nothing).
You could use db.Search('IdField ="' & id & '"', Nothing, 0) instead, as the search does not rely on an index to be rebuilt. This will be slightly slower, but should be way more accurate.

you might want to store the inserted ids in some singleton object or even simply static list. And lock on this list - whoever obtains the lock verifies that the ids it wants to insert are not present and then adds them to the list itself.
You need to keep them only for a short length of time, just so that 2 concurrent posts with the same content does not update plus normal view index gets updated. So rather store timestamp along id, so you can clean out older records if the list grows long.

How to query an SQLite db in batches

I am using C# with .NET 4.5. I am making a scraper which collects specific data. Each time a value is scraped, I need to make sure it hasn't already been added to the SQLite db.
To do this, I am making a call each time a value is scraped to query against the db to check if it contains the value, and if not, I make another call to insert the value into the db.
Since I am scraping multiple values per second, this gets to be very IO-intensive, with constant calls to the db.
My question is, is there any better way to do this? Perhaps I could queue the values scraped and then run a batch query at once? Is that possible?

I see three approaches:
Use INSERT OR IGNORE, which will reject an entry if it is already present (based on primary key and unique fields). Or plainly INSERT (or its equivalent (INSERT or ABORT) which will return SQLITE_CONSTRAINT, a value you will have to catch and manage if you want to count failed insertions.
Accumulate, outside the database, the updates you want to make. When you have accumulated enough/all, start a transaction (BEGIN;), do your insertions (you can use INSERT OR IGNORE here as well), commit the transaction (COMMIT;)
You could pre-fetch a list of items you already have, depending, and check against that list, if your data model allows it.

How to refresh xml cache after one day from the last modified date?

I am developing a web app using ASP.NET 2.0 (C#), where on home page I am displaying recently added records. Adding of records frequency is around 1-5 records per day, so I decided not to put much overhead on the sql server by fetching recent records every time from db server.
So, To make the data cached I have used XML files, I have generated the XML file from dataset, (ds.WriteXML function in .NET), now lets say today (10 Jan 2008 12:30:00) I have created a file recent-cache.xml is created. So, the recent cache file is valid for one day.
Then if the difference between current date and last modified date is greater than or equal 1 day then cache xml file must be generated again, with the new data from the db server.
So, I want the code using which I can get the last modfied date of the xml file and then find the difference between both (current and file last modified-date) dates.
And also please tell me what I thought is the better solution, or we can do anything else, some other easy and speedy technique.
Thanks

You might consider using the ASP.NET Cache API, which exists to do the sort of job you're describing. You can add any object (like an XmlDocument, or a DataSet) to the Cache collection and specify how long you want it in there like so:
Cache.Insert("MyCacheKey", myObjectToCache, null, DateTime.Now.AddDays(1), null);
Then you could get at your cached data with a function like this:
const string CACHE_KEY = "MyCacheKey";
private DataSet RecentlyAdded()
{
if(Cache[CACHE_KEY] == null)
Cache.Insert(CACHE_KEY, GetRecentlyAddedFromDatabase(), null, DateTime.Now.AddDays(1), null);
return Cache[CACHE_KEY];
}
The caching API has lots of other neato features but this would accomplish what you want without having to roll your own file-based solution.
Note that if your app shuts down before the day is up, the Cache will shut down with it, and the "recently added" database query will have to run again the next time the data is requested.
edit: changed cache key to a string constant so it only has to be specified once.