Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 hours ago.
Improve this question
I'm writing an NFL football simulator widget in C# to run in the background and show me simulated games and scores and whatnot. While the normal NFL has 32 teams, I added an additional 8 across 2 new divisions and am now trying to write the schedule generator. Luckily, I've been able to generate a valid set of matchups for every team, now I'm just trying to schedule it. Luckily, by this point, I can express the problem a lot more simply.
There is a 20x16 grid where each column is a game number in a week and each row is a single week of play.
Each grid cell is either empty or can hold a game between two teams. This grid should be completely full once the schedule is complete.
In these games, a team never faces itself (i.e. the Rams never play themselves).
Each team gets 16 matchups, which have been precomputed.
Two teams will face each other multiple times per season (i.e. the Patriots will play the Bills more than once).
Since each team only plays 16 opponents out of a possible 39, many matchups don't occur at all in the 320-game season.
My goal sounds simple:
Fill all 320 slots in the 20x16 grid such that each row has every team appear exactly once.
That's it, there are no further constraints. Except it isn't as simple as it sounds. Often, we might fill 19 of 20 slots in a single row, using 38 of the 40 teams, but the 2 remaining teams never play each other.
My current algorithm is as follows:
There's an array of flags, one for each team. A team's flag is set when a game is scheduled for it this week, and when all 40 flags are set, we know we have a valid week. When we do, this array is cleared and we move on to the next week.
Selecting games for each week is simple: we loop through the list of all 320 games, restarting from the beginning when we reach the end, and choose the first game such that
Neither of its teams are playing yet this week
The game is not part of a previous week
When all 320 games are selected, and when all 16 weeks have 20 games in them, the algorithm is complete and we have a valid NFL season.
This is quite simple, but we quickly run into a problem. Let's say we're scheduling Week 10, and we've picked 19 of 20 matches so far. That's 38 out of the 40 teams who have been scheduled for this week. The remaining teams are the 49ers and the Steelers. However, this season, they don't play each other at all, so my algorithm will cycle through all games forever and never terminate.
Even though there are still 49ers and Steelers games left (6 each), all of those games have opponents who are already playing someone else this week. As a result, we stall out here and cannot continue.
I did look at some round-robin tournament algorithms, but they seem to assume that every team will play every other team.
How do I fill my 20x16 grid such that each row has all 40 teams appearing exactly once?
or:
How do I generate an NFL schedule, given all the matchups that occur in a season, such that all teams play each week and no team plays the same week twice?
Related
I'm in the process of programming a web application that gets data from an inverter to which a PV cell is attached. I read the data from a CSV file. Every 20 seconds, the CSV file gains a line that contains the data at the respective point in time (line contains the following data: timestamp, current performance, energy).
The CSV file is saved to a database when the application is started (when the index action is called in the controller). It's all working.
Since the database now contains data at 20s intervals, it is rapidly increasing in size. Since I use graphs to show the energy that the PV system supplies me over the year on my web application, I have to summarize the 20s data, which also requires computing power. I also do this in the index action.
So whenever the user opens the page, the data is updated. If I e.g. switch from one view to the other and back again, the index action is called again in the associated controller. So it takes time to load the page again. So my application becomes slow.
What do I need to do to solve such a problem?
Ok, then.
In our IT industry, we often come across the term "data warehousing".
What this means (in most cases) is that we have a LOT of transaction data. Think maybe the very high transaction rate generated by people shopping on amazon. HUGE number of transactions.
But, if we want to report on such data? Say we want sales by hour, or maybe even only need by per day.
Well, we don't store each single transaction for that "house" of data, but a sum total, and a sum total over a given "chosen" time period by the developer of that data warehouse system.
So, you probably don't need to capture each 20 second data point. (maybe you do?????).
So, as I stated, every 20 seconds, you get a data point. Given a year has 31 million seconds? then that means you will have 1.5 million data points per year.
However, perhaps you don't need such fine resolution. If you take the data, and sum by say 1 minute intervals, then you now down to only 525,000 data points per year. (and if you report by month, then that is only 43,000 points per month).
However, maybe a resolution of 5 minutes is more then fine for your needs. at that resolution, then a whole year of data becomes only 105,120 data points.
And thus for a graph or display of one month of data, we have only 8,760 data points.
So, if we have to (for example) display a graph for one month, then we are only pulling 8,700 points of data. Not at all a large query for any database system these days.
So, you might want to think of this data as some "miniature" data warehousing project, in which you do loose some data "granularity", but as such, it still is sufficient for your reporting needs.
What time slot or "gap" you choose will be based on YOUR requirements or need.
What the above thus then suggests?
You would need a routine that reads the csv, and then groups the data by that "time slot" you chosen, and then sum into existing data points, and appends for the new ones.
This as a result would not only vast reduce the number of data rows of data, but of course would also significantly speed up reports and graphing on such data.
So, you could easy drop from about 1.5 million rows of data per year down to say 100,000 rows per year. With a index on the date, then reporting on such data be it daily, weekly, or monthly becomes far more manageable, and you reduced the data by a factor of 10x. Thus, you have a lot more headroom in the database, a lot less data, and after 10 years of data, you would only be around 1 million rows of data - not a lot for even the free "express" edition of SQL server.
Also, since you can't control when the "device" triggers adding of data to that csv, I would consider a re-name of the file before you read it, and thus during some read (and you deleting after done), you would reduce the possibility of losing data during your csv read + delete operation.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 2 years ago.
Improve this question
I'm currently working on a point of sales software in which I have a table to record each and every item of a transaction, Since its going to hold hundreds of records each day after its release, I just wanna know the maximum amount of records that can be held by a Table and can anyone pls let me know whether it can slow down the software over time.
For practical day-to-day purposes (where you're inserting hundreds or thousands of rows per day) there is no limit to the size of the table, except if it fills up your disk.
Remember that organisations with userbases larger than yours, use databases with not hundreds of rows per day, but millions of rows per day.
Typically though, you will start to run into performance issues that needs fixing. You can still get good performance, you just need to do more to watch and tweak it.
For example, you may have a typical table with, say,
An ID (autoincrement/identity) that is the Primary Key (and clustered index).
A date/time field recording when it occurred
Some other data e.g., user IDs, amounts, types of action, etc
Each row you insert into the table just puts a new row at the end of that table, which databases typically have no problem doing. Even if the table is already large, adding more rows isn't much of a problem.
However, imagine you have a query/report that gets the data for the last week - for example, SELECT * FROM trn_log WHERE trn_datetime >= DATEADD(day, -7, getdate())
At first that runs fine.
After a while, it slows down. Why? Because the database doesn't know the that the datetimes are sequential, and therefore it must read every row of the table and work out which of the rows are the ones you want to use.
At that point, you start to think about indexes - which is a good next step. But when you add an index, it slows down your new row inserts (by a small amount).
I learned a lot from watching Brent Ozar's videos. I recommend watching his How to Think Like the SQL Server Engine series.
Note that this above is based on my experience with SQL Server - but it's likely (at this fundamental level) most other databases are the same.
The number of rows per page is limited to 255 rows so that works out to 4.1 billion rows per partition. A table can have an unlimited number of partitions and a single server can manage up to 128PB of storage.
https://www.quora.com/How-many-rows-can-recent-SQL-and-NoSQL-databases-reasonably-handle-within-one-table#:~:text=The%20number%20of%20rows%20per,up%20to%20128PB%20of%20storage.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
I am trying to create a simple social networking app starting with back-end.
App Description
When a user opens the app a list of users will be displayed based on his/her Geo-Location, age and gender.
One a user has been viewed he won't be showed ever again.
Technologies
I am using Azure CosmosDB(MongoDB implementation) and Azure Redis Cache to store the documents.
My Approach to deal with the problem
I save all the user db in cosmosdb. I query for user Ids based on geocoordinate and age and gender preference filters and limit the results by 5000.
I also apply one more filter which is if a user has already been viewed filter. I am maintaining collection where for each user all the user Ids that he viewed will be saved as document.
For the first time I'll get 5000 id's from cosmosdb and put 4950 in redis cache(with an expiry time). Using the remaining 50 ids I'll fetch users from cosmosdb and return it as response to the api call. For subsequent calls I get next 50 ids from redis cache and fetch those users and return as response.
Problem I'm facing
Fetching the 5000 users is a time taking step as it involves geolocation computation and other filtering. I created a sample user database where there are nearly 2 million users in 100 mile radius and I am supposed to get 100,000 users based on my preference i.e, age and gender had I not applied 5000 Limit.
It would take around 25 seconds to do so.
Applying 5000 limit would run the query for 1 - 1.5 seconds only initially. Buy as the users get viewed i.e, when Not-In ($nin) filter will exclude those 5000 ids time taken would eventually increase. Time taken to get from cache will be fast but when the cache gets exhausted or expired and we have to hit cosmos db to query for 5000 more users it would take more time as the users he as already viewed keep increasing.
Stats
Time format is in hrs:min:sec.
It is performed just for performance stats. Actual Api request will provide 50 users each time(most of the time from cache).
first time
Time taken to get 5000 matches is 00:00:01.22
Time taken to set Viewed Ids is 00:00:00.06
second time
Time taken to get 5000 matches is 00:00:02.49
Time taken to set Viewed Ids is 00:00:00.67
:
:
Fifteenth time
Time taken to get 5000 matches is 00:00:23.05
Time taken to set Viewed Ids is 00:00:09.23
Question
How can the architecture be improved for better performance ? How apps like Uber, Tinder etc that involve users Geo-Location computations architect their application ? Is there a better way to model the problem or model the data ?
Any help would be appreciated. Thank you.
2 million users is enough that you need to start having a good indexing strategy for database queries to work. Geography queries provide a unique indexing problem because they are searches over two related variabled (namely longitude and latitude).
There's a good description of how Microsoft SQL server does their spacial index over here, which also nicely summarizes the indexing problem more generally.
While I've not personally used it, CosmoDB seems to now have some support for this too. See this and this.
The first thing I would do is slightly rethink your expectations - simply finding the 50 or 5000 (or whatever n) nearest items can involve a lengthy search if there are no nearby matches (or even if there are), but if your database is properly indexed, you can search very efficiently within some radius r of a point, and then sort those results by distance. If you have or expect to have a large number of coordinates, I would suggest doing that several times, in other words searching for all matches with 100m, sort by distance, and then if you need more, search for all matches within 500m and exclude the ones you've already seen, and so on up to 10km or 25km or whatever your app calls for.
MongoDB has a quite efficient index available for geospatial coordinates (basically a world map partitioned into B+ trees). The '$near' query allows you to specify both minimum and maximum distance, and sorts by distance by default, so it'a very convenient for this kind of tiered, distance based searches. You will have to format your coordinates (both in the DB and in the query) as GeoJSON Point objects if they aren't already, though.
i want to make a web-based game for my study.
The game is simple: after registration you have 100$ and you can buy let's say one tree, this tree will give you 10 apples per hour and you can sell one apple for 1$
My question is: how do i make this tree drop 10 apples per hour? Do I need to write another C# Console programm (aka cronjob) that connects to database and updates everything? Catching users, gettings their trees and giving them apples every hour? I have tried it that way but it is a performance issue
I have tried to do that and filled my database with dummy 1000 users and each user had 10000 trees (so 10000000 trees in total) but my C# program (so-called cronjob) is taking more than 5 minutes to reward all fake players with apples
I am using entity framework and foreach loop to catch every player and count their trees then add apples to them and save database changes
First, if a request to your server is the only way anyone can see apple count updates, I don't see the need to make those updates before the next request. You can store latest_apple_drop_datetime, and on request, calculate how many times apples were dropped and when the most recent drop was. That way you don't need any cronjob. Next, does a player see other players' activity? If not, you have to process only the requesting player's trees. Finally, look into implementing all updates in a single UPDATE statement (I assume you are using an SQL database), without any loops in your code, letting the database itself optimize performance
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
Greeting all,
I'm currently coming up against a logic problem that I've yet to figure out after two days due that I keep second guessing myself. Here is the gist of what I'm doing.
I have a orders that come into a site. Logically, it has already determined how many of a particular item to send on an order based on what is in stock. If the stock is high enough, it issues that many items requested.
However, there is a twist. The admins can adjust these order numbers. They can change the amount requested along with the amount issued. One rule is that they cannot issue more than requested (issued <= requested in other words). Here is an example of an order.
A request comes in for a quantity of 5 items. The number in stock at the time of the order was 4. The system takes what is in stock and issues it for the order and places a backorder for the remaining item.
Qty = 5
Stock = 4
5 - 4 = 1
Issue Qty = 4
Backorder = 1
There are 4 scenerios that I've come up with to account for all the possible changes.
1.) Admin changes Qty from 5 to 4. The admin leaves the amount issued alone due that the stock is still 4 and cannot go any higher. This causes the backorder to need removed.
2.) Admin changes the Qty from 5 to 6, leaves the issued amount alone (in turn due that the stock is empty). This causes the backorder to need updated to 2.
3.) Admin doesn't change the Qty (5). The admin changes the issued amount from 4 to 3. This adds one item back into stock (1). This causes the backorder to remain 2.
4.) Admin changes both Qty and issued amount to 4. This leaves the stock at 0. The backorder is also removed due that it goes down to 0.
My problem is creating logic in code to handle this. I started by checking the quantity to see if it exceeded the original quantity's. If it did, I checked the issued amount to see if it was greater than the original amount issued. And then I keep getting lost.
I'm hoping maybe someone has some ideas. I really appreciate any help in figuring out this logic. Thanks.
If there is already a system for fulfilling the orders (calculates issued & back order numbers), then one way to handle this would be to reverse the operation, apply the admin's changes and re-fulfill the order with the changes.
You'd have to add a field or two to the order to account for the admin's changes, and update the order fulfillment system to account for the specified issued amount.
In this way, you wouldn't need complicated logic for altering what is issued, backordered, etc.