How do I group T-SQL set by a flag bigint column - c#

I'm working on a project where efficiency of the search functionality is critical.
I have several flag columns (like enum flags in c#). Searching on this data is super fast (3 milliseconds round trip) but I've come-a-cropper now I have to perform group counts.
So, I have an item 'A' that contains Red (1), White (8) and blue (64) so the 'Colours' column holds the number 73.
To search I can search for items with red with this
Declare #colour int
set #colour = 1
Select *
From Items
Where (Colour & #colour) > 0
That works great. Now I have to group it (also super fast)
So if I have 8 items in total, 5 contain red, 3 contain white and 7 contain blue the results would look like:
Colour Qty
------------------
1 5
8 3
64 7 ( I don't have to worry about the name )
So: Is there any way I can take the number 73 and bitwise split it into groups?
(Part 2: How do I translate that into Linq to SQL?)
Any advise would be appreciated
Thanks ^_^

Ok - I think I've worked out the best solution:
I tried a view with a cte:
with cte as (
select cast(1 as bigint) as flag, 1 pow
union all
select POWER(cast(2 as bigint),pow), pow + 1
from cte
where flag < POWER(cast(2 as bigint),62)
)
, cte2 as (
select flag from cte
union select -9223372036854775808
)
but that was too slow so now I have made it into a static table. I join with a bitwise '&':
select Flag, Count(*)
From FlagValues fv
inner join Items i on (fv.Flag & i.Colour)
Much faster ^_^

Related

Update all rows in sql table with unique random value without using primary key or unique key in c#

In my application, I fetch all tables in Database.
User will select table name and colum names to be masked.
Now i want to update sql table-columns with random generate string , which must be unique for each row without using primary key or unique key.
For example, In my Employeedb i have a table Employee.
Out of columns in Employee table, i want to mask data in name and city columns.
If table conatins 1000 rows, i want change name and city columns with 1000 unique values each. That means i want to update row by row.
Name Address City
Raghav flatno34 mumbai
Ranveer flatno23 chennai
This is orignal data
Name Adress City
Sbgha flatno34 mmjgujj
Lkhhvh flatno23 huughh
This is expected out
The table have primarykey sometimes.. There may be chances of not having primary key.
I have one more qn, I have this expected output in a datatable. Since i cannot predefine the table name and number of fields how will i write an update qry.
I think you will find my blog post entitled How to pre-populate a random strings pool very helpful for this requirement.
(Inspired by this SO answer from Martin Smith, to give credit where credit is due)
It describes an inline table valued user defined function that generates a table of random values, which you can use to update your data.
However, it does not guarantee uniqueness of these values. For that, you must use DISTINCT when selecting from it.
One problem you might encounter because of that is having a result with less values than you generated, but for 1,000 records per table as you wrote in the question it's probably not going to be a problem, since the function can generate up to 1,000,000 records each time you call it.
For the sake of completeness, I'll post the code here as well, but you should probably read the post at my blog.
Also, there's another version of this function in another blog post entitled A more controllable random string generator function for SQL Server - which gives you better control over the content of the random strings - i.e a string containing only numbers, or only lower digits.
The first thing you need to do is create a view that will generate a new guid for you, because this can't be done inside a user-defined function:
CREATE VIEW GuidGenerator
AS
SELECT Newid() As NewGuid
Then, the function code: (Note: this is the simpler version)
CREATE FUNCTION dbo.RandomStringGenerator
(
#Length int,
#Count int -- Note: up to 1,000,000 rows
)
RETURNS TABLE
AS
RETURN
-- An inline tally table with 1,000,000 rows
WITH E1(N) AS (SELECT N FROM (VALUES (1), (2), (3), (4), (5), (6), (7), (8), (9), (10)) V(N)), -- 10
E2(N) AS (SELECT 1 FROM E1 a, E1 b), --100
E3(N) AS (SELECT 1 FROM E2 a, E2 b), --10,000
Tally(N) AS (SELECT ROW_NUMBER() OVER (ORDER BY ##SPID) FROM E3 a, E2 b) --1,000,000
SELECT TOP(#Count) (
SELECT TOP (#Length) CHAR(
-- create a random number from a guid using the GuidGenerator view, mod 3.
CASE Abs(Checksum(NewGuid)) % 3
WHEN 0 THEN 65 + Abs(Checksum(NewGuid)) % 26 -- Random upper case letter
WHEN 1 THEN 97 + Abs(Checksum(NewGuid)) % 26 -- Random lower case letter
ELSE 48 + Abs(Checksum(NewGuid)) % 10 -- Random digit
END
)
FROM Tally As t0
CROSS JOIN GuidGenerator
WHERE t0.n != -t1.n -- Needed for the subquery to get re-evaluated for each row
FOR XML PATH('')
) As RandomString
FROM Tally As t1
Then, you can use it like this to get a distinct random string:
SELECT DISTINCT RandomString
FROM dbo.RandomStringGenerator(50, 5000);

Get empty values from database table [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 3 years ago.
Improve this question
I have a database table which has columns with values 1 till 999
But it has some spaces e.g. 1,2,3,4,5,6,11,15 etc...
What would be the best to get the "next number" from this table?
Thanks in advance for your help
one way to do this is to get for every row the prior row, and then check where you are making a step.
This will not perform great, and it is NOT SAFE when more then 1 user is adding new rows !
declare #t table (number int)
insert into #t values (1), (2), (3), (4), (5), (6), (11), (12)
select top 1
(select top 1 t2.number + 1 from #t t2 where t2.number < t.number order by t2.number desc) as prior
from #t t
where number <> (select top 1 t2.number + 1 from #t t2 where t2.number < t.number order by t2.number desc)
order by t.number
The result would be 7
Another option is this
select top 1
t.number + 1
from #t t
left join #t t2 on t.number = t2.number - 1
where t2.number is null
order by t.number
This method might even be faster then the solution of Robin
EDIT
As Daniel pointed out in a comment, this will never return 1 in case the gap happens to be the first row.
To fix this, we can retrieve a value for the first missing row, and add it to our result by use of a union.
select top 1 number
from ( select top 1
t.number + 1 as number
from #t t
left join #t t2 on t.number = t2.number - 1
where t2.number is null
union
select 1 as number
from #t t
where not exists (select 1 from #t t3 where t3.number = 1)
) t
order by t.number
Since the extra query can only retrieve exact one row by an index, this should not affect performance much
You can use a CTE to generate the numbers and then get the first one that does not match with a record....
This work fine as you mentioned that it is not a large table
I have a datatable which has columns with values 1 till 999
Regard the other answers, both are too much faster than this with large tables, but none of them will return the correct value (1) if your input starts on 2 or greater.
I don't know the purpose of this request, but be aware that calculating values this way two users working at same time can get the same value. It can be an issue specially if you want to use this value to be part of a primary key or unique index
;with numbers as (
SELECT 1 as nrstart, MAX(yourcolumn) as nrend FROM yourTable
UNION ALL
SELECT nrstart+1, nrend FROM numbers
WHERE nrstart <= nrend
)
SELECT TOP 1 nrstart
FROM numbers
WHERE NOT EXISTS (SELECT 1 FROM yourTable WHERE yourcolumn = numbers.nrstart)
ORDER BY nrStart
OPTION (MAXRECURSION 0);
You want the first number, where that number plus one is not in the table.
SELECT TOP 1 (Number + 1) FROM myTable a WHERE NOT EXISTS
(SELECT * FROM myTable b WHERE b.Number = a.Number + 1)
ORDER By Number
As mentioned in various comments, this sort of thing should be done in a transaction if there's any risk of a second user filling the gap while the first is looking for it.

Algorithm in SQL / C# to get an exact number of rows depending on variable?

I need to get an automatic method to link certain number consumptions with a payment.
I have for example in my first table.
Payments
ID Amount
1 $5,000
Then I have the following Table:
Consumptions
ID Amount CreatedDate
1 1000 2015-07-01 13:59
2 1000 2015-07-01 19:15
3 1000 2015-07-02 01:01
4 1500 2015-07-02 08:44
5 1000 2015-07-03 05:00
6 800 2015-07-03 19:57
7 200 2015-07-03 21:32
8 500 2015-07-03 23:48
I want to have a way that considering the $5000 payment amount, it automatically chooses the best combination of consumptions that make up the $5,000 sum.
If SQL is difficult, it can also be done on C#.
You will need a CTE that recursively joins with itself and checks all possible combinations.
with Anchor
(
Select Id, Amount, 0 as Level
From Consumptions
),
with RecursiveJoin
(
Select Id, Amount, Level From Anchor
Where Level = 0
UNION ALL
-- generate all possible permutations of amounts
Select c2.Id, c1.Amount + c2.Amount, Level + 1
From RecursiveJoin c1
Left Join RecursiveJoin c2 on c1.Id < c2.Id -- eliminate self joins and palindromes
)
Select *
From RecursiveJoin c
Join Payments p on c.Amount = p.Amount
I'm not completely confident that my recursion is correct. Hopefully that gets you started on the concept.
I'll leave it to you to add fields that concatenate lists of Id's of consumptions in each recursion.
It will give you multiple matches since multiple combinations of consumptions could match a payment. You could find the one with fewest combinations by keeping count in the recursion.

Select unique random rows from SQL Server table but always duplicates

i am not the best with sql but i try my best to get my Problems done. I have a table "just" which is filled with Columns ( ID (PK, Identity), Char, Serv , Random ) . Now i want to select a random row from this table and insert it into the Table "f_WinGet". So far all my Procedures take this Step fine , but i always get Duplicates in the second table.
First table : 84 rows
Second table: needed 35 random out of 84.
I have tried many other ways but i always get the same result. All my Procedure for random are binded to a Button i a C# Programm. All is working fine so far , but i always have some Duplicates of Rows in my Table.
INSERT INTO f_TWinGet
SELECT TOP 1 Percent Char, Serv, Random
FROM ( select distinct Char, Serv, Random from dbo.just) as derived1
ORDER BY CHECKSUM(NEWID())
It would be nice , if anyone hase an Idea how i can fix my Problem. I am still trying , but all what i get are always the same result.
It you want to insert 35 rows, do it all at once:
INSERT INTO f_TWinGet(char, serv, random)
SELECT TOP 35 Char, Serv, Random
FROM (select distinct Char, Serv, Random
from dbo.just
) derived1
ORDER BY CHECKSUM(NEWID());
If you really want to do them one at a time, I would suggest using not exists:
INSERT INTO f_TWinGet(char, serv, random)
SELECT TOP 1 Char, Serv, Random
FROM (select distinct Char, Serv, Random
from dbo.just
) d
WHERE not exists (select 1 from f_TWinGet f where t.char = f.char and t.serv = f.serv and t.random = f.random)
ORDER BY CHECKSUM(NEWID());
Note that char is a reserved word, so it should be in square braces. I am leaving the names you have have them in your query.
With a table as small as yours you can use something like:
INSERT INTO f_TWinGet
SELECT TOP 1 j.Char, j.Serv, j.Random
FROM dbo.just j
LEFT JOIN f_TWinGet f
ON f.Char = j.Char
AND j.Serv = f.Serv
AND j.Random = f.Random
WHERE f.Char IS NULL
ORDER BY NEWID()
This way making sure that the values you're trying to insert is not on the final table.

Get the Rows separately based on the ID in sql

I have a table as follows,
TypeID Name Date
-------------------------------
1 Carrot 1-1-2013
1 Beetroot 1-1-2013
1 Beans 1-1-2013
2 cabbage 1-1-2013
2 potato 1-1-2013
2 tomato 1-1-2013
2 onion 1-1-2013
If need 2 rows then it should return 2 rows from TypeId 1 and 2 rows from TypeId 2.If need the only 4 rows, means I have to get 4 rows from TypeId 1 and 4 rows from TypeId 2
but TypeId 1 has only 3 rows so we need to get only 3 rows for typeId 1
How to do that? Shall I add RowNumber?
For SQL Server;
EDIT: Your question changed slightly;
If you want want a maximum of x items per category, you can use ROW_NUMBER();
WITH cte AS (
SELECT *, ROW_NUMBER() OVER (PARTITION BY TypeID ORDER BY Name) rn FROM Table1
)
SELECT TypeID, Name, [Date] FROM cte
WHERE rn <=3 -- here is where your x goes
ORDER BY TypeID;
An SQLfiddle to test with.
You can write your query to order by the TypeID.
Then, if you're using SQL, you could use SELECT TOP N or LIMIT N (depending on the DB), or with TSQL and SQL Server, use TOP(N) to take the top N rows.
If you're using a LINQ based ORM from your C# code, then you can use Take(N), which automatically creates the appropriate query based on the provider details to limit the number of results.
I think you should use a query to select your 3 rows from type 1.....and then the however many rows from type 2 and then add the results together.
;With CTE(TypeID,Name,Date,RowNo)
AS
(
select *, ROW_NUMBER() OVER(PARTITION BY ID ORDER BY ID) from TableVEG
)
Select Top(#noofRows*2) * from CTE where RowNo<=#noofRows order by rowno
The above query worked.. Thank u all... :-)

Categories

Resources