Background Information
So here is the background to this question. Currently we have multiple bill of material number identifiers( UNPN,UDEN,UXXN,etc.) Each with its own sequence number (UNPN0001,UNPN0002,UNPN0003,etc.)
When a user requests a new bill of material number, lets say a new UNPN number. If the last number was UNPN0003 then the user should be able to reserve UNPN0004.
My Initial Thought:
So initially I figured, I could simply create a sequence column in a table. So if we had a UNPN table, we could easily add a new record and sequence it automatically and return the combination of UNPN + Generated Sequence number.
The Problem:
So the problem I identified is that for this method, we currently have 50+ different bill of material identifiers. So this would mean I would have to create 1 table for each identifier with its own auto sequence column.
What I am looking for:
I would prefer to avoid creating 50+ different tables. I am not sure if there is a different schema I could use for my database or if I should abandon this method all together for something else. I don't believe you can have multiple sequence columns in a table that only sequence based on what is input, of course I don't have a full understanding of what all is possible
Any suggestions would be appreciated.
Thank you,
You've indicated that you are using Oracle. Oracle has a database level objects known as sequences made for this sort of thing. They are very performant. You would want to make one sequence for each of your identifiers. You only need to do this once for each identifier.
CREATE SEQUENCE UNPN; -- Starts a 1 by default
CREATE SEQUENCE UDEN START WITH 50;
CREATE SEQUENCE UXXN MAXVALUE 9999 CYCLE; -- This one will recycle back to 0001 after it hits 9999
Next when ever you need a new value for a given sequence you can just select it:
SELECT 'UNPN'||TO_CHAR(UNPN.NEXTVAL,'FM0000') FROM DUAL;
To make it easier/more dynamic you can define a function to get the new value by name:
create or replace function get_named_seq(p_sequence varchar2) return varchar2
is
l_sql VARCHAR2(4000);
l_Result varchar2(30);
begin
select 'select '''||sequence_name||'''||to_char('||sequence_name||'.nextval,''FM0000'') from dual'
into l_sql
from user_sequences
where sequence_name = upper(p_sequence);
execute immediate l_sql
into l_Result;
return l_result;
end;
/
The above function will throw a NO_DATA_FOUND exception if the requested sequence doesn't exist, and even though it's using dynamic SQL it should be safe from SQL Injection since the actual dynamic SQL statement never directly touches the input parameter. you would use it like this or any where you can use a function:
INSERT into MyTable (ID, Data) values (get_named_seq('UXXN'), 'some data');
Related
I have a requirement to generate a semi-random code in C#/ASP.NET that has to be unique in the SQL Server database.
These codes need to be generated in batches of up to 100 codes per run.
Given the requirements, I'm not sure how I can do this without generating a code and then checking the database to see if it exists, which seems like a horrible way of doing it.
Here are the requirements:
Maximum 10 characters long (alpha-numeric only)
Must not be case sensitive
User can specify an optional 3 character prefix for the code
Must not violate 2 column unique constraint in the database, i.e. must be a unique "code text" within the "category" (CONSTRAINT ucCodes UNIQUE (ColumnCodeText, ColumnCategoryId))
So, given the 10 character limit, GUIDs are not an option. Given the case insensitivity requirement, the mathematical probability for database collisions are fairly high, I think.
At the same time, there are enough possible combinations that a straight look-up table in the DB would be prohibitive, I believe.
Is there a reasonably performant way of generating codes with these requirements that doesn't involve saving them to the DB one code at a time and waiting for a unique key violation to see if it goes through?
You have two options here.
You generate a new ID and insert it. If it throws dup unique key exception then try again until you succeed or bail if you run out of IDs. The performance will stink if most of the IDs are used up.
You pregenerate all the possible IDs and store them in a table. Whenever you need to get one you can remove one from a random row index and use that as the ID. Database will take care of the concurrency for you so its guarantee unique. if the first three letters are given then you can simply add a where clause to restrict the rows to match that constraint.
I created a new table and a new sequence, I have two C# web services trying to insert records into this table using same query utilizing mySequence.nextval (and yes I checked it many times, they both use mySequence.nextval).
The two web services are inserting rows to the table, but the mySequence.nextval is returning numbers out of sequence
Here is how the records were created, showing PrimaryKey which gets its value from mySequence.nextval
1 21 22 23 2 3 24 25 4 27 28 5
So far no duplicates but why is mySequence.nextval jumping back and forth? and should I worry about it
Update:
The sequence is created with cache_size = 20
I will wager that your database is running RAC (Real Application Clusters). Assuming that is the case and that you create the sequence with all the default settings, that's the expected behavior.
The default setting is to cache 20 values. Each node in the RAC cluster, by default, will have a separate cache. Assuming that you have a cluster with two nodes A and B, the first time a nextval is requested on A, A will cache values 1-20 and return a value of 1. If the next request for a nextval is made on B, B will cache values 21-40 and return a value of 21. From there, the value you get will depend on the node that your connection happens to be running on.
Generally, this shouldn't be a problem. Sequences generate unique numbers. The numbers generally need not be consecutive. If you really need values to be returned sequentially because you are doing something like ordering by the sequence-generated value to determine the "first" or "last" row, you can use the ORDER clause when you create the sequence to force values to be returned in order. That has a negative performance implication in a RAC database, however, because it increases the amount of communication that needs to go on between the nodes to synchronize the values being returned. If you need to determine the "first" or "last" row, it's generally better to add a date or a timestamp column to the table and order by that rather than assuming that the primary key is generated sequentially.
From the docs...
Sequence numbers are generated independently of tables, so the same sequence can be used for one or for multiple tables. It is possible that individual sequence numbers will appear to be skipped, because they were generated and used in a transaction that ultimately rolled back. Additionally, a single user may not realize that other users are drawing from the same sequence.
http://docs.oracle.com/cd/B28359_01/server.111/b28286/statements_6015.htm#SQLRF01314
I need to generate an id with the
following features:
Id must be unique
Id consist of two parts 'type' and 'auto incremented' number
'type' is integer and value can be 1, 2 or 3
'auto incremented' number starts with 10001 and incremented each time id
is generated.
type is selected from a web form and auto incremented number
is from the database.
Example: if type is selected 2 and auto incremented number is 10001
then the generated id is = 210001
There may be hundrads of users generating id. Now my question is,
Can this be done without stored procedure so that there is no id confict.
I am using ASP.Net(C#), Oracle, NHibernate
As you use Oracle, you can use a Sequence for that.
Each time you call your_sequence.NEXTVAL, a unique number is returned.
Why isn't the NHibernate implementation of Hi-Lo acceptable?
What’s the Hi/Lo Algorithm
What's the point in having the first digit of the ID to define the type? You should use a separate column for this, and then just use a plain auto-incrementing primary key for the actual ID.
The cleanest way is - as Scott Anderson also said - to use two columns. Each attribute should be atomic, i.e. have only one meaning. With a multi-valued column you'll have to apply functions (substr) to reveal for example the type. Constraints will be harder to define. Nothing beats a simple "check (integer_type in (1,2,3))" or "check (id > 10000)".
As for defining your second attribute - let's call it "id" - the number starting from 10001, you have two good strategies:
1) use one sequence, start with 1, and for display use the expression "10000 + row_number() over (partition by integer_type order by id)", to let the users see the number they want.
2) use three sequences, one for each integer_type, and let them have a start with clause of 10001.
The reason why you should definitely use sequences, is scalability. If you don't use sequences, you'll have to store the current value in some table, and serialize access to that table. And that's not good in a multi user system. With sequences you can set the cache property to reduce almost all contention issues.
Hope this helps.
Regards,
Rob.
If you can't use auto incrementing types such as sequences, have a table containing each type and keeping score of its current value. Be careful to control access to this table and use it to generate new numbers. It is likely it will be a hot spot in your db though.
We have a large database with enquiries, each enquirys is referenced using a Guid. The Guid isn't very customer friendly so we want to the additional 5 digit "human id" (ok as we'll very likely won't have more than 99999 enquirys active at any time, and it's ok if a humanuid reference multiple enquirys as they aren't used for anything important).
1) Is there any way to have a IDENTITY column reset to 1 after 99999?
My current workaround to this is to use a INT IDENTITY(1,1) NOT NULL column and when presenting a HumanId take HumanId % 100000.
2) Is there any way to automatically "randomly distribute" the ids over [0..99999] so that two enquirys created after each other don't get the adjacent ids? I guess I'm looking for a two-way one-to-one hash function??
... Ideally I'd like to create this using T-SQL automatically creating these id's when a enquiry is created.
If performance and concurrency isn't too much of an issue, you can use triggers and the MAX() function to calculate a 'next human ID' value. You probably would want to keep your IDENTITY column as is, and have the 'human ID' in a separate column.
EDIT: On a side note, this sounds like a 'presentation layer' issue, which shouldn't be in your database. Your presentation layer of your application should have the code to worry about presenting a record in a human readable manner. Just a thought...
If you absolutely need to do this in the database, then why not derive your human-friendly value directly from the GUID column?
-- human_id doesn't have to be calculated when you retrieve the data
-- you could create a computed column on the table itself if you prefer
SELECT (CAST(your_guid_column AS BINARY(3)) % 100000) AS human_id
FROM your_table
This will give you a random-ish value between 0 and 99999, derived from the first 3 bytes of the GUID. If you want a larger, or smaller, range then adjust the divisor accordingly.
I would strongly recommend relooking at your logic. Your approach has a few dangers, including:
It is always a bad idea to re-use ID's, even if the original record has become "obsolete" - do you lose anything by continuing to grow ID's beyond 99999? The problem here is more likely to be with long term maintenance, especially if there is any danger of the system developing over time. Another thing to consider - is there any chance a user will take this reference number, and use it to reference your system at some stage in the future?
With manually assigning a generated / random ID, you will need to ensure that multiple records are not assigned the same ID. There are a few options that you have to follow this (for example, using transactions), however you should ensure that the scope of the transactions is not going to leave you open to problems with concurrent transactions being blocked - this may cause a few problems eg. Performance. You may be best served by generating your ID externally (as SQL does not do random especially well), and then enforcing a unique constraint on your DB, perhaps in the way suggested by Firoz Ansari.
If you still want to reset the identity column, this can be done with the DBCC CHECKIDENT command.
An example of generating random seeds in SQL server can be found here:
http://weblogs.sqlteam.com/jeffs/archive/2004/11/22/2927.aspx
You can create composite primary key with two columns, say..BatchId and HumanId.
Records in these columns will look like this:
BatchId, HumanId
1, 1
1, 2
1, 3
.
.
1, 99998
1, 99999
2, 1
2, 2
3, 3
use MAX or ORDER BY DESC to get next available HumanId with condition with BachId
SELECT TOP 1 #NextHumanId=HumanId
FROM [THAT_TABLE]
ORDER BY BatchId DESC, HumanID DESC
IF #NextHumanId>=99999 THEN SET #NextHumanId=1
Hope this help.
You could have a table of available HUMANIDs, each time you add an enquiry you could randomly pull a HUMANID from the table (and DELETE it), and each time you delete the enquiry you could add it back (by INSERTing).
I have a database that is part of a Merge Replication scheme that has a GUID as it's PK. Specifically the Data Type is uniqueidentifier, Default Value (newsequentialid()), RowGUID is set to Yes. When I do a InsertOnSubmit(CaseNote) I thought I would be able to leave CaseNoteID alone and the database would input the next Sequential GUID like it does if you manually enter a new row in MSSMS. Instead it sends 00000000-0000-0000-0000-000000000000. If I add CaseNoteID = Guid.NewGuid(), the I get a GUID but not a Sequential one (I'm pretty sure).
Is there a way to let SQL create the next sequential id on a LINQ InsertOnSubmit()?
For reference below is the code I am using to insert a new record into the database.
CaseNote caseNote = new CaseNote
{
CaseNoteID = Guid.NewGuid(),
TimeSpentUnits = Convert.ToDecimal(tbxTimeSpentUnits.Text),
IsCaseLog = chkIsCaseLog.Checked,
ContactDate = Convert.ToDateTime(datContactDate.Text),
ContactDetails = memContactDetails.Text
};
caseNotesDB.CaseNotes.InsertOnSubmit(caseNote);
caseNotesDB.SubmitChanges();
Based on one of the suggestions below I enabled the Autogenerated in LINQ for that column and now I get the following error --> The target table of the DML statement cannot have any enabled triggers if the statement contains an OUTPUT clause without INTO clause.
Ideas?
In the Linq to Sql designer, set the Auto Generated Value property to true for that column.
This is equivalent to the IsDbGenerated property for a column. The only limitation is that you can't update the value using Linq.
From the top of the "Related" box on the right:
Sequential GUID in Linq-to-Sql?
If you really want the "next" value, use an int64 instead of GUID. COMB guid will ensure that the GUIDs are ordered.
In regards to your "The target table of the DML statement cannot have any enabled triggers if the statement contains an OUTPUT clause without INTO clause", check out this MS KB article, it appears to be a bug in LINQ:
http://support.microsoft.com/kb/961073
You really needed to do a couple of things.
Remove any assignment to the GUID type property
Change the column to autogenerated
Create a constraint in the database to default the column to NEWSEQUENTIALID()
Do insert on submit just like you were before.
On the insert into the table the ID will be created and will be sequential. Performance comparison of NEWSEQUENTIALID() vs. other methods
There is a bug in Linq2Sql when using an auto-generated (guid/sequential guid) primary key and having a trigger on the table.. that is what is causing your error. There is a hotfix for the problem:
http://support.microsoft.com/default.aspx?scid=kb;en-us;961073&sd=rss&spid=2855
Masstransit uses a combguid :
https://github.com/MassTransit/MassTransit/blob/master/src/MassTransit/NewId/NewId.cs
is this what you're looking for?
from wikipedia:
Sequential algorithms
GUIDs are commonly used as the primary key of database tables, and
with that, often the table has a clustered index on that attribute.
This presents a performance issue when inserting records because a
fully random GUID means the record may need to be inserted anywhere
within the table rather than merely appended near the end of it. As a
way of mitigating this issue while still providing enough randomness
to effectively prevent duplicate number collisions, several algorithms
have been used to generate sequential GUIDs. The first technique,
described by Jimmy Nilsson in August 2002[7] and referred to as a
"COMB" ("combined guid/timestamp"), replaces the last 6 bytes of Data4
with the least-significant 6 bytes of the current system date/time.
While this can result in GUIDs that are generated out of order within
the same fraction of a second, his tests showed this had little
real-world impact on insertion. One side effect of this approach is
that the date and time of insertion can be easily extracted from the
value later, if desired. Starting with Microsoft SQL Server version
2005, Microsoft added a function to the Transact-SQL language called
NEWSEQUENTIALID(),[8] which generates GUIDs that are guaranteed to
increase in value, but may start with a lower number (still guaranteed
unique) when the server restarts. This reduces the number of database
table pages where insertions can occur, but does not guarantee that
the values will always increase in value. The values returned by this
function can be easily predicted, so this algorithm is not well-suited
for generating obscure numbers for security or hashing purposes. In
2006, a programmer found that the SYS_GUID function provided by Oracle
was returning sequential GUIDs on some platforms, but this appears to
be a bug rather than a feature.[9]
You must handle OnCreated() method
Partial Class CaseNote
Sub OnCreated()
id = Guid.NewGuid()
End Sub
End Class