Generate Unique UInt32 ID for Couchbase

5
votes

I am looking for a way to generate a unique ID for nosql database. Unlike relational database there is no idea of rows which means there is no last row to increment from.

The most common way to handle this is to use UUID's. But my problem is I need to add another ID (other than the UUID) which needs to be:

  • Unique
  • Unsigned Int32

Total data could reach around 50,000,000. So how would you generate somewhat unique uint32 ID's?

The UInt32 value type represents unsigned integers with values ranging from 0 to 4,294,967,295.

  • Only generated when a new user registers.
  • 3 Id's are given to each new user.
  • Currently using Couchbase Server.
3
javascriptnode.jscouchbase
how frequently will they be generated? You might be able to use ms since unix epoch.. also, CRC32? - Paul S.
Maybe once a few minutes. There could be times that they can happen at the same time, but unlikely. - majidarif
@PaulS.: Seconds, rather? - Ry-♦
@PaulS. I could CRC32 usernames right? as usernames will be unique, and this ids are given to each user on register, problem is, 3 numbers should be given to each new registered user. Just realized crc can contain letters so it wont work? - majidarif
@minitech this looks promising. Any idea if it might be a fit for my use case? You might want to add it as an answer. I'll accept it. Thanks. - majidarif

3 Answers

4
votes

This problem has already been solved - I would suggest using the atomic Increment (or Decrement) functions in Couchbase - these are a common pattern to generate unique IDs.

Whenever the incr() method is called, it atomically increments the counter by the specified value, and returns the old value, therefore it's safe if two clients try to increment at the same time.

Pseudocode example (I'm no Node.JS expert!):

// Once, at the beginning of time we init the counter:
client.set("user::count", 0);
...
// Then, whenever a new user is needed: 
nextID = client.incr("user::count", 1); // increments counter and returns 'old' value.
newKey = "user_" + nextID;
client.add(newKey, value);

See the Node.JS SDK for reference, and see Using Reference Doucments for Lookups section in the Couchbase Developer Guide for a complete usage example.

1
votes

Here's a function that returns a unique identifier each time it's called. It should be fine as long as the number of items does not exceed the range of 32-bit integers, which seems to be the case given the described requirements. (warning: once the array of UIDs fills up, this enters an infinite loop. You may also want to create some sort of a reset function that can empty the array and thus reset the UID when necessary.)

var getUID = (function() {
    var UIDs = [];

    return function() {
        var uid;

        do {
            uid = Math.random() * Math.pow(2, 32) | 0x0;
        } while (UIDs[uid] !== undefined);

        return UIDs[uid] = uid;
    };
}());
0
votes

if you will call this insert method by passing "user" as key then your docId will be auto increment as user_0 user_1 user_2 etc... Please note that couchbase will show one extra row in your bucket with key as meta id and next counter value as doc value. Do not get surprised if you use query like select count(*) total from table; as it will show one more than real count, to avoid use where clause so that this row won't be counted.

public insert(data: any, key: string) {
  return new Promise((resolve, reject) => {
    let bucket = CouchbaseConnectionManager.getBucket(`${process.env.COUCHBASE_BUCKET}`)
    bucket.counter(key, 1, {initial:0}, (err:any, res:any)=>{
      if(err){
        this.responseHandler(err, res, reject, resolve);
      }
      const docId = key + "_" + res.value;
      bucket.insert(docId, data, (error:any, result:any) => {
        this.responseHandler(error, result, reject, resolve);
      });
    });
  });
}