Generating Millions of Unique IDs in TypeScript — Fast, Scalable & Collision-Free

This content originally appeared on DEV Community and was authored by Srashti Gupta

Why I Wrote This

While scaling a backend service recently, I hit a challenge — generating billions of unique IDs efficiently, without collisions, and without blowing up memory.
UUIDs were too long, and auto-increment IDs broke under distributed systems.

That’s when I explored Snowflake algorithms and combined them with Base62 encoding — producing short, globally unique IDs that can be generated in crores per minute.

Here’s what I learned

1. Introduction

Ever wondered how Twitter, YouTube, or TikTok generate billions of unique IDs every day — instantly and safely?

Traditional methods like UUID or AUTO_INCREMENT are not scalable across distributed databases.
As systems grow, we need an ID generator that’s:

• Fast
• Memory-efficient
• Collision-free
• Compact & Sortable

Let’s explore how to build one — and even generate crores of unique IDs in seconds using TypeScript.

2. Key Requirements for a Scalable ID Generator

An ideal ID generator should be:

• Unique — zero collision risk
• Sortable — time-based for sequencing
• Fast — able to generate millions/sec
• Compact — fixed length (~13 chars)
• Flexible — predictable or opaque

3. Popular Approaches

a) UUID / NanoID

• Simple, plug-and-play
• Not sortable
• UUIDv4 = 36 chars long
• NanoID shorter but slower at massive scale

b) Snowflake Algorithm (used by Twitter, TikTok)

Snowflake generates a 64-bit integer using:

• Timestamp
• Machine ID
• Process ID
• Sequence Counter

It’s ultra-fast, distributed-safe, and scales horizontally across servers.

Example in TypeScript:

class Snowflake {
  private lastTimestamp = 0n;
  private sequence = 0n;
  private machineId: bigint;

  constructor(machineId: number) {
    this.machineId = BigInt(machineId) & 0b11111n; // 5 bits
  }

  private timestamp() {
    return BigInt(Date.now());
  }

  nextId() {
    let now = this.timestamp();
    if (now === this.lastTimestamp) {
      this.sequence = (this.sequence + 1n) & 0b111111111111n; // 12 bits
      if (this.sequence === 0n) now++;
    } else {
      this.sequence = 0n;
    }
    this.lastTimestamp = now;

    const id =
      ((now - 1700000000000n) << 17n) | (this.machineId << 12n) | this.sequence;
    return id.toString();
  }
}

4. Enhancing Snowflake with Base36/Base62 Encoding

To make IDs shorter, URL-safe, and more random-looking, encode them in Base62.

function toBase62(num: bigint): string {
  const chars = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";
  let str = "";
  while (num > 0n) {
    const rem = num % 62n;
    str = chars[Number(rem)] + str;
    num = num / 62n;
  }
  return str.padStart(13, "0"); // fixed length 13
}

Now combine both:

const snow = new Snowflake(1);
const id = toBase62(BigInt(snow.nextId()));
console.log(id); // e.g. "00bX5ztuG8p3L"

Result:
Short, unique, URL-safe, and crazy fast

5. Performance Comparison

Method	Length	Sortable	Collision Risk	Speed	Memory Use
UUIDv4	36		Low	Medium	Medium
NanoID	~21		Low	Medium	Medium
Snowflake	19 digits		Very Low	Very Fast	Low
Snowflake + Base62	11–13 chars		Very Low	Blazing	Low

6. Predictability vs Randomness

• Snowflake IDs are time-sequential → predictable but easy to sort.
• Adding Base62 encoding makes them opaque.
• For extra randomness, add a salt or hash layer (SHA1 / MD5 hash slice).

7. Real-world Usage

• Twitter, Discord, Instagram → use Snowflake-like systems
• TinyURL, Bitly → use Base62 for compact links
• Firebase / Firestore → time-based + random push IDs

8. Generating in Chunks (for Crores of IDs)

You can generate crores of IDs efficiently by batching them into chunks.
This avoids memory overload and stays lightning fast

import fs from "fs";

const snow = new Snowflake(1);
const CHUNK_SIZE = 100_000; // 1 lakh IDs
const TOTAL = 10_000_000;   // 1 crore IDs

const output = fs.createWriteStream("ids.txt", { flags: "a" });

async function generateIDs() {
  console.time("Generation");
  for (let i = 0; i < TOTAL; i += CHUNK_SIZE) {
    let chunk = "";
    for (let j = 0; j < CHUNK_SIZE; j++) {
      const id = toBase62(BigInt(snow.nextId()));
      chunk += id + "\n";
    }
    output.write(chunk);
    console.log(`✅ Generated ${i + CHUNK_SIZE} IDs`);
  }
  output.end();
  console.timeEnd("Generation");
}

generateIDs();

This can easily scale to tens of crores of unique IDs with constant memory usage.

9. Best Choice

10. Conclusion

A strong ID generation system is the backbone of scalability.

By combining Snowflake with Base62 encoding, we get IDs that are:

• Fast
• Compact
• Globally unique
• Time-sortable

Perfect for e-commerce, URL shorteners, and microservice architectures.

Would you like me to write a follow-up on multi-threaded ID generation (using Node.js Workers) to scale to 100+ crore IDs? Comment below!

If you found this helpful, follow me for more backend & TypeScript scalability insights!

This content originally appeared on DEV Community and was authored by Srashti Gupta