手记

Java分布式ID入门教程

素胚勾勒不出你
2024-11-18 20:57:11浏览 37
概述

本文介绍了java分布式id入门的相关知识，包括分布式ID的基本概念、生成方法以及在实际项目中的应用实例。文中详细讲解了基于数据库、雪花算法和UUID的生成方法，并提供了完整的Java实现代码。此外，还讨论了在分布式系统中使用这些方法时可能遇到的问题及解决方案。

分布式ID的基本概念
什么是分布式ID

分布式ID是一种在分布式系统中生成全局唯一标识符的方法。在分布式环境中，由于数据分布在多个节点上，传统的单机系统中使用的自增ID或者其他依赖于本地数据库的生成方式不再适用。分布式ID需要确保每个生成的ID在全球范围内唯一，并且具有良好的可扩展性。

分布式ID的作用和应用场景

分布式ID的主要作用是生成全局唯一的标识符，确保在分布式系统中各个组件和节点之间能够正确无误地识别和通信。以下是一些典型的应用场景：

  1. 微服务架构：在微服务架构中，每个服务都有自己的数据库，需要一种方式生成全局唯一的ID来确保各服务之间数据的一致性和唯一性。
  2. 分布式系统：在分布式系统中，数据可能会分布在多个服务器上，每个服务器都需要生成唯一的ID。
  3. 消息系统：消息中间件中，每个消息都需要一个唯一的ID来标识，分布式ID可以确保消息ID的唯一性。
  4. 缓存系统：缓存系统中，每个缓存项都需要一个唯一的Key，使用分布式ID可以确保缓存项的唯一性。

示例代码

public class SimpleIdGenerator {
    public static void main(String[] args) {
        System.out.println("生成一个简单的全局唯一ID");
    }
}
分布式ID的生成方法
基于数据库的生成方法

基于数据库的生成方法通常依赖于数据库的自增ID。这是最简单的方法之一，但是它有一个明显的缺点：当数据库发生故障或者网络延迟时，自增ID的生成会受到影响。此外，当多个数据库实例时，自增ID可能会出现冲突。

示例代码

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;

public class DatabaseIdGenerator {
    private static final String SQL_INSERT = "INSERT INTO id_generator (id) VALUES (null)";
    private static final String SQL_SELECT = "SELECT LAST_INSERT_ID() AS id";

    public static void main(String[] args) {
        Connection conn = null;
        PreparedStatement stmt = null;
        ResultSet rs = null;
        try {
            Class.forName("com.mysql.jdbc.Driver");
            conn = DriverManager.getConnection("jdbc:mysql://localhost:3306/test", "root", "password");
            stmt = conn.prepareStatement(SQL_INSERT);
            stmt.executeUpdate();
            stmt = conn.prepareStatement(SQL_SELECT);
            rs = stmt.executeQuery();
            if (rs.next()) {
                int id = rs.getInt("id");
                System.out.println("Generated ID: " + id);
            }
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            try {
                if (rs != null) rs.close();
                if (stmt != null) stmt.close();
                if (conn != null) conn.close();
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
    }
}
基于雪花算法的生成方法

雪花算法（Snowflake Algorithm）是由Twitter开源的一种分布式ID生成算法。它使用时间戳、机器ID和序列号来生成全局唯一的ID。这种算法生成的ID是64位长，其中41位用于时间戳，10位用于机器ID，12位用于序列号。

示例代码

import java.util.concurrent.atomic.AtomicInteger;

public class SnowflakeIdGenerator {
    private static final long EPOCH = 1288834974657L; // 2010-12-31T11:35:57.465Z
    private static final long SEQUENCE_BIT = 12L;
    private static final long MACHINE_BIT = 10L;
    private static final long DATACENTER_BIT = 5L;
    private static final long MAX_DATACENTER_NUM = -1L ^ (-1L << DATACENTER_BIT);
    private static final long MAX_MACHINE_NUM = -1L ^ (-1L << MACHINE_BIT);
    private static final long MAX_SEQUENCE = -1L ^ (-1L << SEQUENCE_BIT);
    private static final long DATACENTER_ID_SHIFT = SEQUENCE_BIT;
    private static final long MACHINE_ID_SHIFT = SEQUENCE_BIT + DATACENTER_BIT;
    private static final long TIMESTAMP_SHIFT = SEQUENCE_BIT + MACHINE_BIT + DATACENTER_BIT;
    private static final long SEQUENCE_MASK = ~(-1L << SEQUENCE_BIT);

    private static final AtomicInteger sequence = new AtomicInteger(0);
    private static long lastTimestamp = -1L;
    private long datacenterId;
    private long machineId;

    public SnowflakeIdGenerator(long datacenterId, long machineId) {
        if (datacenterId < 0 || datacenterId > MAX_DATACENTER_NUM || machineId < 0 || machineId > MAX_MACHINE_NUM) {
            throw new IllegalArgumentException("数据中心ID和机器ID必须在合法范围内");
        }
        this.datacenterId = datacenterId;
        this.machineId = machineId;
    }

    public synchronized long nextId() {
        long timestamp = timeGen();
        if (timestamp < lastTimestamp) {
            throw new RuntimeException("时钟回拨");
        }
        if (lastTimestamp == timestamp) {
            sequence.compareAndSet(MAX_SEQUENCE, 0);
            timestamp = tilNextMillis(lastTimestamp);
        } else {
            sequence.set(0);
        }
        lastTimestamp = timestamp;
        return ((timestamp - EPOCH) << TIMESTAMP_SHIFT) | (datacenterId << DATACENTER_ID_SHIFT) | (machineId << MACHINE_ID_SHIFT) | sequence.getAndIncrement();
    }

    private long tilNextMillis(long lastTimestamp) {
        long timestamp = timeGen();
        while (timestamp <= lastTimestamp) {
            timestamp = timeGen();
        }
        return timestamp;
    }

    private long timeGen() {
        return System.currentTimeMillis();
    }

    public static void main(String[] args) {
        SnowflakeIdGenerator idGenerator = new SnowflakeIdGenerator(1, 1);
        long id = idGenerator.nextId();
        System.out.println("Generated ID: " + id);
    }
}
基于UUID的生成方法

UUID（Universally Unique Identifier）是全局唯一标识符的一种实现，生成的是128位的数字，一般表示为32字符的无符号整数。虽然UUID的生成是全局唯一的，但由于其长度较长，不适合用作数据库的主键。此外，UUID的生成速度较慢，性能不如雪花算法。

示例代码

import java.util.UUID;

public class UUIDIdGenerator {
    public static void main(String[] args) {
        UUID uuid = UUID.randomUUID();
        long mostSignificantBits = uuid.getMostSignificantBits();
        long leastSignificantBits = uuid.getLeastSignificantBits();
        System.out.println("Generated UUID: " + uuid.toString());
        System.out.println("Most Significant Bits: " + mostSignificantBits);
        System.out.println("Least Significant Bits: " + leastSignificantBits);
    }
}
Java实现分布式ID生成器
引入相关依赖

在Java项目中，可以通过Maven或Gradle来引入所需的依赖。这里以Maven为例，引入Spring Boot的依赖。

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-actuator</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-aop</artifactId>
    </dependency>
    <dependency>
        <groupId>org.apache.commons</groupId>
        <artifactId>commons-lang3</artifactId>
        <version>3.9</version>
    </dependency>
</dependencies>
使用雪花算法实现分布式ID生成器

在实际应用中，通常使用雪花算法来生成分布式ID。下面是一个简单的雪花算法实现。

示例代码

public class SnowflakeIdGenerator {
    private static final long EPOCH = 1288834974657L; // 2010-12-31T11:35:57.465Z
    private static final long SEQUENCE_BIT = 12L;
    private static final long MACHINE_BIT = 10L;
    private static final long DATACENTER_BIT = 5L;
    private static final long MAX_DATACENTER_NUM = -1L ^ (-1L << DATACENTER_BIT);
    private static final long MAX_MACHINE_NUM = -1L ^ (-1L << MACHINE_BIT);
    private static final long MAX_SEQUENCE = -1L ^ (-1L << SEQUENCE_BIT);
    private static final long DATACENTER_ID_SHIFT = SEQUENCE_BIT;
    private static final long MACHINE_ID_SHIFT = SEQUENCE_BIT + DATACENTER_BIT;
    private static final long TIMESTAMP_SHIFT = SEQUENCE_BIT + MACHINE_BIT + DATACENTER_BIT;
    private static final long SEQUENCE_MASK = ~(-1L << SEQUENCE_BIT);

    private static final AtomicInteger sequence = new AtomicInteger(0);
    private static long lastTimestamp = -1L;
    private long datacenterId;
    private long machineId;

    public SnowflakeIdGenerator(long datacenterId, long machineId) {
        if (datacenterId < 0 || datacenterId > MAX_DATACENTER_NUM || machineId < 0 || machineId > MAX_MACHINE_NUM) {
            throw new IllegalArgumentException("数据中心ID和机器ID必须在合法范围内");
        }
        this.datacenterId = datacenterId;
        this.machineId = machineId;
    }

    public synchronized long nextId() {
        long timestamp = timeGen();
        if (timestamp < lastTimestamp) {
            throw new RuntimeException("时钟回拨");
        }
        if (lastTimestamp == timestamp) {
            sequence.compareAndSet(MAX_SEQUENCE, 0);
            timestamp = tilNextMillis(lastTimestamp);
        } else {
            sequence.set(0);
        }
        lastTimestamp = timestamp;
        return ((timestamp - EPOCH) << TIMESTAMP_SHIFT) | (datacenterId << DATACENTER_ID_SHIFT) | (machineId << MACHINE_ID_SHIFT) | sequence.getAndIncrement();
    }

    private long tilNextMillis(long lastTimestamp) {
        long timestamp = timeGen();
        while (timestamp <= lastTimestamp) {
            timestamp = timeGen();
        }
        return timestamp;
    }

    private long timeGen() {
        return System.currentTimeMillis();
    }

    public static void main(String[] args) {
        SnowflakeIdGenerator idGenerator = new SnowflakeIdGenerator(1, 1);
        long id = idGenerator.nextId();
        System.out.println("Generated ID: " + id);
    }
}
使用第三方库实现分布式ID生成

除了自己实现雪花算法之外，还可以使用Java中的一些第三方库，例如facebook/jwt、twitter/snowflake等。

示例代码

以twitter/snowflake为例：

<dependency>
    <groupId>com.twitter</groupId>
    <artifactId>java-util</artifactId>
    <version>1.15.0</version>
</dependency>
import com.twitter.util.Time;
import com.twitter.util.Future;
import com.twitter.util.RandomGenerator;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicLong;

public class TwitterSnowflakeIdGenerator {
    private static final long EPOCH = 1288834974657L; // 2010-12-31T11:35:57.465Z
    private static final long SEQUENCE_BIT = 12L;
    private static final long MACHINE_BIT = 10L;
    private static final long DATACENTER_BIT = 5L;
    private static final long MAX_DATACENTER_NUM = -1L ^ (-1L << DATACENTER_BIT);
    private static final long MAX_MACHINE_NUM = -1L ^ (-1L << MACHINE_BIT);
    private static final long MAX_SEQUENCE = -1L ^ (-1L << SEQUENCE_BIT);
    private static final long DATACENTER_ID_SHIFT = SEQUENCE_BIT;
    private static final long MACHINE_ID_SHIFT = SEQUENCE_BIT + DATACENTER_BIT;
    private static final long TIMESTAMP_SHIFT = SEQUENCE_BIT + MACHINE_BIT + DATACENTER_BIT;
    private static final long SEQUENCE_MASK = ~(-1L << SEQUENCE_BIT);

    private static final AtomicLong sequence = new AtomicLong(0);
    private static long lastTimestamp = -1L;
    private long datacenterId;
    private long machineId;

    public TwitterSnowflakeIdGenerator(long datacenterId, long machineId) {
        if (datacenterId < 0 || datacenterId > MAX_DATACENTER_NUM || machineId < 0 || machineId > MAX_MACHINE_NUM) {
            throw new IllegalArgumentException("数据中心ID和机器ID必须在合法范围内");
        }
        this.datacenterId = datacenterId;
        this.machineId = machineId;
    }

    public synchronized long nextId() {
        long timestamp = timeGen();
        if (timestamp < lastTimestamp) {
            throw new RuntimeException("时钟回拨");
        }
        if (lastTimestamp == timestamp) {
            sequence.compareAndSet(MAX_SEQUENCE, 0);
            timestamp = tilNextMillis(lastTimestamp);
        } else {
            sequence.set(0);
        }
        lastTimestamp = timestamp;
        return ((timestamp - EPOCH) << TIMESTAMP_SHIFT) | (datacenterId << DATACENTER_ID_SHIFT) | (machineId << MACHINE_ID_SHIFT) | sequence.getAndIncrement();
    }

    private long tilNextMillis(long lastTimestamp) {
        long timestamp = timeGen();
        while (timestamp <= lastTimestamp) {
            timestamp = timeGen();
        }
        return timestamp;
    }

    private long timeGen() {
        return Time.fromCurrent().as(TimeUnit.MILLISECONDS).inMicros();
    }

    public static void main(String[] args) {
        TwitterSnowflakeIdGenerator idGenerator = new TwitterSnowflakeIdGenerator(1, 1);
        long id = idGenerator.nextId();
        System.out.println("Generated ID: " + id);
    }
}
分布式ID生成器的测试
单元测试

单元测试主要是测试生成的ID是否符合预期，包括唯一性、连续性等特性。可以使用JUnit框架来编写单元测试。

示例代码

import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;

public class SnowflakeIdGeneratorTest {
    private SnowflakeIdGenerator idGenerator = new SnowflakeIdGenerator(1, 1);

    @Test
    public void testUnique() {
        long id1 = idGenerator.nextId();
        long id2 = idGenerator.nextId();
        assertTrue(id1 != id2);
    }

    @Test
    public void testSequential() {
        long id1 = idGenerator.nextId();
        long id2 = idGenerator.nextId();
        assertTrue(id1 < id2);
    }

    @Test
    public void testClockBackward() {
        long lastTimestamp = System.currentTimeMillis();
        long id = idGenerator.nextId();
        long timestamp = timeGen();
        assertTrue(timestamp >= lastTimestamp);
    }

    public long timeGen() {
        return System.currentTimeMillis();
    }
}
性能测试

性能测试主要是测试生成ID的速度和吞吐量。可以使用JMeter或Apache Benchmark等工具进行性能测试。

示例代码

import org.junit.jupiter.api.Test;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;

public class SnowflakeIdGeneratorTest {
    private SnowflakeIdGenerator idGenerator = new SnowflakeIdGenerator(1, 1);

    @Test
    public void testPerformance() {
        ExecutorService executor = Executors.newFixedThreadPool(10);
        for (int i = 0; i < 10000; i++) {
            final int index = i;
            executor.submit(() -> {
                long id = idGenerator.nextId();
                System.out.println("Generated ID: " + id);
            });
        }
        executor.shutdown();
        try {
            executor.awaitTermination(1, TimeUnit.MINUTES);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}
实际项目中的应用
在微服务架构中的应用实例

在微服务架构中，每个服务都有自己的数据库，需要一种方式生成全局唯一的ID来确保各服务之间数据的一致性和唯一性。可以使用雪花算法来生成分布式ID，然后在每个服务中使用相同的算法生成ID。

示例代码

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@SpringBootApplication
public class MicroserviceApplication {
    public static void main(String[] args) {
        SpringApplication.run(MicroserviceApplication.class, args);
    }
}

@RestController
class GeneratorController {
    private SnowflakeIdGenerator idGenerator = new SnowflakeIdGenerator(1, 1);

    @GetMapping("/generate")
    public long generateId() {
        return idGenerator.nextId();
    }
}
在数据库主键生成中的应用实例

在数据库主键生成中，可以使用雪花算法来生成全局唯一的ID。这样可以在不依赖数据库自增ID的情况下，确保主键的全局唯一性。

示例代码

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.jdbc.datasource.DriverManagerDataSource;

@SpringBootApplication
public class DatabaseApplication {
    public static void main(String[] args) {
        SpringApplication.run(DatabaseApplication.class, args);
    }

    public static class ApplicationRunner implements org.springframework.boot.CommandLineRunner {
        private JdbcTemplate jdbcTemplate;
        private SnowflakeIdGenerator idGenerator = new SnowflakeIdGenerator(1, 1);

        public ApplicationRunner(JdbcTemplate jdbcTemplate) {
            this.jdbcTemplate = jdbcTemplate;
        }

        @Override
        public void run(String... args) {
            jdbcTemplate.execute("CREATE TABLE IF NOT EXISTS user(id BIGINT PRIMARY KEY, name VARCHAR(100))");
            jdbcTemplate.execute("INSERT INTO user(id, name) VALUES (?, 'User 1')", idGenerator.nextId());
            jdbcTemplate.execute("INSERT INTO user(id, name) VALUES (?, 'User 2')", idGenerator.nextId());
        }
    }

    @org.springframework.boot.SpringApplicationConfiguration
    public static class ApplicationConfig {
        public JdbcTemplate jdbcTemplate() {
            DriverManagerDataSource dataSource = new DriverManagerDataSource();
            dataSource.setDriverClassName("com.mysql.jdbc.Driver");
            dataSource.setUrl("jdbc:mysql://localhost:3306/test");
            dataSource.setUsername("root");
            dataSource.setPassword("password");
            return new JdbcTemplate(dataSource);
        }
    }
}
常见问题及解决方案
常见问题分析
  1. 时钟回拨：当系统时钟发生回拨时，雪花算法会抛出异常。
  2. 序列号溢出：当序列号达到最大值时，需要等待下一毫秒才能生成新的ID。
  3. 性能问题：在高并发环境下，生成ID的性能可能成为瓶颈。
解决方案与最佳实践
  1. 时钟回拨：可以通过设置缓冲区来解决时钟回拨问题。
  2. 序列号溢出：序列号溢出时，可以等待下一毫秒再生成新的ID。
  3. 性能问题：可以通过增加机器ID和数据中心ID的范围来提高性能。

示例代码

import java.util.concurrent.atomic.AtomicLong;

public class ImprovedSnowflakeIdGenerator extends SnowflakeIdGenerator {
    private static final long EPOCH = 1288834974657L; // 2010-12-31T11:35:57.465Z
    private static final long SEQUENCE_BIT = 12L;
    private static final long MACHINE_BIT = 10L;
    private static final long DATACENTER_BIT = 5L;
    private static final long MAX_DATACENTER_NUM = -1L ^ (-1L << DATACENTER_BIT);
    private static final long MAX_MACHINE_NUM = -1L ^ (-1L << MACHINE_BIT);
    private static final long MAX_SEQUENCE = -1L ^ (-1L << SEQUENCE_BIT);
    private static final long DATACENTER_ID_SHIFT = SEQUENCE_BIT;
    private static final long MACHINE_ID_SHIFT = SEQUENCE_BIT + DATACENTER_BIT;
    private static final long TIMESTAMP_SHIFT = SEQUENCE_BIT + MACHINE_BIT + DATACENTER_BIT;
    private static final long SEQUENCE_MASK = ~(-1L << SEQUENCE_BIT);

    private static final AtomicLong sequence = new AtomicLong(0);
    private static long lastTimestamp = -1L;
    private long datacenterId;
    private long machineId;

    public ImprovedSnowflakeIdGenerator(long datacenterId, long machineId) {
        if (datacenterId < 0 || datacenterId > MAX_DATACENTER_NUM || machineId < 0 || machineId > MAX_MACHINE_NUM) {
            throw new IllegalArgumentException("数据中心ID和机器ID必须在合法范围内");
        }
        this.datacenterId = datacenterId;
        this.machineId = machineId;
    }

    public synchronized long nextId() {
        long timestamp = timeGen();
        if (timestamp < lastTimestamp) {
            throw new RuntimeException("时钟回拨");
        }
        if (lastTimestamp == timestamp) {
            sequence.compareAndSet(MAX_SEQUENCE, 0);
            timestamp = tilNextMillis(lastTimestamp);
        } else {
            sequence.set(0);
        }
        lastTimestamp = timestamp;
        return ((timestamp - EPOCH) << TIMESTAMP_SHIFT) | (datacenterId << DATACENTER_ID_SHIFT) | (machineId << MACHINE_ID_SHIFT) | sequence.getAndIncrement();
    }

    private long tilNextMillis(long lastTimestamp) {
        long timestamp = timeGen();
        while (timestamp <= lastTimestamp) {
            timestamp = timeGen();
        }
        return timestamp;
    }

    private long timeGen() {
        return System.currentTimeMillis();
    }

    public static void main(String[] args) {
        ImprovedSnowflakeIdGenerator idGenerator = new ImprovedSnowflakeIdGenerator(1, 1);
        long id = idGenerator.nextId();
        System.out.println("Generated ID: " + id);
    }
}
相关标签
Java
算法与数据结构
0人推荐
随时随地看视频
慕课网APP