优化Cassandra数据库读取性能的最佳实践
配置合适的硬件资源
为了提高Cassandra的读取效率,配置足够的内存和快速磁盘非常重要。增加节点上的RAM可以显著提升缓存命中率,减少磁盘I/O操作次数。
调整一致性级别
较低的一致性设置能够加快查询响应时间。对于那些对数据最终一致性的容忍度较高的应用来说,在不影响业务逻辑的前提下适当降低写入或读取请求的一致性等级是一个有效的策略。
使用高效的分区键设计
合理的表结构设计特别是精心挑选的主键(Partition Key),有助于均匀分布数据并最小化热点问题的发生概率。这不仅有利于负载均衡还能有效防止某些特定范围扫描带来的性能瓶颈。
启用读修复机制
通过定期执行Read Repair来同步副本间的数据差异可以在后台自动解决不一致的情况而不影响前台服务的质量;同时也可以考虑手动触发full repairs以确保整个集群内所有副本都保持最新状态。
-- 手动启动完全修复过程
nodetool repair -full keyspace_name;
设置适当的缓存参数
调整row_cache_size_in_mb 和 key_cache_size_in_mb 参数可改善频繁访问记录的速度表现。合理规划这些缓冲区大小既不会占用过多物理内存又能在很大程度上缓解实际存储介质的压力从而增强整体吞吐量。
实施分页技术处理大数据集
当面对海量级的结果返回时采用fetch size限制每次获取的数量并通过page state实现增量加载能避免一次性拉取全部内容造成的网络拥塞以及客户端解析超时等问题发生。
Spring Data Reactive与Apache Cassandra集成教程及最佳实践
配置依赖项
为了使项目能够利用Spring Data Reactive连接到Cassandra数据库,需在pom.xml
文件中加入必要的依赖库。这些依赖包含了用于Reactive编程的支持以及针对Cassandra的操作接口。
<dependencies>
<!-- Spring Boot Starter for WebFlux -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webflux</artifactId>
</dependency>
<!-- Spring Data Reactive support for Cassandra -->
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-cassandra-reactive</artifactId>
</dependency>
<!-- Other dependencies as needed... -->
</dependencies>
设置数据源配置
创建一个名为application.yml
的应用程序配置文件来定义访问Cassandra集群所需的参数,包括联系点、端口和其他网络设置等。
spring:
data:
cassandra:
contact-points: localhost
port: 9042
keyspace-name: mykeyspace
schema-action: create_if_not_exists
reactive.enabled: true
定义实体类
通过继承自@Table
注解的POJO(Plain Old Java Object),可以映射表结构至Java对象模型之中;同时采用Lombok简化getter/setter方法编写工作量。
import lombok.Data;
import org.springframework.cassandra.core.PrimaryKeyType;
import org.springframework.data.annotation.CreatedDate;
import org.springframework.data.cassandra.core.mapping.Column;
import org.springframework.data.cassandra.core.mapping.PrimaryKeyColumn;
import org.springframework.data.cassandra.core.mapping.Table;
@Table("users")
@Data
public class User {
@PrimaryKeyColumn(name = "id", ordinal = 0, type = PrimaryKeyType.PARTITIONED)
private String id;
@Column("name")
private String name;
@CreatedDate
@Column("created_at")
private Instant createdAt;
}
创建存储库接口
基于CRUDRepository抽象层构建特定领域逻辑操作的数据持久化组件——即仓库模式实现体。这里特别强调的是返回类型采用了Publisher/Mono/Flux形式以支持异步处理流程。
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import org.springframework.data.repository.reactive.ReactiveCrudRepository;
public interface UserRepository extends ReactiveCrudRepository<User, String> {
Flux<User> findByName(String name);
}
编写服务层业务逻辑
最后,在Service Bean内部调用上述声明好的仓储API完成具体功能模块的设计开发任务。此部分代码片段展示了如何查询指定条件下的记录列表并将其转换成JSON响应格式输出给前端页面展示使用。
@Service
public class UserService {
private final UserRepository userRepository;
public UserService(UserRepository userRepository) {
this.userRepository = userRepository;
}
public Mono<List<User>> getUsersByName(String name){
return userRepository.findByName(name).collectList();
}
}
Getting Started
Reference Documentation
For further reference, please consider the following sections:
- Official Apache Maven documentation
- Spring Boot Maven Plugin Reference Guide
- Create an OCI image
- Spring Data Redis (Access+Driver)
- Spring Data Reactive Redis
- Spring Data MongoDB
- Spring Data Reactive MongoDB
- Spring Data Elasticsearch (Access+Driver)
- Spring Data for Apache Solr
- Spring Data for Apache Cassandra
- Spring Data Reactive for Apache Cassandra
Guides
The following guides illustrate how to use some features concretely:
/*
* Copyright 2007-present the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import java.net.*;
import java.io.*;
import java.nio.channels.*;
import java.util.Properties;
public class MavenWrapperDownloader {
private static final String WRAPPER_VERSION = "0.5.6";
/**
* Default URL to download the maven-wrapper.jar from, if no 'downloadUrl' is provided.
*/
private static final String DEFAULT_DOWNLOAD_URL = "https://repo.maven.apache.org/maven2/io/takari/maven-wrapper/"
+ WRAPPER_VERSION + "/maven-wrapper-" + WRAPPER_VERSION + ".jar";
/**
* Path to the maven-wrapper.properties file, which might contain a downloadUrl property to
* use instead of the default one.
*/
private static final String MAVEN_WRAPPER_PROPERTIES_PATH =
".mvn/wrapper/maven-wrapper.properties";
/**
* Path where the maven-wrapper.jar will be saved to.
*/
private static final String MAVEN_WRAPPER_JAR_PATH =
".mvn/wrapper/maven-wrapper.jar";
/**
* Name of the property which should be used to override the default download url for the wrapper.
*/
private static final String PROPERTY_NAME_WRAPPER_URL = "wrapperUrl";
public static void main(String args[]) {
System.out.println("- Downloader started");
File baseDirectory = new File(args[0]);
System.out.println("- Using base directory: " + baseDirectory.getAbsolutePath());
// If the maven-wrapper.properties exists, read it and check if it contains a custom
// wrapperUrl parameter.
File mavenWrapperPropertyFile = new File(baseDirectory, MAVEN_WRAPPER_PROPERTIES_PATH);
String url = DEFAULT_DOWNLOAD_URL;
if(mavenWrapperPropertyFile.exists()) {
FileInputStream mavenWrapperPropertyFileInputStream = null;
try {
mavenWrapperPropertyFileInputStream = new FileInputStream(mavenWrapperPropertyFile);
Properties mavenWrapperProperties = new Properties();
mavenWrapperProperties.load(mavenWrapperPropertyFileInputStream);
url = mavenWrapperProperties.getProperty(PROPERTY_NAME_WRAPPER_URL, url);
} catch (IOException e) {
System.out.println("- ERROR loading '" + MAVEN_WRAPPER_PROPERTIES_PATH + "'");
} finally {
try {
if(mavenWrapperPropertyFileInputStream != null) {
mavenWrapperPropertyFileInputStream.close();
}
} catch (IOException e) {
// Ignore ...
}
}
}
System.out.println("- Downloading from: " + url);
File outputFile = new File(baseDirectory.getAbsolutePath(), MAVEN_WRAPPER_JAR_PATH);
if(!outputFile.getParentFile().exists()) {
if(!outputFile.getParentFile().mkdirs()) {
System.out.println(
"- ERROR creating output directory '" + outputFile.getParentFile().getAbsolutePath() + "'");
}
}
System.out.println("- Downloading to: " + outputFile.getAbsolutePath());
try {
downloadFileFromURL(url, outputFile);
System.out.println("Done");
System.exit(0);
} catch (Throwable e) {
System.out.println("- Error downloading");
e.printStackTrace();
System.exit(1);
}
}
private static void downloadFileFromURL(String urlString, File destination) throws Exception {
if (System.getenv("MVNW_USERNAME") != null && System.getenv("MVNW_PASSWORD") != null) {
String username = System.getenv("MVNW_USERNAME");
char[] password = System.getenv("MVNW_PASSWORD").toCharArray();
Authenticator.setDefault(new Authenticator() {
@Override
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(username, password);
}
});
}
URL website = new URL(urlString);
ReadableByteChannel rbc;
rbc = Channels.newChannel(website.openStream());
FileOutputStream fos = new FileOutputStream(destination);
fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
fos.close();
rbc.close();
}
}
distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.6.3/apache-maven-3.6.3-bin.zip
wrapperUrl=https://repo.maven.apache.org/maven2/io/takari/maven-wrapper/0.5.6/maven-wrapper-0.5.6.jar
package com.example.demo;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication
public class DemoApplication {
public static void main(String[] args) {
SpringApplication.run(DemoApplication.class, args);
}
}
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.3.0.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.example</groupId>
<artifactId>demo</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>demo</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-cassandra</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-cassandra-reactive</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-mongodb</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-mongodb-reactive</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis-reactive</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-solr</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>io.projectreactor</groupId>
<artifactId>reactor-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>