学习方式:
- 上手就用(是什么、怎么用!)
- 基本的理论先学习,然后将知识融汇贯通!
redis课程安排:
- nosql讲解
- 阿里巴巴架构演进
- nosql数据模型
- Nosql四大分类
- CAP
- BASE
- Redis 入门
- Redis安装(Window & Linux服务器)五大基本数据类型
- String
- List
- Set
- Hash
- Zset
- 三种特殊数据类型
- geo
- hyperloglog
- bitmap
- Redis配置详解
- Redis持久化
- RDB
- AOF
- Redis事务操作
- Redis实现订阅发布
- Redis主从复制
- Redis哨兵模式
- 缓存穿透及解决方案
- 缓存击穿及解决方案
- 缓存雪崩及解决方案
- 基础API之Jedis 详解
- SpringBoot集成Redis操作
- Redis的实践分析
一、Nosql概述
为什么使用Nosql
1.1、单机Mysql时代
DAL:数据访问层(Data Access Layer)
90年代,一个网站的访问量一般不会太大,单个数据库完全够用。随着用户增多,网站出现以下问题:
-
数据量增加到一定程度,单机数据库就放不下了
-
数据的索引(B+ Tree),一个机器内存也存放不下
-
访问量变大后(读写混合),一台服务器承受不住。
只要你开始出现以上的三种情况之一,那么你就必须要晋级!
1.2、Memcached(缓存) + Mysql + 垂直拆分(读写分离)
网站80%的情况都是在读,每次都要去查询数据库的话就十分的麻烦!所以说我们希望减轻数据库的压力,我们可以使用缓存来保证效率!
优化过程经历了以下几个过程:
(1)优化数据库的数据结构和索引(难度大)
(2)文件缓存,通过IO流获取比每次都访问数据库效率略高,但是流量爆炸式增长时候,IO流也承受不了
(3)MemCache,当时最热门的技术,通过在数据库和数据库访问层之间加上一层缓存,第一次访问时查询数据库,将结果保存到缓存,后续的查询先检查缓存,若有直接拿去使用,效率显著提升。
1.3、分库分表 + 水平拆分 + Mysql集群
技术和业务在发展的同时,对人的要求也越来越高!
本质:数据库(读,写)
早些年MyISAM:表锁(即在查询的时候锁定整个表),十分影响效率!高并发下就会出现严重的锁问题
转战Innodb:行锁(在查询的时候锁定表的某一行)
慢慢的就开始使用分库分表来解决写的压力! MySQL在哪个年代推出了表分区!这个并没有多少公司使用!
MySQL的集群,很好满足那个年代的所有需求!
1.4、如今最近的年代
2010–2020十年之间,世界已经发生了翻天覆地的变化;(定位,也是一种数据,音乐,热榜!)
MySQL等关系型数据库就不够用了!数据量很多,变化很快~!
MySQL有的使用它来存储一些比较大的文件,博客,图片!数据库表很大,效率就低了!如果有一种数据库来专门处理这种数据,MySQL压力就变得十分小(研究如何处理这些问题!)大数据的IO压力下,表几乎没法更大!
目前一个基本的互联网项目架构模型
为什么要用NoSQL ?
用户的个人信息,社交网络,地理位置。用户自己产生的数据,用户日志等等爆发式增长!
这时候我们就需要使用NoSQL数据库的,Nosql可以很好的处理以上的情况!
1.5、什么是Nosql
NoSQL = Not Only SQL(不仅仅是SQL) Not Only Structured Query Language
关系型数据库:列+行,同一个表下数据的结构是一样的。
非关系型数据库:数据存储没有固定的格式,并且可以进行横向扩展。
NoSQL泛指非关系型数据库,随着web2.0互联网的诞生,传统的关系型数据库很难对付web2.0时代!尤其是超大规模的高并发的社区,暴露出来很多难以克服的问题,NoSQL在当今大数据环境下发展的十分迅速,Redis是发展最快的。
1.5.1、Nosql特点
(1)方便扩展(数据之间没有关系,很好扩展!)
(2)大数据量高性能(Redis一秒可以写8万次,读11万次,NoSQL的缓存记录级,是一种细粒度的缓存,性能会比较高!)
(3)数据类型是多样型的!(不需要事先设计数据库,随取随用)
(4)传统的 RDBMS(关系型数据库:Relational Database Management System) 和 NoSQL
传统的 RDBMS(关系型数据库)
- 结构化组织
- SQL
- 数据和关系都存在单独的表中 row col
- 数据操作语言,数据定义语言
- 严格的一致性
- 基础的事务
- ...
Nosql
- 不仅仅是数据
- 没有固定的查询语言
- 多种存储方式:键值对存储,列存储,文档存储,图形数据库(社交关系)
- 最终一致性
- CAP定理和BASE
- 高性能,高可用,高扩展
- ...
1.5.2、了解:3V + 3高
大数据时代的3V :主要是描述问题的
- 海量Velume
- 多样Variety
- 实时Velocity
大数据时代的3高 : 主要是对程序的要求
- 高并发
- 高可扩(随时水平拆分,机器不够了,可以扩展机器来解决)
- 高性能(保证用户体验和性能)
真正在公司中的实践:NoSQL + RDBMS 一起使用才是最强的。
1.6、阿里巴巴演进分析
推荐阅读:阿里云的这群疯子
# 商品信息
- 一般存放在关系型数据库:Mysql,阿里巴巴使用的Mysql都是经过内部改动的。
# 商品描述、评论(文字居多)
- 文档型数据库:MongoDB
# 图片
- 分布式文件系统 FastDFS
- 淘宝:TFS
- Google: GFS
- Hadoop: HDFS
- 阿里云: oss
# 商品关键字 用于搜索
- 搜索引擎:solr,elasticsearch
- 阿里:Isearch 多隆(技术大佬)
# 商品热门的波段信息
- 内存数据库:Redis,Memcache
# 商品交易,外部支付接口
- 第三方应用
要知道,一个简单地网页背后的技术一定不是大家所想的那么简单!
大型互联网应用问题︰
- 数据类型太多了!
- 数据源繁多,经常重构!
- 数据要改造,大面积改造?
解决办法:
这里以上都是NoSQL入门概述,不仅能够提高大家的知识,还可以帮助大家了解大厂的工作内容!
1.7、Nosql的四大分类
1.7.1、KV键值对
- 新浪:Redis
- 美团:Redis + Tair
- 阿里、百度:Redis + Memcache
1.7.2、文档型数据库(bson数据格式,类似与json)
- MongoDB(掌握)
- 基于分布式文件存储的数据库。C++编写,用于处理大量文档。
- MongoDB是RDBMS和NoSQL的中间产品。MongoDB是非关系型数据库中功能最丰富的,NoSQL中最像关系型数据库的数据库。
- ConthDB
1.7.3、列存储数据库
- HBase(大数据必学)
- 分布式文件系统
1.7.4、图关系数据库
用于广告推荐,社交网络
- Neo4j、InfoGrid
1.7.5、总结
分类 | Examples举例 | 典型应用场景 | 数据模型 | 优点 | 缺点 |
---|---|---|---|---|---|
键值对(key-value) | Tokyo Cabinet/Tyrant, Redis, Voldemort, Oracle BDB | 内容缓存,主要用于处理大量数据的高访问负载,也用于一些日志系统等等。 | Key 指向 Value 的键值对,通常用hash table来实现 | 查找速度快 | 数据无结构化,通常只被当作字符串或者二进制数据 |
列存储数据库 | Cassandra, HBase, Riak | 分布式的文件系统 | 以列簇式存储,将同一列数据存在一起 | 查找速度快,可扩展性强,更容易进行分布式扩展 | 功能相对局限 |
文档型数据库 | CouchDB, MongoDb | Web应用(与Key-Value类似,Value是结构化的,不同的是数据库能够了解Value的内容) | Key-Value对应的键值对,Value为结构化数据 | 数据结构要求不严格,表结构可变,不需要像关系型数据库一样需要预先定义表结构 | 查询性能不高,而且缺乏统一的查询语法。 |
图形(Graph)数据库 | Neo4J, InfoGrid, Infinite Graph | 社交网络,推荐系统等。专注于构建关系图谱 | 图结构 | 利用图结构相关算法。比如最短路径寻址,N度关系查找等 | 很多时候需要对整个图做计算才能得出需要的信息,而且这种结构不太好做分布式的集群 |
二、Redis入门
1、概述
1.1、Redis是什么?
Redis(Remote Dictionary Server ),即远程字典服务。
是一个开源的使用ANSI C语言编写、支持网络、可基于内存亦可持久化的日志型、Key-Value数据库,并提供多种语言的API。
与memcached一样,为了保证效率,数据都是缓存在内存中。区别的是redis会周期性的把更新的数据写入磁盘或者把修改操作写入追加的记录文件,并且在此基础上实现了master-slave(主从)同步。
1.2、Redis能该干什么?
(1)内存存储、持久化,内存是断电即失的,所以需要持久化(RDB、AOF)
(2)高效率、用于高速缓冲
(3)发布订阅系统
(4)地图信息分析
(5)计时器、计数器(eg:浏览量)
(6)。。。
1.3、特性
(1)多样的数据类型
(2)持久化
(3)集群
(4)事务
2、环境搭建
官网:https://redis.io/
中文官网:http://www.redis.cn/
推荐使用Linux服务器学习。
windows版本的Redis已经停更很久了,不推荐使用
Linux安装
(1)下载安装包!redis-7.0.4.tar.gz
(2)解压Redis的安装包!程序一般放在 /opt
目录下
cd /opt/redis
tar -zvxf redis-7.0.4.tar.gz
(3)基本环境安装
yum install gcc-c++
# 然后进入redis目录下执行
cd redis-7.0.4
make
# 然后执行
make install #查看是否安装完成
1)redis默认安装路径 /usr/local/bin
2)将redis的配置文件复制到 程序安装目录 /usr/local/bin/kconfig
下,以后就操作该目录下的配置文件,源配置文件保留
3)redis默认不是后台启动的,需要修改配置文件!
4)通过制定的配置文件启动redis服务
redis-server yang-config/redis.conf
5)使用redis-cli连接指定的端口号测试,Redis的默认端口6379
redis-cli -p 6379
6)查看redis进程是否开启
7)关闭Redis服务 shutdown
8)再次查看进程是否存在
9)后面我们会使用单机多Redis启动集群测试
测试性能
redis-benchmark:Redis官方提供的性能测试工具,参数选项如下:
序号 | 选项 | 描述 | 默认值 |
---|---|---|---|
1 | -h | 指定服务器主机名 | 127.0.0.1 |
2 | -p | 指定服务器端口 | 6379 |
3 | -s | 指定服务器 socket | |
4 | -c | 指定并发连接数 | 50 |
5 | -n | 指定请求数 | 10000 |
6 | -d | 以字节的形式指定 SET/GET 值的数据大小 | 2 |
7 | -k | 1=keep alive 0=reconnect | 1 |
8 | -r | SET/GET/INCR 使用随机 key, SADD 使用随机值 | |
9 | -P | 通过管道传输 请求 | 1 |
10 | -q | 强制退出 redis。仅显示 query/sec 值 | |
11 | –csv | 以 CSV 格式输出 | |
12 | -l(L 的小写字母) | 生成循环,永久执行测试 | |
13 | -t | 仅运行以逗号分隔的测试命令列表。 | |
14 | -I(i 的大写字母) | Idle 模式。仅打开 N 个 idle 连接并等待。 |
简单测试:
# 测试:100个并发连接 100000请求
redis-benchmark -h localhost -p 6379 -c 100 -n 100000
docker安装运行redis
[root@mycentos harbor]# docker pull redis:7.0.4
[root@mycentos harbor]# docker run -d --name redis -p 6379:6379 redis:7.0.4
[root@mycentos harbor]# docker exec -it redis redis-cli
3、基础知识
redis默认有16个数据库。
默认使用的第0个;
16个数据库为:DB 0~DB 15
默认使用DB 0 ,可以使用select n
切换到DB n,dbsize
可以查看当前数据库的大小,与key数量相关。
127.0.0.1:6379> config get databases # 命令行查看数据库数量databases
1) "databases"
2) "16"
127.0.0.1:6379> select 8 # 切换数据库 DB 8
OK
127.0.0.1:6379[8]> dbsize # 查看数据库大小
(integer) 0
# 不同数据库之间 数据是不能互通的,并且dbsize 是根据库中key的个数。
127.0.0.1:6379> set name aa
OK
127.0.0.1:6379> SELECT 8
OK
127.0.0.1:6379[8]> get name # db8中并不能获取db0中的键值对。
(nil)
127.0.0.1:6379[8]> DBSIZE
(integer) 0
127.0.0.1:6379[8]> SELECT 0
OK
127.0.0.1:6379> keys * # 查看当前数据库中所有的key。
1) "counter:__rand_int__"
2) "mylist"
3) "name"
4) "key:__rand_int__"
5) "myset:__rand_int__"
127.0.0.1:6379> DBSIZE # size和key个数相关
(integer) 5
127.0.0.1:6379> flushdb #清空当前数据库中的键值对。
OK
127.0.0.1:6379> keys *
(empty array)
127.0.0.1:6379> flushall #清空所有数据库的键值对。
OK
127.0.0.1:6379>
Redis是单线程的!
Redis是很快的,官方表示,Redis是基于内存操作,CPU不是Redis性能瓶颈,Redis的瓶颈是根据机器的内存和网络带宽,既然可以使用单线程来实现,就使用单线程了!所以就使用了单线程了!
Redis是C语言写的,官方提供的数据为100000+的QPS,完全不比同样是使用key-value的Memecache差!
Redis为什么单线程还这么快?
误区1:高性能的服务器一定是多线程的?
误区2:多线程(CPU上下文会切换!)一定比单线程效率高!
核心:Redis是将所有的数据放在内存中的,所以说使用单线程去操作效率就是最高的,多线程(CPU上下文会切换:耗时的操作!),对于内存系统来说,如果没有上下文切换效率就是最高的,多次读写都是在一个CPU上的,在内存存储数据情况下,单线程就是最佳的方案。
三、五大数据类型
Redis 是一个开源(BSD许可)的,内存中的数据结构存储系统,它可以用作数据库、缓存和消息中间件。 它支持多种类型的数据结构,如 字符串(strings), 散列(hashes), 列表(lists), 集合(sets), 有序集合(sorted sets) 与范围查询, bitmaps, hyperloglogs 和 地理空间(geospatial) 索引半径查询。 Redis 内置了 复制(replication),LUA脚本(Lua scripting), LRU驱动事件(LRU eviction),事务(transactions) 和不同级别的 磁盘持久化(persistence), 并通过 Redis哨兵(Sentinel)和自动 分区(Cluster)提供高可用性(high availability)。
1、Redis-key
在redis中无论什么数据类型,在数据库中都是以key-value形式保存,通过进行对Redis-key的操作,来完成对数据库中数据的操作。
下面学习的命令:
exists key
:判断键是否存在del key
:删除键值对move key db
:将键值对移动到指定数据库expire key second
:设置键值对的过期时间type key
:查看value的数据类型
127.0.0.1:6379> keys * # 查看当前数据库所有key
(empty list or set)
127.0.0.1:6379> set name qinjiang # set key
OK
127.0.0.1:6379> set age 20
OK
127.0.0.1:6379> keys *
1) "age"
2) "name"
127.0.0.1:6379> move age 1 # 将键值对移动到指定数据库
(integer) 1
127.0.0.1:6379> EXISTS age # 判断键是否存在
(integer) 0 # 不存在
127.0.0.1:6379> EXISTS name
(integer) 1 # 存在
127.0.0.1:6379> SELECT 1
OK
127.0.0.1:6379[1]> keys *
1) "age"
127.0.0.1:6379[1]> del age # 删除键值对
(integer) 1 # 删除个数
127.0.0.1:6379> set age 20
OK
127.0.0.1:6379> EXPIRE age 15 # 设置键值对的过期时间
(integer) 1 # 设置成功 开始计数
127.0.0.1:6379> ttl age # 查看key的过期剩余时间
(integer) 13
127.0.0.1:6379> ttl age
(integer) -2 # -2 表示key过期,-1表示key未设置过期时间
127.0.0.1:6379> get age # 过期的key 会被自动delete
(nil)
127.0.0.1:6379> keys *
1) "name"
127.0.0.1:6379> type name # 查看value的数据类型
string
2、String(字符串)
命令:
命令 | 描述 |
---|---|
APPEND key value | 向指定的key的value后追加字符串 |
DECR/INCR key | 将指定key的value数值进行+1/-1(仅对于数字) |
INCRBY/DECRBY key n | 按指定的步长对数值进行加减 |
INCRBYFLOAT key n | 按浮点数增加步长 |
STRLEN key | 获取key保存值的字符串长度 |
GETRANGE key start end | 按起止位置获取字符串(闭区间,起止位置都取) |
SETRANGE key offset value | 用指定的value 替换key中 offset开始的值 |
GETSET key value | 将给定 key 的值设为 value ,并返回 key 的旧值(old value)。 |
SETNX key value | 仅当key不存在时进行set |
SETEX key seconds value | set 键值对并设置过期时间 |
MSET key1 value1 [key2 value2…] | 批量set键值对 |
MSETNX key1 value1 [key2 value2…] | 批量设置键值对,仅当参数中所有的key都不存在时执行 |
MGET key1 [key2…] | 批量获取多个key保存的值 |
PSETEX key milliseconds value | 和 SETEX 命令相似,但它以毫秒为单位设置 key 的生存时间 |
getset key value | 如果不存在值,则返回nil,如果存在值,获取原来的值,并设置新的值 |
示例:
127.0.0.1:6379> set key1 va
OK
127.0.0.1:6379> get key1
"va"
127.0.0.1:6379> append key1 beijing
(integer) 9
127.0.0.1:6379> get key1
"vabeijing"
127.0.0.1:6379> append name1 yangyang # 如果当前key不存在,则相当于set key
(integer) 8
127.0.0.1:6379> strlen key1 # 获取字符串的长度
(integer) 16
###################################################################
# i++操作
127.0.0.1:6379> set views 0
OK
127.0.0.1:6379> get views
"0"
127.0.0.1:6379> incr views # 自增1,默认
(integer) 1
127.0.0.1:6379> incr views
(integer) 2
127.0.0.1:6379> decr views # 自减1,默认
(integer) 1
127.0.0.1:6379> INCRBY views 5 # 按步长自增
(integer) 6
127.0.0.1:6379> INCRBY views 5
(integer) 11
127.0.0.1:6379> DECRBY views 3 # 按步长自减
(integer) 8
127.0.0.1:6379> DECRBY views 3
(integer) 5
#####################################################
127.0.0.1:6379> set key1 hello,world
OK
127.0.0.1:6379> GETRANGE key1 0 6 # 获取指定长度的字符串 [0,6]
"hello,w"
127.0.0.1:6379> GETRANGE key1 0 -1 # 获取全部的字符串
"hello,world"
################################################################
127.0.0.1:6379> SET key2 abcdef
OK
127.0.0.1:6379> get key2
"abcdef"
127.0.0.1:6379> SETRANGE key2 2 xx #从指定索引处替换
(integer) 6
127.0.0.1:6379> get key2
"abxxef"
####################################################################
#setex (set with expire) #设置过期时间
#setnx (set if not exist) #不存在再设置
127.0.0.1:6379> SETEX key3 30 hello # 设置key3值为hello,30s后过期
OK
127.0.0.1:6379> setnx key2 djfheuw # 如果不存在key2创建key2值,
(integer) 0
#################################################################
127.0.0.1:6379> mset k1 v1 k2 v2 k3 v3 # 批量创建
OK
127.0.0.1:6379> mget k1 k2 k3 # 批量获取
1) "v1"
2) "v2"
3) "v3"
127.0.0.1:6379> msetnx k1 vvv1 k4 v4 # msetnx:原子性操作,要么都成功,要么都失败。
(integer) 0
##################################################################
# 对象
set user:1 {name:zhangsan,age:3} # 设置一个user:1 对象 值为 json字符来保存一个对象!
# 这里的key是一个巧妙的设计: user:{id}:{field} ,如此设计在Redis中是完全OK了!
127.0.0.1:6379> mset user:1:name zhangsan user:1:age 2
OK
127.0.0.1:6379> mget user:1:name user:1:age
1) "zhangsan"
2) "2"
################################################################
127.0.0.1:6379> getset db redis # 获取并设置键值对,如果不存在则返回nil
(nil)
127.0.0.1:6379> get db
"redis"
127.0.0.1:6379> getset db mongobd # 获取并设置键值对,如果存在则返回旧值并设置新值
"redis"
127.0.0.1:6379> get db
"mongobd"
数据结构都是相同的!
String类似的使用场景:value除了是字符串还可以是数字,用途举例:
- 计数器
- 统计多单位的数量:uid:123666:follow 0
- 粉丝数
- 对象存储缓存
3、List(列表)
在redis里面,可以把list做成栈、队列、阻塞队列。
所有的list的命令都是用l开头的。
命令:
序号 | 命令 | 描述 |
---|---|---|
1 | BLPOP key1 [key2] timeout | 移出并获取列表的第一个元素, 如果列表没有元素会阻塞列表直到等待超时或发现可弹出元素为止。 |
2 | BRPOP key1 [key2] timeout | 移出并获取列表的最后一个元素, 如果列表没有元素会阻塞列表直到等待超时或发现可弹出元素为止。 |
3 | BRPOPLPUSH source destination | 从列表中弹出一个值,将弹出的元素插入到另外一个列表中并返回它. |
4 | LINDEX key index | 通过索引获取列表中的元素 |
5 | LINSERT key BEFORE|AFTER pivot value | 在列表的元素前或者后插入元素 |
6 | LLEN key | 获取列表长度 |
7 | LPOP key | 移出并获取列表的第一个元素 |
8 | LPUSH key value1 [value2] | 将一个或多个值插入到列表头部 |
9 | LPUSHX key value | 将一个值插入到已存在的列表头部 |
10 | LRANGE key start stop | 获取列表指定范围内的元素 |
11 | LREM key count value | 移除列表元素 |
12 | LSET key index value | 通过索引设置列表元素的值 |
13 | LTRIM key start stop | 对一个列表进行修剪(trim),就是说,让列表只保留指定区间内的元素,不在指定区间之内的元素都将被删除。 |
14 | RPOP key | 移除列表的最后一个元素,返回值为移除的元素。 |
15 | RPOPLPUSH source destination | 移除列表的最后一个元素,并将该元素添加到另一个列表并返回 |
16 | RPUSH key value1 [value2] | 在列表中添加一个或多个值 |
17 | RPUSHX key value | 为已存在的列表添加值 |
示例:
127.0.0.1:6379> lpush list one # 将一个值或者多个值,插入列表头部,即左侧
(integer) 1
127.0.0.1:6379> lpush list two # list,列表的名称,可以是任意名称,如list1,list2等
(integer) 2
127.0.0.1:6379> lpush list three
(integer) 3
127.0.0.1:6379> lrange list 0 -1 # 获取list中的所有值
1) "three"
2) "two"
3) "one"
127.0.0.1:6379> lrange list 0 1 # 通过区间获取具体的值
1) "three"
2) "two"
127.0.0.1:6379> rpush list four # 将一个值或者多个值,插入列表尾部,即右侧
(integer) 4
127.0.0.1:6379> lrange list 0 -1
1) "three"
2) "two"
3) "one"
4) "four"
##################################################
127.0.0.1:6379> lpop list # 从左删除列表一个元素
"three"
127.0.0.1:6379> rpop list # 从右删除列表一个元素
"four"
127.0.0.1:6379> lpop list 2 # 从左删除列表中两个元素,不写具体数则默认删除1个,不能写范围
1) "three"
2) "two"
##################################################
127.0.0.1:6379> lindex list 1 # 通过坐标获取列表的值
"two"
##################################################
127.0.0.1:6379> llen list # 返回列表的长度
(integer) 4
##################################################
127.0.0.1:6379> lrem list 1 three # 移除列表
(integer) 1
127.0.0.1:6379> lrange list 0 -1
1) "two"
2) "two"
3) "one"
127.0.0.1:6379> lrem list 2 two
(integer) 2
127.0.0.1:6379> lrange list 0 -1
1) "one"
127.0.0.1:6379>
##################################################
127.0.0.1:6379> lrange mylist 0 -1
1) "hello"
2) "hello1"
3) "hello2"
4) "hello3"
127.0.0.1:6379> ltrim mylist 1 2 # 通过下标截取列表,获取指定区间的值
OK
127.0.0.1:6379> lrange mylist 0 -1
1) "hello1"
2) "hello2"
##################################################
127.0.0.1:6379> rpoplpush mylist myotherlist # 移除列表的最后一个元素并将该元素添加到另一个列表
"hello2"
127.0.0.1:6379> lrange mylist 0 -1
1) "hello1"
127.0.0.1:6379> lrange myotherlist 0 -1 #查看新列表中确实存在该值
1) "hello2"
##################################################
127.0.0.1:6379> lrange list 0 -1
1) "one"
127.0.0.1:6379> lset list1 0 value
(error) ERR no such key
127.0.0.1:6379> lset list 0 value # 将列表中指定下标的值替换为指定的值,相当于更新操作
OK
127.0.0.1:6379> lrange list 0 0
1) "value"
127.0.0.1:6379> lset list 1 other # 替换时列表不存在或者超出列表下标值的,会报错
(error) ERR index out of range
##################################################
127.0.0.1:6379> lrange list 0 0
1) "value"
127.0.0.1:6379> linsert list before "value" value1 # 在指定列表中的值的左侧插入value1,也可以右侧插入,after
(integer) 2
127.0.0.1:6379> lrange list 0 -1
1) "value1"
2) "value"
小结
- 他实际上是一个链表,before Node after , left , right 都可以插入值
- 如果key不存在,创建新的链表
- 如果key存在,新增内容
- 如果移除了所有值,空链表,也代表不存在!
- 在两边插入或者改动值,效率最高!中间元素,相对来说效率会低一点~
消息排队!消息队列(Lpush Rpop) 栈(Lpush Lpop)
4、Set(集合)
Redis的Set是string类型的无序集合。集合成员是唯一的,这就意味着集合中不能出现重复的数据。
Redis 中 集合是通过哈希表实现的,所以添加,删除,查找的复杂度都是O(1)。
集合中最大的成员数为 2^32 - 1 (4294967295, 每个集合可存储40多亿个成员)。
命令:
命令 | 描述 |
---|---|
SADD key member1[member2…] | 向集合中无序增加一个/多个成员 |
SCARD key | 获取集合的成员数 |
SMEMBERS key | 返回集合中所有的成员 |
SISMEMBER key member | 查询member元素是否是集合的成员,结果是无序的 |
SRANDMEMBER key [count] | 随机返回集合中count个成员,count缺省值为1 |
SPOP key [count] | 随机移除并返回集合中count个成员,count缺省值为1 |
SMOVE source destination member | 将source集合的成员member移动到destination集合 |
SREM key member1[member2…] | 移除集合中一个/多个成员 |
SDIFF key1[key2…] | 返回所有集合的差集 key1- key2 - … |
SDIFFSTORE destination key1[key2…] | 在SDIFF的基础上,将结果保存到集合中==(覆盖)==。不能保存到其他类型key噢! |
SINTER key1 [key2…] | 返回所有集合的交集 |
SINTERSTORE destination key1[key2…] | 在SINTER的基础上,存储结果到集合中。覆盖 |
SUNION key1 [key2…] | 返回所有集合的并集 |
SUNIONSTORE destination key1 [key2…] | 在SUNION的基础上,存储结果到及和张。覆盖 |
SSCAN KEY [MATCH pattern] [COUNT count] | 在大量数据环境下,使用此命令遍历集合中元素,每次遍历部分 |
示例:
127.0.0.1:6379> sadd myset "hello" # set集合中添加元素
(integer) 1
127.0.0.1:6379> sadd myset "world"
(integer) 1
127.0.0.1:6379> sadd myset "I love world"
(integer) 1
127.0.0.1:6379> smembers myset # 查看set集合中是所有元素
1) "world"
2) "I love world"
3) "hello"
127.0.0.1:6379> sismember myset hello # 查询某个值是否在set集合中
(integer) 1
########################################################
127.0.0.1:6379> scard myset # 查看set集合中元素的个数
(integer) 3
########################################################
127.0.0.1:6379> srem myset hello # 移除set集合中指定的元素
(integer) 1
########################################################
set集合中元素都是无序非重复的
127.0.0.1:6379> SRANDMEMBER myset 1 # 随机抽取一个元素
1) "world"
127.0.0.1:6379> SRANDMEMBER myset 1
1) "I love world"
127.0.0.1:6379> SRANDMEMBER myset 2 # 随机抽取两个元素
1) "world"
2) "I love world"
########################################################
127.0.0.1:6379> SMEMBERS myset
1) "world"
2) "I love world"
3) "hello1"
4) "hello"
5) "hello2"
127.0.0.1:6379> spop myset # 随机删除元素
"world"
127.0.0.1:6379> spop myset
"hello1"
########################################################
127.0.0.1:6379> SMEMBERS myset
1) "I love world"
2) "hello"
127.0.0.1:6379> smove myset myset2 "hello" # 移动myset中hello到myset2中
(integer) 1
127.0.0.1:6379> SMEMBERS myset
1) "I love world"
127.0.0.1:6379> SMEMBERS myset2
1) "hello"
########################################################
数字集合类:
- 差集
- 并集
- 交集
key1: a b c key2: c d e
127.0.0.1:6379> sdiff key1 key2 # 差集
1) "b"
2) "a"
127.0.0.1:6379> sinter key1 key2 # 交集
1) "c"
127.0.0.1:6379> sunion key1 key2 # 并集
1) "b"
2) "c"
3) "a"
4) "d"
5) "e"
应用:
-
如微博,A用户将所有关注的人放到一个集合中,也可以将他的粉丝也放到一个集合中。
-
两个好友之间的共同关注,共同爱好
-
二度好友:两个人之间存在一个共同好友------>推荐好友的功能
5、Hash(哈希)
是map集合类型,key-value,本质上与String类型没有太大的区别
命令:
命令 | 描述 |
---|---|
HSET key field value | 将哈希表 key 中的字段 field 的值设为 value 。重复设置同一个field会覆盖,返回0 |
HMSET key field1 value1 [field2 value2…] | 同时将多个 field-value (域-值)对设置到哈希表 key 中。 |
HSETNX key field value | 只有在字段 field 不存在时,设置哈希表字段的值。 |
HEXISTS key field | 查看哈希表 key 中,指定的字段是否存在。 |
HGET key field value | 获取存储在哈希表中指定字段的值 |
HMGET key field1 [field2…] | 获取所有给定字段的值 |
HGETALL key | 获取在哈希表key 的所有字段和值 |
HKEYS key | 获取哈希表key中所有的字段 |
HLEN key | 获取哈希表中字段的数量 |
HVALS key | 获取哈希表中所有值 |
HDEL key field1 [field2…] | 删除哈希表key中一个/多个field字段 |
HINCRBY key field n | 为哈希表 key 中的指定字段的整数值加上增量n,并返回增量后结果 一样只适用于整数型字段 |
HINCRBYFLOAT key field n | 为哈希表 key 中的指定字段的浮点数值加上增量 n。 |
HSCAN key cursor [MATCH pattern] [COUNT count] | 迭代哈希表中的键值对。 |
示例:
127.0.0.1:6379> hset myhash field hello #创建一个hash集合,key为field,value为hello
(integer) 1
127.0.0.1:6379> hget myhash field #获取hash集合myhash中key为field的value值
"hello"
127.0.0.1:6379> hmset myhash field1 world1 field2 world2 #给myhash批量添加key-value
OK
127.0.0.1:6379> hmget myhash field1 field2 #获取myhash中多个key的value值
1) "world1"
2) "world2"
127.0.0.1:6379> hgetall myhash #获取myhash的所有key和value
1) "field"
2) "hello"
3) "field1"
4) "world1"
5) "field2"
6) "world2"
127.0.0.1:6379> hset myhash field world #更新
(integer) 0
127.0.0.1:6379> hget myhash field
"world"
########################################################
127.0.0.1:6379> hdel myhash field1 #删除myhash中key为field1的字段,field1对应的value也会被删除
(integer) 1
########################################################
127.0.0.1:6379> hlen myhash #获取hash集合中键值对的个数
(integer) 2
########################################################
127.0.0.1:6379> hexists myhash field1 #判断hash集合中某个key是否存在,存在则返回1,不存在则返回0
(integer) 0
127.0.0.1:6379> hexists myhash field
(integer) 1
########################################################
127.0.0.1:6379> hkeys myhash #获取所有的key
1) "field"
2) "field2"
127.0.0.1:6379> hvals myhash #获取所有的value
1) "world"
2) "world2"
########################################################
127.0.0.1:6379> hset myhash field3 5
(integer) 1
127.0.0.1:6379> hincrby myhash field3 2 #自增
(integer) 7
127.0.0.1:6379> hincrby myhash field3 -2 #自减,没有hdecrby命令
(integer) 5
########################################################
127.0.0.1:6379> hsetnx myhash field4 value4 #如果不存在则创建field4,成功返回1,若已经存在则命名不生效,并返回0
(integer) 1
127.0.0.1:6379> hsetnx myhash field4 value44
(integer) 0
应用:hash更适合存储变更的数据,如user name age,尤其是是用户信息之类的,经常变动的信息! hash更适合于对象的存储,String更加适合字符串存储!
127.0.0.1:6379> hset user:1 name yang
(integer) 1
127.0.0.1:6379> hset user:2 age 20
(integer) 1
127.0.0.1:6379> hget user:1 name
"yang"
127.0.0.1:6379> hget user:2 age
"20"
6、Zset(有序集合)
不同的是每个元素都会关联一个double类型的分数(score)。redis正是通过分数来为集合中的成员进行从小到大的排序。
score相同:按字典顺序排序
有序集合的成员是唯一的,但分数(score)却可以重复。
命令 | 描述 |
---|---|
ZADD key score member1 [score2 member2] | 向有序集合添加一个或多个成员,或者更新已存在成员的分数 |
ZCARD key | 获取有序集合的成员数 |
ZCOUNT key min max | 计算在有序集合中指定区间score的成员数 |
ZINCRBY key n member | 有序集合中对指定成员的分数加上增量 n |
ZSCORE key member | 返回有序集中,成员的分数值 |
ZRANK key member | 返回有序集合中指定成员的索引 |
ZRANGE key start end | 通过索引区间返回有序集合成指定区间内的成员 |
ZRANGEBYLEX key min max | 通过字典区间返回有序集合的成员 |
ZRANGEBYSCORE key min max | 通过分数返回有序集合指定区间内的成员,区间的取值默认是闭区间 |
ZLEXCOUNT key min max | 在有序集合中计算指定字典区间内成员数量 |
ZREM key member1 [member2…] | 移除有序集合中一个/多个成员 |
ZREMRANGEBYLEX key min max | 移除有序集合中给定的字典区间的所有成员 |
ZREMRANGEBYRANK key start stop | 移除有序集合中给定的排名区间的所有成员 |
ZREMRANGEBYSCORE key min max | 移除有序集合中给定的分数区间的所有成员 |
ZREVRANGE key start end | 返回有序集中指定区间内的成员,通过索引,分数从高到底 |
ZREVRANGEBYSCORRE key max min | 返回有序集中指定分数区间内的成员,分数从高到低排序 |
ZREVRANGEBYLEX key max min | 返回有序集中指定字典区间内的成员,按字典顺序倒序 |
ZREVRANK key member | 返回有序集合中指定成员的排名,有序集成员按分数值递减(从大到小)排序 |
ZINTERSTORE destination numkeys key1 [key2 …] | 计算给定的一个或多个有序集的交集并将结果集存储在新的有序集合 key 中,numkeys:表示参与运算的集合数,将score相加作为结果的score |
ZUNIONSTORE destination numkeys key1 [key2…] | 计算给定的一个或多个有序集的交集并将结果集存储在新的有序集合 key 中 |
ZSCAN key cursor [MATCH pattern] [COUNT count] | 迭代有序集合中的元素(包括元素成员和元素分值) |
示例:
127.0.0.1:6379> zadd myzset 1 one #添加一个值
(integer) 1
127.0.0.1:6379> zadd myzset 2 two 3 three #添加多个值
(integer) 2
127.0.0.1:6379> zrange myzset 0 -1
1) "one"
2) "two"
3) "three"
########################################################
127.0.0.1:6379> zadd salary 2500 zhangsan
(integer) 1
127.0.0.1:6379> zadd salary 4000 lisi
(integer) 1
127.0.0.1:6379> zadd salary 400 wangwu
(integer) 1
127.0.0.1:6379> zrangebyscore salary -inf +inf #按照从小到大的顺序,把salary集合中从负无穷到正无穷区间的值排序
1) "wangwu"
2) "zhangsan"
3) "lisi"
127.0.0.1:6379> ZREVRANGE salary 0 -1 #从大到小排序
1) "lisi"
2) "zhangsan"
127.0.0.1:6379> zrangebyscore salary 0 2500 withscores #对0-2500的salary集合的值排序,并且附带score
1) "wangwu"
2) "400"
3) "zhangsan"
4) "2500"
127.0.0.1:6379> zrangebyscore salary 0 (2500 withscores #排序时默认是闭区间,要想开区间需要加(,即0 (2500表示0<=salary<2500
1) "wangwu"
2) "400"
########################################################
127.0.0.1:6379> zrem salary wangwu #移除zset中的指定元素
(integer) 1
########################################################
127.0.0.1:6379> zcard salary #获取有序集合中的元素个数
(integer) 2
########################################################
127.0.0.1:6379> zcount myzset 0 3 # 获取指定区间内的元素数量
(integer) 3
应用案例:
- set排序存储班级成绩表工资表排序!
- 普通消息1. 重要消息2. 带权重进行判断。
- 排行榜应用实现,取Top N测试
四、三种特殊数据类型
1、Geospatial(地理位置)
朋友的定位,附近的人,打车距离计算?
Redis的Geo在Redis3.2版本就推出了!这个功能可以推算地理位置的信息,两地之间的距离,方圆几里的人!
城市经纬度查询:http://www.jsons.cn/lngcode/
命令:
命令 | 描述 |
---|---|
geoadd key longitud(经度) latitude(纬度) member […] | 将具体经纬度的坐标存入一个有序集合 |
geopos key member [member…] | 获取集合中的一个/多个成员坐标 |
geodist key member1 member2 [unit] | 返回两个给定位置之间的距离。默认以米作为单位。 |
georadius key longitude latitude radius m/km/… | 返回指定经纬度的给定距离范围内的成员 |
GEORADIUSBYMEMBER key member radius… | 功能与GEORADIUS相同,只是中心位置不是具体的经纬度,而是使用结合中已有的成员作为中心点。 |
geohash key member1 [member2…] | 返回一个或多个位置元素的Geohash表示。使用Geohash位置52点整数编码。 |
geoadd
添加地理位置
# geoadd 添加地理位置 可以添加一个,也可以多个
# 规则:地球南北极是无法直接添加,并且我们一般会下载城市数据,直接通过java程序一次性导入即可
127.0.0.1:6379> geoadd china:city 116.40 39.90 beijing #添加北京的地理位置
(integer) 1
127.0.0.1:6379> geoadd china:city 121.47 31.23 shanghai
(integer) 1
127.0.0.1:6379> geoadd china:city 106.50 29.53 chongqing
(integer) 1
127.0.0.1:6379> geoadd china:city 114.05 22.52 shenzhen 120.16 30.24 hangzhou 108.96 34.26 xian
注意:
- 有效的经度从-180度到180度。
- 有效的纬度从-85.05112878度到85.05112878度。
若超出范围,则返回如下范围:
127.0.0.1:6379> geoadd china:city 22.52 114.05 tiandi
(error) ERR invalid longitude,latitude pair 22.520000,114.050000
geopos
获取指定城市的经纬度
127.0.0.1:6379> geopos china:city shanghai #获取上海的经纬度
1) 1) "121.47000163793563843"
2) "31.22999903975783553"
127.0.0.1:6379> geopos china:city shanghai shenzhen
1) 1) "121.47000163793563843"
2) "31.22999903975783553"
2) 1) "114.04999762773513794"
2) "22.5200000879503861"。
geodist
返回两个给定位置之间的直线距离
如果两个位置之间的其中一个不存在, 那么命令返回空值。
指定单位的参数 unit 必须是以下单位的其中一个:
- m 表示单位为米。
- km 表示单位为千米。
- mi 表示单位为英里。
- ft 表示单位为英尺。
如果用户没有显式地指定单位参数, 那么 GEODIST
默认使用米作为单位。
GEODIST
命令在计算距离时会假设地球为完美的球形, 在极限情况下, 这一假设最大会造成 0.5% 的误差。
127.0.0.1:6379> geodist china:city shanghai shenzhen #查看上海到深圳的直线距离,不写单位默认为米
"1215922.3698"
127.0.0.1:6379> geodist china:city shanghai beijing km #查看上海到北京的直线距离,单位千米
"1067.3788"
georadius
以给定的经纬度为中心, 返回键包含的位置元素当中, 与中心的距离不超过给定最大距离的所有位置元素。
在给定以下可选项时, 命令会返回额外的信息:
WITHDIST
: 在返回位置元素的同时, 将位置元素与中心之间的距离也一并返回。 距离的单位和用户给定的范围单位保持一致。WITHCOORD
: 将位置元素的经度和维度也一并返回。WITHHASH
: 以 52 位有符号整数的形式, 返回位置元素经过原始 geohash 编码的有序集合分值。 这个选项主要用于底层应用或者调试, 实际中的作用并不大。
命令默认返回未排序的位置元素。 通过以下两个参数, 用户可以指定被返回位置元素的排序方式:
ASC
: 根据中心的位置, 按照从近到远的方式返回位置元素。DESC
: 根据中心的位置, 按照从远到近的方式返回位置元素。
127.0.0.1:6379> georadius china:city 110 30 500 km #查询经纬度为(110,30)为中心,集合中500km范围内的城市
1) "chongqing"
2) "xian"
127.0.0.1:6379> georadius china:city 110 30 500 km withdist #查询经纬度为(110,30)为中心,集合中500km范围内的城市,并显示直线距离
1) 1) "chongqing"
2) "341.9374"
2) 1) "xian"
2) "483.8340"
127.0.0.1:6379> georadius china:city 110 30 500 km withcoord#查询经纬度为(110,30)为中心,集合中500km范围内的城市,并显示城市的经纬度
1) 1) "chongqing"
2) 1) "106.49999767541885376"
2) "29.52999957900659211"
2) 1) "xian"
2) 1) "108.96000176668167114"
2) "34.25999964418929977"
127.0.0.1:6379> georadius china:city 110 30 500 km withcoord count 1 #查询经纬度为(110,30)为中心,集合中500km范围内的1个城市。count会默认按距离从小到达返回。
1) 1) "chongqing"
2) 1) "106.49999767541885376"
2) "29.52999957900659211"
127.0.0.1:6379> georadius china:city 110 30 1000 km withdist count 4
1) 1) "chongqing"
2) "341.9374"
2) 1) "xian"
2) "483.8340"
3) 1) "shenzhen"
2) "924.6408"
4) 1) "hangzhou"
2) "977.5143"
georadiusbymember
这个命令和 GEORADIUS命令一样, 都可以找出位于指定范围内的元素, 但是 GEORADIUSBYMEMBER
的中心点是由给定的位置元素决定的, 而不是像 GEORADIUS那样, 使用输入的经度和纬度来决定中心点
127.0.0.1:6379> georadiusbymember china:city chongqing 1000 km #获取重庆1000km范围内的城市
1) "chongqing"
2) "xian"
127.0.0.1:6379> georadiusbymember china:city shanghai 400 km
1) "hangzhou"
2) "shanghai"
geohash
返回一个或多个位置元素的 Geohash表示。
# 将经纬度转换成一维的字符串,两个字符串越相似,表面距离越接近,使用较少
127.0.0.1:6379> geohash china:city beijing
1) "wx4fbxxfke0"
127.0.0.1:6379> geohash china:city beijing shanghai
1) "wx4fbxxfke0"
2) "wtw3sj5zbj0"
注:
- geo底层的实现原理其实就是Zset!因此可以使用Zset命令来操作geo
127.0.0.1:6379> zrange china:city 0 -1
1) "chongqing"
2) "xian"
3) "shenzhen"
4) "hangzhou"
5) "shanghai"
6) "beijing"
127.0.0.1:6379> zrem china:city beijing
(integer) 1
127.0.0.1:6379> zrange china:city 0 -1
1) "chongqing"
2) "xian"
3) "shenzhen"
4) "hangzhou"
2、Hyperloglog(基数统计)
什么是基数?
即数据集中不重复的元素的个数。
如:
- A {1,2,3,5,7,9,7} 基数=5
- B {1,2,3,5,7,9} 基数=5
简介
Redis 2.8.9版本就更新了Hyperloglog数据结构!Redis Hyperloglog基数统计的算法!
优点∶占用的内存是固定,存储2^64不同的元素的基数,只需要12KB内存!如果要从内存角度来比较的话Hyperloglog首选!
缺点:Hyperloglog会有0.81%错误率,如果允许容错,那么一定可以使用Hyperloglog;如果不允许容错,就使用set或者自己的数据类型即可 !
应用
网页的UV(一个人访问一个网站多次,但是还是算作一个人! )
传统的方式,set保存用户的id,然后就可以统计set中的元素数量作为标准判断!
这个方式如果保存大量的用户id,就会比较麻烦!我们的目的是为了计数,而不是保存用户id ;统计UV任务,0.81%错误率是可以忽略不计的!
命令:
命令 | 描述 |
---|---|
PFADD key element1 [elememt2…] | 添加指定元素到 HyperLogLog 中 |
PFCOUNT key [key] | 返回给定 HyperLogLog 的基数估算值。 |
PFMERGE destkey sourcekey [sourcekey…] | 将多个 HyperLogLog 合并为一个 HyperLogLog |
示例:
127.0.0.1:6379> pfadd mykey a b c d e f j h l #创建第一组元素 mykey
(integer) 1
127.0.0.1:6379> pfcount mykey #统计mykey中的基数数量
(integer) 9
127.0.0.1:6379> pfadd mykey2 j k h c v b n m z o p #创建第二组元素 mykey2
(integer) 1
127.0.0.1:6379> pfcount mykey2
(integer) 11
127.0.0.1:6379> pfadd mykeyn a b c d e a d #创建第三组元素 mykeyn,元素有重复的
(integer) 1
127.0.0.1:6379> pfcount mykeyn #统计基数,发现重复元素只统计了一次
(integer) 5
127.0.0.1:6379> pfmerge mykey3 mykey mykey2 #将mykey和mykey2合并成mykey3
OK
127.0.0.1:6379> pfcount mykey3 #查看并集的基数数量
(integer) 16
3、BitMaps(位图)
应用场景:
-
统计疫情人数:中国14亿人口,可以放14亿个0,有一人感染就由一个0变成1
-
统计用户信息:活跃用户和不活跃用户;登陆未登录;打卡未打卡等两个状态的都可以使用
Bitmaps位图,都是操作二进制位来进行记录,就只有0和1两个状态!除了0和1,写入其他数字会报错!
命令:
命令 | 描述 |
---|---|
setbit key offset value | 为指定key的offset位设置值 |
getbit key offset | 获取offset位的值 |
bitcount key [start end] | 统计字符串被设置为1的bit数,也可以指定统计范围按字节 |
bitop operration destkey key[key…] | 对一个或多个保存二进制位的字符串 key 进行位元操作,并将结果保存到 destkey 上。 |
BITPOS key bit [start] [end] | 返回字符串里面第一个被设置为1或者0的bit位。start和end只能按字节,不能按位 |
示例:
假如1代表了打卡,0代表了没有打卡。用bitmaps记录周一到周日的打卡情况。
127.0.0.1:6379> setbit status 0 1 #周一,状态1,打卡
(integer) 0
127.0.0.1:6379> setbit status 1 1 #周二,状态1,打卡
(integer) 0
127.0.0.1:6379> setbit status 2 0 #周三,状态0,未打卡
(integer) 0
127.0.0.1:6379> setbit status 3 0 #周四,状态0,未打卡
(integer) 0
127.0.0.1:6379> setbit status 4 0 #周五,状态0,未打卡
(integer) 0
127.0.0.1:6379> setbit status 5 1 #周六,状态1,打卡
(integer) 0
127.0.0.1:6379> setbit status 6 1 #周日,状态1,打卡
(integer) 0
127.0.0.1:6379> getbit status 5 #查看周六打开状态
(integer) 1
127.0.0.1:6379> bitcount status #统计
(integer) 4
五、事务
Redis事务本质:一组命令的集合。
redis一个事务中的所有命令都会被序列化,在事务执行的过程中,会按照顺序执行。具有一次性、顺序性、排他性,来保证一系列的命令执行。
如下展示:命令会先一个一个按顺序入队列,所有命令在事务中,并没有直接被执行!而是在发起执行命令的时候才会执行。
--- 队列 set set set 执行 ---
Redis事务没有隔离级别的概念
Redis的单条命令是保证原子性的,但是redis事务不能保证原子性
Redis事务操作过程
开启事务(multi
)---->命令入队---->执行事务(exec
)
所以事务中的命令在加入时都没有被执行,直到提交时才会开始执行(Exec)一次性完成。
127.0.0.1:6379> multi #开启事务
OK
127.0.0.1:6379(TX)> set k1 v1
QUEUED
127.0.0.1:6379(TX)> set k2 v2
QUEUED
127.0.0.1:6379(TX)> get k2
QUEUED
127.0.0.1:6379(TX)> set k3 v3
QUEUED
127.0.0.1:6379(TX)> exec #执行事务
1) OK
2) OK
3) "v2"
4) OK
#############################################
127.0.0.1:6379> multi
OK
127.0.0.1:6379(TX)> set k4 v4
QUEUED
127.0.0.1:6379(TX)> discard #取消事务
OK
127.0.0.1:6379> get k4 #事务中的命令并没有被执行
(nil)
事务错误:
编译型异常(即代码有问题,命令有错误),此时命令中所有的命令都不会执行
127.0.0.1:6379(TX)> set k1 v1
QUEUED
127.0.0.1:6379(TX)> set k2 v3
QUEUED
127.0.0.1:6379(TX)> getset k3 #命令错误,会抛出异常
(error) ERR wrong number of arguments for 'getset' command
127.0.0.1:6379(TX)> set k4 v4
QUEUED
127.0.0.1:6379(TX)> exec
(error) EXECABORT Transaction discarded because of previous errors.
127.0.0.1:6379> get k1 #抛出异常后,所有的命令都没有执行
(nil)
127.0.0.1:6379> get k4
(nil)
运行时异常,如果事务队列中存在语法性错误,如
1/0
,那么执行命令时,除该命令外其他命令是可以正常执行的,错误命令会抛出异常。
127.0.0.1:6379> set k1 v1 #创建一个key
OK
127.0.0.1:6379> multi
OK
127.0.0.1:6379(TX)> incr k1 #让v1自增,会报错,因为字符串不能自增,只有数字才可以
QUEUED
127.0.0.1:6379(TX)> set k2 v2
QUEUED
127.0.0.1:6379(TX)> get k2
QUEUED
127.0.0.1:6379(TX)> exec #发现虽然报错了,但剩下的命令都执行了
1) (error) ERR value is not an integer or out of range
2) OK
3) "v2"
127.0.0.1:6379> get k2
"v2"
六、监控(面试常问)
悲观锁:
- 很悲观,认为什么时候都会出现问题,无论做什么都会加锁
乐观锁:
- 很乐观,认为什么时候都不会出现问题,所以不会上锁!更新数据的时候去判断一下,在此期间是否有人修改过这个数据
- mysql会获取version,更新的时候比较version,redis使用watch
redis监测测试:
正常执行成功:
127.0.0.1:6379> set money 100
OK
127.0.0.1:6379> set out 0
OK
127.0.0.1:6379> watch money #监控money对象
OK
127.0.0.1:6379> multi #事务开始
OK
127.0.0.1:6379(TX)> decrby money 20
QUEUED
127.0.0.1:6379(TX)> incrby out 20
QUEUED
127.0.0.1:6379(TX)> exec #事务正常结束
1) (integer) 80
2) (integer) 20
执行失败情况:
测试多线程修改值,使用watch可以当做redis的乐观锁操作(相当于get version)
线程1:
127.0.0.1:6379> watch money
OK
127.0.0.1:6379> multi
OK
127.0.0.1:6379(TX)> decrby money 10
QUEUED
127.0.0.1:6379(TX)> incrby out 10
QUEUED
模拟另一线程在线程1的事务没有执行完毕的时候插队:
127.0.0.1:6379> get money
"80"
127.0.0.1:6379> set money 1000 #此时将money设置未1000
OK
此时回到线程1执行事务:
127.0.0.1:6379(TX)> exec #执行事务,失败
(nil)
执行失败以后解决办法:
127.0.0.1:6379> unwatch #若执行失败先解锁
OK
127.0.0.1:6379> watch money #再次开启监视最新的值,相当于mysql的select version
OK
127.0.0.1:6379> multi
OK
127.0.0.1:6379(TX)> decrby money 20
QUEUED
127.0.0.1:6379(TX)> incrby out 20
QUEUED
127.0.0.1:6379(TX)> exec #若在开启监视以后,监视的元素没有另外发生变化,那么可以执行成功。如果别的地方修改了该元素,会执行失败,重复以后步骤直至成功即可。
1) (integer) 980
2) (integer) 40
七、Jedis
使用Java来操作Redis,Jedis是Redis官方推荐使用的Java连接redis的客户端。
测试
(1)导入依赖
<dependencies>
<dependency>
<groupId>redis.clients</groupId>
<artifactId>jedis</artifactId>
<version>4.2.3</version>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>2.0.7</version>
</dependency>
</dependencies>
(2)编码测试
连接数据库:
public class TestPing {
public static void main(String[] args) {
Jedis jedis = new Jedis("43.143.140.141",6379);
//jedis所有的指令都是之前学习的redis命令
System.out.println(jedis.ping());
}
}
注:
- 若有防火墙需要防火墙开放6379端口:
firewall-cmd --zone=public --add-port=6379/tcp --permanet
- 若是厂商的服务器,还需要在该服务器的防火墙上添加6379规则
连接成功以后,输出结果:PONG
继续测试:
public class TestPing {
public static void main(String[] args) {
Jedis jedis = new Jedis("43.143.140.141",9001);
//jedis所有的指令都是之前学习的redis命令
System.out.println(jedis.ping());
}
public class TestString {
public static void main(String[] args) {
Jedis jedis = new Jedis("43.143.140.141",9001);
System.out.println("清空数据:"+jedis.flushDB());
System.out.println("判断某个键是否存在:"+jedis.exists("key1"));
System.out.println("新增<'username','kuangshen'>的键值对:"+jedis.set("username","kuangshen"));
System.out.println("新增<'password','password'>的键值对:"+jedis.set("password","password"));
System.out.println("系统中所有的键如下");
Set<String> keys = jedis.keys("*");
System.out.println(keys);
System.out.println("删除键password:"+jedis.del("password"));
System.out.println("判断password键是否存在:"+jedis.exists("password"));
System.out.println("查看键username所存储的值的类型"+jedis.type("username"));
System.out.println("随机返回keyJon关键的一个:"+jedis.randomKey());
System.out.println("重命名key:"+jedis.rename("username","name"));
System.out.println("取出修改后的name:"+jedis.get("name"));
System.out.println("按索引查询:"+jedis.select(0));
System.out.println("删除当前数据库多余key:"+jedis.flushDB());
}
}
(3)测试事务
public class TestTransaction {
public static void main(String[] args) {
Jedis jedis = new Jedis("43.143.140.141", 9001);
jedis.flushDB();
JSONObject jsonObject = new JSONObject();
jsonObject.put("password","123456");
jsonObject.put("username","admin");
//开启事务
Transaction multi = jedis.multi();
String result = jsonObject.toJSONString();
//jedis.watch(result);//监测
try {
multi.set("user1", result);
multi.set("user2", result);
//int i =1/0; //代码抛出异常,执行失败
multi.exec(); //执行事务
} catch (Exception e) {
multi.discard(); //有问题则放弃事务
} finally {
System.out.println(jedis.get("user1"));
System.out.println(jedis.get("user2"));
jedis.close(); //关闭连接
}
}
}
八、SpringBoot整合redis
SpringBoot操作数据: spring-data jpa jdbc mongodb redis !
SpringData也是和SpringBoot齐名的项目!
说明:
- 在SpringBoot2.x之后,原来使用的jedis被替换为了lettuce,原因:
- jedis:采用的直连,多个线程操作的话,是不安全的。如果要避免不安全,使用jedis pool连接池!更像BIO模式
- lettuce:采用netty,实例可以在多个线程中共享,不存在线程不安全的情况!可以减少线程数据了,更像NIO模式
- NIO/BIO区别
我们在学习SpringBoot自动配置的原理时,整合一个组件并进行配置一定会有一个自动配置类xxxAutoConfiguration,并且在spring.factories中也一定能找到这个类的完全限定名。Redis也不例外。
注:2.7.x以后,自动配置注册由
META-INF/spring.factories
路径改为META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports
,且无法再按CTRL+鼠标跳转到该类中,需要在该META-INF中按imports中的路径查找。
那么就一定还存在一个RedisProperties类
源码分析:
@Bean
@ConditionalOnMissingBean(name = {"redisTemplate"}) //当给定的在bean不存在时,则实例化当前Bean,因此我们可以自己定义一个Template来替换默认的redisTemplate
//如图,“redisTemplate”这个Bean在容器中不存在的时候,下面的redisTemplate才会被注入:
@ConditionalOnSingleCandidate(RedisConnectionFactory.class)
public RedisTemplate<Object, Object> redisTemplate(RedisConnectionFactory redisConnectionFactory) {
//使用默认的RedisTemplate,没有过多的设置。但是redis对象都需要序列化,因此需要我们在使用的时候进行序列化
//两个泛型都是<Object, Object>,需要强转成期望的<String, Object>类型
RedisTemplate<Object, Object> template = new RedisTemplate();
template.setConnectionFactory(redisConnectionFactory);
return template;
}
@Bean
@ConditionalOnMissingBean //不带参则表示直接注入该容器。
//由于String类型是redis中最常使用的类型,所以单独提出了一个bean
@ConditionalOnSingleCandidate(RedisConnectionFactory.class)
public StringRedisTemplate stringRedisTemplate(RedisConnectionFactory redisConnectionFactory) {
return new StringRedisTemplate(redisConnectionFactory);
}
之前我们说SpringBoot2.x后默认使用Lettuce来替换Jedis,现在我们就能来验证了。
在上述redisTemplate方法中,传递的参数为RedisConnectionFactory类型,跳转到RedisConnectionFactory,
会发现由两个方法实现了该接口:
点击JedisConnectionFactory,download,然后会发现有大量爆红的代码:
源代码有些内容已经不存在了,所以Jedis已经不生效了。因此使用Lettuce。
测试:
(1)导入依赖:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
(2)配置连接:
spring.redis.host=主机ip
spring.redis.port=开放端口
(3)测试类
@SpringBootTest
class Redis02SpringbootApplicationTests {
@Autowired
private RedisTemplate redisTemplate;
@Test
void contextLoads() {
//操作不同的数据类型,api和redis的指令大致相同
//opsForValue 操作字符串 类似于String
//opsForList 操作List 类似于List
//...
//除了基本的操作,常用的方法都可以直接使用redisTemplate来进行操作,比如事务和基本的CRUD。
redisTemplate.opsForValue().set("k1","天地有正气");
System.out.println(redisTemplate.opsForValue().get("k1"));
//获取redis的连接对象
/*RedisConnection connection = redisTemplate.getConnectionFactory().getConnection();
connection.flushDb();
connection.set();*/
}
}
可以看到,在idea中,可以正常显示中文:
但是在redis中,则显示是乱码:
原因:
在RedisTemplate中,最开始的位置,有几个序列化参数:
可以看到,默认的序列化器是采用JDK序列化器,而默认的RedisTemplate中的所有序列化器都是使用这个序列化器:
后续我们定制RedisTemplate就可以对其进行修改。
(4)简单定制RedisTemplate的模板:
@Configuration
public class RedisConfig {
//编写我们自己的配置
@Bean
public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory redisConnectionFactory) {
RedisTemplate<String, Object> template = new RedisTemplate();
template.setConnectionFactory(redisConnectionFactory);
return template;
}
}
(5)再次测试:
创建一个实体类User:
@Component
@Data
@AllArgsConstructor
@NoArgsConstructor
public class User {
private String name;
private int age;
}
测试:
@Test
void test() throws JsonProcessingException {
//按照刚才的思路,我们存值应该是使用redisTemplate.opsForValue().set("user",new User());
//但是在真实开发中,我们一般使用json来传递对象
User user = new User("天天",3);
//ObjectMapper类(com.fasterxml.jackson.databind.ObjectMapper)是Jackson的主要类,它可以帮助我们快速的进行各个类型和Json类型的相互转换
String jsonUser = new ObjectMapper().writeValueAsString(user);//将user变成json对象。
redisTemplate.opsForValue().set("user",jsonUser);
System.out.println(redisTemplate.opsForValue().get("user"));
}
此时结果为:
若在上述代码中,直接传递对象如下图,会报没有序列化的错:
但是在实体类上实现序列化接口就会执行成功:
因此一般实体类都是加上序列化。
(6)复杂性的定制RedisTemplate模板
@Configuration
public class RedisConfig {
//固定模板,实际开发工作中可以拿过来直接用
//编写我们自己的配置
@Bean
@SuppressWarnings("all")
public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory factory) {
// 为了开发方便,一般直接使用<String, Object>
RedisTemplate<String, Object> template = new RedisTemplate();
template.setConnectionFactory(factory);
// 配置具体的序列化方式
// json序列化配置通过json去解析任意的对象
Jackson2JsonRedisSerializer jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer(Object.class);
// ObjectMapper类是Jackson的主要类,它可以帮助我们快速的进行各个类型和Json类型的相互转换
ObjectMapper om = new ObjectMapper();
om.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY);
//om.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL);//过期,修改为activateDefaultTyping方法
om.activateDefaultTyping(LaissezFaireSubTypeValidator.instance, ObjectMapper.DefaultTyping.NON_FINAL);
jackson2JsonRedisSerializer.setObjectMapper(om);
//String序列化配置
StringRedisSerializer stringRedisSerializer = new StringRedisSerializer();
// key采用String的序列化方式
template.setKeySerializer(stringRedisSerializer);
// hash的key也采用String的序列化方式
template.setHashKeySerializer(stringRedisSerializer);
// value采用json的序列化方式
template.setValueSerializer(jackson2JsonRedisSerializer);
// hash的value采用json的序列化方式
template.setHashValueSerializer(jackson2JsonRedisSerializer);
// 在bean初始化时将设置加入进去
template.afterPropertiesSet();
return template;
}
}
再次执行测试:
去redis查看,不再是乱码:
(7)工具类
但是在实际开发过程中,在80%的情况下,我们并不会使用opsForList、opsForValue等原生的方法去写代码,会自己编写工具类:
参考地址:
https://www.cnblogs.com/zeng1994/p/03303c805731afc9aa9c60dbbd32a323.html
https://www.cnblogs.com/zhzhlong/p/11434284.html
package com.yang.utils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.stereotype.Component;
import org.springframework.util.CollectionUtils;
import java.util.Collection;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.TimeUnit;
@Component
public final class RedisUtil {
@Autowired
private RedisTemplate<String, Object> redisTemplate;
// =============================common============================
/**
* 指定缓存失效时间
* @param key 键
* @param time 时间(秒)
* @return
*/
public boolean expire(String key, long time) {
try {
if (time > 0) {
redisTemplate.expire(key, time, TimeUnit.SECONDS);
}
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 根据key 获取过期时间
* @param key 键 不能为null
* @return 时间(秒) 返回0代表为永久有效
*/
public long getExpire(String key) {
return redisTemplate.getExpire(key, TimeUnit.SECONDS);
}
/**
* 判断key是否存在
* @param key 键
* @return true 存在 false不存在
*/
public boolean hasKey(String key) {
try {
return redisTemplate.hasKey(key);
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 删除缓存
* @param key 可以传一个值 或多个
*/
@SuppressWarnings("unchecked")
public void del(String... key) {
if (key != null && key.length > 0) {
if (key.length == 1) {
redisTemplate.delete(key[0]);
} else {
redisTemplate.delete(String.valueOf(CollectionUtils.arrayToList(key)));
}
}
}
// ============================String=============================
/**
* 普通缓存获取
* @param key 键
* @return 值
*/
public Object get(String key) {
return key == null ? null : redisTemplate.opsForValue().get(key);
}
/**
* 普通缓存放入
* @param key 键
* @param value 值
* @return true成功 false失败
*/
public boolean set(String key, Object value) {
try {
redisTemplate.opsForValue().set(key, value);
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 普通缓存放入并设置时间
* @param key 键
* @param value 值
* @param time 时间(秒) time要大于0 如果time小于等于0 将设置无限期
* @return true成功 false 失败
*/
public boolean set(String key, Object value, long time) {
try {
if (time > 0) {
redisTemplate.opsForValue().set(key, value, time, TimeUnit.SECONDS);
} else {
set(key, value);
}
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 递增
* @param key 键
* @param delta 要增加几(大于0)
* @return
*/
public long incr(String key, long delta) {
if (delta < 0) {
throw new RuntimeException("递增因子必须大于0");
}
return redisTemplate.opsForValue().increment(key, delta);
}
/**
* 递减
* @param key 键
* @param delta 要减少几(小于0)
* @return
*/
public long decr(String key, long delta) {
if (delta < 0) {
throw new RuntimeException("递减因子必须大于0");
}
return redisTemplate.opsForValue().increment(key, -delta);
}
// ================================Map=================================
/**
* HashGet
* @param key 键 不能为null
* @param item 项 不能为null
* @return 值
*/
public Object hget(String key, String item) {
return redisTemplate.opsForHash().get(key, item);
}
/**
* 获取hashKey对应的所有键值
* @param key 键
* @return 对应的多个键值
*/
public Map<Object, Object> hmget(String key) {
return redisTemplate.opsForHash().entries(key);
}
/**
* HashSet
* @param key 键
* @param map 对应多个键值
* @return true 成功 false 失败
*/
public boolean hmset(String key, Map<String, Object> map) {
try {
redisTemplate.opsForHash().putAll(key, map);
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* HashSet 并设置时间
* @param key 键
* @param map 对应多个键值
* @param time 时间(秒)
* @return true成功 false失败
*/
public boolean hmset(String key, Map<String, Object> map, long time) {
try {
redisTemplate.opsForHash().putAll(key, map);
if (time > 0) {
expire(key, time);
}
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 向一张hash表中放入数据,如果不存在将创建
* @param key 键
* @param item 项
* @param value 值
* @return true 成功 false失败
*/
public boolean hset(String key, String item, Object value) {
try {
redisTemplate.opsForHash().put(key, item, value);
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 向一张hash表中放入数据,如果不存在将创建
* @param key 键
* @param item 项
* @param value 值
* @param time 时间(秒) 注意:如果已存在的hash表有时间,这里将会替换原有的时间
* @return true 成功 false失败
*/
public boolean hset(String key, String item, Object value, long time) {
try {
redisTemplate.opsForHash().put(key, item, value);
if (time > 0) {
expire(key, time);
}
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 删除hash表中的值
* @param key 键 不能为null
* @param item 项 可以使多个 不能为null
*/
public void hdel(String key, Object... item) {
redisTemplate.opsForHash().delete(key, item);
}
/**
* 判断hash表中是否有该项的值
* @param key 键 不能为null
* @param item 项 不能为null
* @return true 存在 false不存在
*/
public boolean hHasKey(String key, String item) {
return redisTemplate.opsForHash().hasKey(key, item);
}
/**
* hash递增 如果不存在,就会创建一个 并把新增后的值返回
* @param key 键
* @param item 项
* @param by 要增加几(大于0)
* @return
*/
public double hincr(String key, String item, double by) {
return redisTemplate.opsForHash().increment(key, item, by);
}
/**
* hash递减
* @param key 键
* @param item 项
* @param by 要减少记(小于0)
* @return
*/
public double hdecr(String key, String item, double by) {
return redisTemplate.opsForHash().increment(key, item, -by);
}
// ============================set=============================
/**
* 根据key获取Set中的所有值
* @param key 键
* @return
*/
public Set<Object> sGet(String key) {
try {
return redisTemplate.opsForSet().members(key);
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
/**
* 根据value从一个set中查询,是否存在
* @param key 键
* @param value 值
* @return true 存在 false不存在
*/
public boolean sHasKey(String key, Object value) {
try {
return redisTemplate.opsForSet().isMember(key, value);
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 将数据放入set缓存
* @param key 键
* @param values 值 可以是多个
* @return 成功个数
*/
public long sSet(String key, Object... values) {
try {
return redisTemplate.opsForSet().add(key, values);
} catch (Exception e) {
e.printStackTrace();
return 0;
}
}
/**
* 将set数据放入缓存
* @param key 键
* @param time 时间(秒)
* @param values 值 可以是多个
* @return 成功个数
*/
public long sSetAndTime(String key, long time, Object... values) {
try {
Long count = redisTemplate.opsForSet().add(key, values);
if (time > 0)
expire(key, time);
return count;
} catch (Exception e) {
e.printStackTrace();
return 0;
}
}
/**
* 获取set缓存的长度
* @param key 键
* @return
*/
public long sGetSetSize(String key) {
try {
return redisTemplate.opsForSet().size(key);
} catch (Exception e) {
e.printStackTrace();
return 0;
}
}
/**
* 移除值为value的
* @param key 键
* @param values 值 可以是多个
* @return 移除的个数
*/
public long setRemove(String key, Object... values) {
try {
Long count = redisTemplate.opsForSet().remove(key, values);
return count;
} catch (Exception e) {
e.printStackTrace();
return 0;
}
}
// ===============================list=================================
/**
* 获取list缓存的内容
* @param key 键
* @param start 开始
* @param end 结束 0 到 -1代表所有值
* @return
*/
public List<Object> lGet(String key, long start, long end) {
try {
return redisTemplate.opsForList().range(key, start, end);
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
/**
* 获取list缓存的长度
* @param key 键
* @return
*/
public long lGetListSize(String key) {
try {
return redisTemplate.opsForList().size(key);
} catch (Exception e) {
e.printStackTrace();
return 0;
}
}
/**
* 通过索引 获取list中的值
* @param key 键
* @param index 索引 index>=0时, 0 表头,1 第二个元素,依次类推;index<0时,-1,表尾,-2倒数第二个元素,依次类推
* @return
*/
public Object lGetIndex(String key, long index) {
try {
return redisTemplate.opsForList().index(key, index);
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
/**
* 将list放入缓存
* @param key 键
* @param value 值
* @return
*/
public boolean lSet(String key, Object value) {
try {
redisTemplate.opsForList().rightPush(key, value);
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 将list放入缓存
* @param key 键
* @param value 值
* @param time 时间(秒)
* @return
*/
public boolean lSet(String key, Object value, long time) {
try {
redisTemplate.opsForList().rightPush(key, value);
if (time > 0)
expire(key, time);
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 将list放入缓存
* @param key 键
* @param value 值
* @param time 时间(秒)
* @return
*/
public boolean lSet(String key, List<Object> value) {
try {
redisTemplate.opsForList().rightPushAll(key, value);
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 将list放入缓存
*
* @param key 键
* @param value 值
* @param time 时间(秒)
* @return
*/
public boolean lSet(String key, List<Object> value, long time) {
try {
redisTemplate.opsForList().rightPushAll(key, value);
if (time > 0)
expire(key, time);
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 根据索引修改list中的某条数据
* @param key 键
* @param index 索引
* @param value 值
* @return
*/
public boolean lUpdateIndex(String key, long index, Object value) {
try {
redisTemplate.opsForList().set(key, index, value);
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 移除N个值为value
* @param key 键
* @param count 移除多少个
* @param value 值
* @return 移除的个数
*/
public long lRemove(String key, long count, Object value) {
try {
Long remove = redisTemplate.opsForList().remove(key, count, value);
return remove;
} catch (Exception e) {
e.printStackTrace();
return 0;
}
}
}
测试:
@Autowired
private RedisUtil redisUtil;
@Test
public void test1(){
redisUtil.set("name","aaaa");
System.out.println(redisUtil.get("name"));
}
redis查看:
127.0.0.1:6379> keys *
1) "user"
2) "name"
127.0.0.1:6379> get name
"\"aaaa\""
九、Redis.conf详解
在工作中,一些小小的配置,可以让你脱颖而出!
1、启动命令:
docker run -it -p 9001:6379 --name myredis -v /home/config/redis/redis6379.conf:/etc/redis/redis6379.conf -v /home/appdata/redis/data6379:/data -d redis:7.0.4 redis-server /etc/redis/redis6379.conf
docker run -it -p 9002:6380 --name myredis2 -v /home/config/redis/redis6380.conf:/etc/redis/redis6380.conf -v /home/appdata/redis/data6380:/data -d redis:7.0.4 redis-server /etc/redis/redis6380.conf
docker run -it -p 9003:6381 --name myredis3 -v /home/config/redis/redis6381.conf:/etc/redis/redis6381.conf -v /home/appdata/redis/data6381:/data -d redis:7.0.4 redis-server /etc/redis/redis6381.conf
启动失败但没有报错:
原因:因为redis.conf文件中的daemonize为yes,意思是redis服务在后台运行,与docker中的-d参数冲突了。
只要把daemonize的参数值改为no就可以了,再次执行以上命令,容器启动成功。
远程连接报错:
原因:配置文件中开启了保护模式,需要关闭。将protected-mode yes修改为protected-mode no即可!
2、配置详细内容:
# Redis configuration file example.
#
# Note that in order to read the configuration file, Redis must be
# started with the file path as first argument:
#
# ./redis-server /path/to/redis.conf
# Note on units: when memory size is needed, it is possible to specify
# it in the usual form of 1k 5GB 4M and so forth:
#
# 1k => 1000 bytes
# 1kb => 1024 bytes
# 1m => 1000000 bytes
# 1mb => 1024*1024 bytes
# 1g => 1000000000 bytes
# 1gb => 1024*1024*1024 bytes
#
# units are case insensitive so 1GB 1Gb 1gB are all the same.
################################## INCLUDES ###################################
# Include one or more other config files here. This is useful if you
# have a standard template that goes to all Redis servers but also need
# to customize a few per-server settings. Include files can include
# other files, so use this wisely.
#
# Notice option "include" won't be rewritten by command "CONFIG REWRITE"
# from admin or Redis Sentinel. Since Redis always uses the last processed
# line as value of a configuration directive, you'd better put includes
# at the beginning of this file to avoid overwriting config change at runtime.
#
# If instead you are interested in using includes to override configuration
# options, it is better to use include as the last line.
#
# include /path/to/local.conf
# include /path/to/other.conf
################################## NETWORK #####################################
# By default, if no "bind" configuration directive is specified, Redis listens
# for connections from all the network interfaces available on the server.
# It is possible to listen to just one or multiple selected interfaces using
# the "bind" configuration directive, followed by one or more IP addresses.
#
# Examples:
#
# bind 192.168.1.100 10.0.0.1
# bind 127.0.0.1 ::1
#
# ~~~ WARNING ~~~ If the computer running Redis is directly exposed to the
# internet, binding to all the interfaces is dangerous and will expose the
# instance to everybody on the internet. So by default we uncomment the
# following bind directive, that will force Redis to listen only into
# the IPv4 lookback interface address (this means Redis will be able to
# accept connections only from clients running into the same computer it
# is running).
#
# IF YOU ARE SURE YOU WANT YOUR INSTANCE TO LISTEN TO ALL THE INTERFACES
# JUST COMMENT THE FOLLOWING LINE.
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
bind 127.0.0.1
# Protected mode is a layer of security protection, in order to avoid that
# Redis instances left open on the internet are accessed and exploited.
#
# When protected mode is on and if:
#
# 1) The server is not binding explicitly to a set of addresses using the
# "bind" directive.
# 2) No password is configured.
#
# The server only accepts connections from clients connecting from the
# IPv4 and IPv6 loopback addresses 127.0.0.1 and ::1, and from Unix domain
# sockets.
#
# By default protected mode is enabled. You should disable it only if
# you are sure you want clients from other hosts to connect to Redis
# even if no authentication is configured, nor a specific set of interfaces
# are explicitly listed using the "bind" directive.
protected-mode yes
# Accept connections on the specified port, default is 6379 (IANA #815344).
# If port 0 is specified Redis will not listen on a TCP socket.
port 6379
# TCP listen() backlog.
#
# In high requests-per-second environments you need an high backlog in order
# to avoid slow clients connections issues. Note that the Linux kernel
# will silently truncate it to the value of /proc/sys/net/core/somaxconn so
# make sure to raise both the value of somaxconn and tcp_max_syn_backlog
# in order to get the desired effect.
tcp-backlog 511
# Unix socket.
#
# Specify the path for the Unix socket that will be used to listen for
# incoming connections. There is no default, so Redis will not listen
# on a unix socket when not specified.
#
# unixsocket /tmp/redis.sock
# unixsocketperm 700
# Close the connection after a client is idle for N seconds (0 to disable)
timeout 0
# TCP keepalive.
#
# If non-zero, use SO_KEEPALIVE to send TCP ACKs to clients in absence
# of communication. This is useful for two reasons:
#
# 1) Detect dead peers.
# 2) Take the connection alive from the point of view of network
# equipment in the middle.
#
# On Linux, the specified value (in seconds) is the period used to send ACKs.
# Note that to close the connection the double of the time is needed.
# On other kernels the period depends on the kernel configuration.
#
# A reasonable value for this option is 300 seconds, which is the new
# Redis default starting with Redis 3.2.1.
tcp-keepalive 300
################################# GENERAL #####################################
# By default Redis does not run as a daemon. Use 'yes' if you need it.
# Note that Redis will write a pid file in /var/run/redis.pid when daemonized.
daemonize yes
# If you run Redis from upstart or systemd, Redis can interact with your
# supervision tree. Options:
# supervised no - no supervision interaction
# supervised upstart - signal upstart by putting Redis into SIGSTOP mode
# supervised systemd - signal systemd by writing READY=1 to $NOTIFY_SOCKET
# supervised auto - detect upstart or systemd method based on
# UPSTART_JOB or NOTIFY_SOCKET environment variables
# Note: these supervision methods only signal "process is ready."
# They do not enable continuous liveness pings back to your supervisor.
supervised no
# If a pid file is specified, Redis writes it where specified at startup
# and removes it at exit.
#
# When the server runs non daemonized, no pid file is created if none is
# specified in the configuration. When the server is daemonized, the pid file
# is used even if not specified, defaulting to "/var/run/redis.pid".
#
# Creating a pid file is best effort: if Redis is not able to create it
# nothing bad happens, the server will start and run normally.
pidfile /var/run/redis_6379.pid
# Specify the server verbosity level.
# This can be one of:
# debug (a lot of information, useful for development/testing)
# verbose (many rarely useful info, but not a mess like the debug level)
# notice (moderately verbose, what you want in production probably)
# warning (only very important / critical messages are logged)
loglevel notice
# Specify the log file name. Also the empty string can be used to force
# Redis to log on the standard output. Note that if you use standard
# output for logging but daemonize, logs will be sent to /dev/null
logfile ""
# To enable logging to the system logger, just set 'syslog-enabled' to yes,
# and optionally update the other syslog parameters to suit your needs.
# syslog-enabled no
# Specify the syslog identity.
# syslog-ident redis
# Specify the syslog facility. Must be USER or between LOCAL0-LOCAL7.
# syslog-facility local0
# Set the number of databases. The default database is DB 0, you can select
# a different one on a per-connection basis using SELECT <dbid> where
# dbid is a number between 0 and 'databases'-1
databases 16
################################ SNAPSHOTTING ################################
#
# Save the DB on disk:
#
# save <seconds> <changes>
#
# Will save the DB if both the given number of seconds and the given
# number of write operations against the DB occurred.
#
# In the example below the behaviour will be to save:
# after 900 sec (15 min) if at least 1 key changed
# after 300 sec (5 min) if at least 10 keys changed
# after 60 sec if at least 10000 keys changed
#
# Note: you can disable saving completely by commenting out all "save" lines.
#
# It is also possible to remove all the previously configured save
# points by adding a save directive with a single empty string argument
# like in the following example:
#
# save ""
save 900 1
save 300 10
save 60 10000
# By default Redis will stop accepting writes if RDB snapshots are enabled
# (at least one save point) and the latest background save failed.
# This will make the user aware (in a hard way) that data is not persisting
# on disk properly, otherwise chances are that no one will notice and some
# disaster will happen.
#
# If the background saving process will start working again Redis will
# automatically allow writes again.
#
# However if you have setup your proper monitoring of the Redis server
# and persistence, you may want to disable this feature so that Redis will
# continue to work as usual even if there are problems with disk,
# permissions, and so forth.
stop-writes-on-bgsave-error yes
# Compress string objects using LZF when dump .rdb databases?
# For default that's set to 'yes' as it's almost always a win.
# If you want to save some CPU in the saving child set it to 'no' but
# the dataset will likely be bigger if you have compressible values or keys.
rdbcompression yes
# Since version 5 of RDB a CRC64 checksum is placed at the end of the file.
# This makes the format more resistant to corruption but there is a performance
# hit to pay (around 10%) when saving and loading RDB files, so you can disable it
# for maximum performances.
#
# RDB files created with checksum disabled have a checksum of zero that will
# tell the loading code to skip the check.
rdbchecksum yes
# The filename where to dump the DB
dbfilename dump.rdb
# The working directory.
#
# The DB will be written inside this directory, with the filename specified
# above using the 'dbfilename' configuration directive.
#
# The Append Only File will also be created inside this directory.
#
# Note that you must specify a directory here, not a file name.
dir ./
################################# REPLICATION #################################
# Master-Slave replication. Use slaveof to make a Redis instance a copy of
# another Redis server. A few things to understand ASAP about Redis replication.
#
# 1) Redis replication is asynchronous, but you can configure a master to
# stop accepting writes if it appears to be not connected with at least
# a given number of slaves.
# 2) Redis slaves are able to perform a partial resynchronization with the
# master if the replication link is lost for a relatively small amount of
# time. You may want to configure the replication backlog size (see the next
# sections of this file) with a sensible value depending on your needs.
# 3) Replication is automatic and does not need user intervention. After a
# network partition slaves automatically try to reconnect to masters
# and resynchronize with them.
#
# slaveof <masterip> <masterport>
# If the master is password protected (using the "requirepass" configuration
# directive below) it is possible to tell the slave to authenticate before
# starting the replication synchronization process, otherwise the master will
# refuse the slave request.
#
# masterauth <master-password>
# When a slave loses its connection with the master, or when the replication
# is still in progress, the slave can act in two different ways:
#
# 1) if slave-serve-stale-data is set to 'yes' (the default) the slave will
# still reply to client requests, possibly with out of date data, or the
# data set may just be empty if this is the first synchronization.
#
# 2) if slave-serve-stale-data is set to 'no' the slave will reply with
# an error "SYNC with master in progress" to all the kind of commands
# but to INFO and SLAVEOF.
#
slave-serve-stale-data yes
# You can configure a slave instance to accept writes or not. Writing against
# a slave instance may be useful to store some ephemeral data (because data
# written on a slave will be easily deleted after resync with the master) but
# may also cause problems if clients are writing to it because of a
# misconfiguration.
#
# Since Redis 2.6 by default slaves are read-only.
#
# Note: read only slaves are not designed to be exposed to untrusted clients
# on the internet. It's just a protection layer against misuse of the instance.
# Still a read only slave exports by default all the administrative commands
# such as CONFIG, DEBUG, and so forth. To a limited extent you can improve
# security of read only slaves using 'rename-command' to shadow all the
# administrative / dangerous commands.
slave-read-only yes
# Replication SYNC strategy: disk or socket.
#
# -------------------------------------------------------
# WARNING: DISKLESS REPLICATION IS EXPERIMENTAL CURRENTLY
# -------------------------------------------------------
#
# New slaves and reconnecting slaves that are not able to continue the replication
# process just receiving differences, need to do what is called a "full
# synchronization". An RDB file is transmitted from the master to the slaves.
# The transmission can happen in two different ways:
#
# 1) Disk-backed: The Redis master creates a new process that writes the RDB
# file on disk. Later the file is transferred by the parent
# process to the slaves incrementally.
# 2) Diskless: The Redis master creates a new process that directly writes the
# RDB file to slave sockets, without touching the disk at all.
#
# With disk-backed replication, while the RDB file is generated, more slaves
# can be queued and served with the RDB file as soon as the current child producing
# the RDB file finishes its work. With diskless replication instead once
# the transfer starts, new slaves arriving will be queued and a new transfer
# will start when the current one terminates.
#
# When diskless replication is used, the master waits a configurable amount of
# time (in seconds) before starting the transfer in the hope that multiple slaves
# will arrive and the transfer can be parallelized.
#
# With slow disks and fast (large bandwidth) networks, diskless replication
# works better.
repl-diskless-sync no
# When diskless replication is enabled, it is possible to configure the delay
# the server waits in order to spawn the child that transfers the RDB via socket
# to the slaves.
#
# This is important since once the transfer starts, it is not possible to serve
# new slaves arriving, that will be queued for the next RDB transfer, so the server
# waits a delay in order to let more slaves arrive.
#
# The delay is specified in seconds, and by default is 5 seconds. To disable
# it entirely just set it to 0 seconds and the transfer will start ASAP.
repl-diskless-sync-delay 5
# Slaves send PINGs to server in a predefined interval. It's possible to change
# this interval with the repl_ping_slave_period option. The default value is 10
# seconds.
#
# repl-ping-slave-period 10
# The following option sets the replication timeout for:
#
# 1) Bulk transfer I/O during SYNC, from the point of view of slave.
# 2) Master timeout from the point of view of slaves (data, pings).
# 3) Slave timeout from the point of view of masters (REPLCONF ACK pings).
#
# It is important to make sure that this value is greater than the value
# specified for repl-ping-slave-period otherwise a timeout will be detected
# every time there is low traffic between the master and the slave.
#
# repl-timeout 60
# Disable TCP_NODELAY on the slave socket after SYNC?
#
# If you select "yes" Redis will use a smaller number of TCP packets and
# less bandwidth to send data to slaves. But this can add a delay for
# the data to appear on the slave side, up to 40 milliseconds with
# Linux kernels using a default configuration.
#
# If you select "no" the delay for data to appear on the slave side will
# be reduced but more bandwidth will be used for replication.
#
# By default we optimize for low latency, but in very high traffic conditions
# or when the master and slaves are many hops away, turning this to "yes" may
# be a good idea.
repl-disable-tcp-nodelay no
# Set the replication backlog size. The backlog is a buffer that accumulates
# slave data when slaves are disconnected for some time, so that when a slave
# wants to reconnect again, often a full resync is not needed, but a partial
# resync is enough, just passing the portion of data the slave missed while
# disconnected.
#
# The bigger the replication backlog, the longer the time the slave can be
# disconnected and later be able to perform a partial resynchronization.
#
# The backlog is only allocated once there is at least a slave connected.
#
# repl-backlog-size 1mb
# After a master has no longer connected slaves for some time, the backlog
# will be freed. The following option configures the amount of seconds that
# need to elapse, starting from the time the last slave disconnected, for
# the backlog buffer to be freed.
#
# A value of 0 means to never release the backlog.
#
# repl-backlog-ttl 3600
# The slave priority is an integer number published by Redis in the INFO output.
# It is used by Redis Sentinel in order to select a slave to promote into a
# master if the master is no longer working correctly.
#
# A slave with a low priority number is considered better for promotion, so
# for instance if there are three slaves with priority 10, 100, 25 Sentinel will
# pick the one with priority 10, that is the lowest.
#
# However a special priority of 0 marks the slave as not able to perform the
# role of master, so a slave with priority of 0 will never be selected by
# Redis Sentinel for promotion.
#
# By default the priority is 100.
slave-priority 100
# It is possible for a master to stop accepting writes if there are less than
# N slaves connected, having a lag less or equal than M seconds.
#
# The N slaves need to be in "online" state.
#
# The lag in seconds, that must be <= the specified value, is calculated from
# the last ping received from the slave, that is usually sent every second.
#
# This option does not GUARANTEE that N replicas will accept the write, but
# will limit the window of exposure for lost writes in case not enough slaves
# are available, to the specified number of seconds.
#
# For example to require at least 3 slaves with a lag <= 10 seconds use:
#
# min-slaves-to-write 3
# min-slaves-max-lag 10
#
# Setting one or the other to 0 disables the feature.
#
# By default min-slaves-to-write is set to 0 (feature disabled) and
# min-slaves-max-lag is set to 10.
# A Redis master is able to list the address and port of the attached
# slaves in different ways. For example the "INFO replication" section
# offers this information, which is used, among other tools, by
# Redis Sentinel in order to discover slave instances.
# Another place where this info is available is in the output of the
# "ROLE" command of a masteer.
#
# The listed IP and address normally reported by a slave is obtained
# in the following way:
#
# IP: The address is auto detected by checking the peer address
# of the socket used by the slave to connect with the master.
#
# Port: The port is communicated by the slave during the replication
# handshake, and is normally the port that the slave is using to
# list for connections.
#
# However when port forwarding or Network Address Translation (NAT) is
# used, the slave may be actually reachable via different IP and port
# pairs. The following two options can be used by a slave in order to
# report to its master a specific set of IP and port, so that both INFO
# and ROLE will report those values.
#
# There is no need to use both the options if you need to override just
# the port or the IP address.
#
# slave-announce-ip 5.5.5.5
# slave-announce-port 1234
################################## SECURITY ###################################
# Require clients to issue AUTH <PASSWORD> before processing any other
# commands. This might be useful in environments in which you do not trust
# others with access to the host running redis-server.
#
# This should stay commented out for backward compatibility and because most
# people do not need auth (e.g. they run their own servers).
#
# Warning: since Redis is pretty fast an outside user can try up to
# 150k passwords per second against a good box. This means that you should
# use a very strong password otherwise it will be very easy to break.
#
# requirepass foobared
# Command renaming.
#
# It is possible to change the name of dangerous commands in a shared
# environment. For instance the CONFIG command may be renamed into something
# hard to guess so that it will still be available for internal-use tools
# but not available for general clients.
#
# Example:
#
# rename-command CONFIG b840fc02d524045429941cc15f59e41cb7be6c52
#
# It is also possible to completely kill a command by renaming it into
# an empty string:
#
# rename-command CONFIG ""
#
# Please note that changing the name of commands that are logged into the
# AOF file or transmitted to slaves may cause problems.
################################### LIMITS ####################################
# Set the max number of connected clients at the same time. By default
# this limit is set to 10000 clients, however if the Redis server is not
# able to configure the process file limit to allow for the specified limit
# the max number of allowed clients is set to the current file limit
# minus 32 (as Redis reserves a few file descriptors for internal uses).
#
# Once the limit is reached Redis will close all the new connections sending
# an error 'max number of clients reached'.
#
# maxclients 10000
# Don't use more memory than the specified amount of bytes.
# When the memory limit is reached Redis will try to remove keys
# according to the eviction policy selected (see maxmemory-policy).
#
# If Redis can't remove keys according to the policy, or if the policy is
# set to 'noeviction', Redis will start to reply with errors to commands
# that would use more memory, like SET, LPUSH, and so on, and will continue
# to reply to read-only commands like GET.
#
# This option is usually useful when using Redis as an LRU cache, or to set
# a hard memory limit for an instance (using the 'noeviction' policy).
#
# WARNING: If you have slaves attached to an instance with maxmemory on,
# the size of the output buffers needed to feed the slaves are subtracted
# from the used memory count, so that network problems / resyncs will
# not trigger a loop where keys are evicted, and in turn the output
# buffer of slaves is full with DELs of keys evicted triggering the deletion
# of more keys, and so forth until the database is completely emptied.
#
# In short... if you have slaves attached it is suggested that you set a lower
# limit for maxmemory so that there is some free RAM on the system for slave
# output buffers (but this is not needed if the policy is 'noeviction').
#
# maxmemory <bytes>
# MAXMEMORY POLICY: how Redis will select what to remove when maxmemory
# is reached. You can select among five behaviors:
#
# volatile-lru -> remove the key with an expire set using an LRU algorithm
# allkeys-lru -> remove any key according to the LRU algorithm
# volatile-random -> remove a random key with an expire set
# allkeys-random -> remove a random key, any key
# volatile-ttl -> remove the key with the nearest expire time (minor TTL)
# noeviction -> don't expire at all, just return an error on write operations
#
# Note: with any of the above policies, Redis will return an error on write
# operations, when there are no suitable keys for eviction.
#
# At the date of writing these commands are: set setnx setex append
# incr decr rpush lpush rpushx lpushx linsert lset rpoplpush sadd
# sinter sinterstore sunion sunionstore sdiff sdiffstore zadd zincrby
# zunionstore zinterstore hset hsetnx hmset hincrby incrby decrby
# getset mset msetnx exec sort
#
# The default is:
#
# maxmemory-policy noeviction
# LRU and minimal TTL algorithms are not precise algorithms but approximated
# algorithms (in order to save memory), so you can tune it for speed or
# accuracy. For default Redis will check five keys and pick the one that was
# used less recently, you can change the sample size using the following
# configuration directive.
#
# The default of 5 produces good enough results. 10 Approximates very closely
# true LRU but costs a bit more CPU. 3 is very fast but not very accurate.
#
# maxmemory-samples 5
############################## APPEND ONLY MODE ###############################
# By default Redis asynchronously dumps the dataset on disk. This mode is
# good enough in many applications, but an issue with the Redis process or
# a power outage may result into a few minutes of writes lost (depending on
# the configured save points).
#
# The Append Only File is an alternative persistence mode that provides
# much better durability. For instance using the default data fsync policy
# (see later in the config file) Redis can lose just one second of writes in a
# dramatic event like a server power outage, or a single write if something
# wrong with the Redis process itself happens, but the operating system is
# still running correctly.
#
# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF, that is the file
# with the better durability guarantees.
#
# Please check http://redis.io/topics/persistence for more information.
appendonly no
# The name of the append only file (default: "appendonly.aof")
appendfilename "appendonly.aof"
# The fsync() call tells the Operating System to actually write data on disk
# instead of waiting for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log. Slow, Safest.
# everysec: fsync only one time every second. Compromise.
#
# The default is "everysec", as that's usually the right compromise between
# speed and data safety. It's up to you to understand if you can relax this to
# "no" that will let the operating system flush the output buffer when
# it wants, for better performances (but if you can live with the idea of
# some data loss consider the default persistence mode that's snapshotting),
# or on the contrary, use "always" that's very slow but a bit safer than
# everysec.
#
# More details please check the following article:
# http://antirez.com/post/redis-persistence-demystified.html
#
# If unsure, use "everysec".
# appendfsync always
appendfsync everysec
# appendfsync no
# When the AOF fsync policy is set to always or everysec, and a background
# saving process (a background save or AOF log background rewriting) is
# performing a lot of I/O against the disk, in some Linux configurations
# Redis may block too long on the fsync() call. Note that there is no fix for
# this currently, as even performing fsync in a different thread will block
# our synchronous write(2) call.
#
# In order to mitigate this problem it's possible to use the following option
# that will prevent fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving, the durability of Redis is
# the same as "appendfsync none". In practical terms, this means that it is
# possible to lose up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability.
no-appendfsync-on-rewrite no
# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling
# BGREWRITEAOF when the AOF log size grows by the specified percentage.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a percentage of zero in order to disable the automatic AOF
# rewrite feature.
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
# An AOF file may be found to be truncated at the end during the Redis
# startup process, when the AOF data gets loaded back into memory.
# This may happen when the system where Redis is running
# crashes, especially when an ext4 filesystem is mounted without the
# data=ordered option (however this can't happen when Redis itself
# crashes or aborts but the operating system still works correctly).
#
# Redis can either exit with an error when this happens, or load as much
# data as possible (the default now) and start if the AOF file is found
# to be truncated at the end. The following option controls this behavior.
#
# If aof-load-truncated is set to yes, a truncated AOF file is loaded and
# the Redis server starts emitting a log to inform the user of the event.
# Otherwise if the option is set to no, the server aborts with an error
# and refuses to start. When the option is set to no, the user requires
# to fix the AOF file using the "redis-check-aof" utility before to restart
# the server.
#
# Note that if the AOF file will be found to be corrupted in the middle
# the server will still exit with an error. This option only applies when
# Redis will try to read more data from the AOF file but not enough bytes
# will be found.
aof-load-truncated yes
################################ LUA SCRIPTING ###############################
# Max execution time of a Lua script in milliseconds.
#
# If the maximum execution time is reached Redis will log that a script is
# still in execution after the maximum allowed time and will start to
# reply to queries with an error.
#
# When a long running script exceeds the maximum execution time only the
# SCRIPT KILL and SHUTDOWN NOSAVE commands are available. The first can be
# used to stop a script that did not yet called write commands. The second
# is the only way to shut down the server in the case a write command was
# already issued by the script but the user doesn't want to wait for the natural
# termination of the script.
#
# Set it to 0 or a negative value for unlimited execution without warnings.
lua-time-limit 5000
################################ REDIS CLUSTER ###############################
#
# ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
# WARNING EXPERIMENTAL: Redis Cluster is considered to be stable code, however
# in order to mark it as "mature" we need to wait for a non trivial percentage
# of users to deploy it in production.
# ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
#
# Normal Redis instances can't be part of a Redis Cluster; only nodes that are
# started as cluster nodes can. In order to start a Redis instance as a
# cluster node enable the cluster support uncommenting the following:
#
# cluster-enabled yes
# Every cluster node has a cluster configuration file. This file is not
# intended to be edited by hand. It is created and updated by Redis nodes.
# Every Redis Cluster node requires a different cluster configuration file.
# Make sure that instances running in the same system do not have
# overlapping cluster configuration file names.
#
# cluster-config-file nodes-6379.conf
# Cluster node timeout is the amount of milliseconds a node must be unreachable
# for it to be considered in failure state.
# Most other internal time limits are multiple of the node timeout.
#
# cluster-node-timeout 15000
# A slave of a failing master will avoid to start a failover if its data
# looks too old.
#
# There is no simple way for a slave to actually have a exact measure of
# its "data age", so the following two checks are performed:
#
# 1) If there are multiple slaves able to failover, they exchange messages
# in order to try to give an advantage to the slave with the best
# replication offset (more data from the master processed).
# Slaves will try to get their rank by offset, and apply to the start
# of the failover a delay proportional to their rank.
#
# 2) Every single slave computes the time of the last interaction with
# its master. This can be the last ping or command received (if the master
# is still in the "connected" state), or the time that elapsed since the
# disconnection with the master (if the replication link is currently down).
# If the last interaction is too old, the slave will not try to failover
# at all.
#
# The point "2" can be tuned by user. Specifically a slave will not perform
# the failover if, since the last interaction with the master, the time
# elapsed is greater than:
#
# (node-timeout * slave-validity-factor) + repl-ping-slave-period
#
# So for example if node-timeout is 30 seconds, and the slave-validity-factor
# is 10, and assuming a default repl-ping-slave-period of 10 seconds, the
# slave will not try to failover if it was not able to talk with the master
# for longer than 310 seconds.
#
# A large slave-validity-factor may allow slaves with too old data to failover
# a master, while a too small value may prevent the cluster from being able to
# elect a slave at all.
#
# For maximum availability, it is possible to set the slave-validity-factor
# to a value of 0, which means, that slaves will always try to failover the
# master regardless of the last time they interacted with the master.
# (However they'll always try to apply a delay proportional to their
# offset rank).
#
# Zero is the only value able to guarantee that when all the partitions heal
# the cluster will always be able to continue.
#
# cluster-slave-validity-factor 10
# Cluster slaves are able to migrate to orphaned masters, that are masters
# that are left without working slaves. This improves the cluster ability
# to resist to failures as otherwise an orphaned master can't be failed over
# in case of failure if it has no working slaves.
#
# Slaves migrate to orphaned masters only if there are still at least a
# given number of other working slaves for their old master. This number
# is the "migration barrier". A migration barrier of 1 means that a slave
# will migrate only if there is at least 1 other working slave for its master
# and so forth. It usually reflects the number of slaves you want for every
# master in your cluster.
#
# Default is 1 (slaves migrate only if their masters remain with at least
# one slave). To disable migration just set it to a very large value.
# A value of 0 can be set but is useful only for debugging and dangerous
# in production.
#
# cluster-migration-barrier 1
# By default Redis Cluster nodes stop accepting queries if they detect there
# is at least an hash slot uncovered (no available node is serving it).
# This way if the cluster is partially down (for example a range of hash slots
# are no longer covered) all the cluster becomes, eventually, unavailable.
# It automatically returns available as soon as all the slots are covered again.
#
# However sometimes you want the subset of the cluster which is working,
# to continue to accept queries for the part of the key space that is still
# covered. In order to do so, just set the cluster-require-full-coverage
# option to no.
#
# cluster-require-full-coverage yes
# In order to setup your cluster make sure to read the documentation
# available at http://redis.io web site.
################################## SLOW LOG ###################################
# The Redis Slow Log is a system to log queries that exceeded a specified
# execution time. The execution time does not include the I/O operations
# like talking with the client, sending the reply and so forth,
# but just the time needed to actually execute the command (this is the only
# stage of command execution where the thread is blocked and can not serve
# other requests in the meantime).
#
# You can configure the slow log with two parameters: one tells Redis
# what is the execution time, in microseconds, to exceed in order for the
# command to get logged, and the other parameter is the length of the
# slow log. When a new command is logged the oldest one is removed from the
# queue of logged commands.
# The following time is expressed in microseconds, so 1000000 is equivalent
# to one second. Note that a negative number disables the slow log, while
# a value of zero forces the logging of every command.
slowlog-log-slower-than 10000
# There is no limit to this length. Just be aware that it will consume memory.
# You can reclaim memory used by the slow log with SLOWLOG RESET.
slowlog-max-len 128
################################ LATENCY MONITOR ##############################
# The Redis latency monitoring subsystem samples different operations
# at runtime in order to collect data related to possible sources of
# latency of a Redis instance.
#
# Via the LATENCY command this information is available to the user that can
# print graphs and obtain reports.
#
# The system only logs operations that were performed in a time equal or
# greater than the amount of milliseconds specified via the
# latency-monitor-threshold configuration directive. When its value is set
# to zero, the latency monitor is turned off.
#
# By default latency monitoring is disabled since it is mostly not needed
# if you don't have latency issues, and collecting data has a performance
# impact, that while very small, can be measured under big load. Latency
# monitoring can easily be enabled at runtime using the command
# "CONFIG SET latency-monitor-threshold <milliseconds>" if needed.
latency-monitor-threshold 0
############################# EVENT NOTIFICATION ##############################
# Redis can notify Pub/Sub clients about events happening in the key space.
# This feature is documented at http://redis.io/topics/notifications
#
# For instance if keyspace events notification is enabled, and a client
# performs a DEL operation on key "foo" stored in the Database 0, two
# messages will be published via Pub/Sub:
#
# PUBLISH __keyspace@0__:foo del
# PUBLISH __keyevent@0__:del foo
#
# It is possible to select the events that Redis will notify among a set
# of classes. Every class is identified by a single character:
#
# K Keyspace events, published with __keyspace@<db>__ prefix.
# E Keyevent events, published with __keyevent@<db>__ prefix.
# g Generic commands (non-type specific) like DEL, EXPIRE, RENAME, ...
# $ String commands
# l List commands
# s Set commands
# h Hash commands
# z Sorted set commands
# x Expired events (events generated every time a key expires)
# e Evicted events (events generated when a key is evicted for maxmemory)
# A Alias for g$lshzxe, so that the "AKE" string means all the events.
#
# The "notify-keyspace-events" takes as argument a string that is composed
# of zero or multiple characters. The empty string means that notifications
# are disabled.
#
# Example: to enable list and generic events, from the point of view of the
# event name, use:
#
# notify-keyspace-events Elg
#
# Example 2: to get the stream of the expired keys subscribing to channel
# name __keyevent@0__:expired use:
#
# notify-keyspace-events Ex
#
# By default all notifications are disabled because most users don't need
# this feature and the feature has some overhead. Note that if you don't
# specify at least one of K or E, no events will be delivered.
notify-keyspace-events ""
############################### ADVANCED CONFIG ###############################
# Hashes are encoded using a memory efficient data structure when they have a
# small number of entries, and the biggest entry does not exceed a given
# threshold. These thresholds can be configured using the following directives.
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
# Lists are also encoded in a special way to save a lot of space.
# The number of entries allowed per internal list node can be specified
# as a fixed maximum size or a maximum number of elements.
# For a fixed maximum size, use -5 through -1, meaning:
# -5: max size: 64 Kb <-- not recommended for normal workloads
# -4: max size: 32 Kb <-- not recommended
# -3: max size: 16 Kb <-- probably not recommended
# -2: max size: 8 Kb <-- good
# -1: max size: 4 Kb <-- good
# Positive numbers mean store up to _exactly_ that number of elements
# per list node.
# The highest performing option is usually -2 (8 Kb size) or -1 (4 Kb size),
# but if your use case is unique, adjust the settings as necessary.
list-max-ziplist-size -2
# Lists may also be compressed.
# Compress depth is the number of quicklist ziplist nodes from *each* side of
# the list to *exclude* from compression. The head and tail of the list
# are always uncompressed for fast push/pop operations. Settings are:
# 0: disable all list compression
# 1: depth 1 means "don't start compressing until after 1 node into the list,
# going from either the head or tail"
# So: [head]->node->node->...->node->[tail]
# [head], [tail] will always be uncompressed; inner nodes will compress.
# 2: [head]->[next]->node->node->...->node->[prev]->[tail]
# 2 here means: don't compress head or head->next or tail->prev or tail,
# but compress all nodes between them.
# 3: [head]->[next]->[next]->node->node->...->node->[prev]->[prev]->[tail]
# etc.
list-compress-depth 0
# Sets have a special encoding in just one case: when a set is composed
# of just strings that happen to be integers in radix 10 in the range
# of 64 bit signed integers.
# The following configuration setting sets the limit in the size of the
# set in order to use this special memory saving encoding.
set-max-intset-entries 512
# Similarly to hashes and lists, sorted sets are also specially encoded in
# order to save a lot of space. This encoding is only used when the length and
# elements of a sorted set are below the following limits:
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
# HyperLogLog sparse representation bytes limit. The limit includes the
# 16 bytes header. When an HyperLogLog using the sparse representation crosses
# this limit, it is converted into the dense representation.
#
# A value greater than 16000 is totally useless, since at that point the
# dense representation is more memory efficient.
#
# The suggested value is ~ 3000 in order to have the benefits of
# the space efficient encoding without slowing down too much PFADD,
# which is O(N) with the sparse encoding. The value can be raised to
# ~ 10000 when CPU is not a concern, but space is, and the data set is
# composed of many HyperLogLogs with cardinality in the 0 - 15000 range.
hll-sparse-max-bytes 3000
# Active rehashing uses 1 millisecond every 100 milliseconds of CPU time in
# order to help rehashing the main Redis hash table (the one mapping top-level
# keys to values). The hash table implementation Redis uses (see dict.c)
# performs a lazy rehashing: the more operation you run into a hash table
# that is rehashing, the more rehashing "steps" are performed, so if the
# server is idle the rehashing is never complete and some more memory is used
# by the hash table.
#
# The default is to use this millisecond 10 times every second in order to
# actively rehash the main dictionaries, freeing memory when possible.
#
# If unsure:
# use "activerehashing no" if you have hard latency requirements and it is
# not a good thing in your environment that Redis can reply from time to time
# to queries with 2 milliseconds delay.
#
# use "activerehashing yes" if you don't have such hard requirements but
# want to free memory asap when possible.
activerehashing yes
# The client output buffer limits can be used to force disconnection of clients
# that are not reading data from the server fast enough for some reason (a
# common reason is that a Pub/Sub client can't consume messages as fast as the
# publisher can produce them).
#
# The limit can be set differently for the three different classes of clients:
#
# normal -> normal clients including MONITOR clients
# slave -> slave clients
# pubsub -> clients subscribed to at least one pubsub channel or pattern
#
# The syntax of every client-output-buffer-limit directive is the following:
#
# client-output-buffer-limit <class> <hard limit> <soft limit> <soft seconds>
#
# A client is immediately disconnected once the hard limit is reached, or if
# the soft limit is reached and remains reached for the specified number of
# seconds (continuously).
# So for instance if the hard limit is 32 megabytes and the soft limit is
# 16 megabytes / 10 seconds, the client will get disconnected immediately
# if the size of the output buffers reach 32 megabytes, but will also get
# disconnected if the client reaches 16 megabytes and continuously overcomes
# the limit for 10 seconds.
#
# By default normal clients are not limited because they don't receive data
# without asking (in a push way), but just after a request, so only
# asynchronous clients may create a scenario where data is requested faster
# than it can read.
#
# Instead there is a default limit for pubsub and slave clients, since
# subscribers and slaves receive data in a push fashion.
#
# Both the hard or the soft limit can be disabled by setting them to zero.
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
# Redis calls an internal function to perform many background tasks, like
# closing connections of clients in timeout, purging expired keys that are
# never requested, and so forth.
#
# Not all tasks are performed with the same frequency, but Redis checks for
# tasks to perform according to the specified "hz" value.
#
# By default "hz" is set to 10. Raising the value will use more CPU when
# Redis is idle, but at the same time will make Redis more responsive when
# there are many keys expiring at the same time, and timeouts may be
# handled with more precision.
#
# The range is between 1 and 500, however a value over 100 is usually not
# a good idea. Most users should use the default of 10 and raise this up to
# 100 only in environments where very low latency is required.
hz 10
# When a child rewrites the AOF file, if the following option is enabled
# the file will be fsync-ed every 32 MB of data generated. This is useful
# in order to commit the file to the disk more incrementally and avoid
# big latency spikes.
aof-rewrite-incremental-fsync yes
2.1、单位(unit):
并且单位(unit)不区分大小写。
2.2、允许包含多个配置文件(include):
include标签类似于学习Spring时import标签,可以将其他配置导入进来。
2.3、网络(network):
bind 127.0.0.1 # 绑定的ip
protected-mode yes # 保护模式
port 6379 # 端口设置
2.4、通用配置(general):
daemonize yes/no # 以守护进程的方式运行,默认是no,需要手动开启为yes,否则退出redis进程就会自动关闭
pidfile /var/run/redis_6379.pid #如果以后台的方式运行,即daemonize为yes,则需要指定一个pid文件
# 日志
# debug (a lot of information, useful for development/testing)
# verbose (many rarely useful info, but not a mess like the debug level)
# notice (moderately verbose, what you want in production probably) 生产环境
# warning (only very important / critical messages are logged)
loglevel notice
logfile "" # 生成的日志文件位置,为空则是标准输出
databases 16 # 数据库的数量,默认是16个
2.5、快照(snapshotting):
持久化,在规定的时间内,执行了多少次的操作,则会持久化到文件。两种格式:.rdb .aof
redis是内存数据库,如果没有持久化,则会断电即失!
# 如果900s内,至少有1个key进行了修改,就执行持久化操作
save 900 1
# 如果300s内,至少有10个key进行了修改,就执行持久化操作
save 300 10
# 如果60s内,至少有10000个key进行了修改,就执行持久化操作
save 60 10000
stop-writes-on-bgsave-error yes # 如果持久化出错,是否还需要继续工作
rdbcompression yes # 是否压缩rdb文件,会消耗一些CPU资源
rdbchecksum yes # 保存rdb文件时,进行错误的检查校验
dir ./ # rdb文件保存的目录:当前目录
2.6、复制(replication):
在主从复制是详细讲解。
2.7、安全(security):
可以给redis设置密码,默认是没有密码的
requirepass 123456 #设置密码,但一般不在配置文件中设置,而是在外面设置
示例:
127.0.0.1:6379> config get requirepass # 获取redis密码
1) "requirepass"
2) ""
127.0.0.1:6379> config set requirepass 123456 # 设置redis密码
OK
# 退出重启redis
root@6e35483f88fc:/usr/local/bin# redis-cli
127.0.0.1:6379> ping # 所有命令都没有权限了
(error) NOAUTH Authentication required.
127.0.0.1:6379> config get requirepass
(error) NOAUTH Authentication required.
127.0.0.1:6379> auth 123456 # 认证登陆
OK
127.0.0.1:6379> ping
PONG
127.0.0.1:6379> config get requirepass
1) "requirepass"
2) "123456"
2.8、限制(limits):
maxclients 10000 # 设置能连接redis的最大客户端数量
maxmemory <bytes> # redis配置最大的内存容量
maxmemory-policy noeviction # 内存到达上限的处理策略
volatile-lru:只对设置了过期时间的key进行LRU(默认值)
allkeys-lru:删除lru算法的key
volatile-random:随机删除即将过期key
allkeys-random:随机删除
volatile-ttl:删除即将过期的
noeviction:永不过期,返回错误
2.9、APPEND ONLY模式(aof配置):
appendonly no # 默认是不开启aof模式的,默认使用rdb方式持久化,大部分情况下,rdb是完全够用的。
appendfilename "appendonly.aof" # 持久化文件的名字
appendfsync everysec # 同步策略
everysec # 默认,每秒同步一次,缺点是可能会丢失这1s的数据
always # 每次修改都要sync,消耗性能
no # 不执行sync,这个时候操作系统自己同步数据,速度最快!
具体的配置见后redis持久化!
十、redis持久化
面试和工作,持久化都是重点!
Redis是内存数据库,如果不将内存中的数据库状态保存到磁盘,那么一旦服务器进程退出,服务器中的数据库状态也会消失。所以Redis 提供了持久化功能!
1、RDB (Redis DataBase )
1.1、什么是RDB?
在指定的时间间隔内将内存中的数据集快照写入磁盘,也就是行话讲的Snapshot快照,它恢复时是将快照文件直接读到内存里。
Redis会单独创建 ( fork )一个子进程来进行持久化,会先将数据写入到一个临时文件中,待持久化过程都结束了,再用这个临时文件替换上次持久化好的文件。整个过程中,主进程是不进行任何IO操作的。这就确保了极高的性能。如果需要进行大规模数据的恢复,且对于数据恢复的完整性不是非常敏感,那RDB方式要比AOF方式更加的高效。RDB的缺点是最后一次持久化后的数据可能丢失。我们默认的就是RDB,一般情况下不需要修改这个配置!
默认情况下, Redis 将数据库快照保存在名字为 dump.rdb的二进制文件中。
文件名可以在配置文件中进行自定义,在配置的快照中:
1.2、测试生成rdb文件方法:
方法一:
127.0.0.1:6379> set k1 v1 #60s内设置5个key
OK
127.0.0.1:6379> set k2 v2
OK
127.0.0.1:6379> set k3 v3
OK
127.0.0.1:6379> set k4 v1
OK
127.0.0.1:6379> set k5 v5
OK
在启动命令中可以看到,容器内的数据挂载到了/home/appdata/redis/data目录:
进入该目录(先删除dump.rdb,再执行上述命令):
[root@mycentos data]# ls
dump.rdb
会发现满足save 60 5以后,生成了一个rdb文件。
注:直接执行save命令,不加任何参数,也会生成rdb文件。
方法二:
执行flushall,也会生成rdb文件。
127.0.0.1:6379> flushall
OK
方法三:
直接关闭redis进程,也会生成rdb文件。
如何恢复rdb文件:
-
只需要将rdb文件放到redis启动目录就可以,redis启动的时候会自动检查dump.rdb文件并恢复其中的内容。
-
查看需要存放的目录
127.0.0.1:6379> config get dir 1) "dir" 2) "/data" # 如果在这个目录下存在dump.rdb 文件,启动就会自动恢复其中的数据
如果是直接用linux安装的redis,则只需要将dump.rdb放到
/data
;如果是用docker启动的redis容器,则需要找到
/data
挂载的路径,将rdb放到挂载路径下即可。
使用默认的配置几乎就足够了,但是我们仍需要去学习!
rdb优缺点:
优点:
- 适合大规模的数据恢复!
- 对数据的完整性要不高!
缺点:
- 需要一定的时间间隔进程操作!如果redis意外宕机了,这个最后一次修改数据就没有了!因此在生产环境中,我们通常还会备份dump.rdb。
- fork进程的时候,会占用一定的内容空间!
2、AOF(Append Only File)
2.1、AOF是什么?
以日志的形式来记录每个写操作,将Redis执行过的所有指令记录下来(读操作不记录),只许追加文件但不可以改写文件,redis启动之初会读取该文件重新构建数据,换言之,redis重启的话就根据日志文件的内容将写指令从前到后执行一次以完成数据的恢复工作。
AOF保存的是appendonly.aof文件
在配置文件中的位置:
append
appendonly no # 默认是不开启aof模式的,需要改为yes手动开启
rewrite 重写
AOF 持久化是通过保存被执行的写命令来记录数据库状态的,所以AOF文件的大小随着时间的流逝一定会越来越大;影响包括但不限于:对于Redis服务器,计算机的存储压力;AOF还原出数据库状态的时间增加;
为了解决AOF文件体积膨胀的问题,Redis提供了AOF重写功能:Redis服务器可以创建一个新的AOF文件来替代现有的AOF文件,新旧两个文件所保存的数据库状态是相同的,但是新的AOF文件不会包含任何浪费空间的冗余命令,通常体积会较旧AOF文件小很多。
# 指定 Redis 重写 AOF 文件的条件,默认为 100,它会对比上次生成的 AOF 文件大小。如果当前 AOF 文件的增长量大于上次 AOF 文件的 100%,就会触发重写操作;如果将该选项设置为 0,则不会触发重写操作。
auto-aof-rewrite-percentage 100
# 指定触发重写操作的 AOF 文件的大小,默认为 64MB。如果当前 AOF 文件的大小低于该值,此时就算当前文件的增量比例达到了 auto-aof-rewrite-percentage 选项所设置的条件,也不会触发重写操作。换句话说,只有同时满足以上这两个选项所设置的条件,才会触发重写操作。
auto-aof-rewrite-min-size 64mb
测试:docker版redis容器
# 修改完配置文件以后,需要重启redis容器
docker restart myredis
# 或者关停redis容器再启动
docker stop myredis # 关停redis容器
docker start myredis # 开始redis容器
此时redis容器挂载出来的数据路径(见九、1、启动命令)会生成aof文件。
注:
-
如果使用的是redis7以及以上版本,在bin目录下我们找不到appendonly.aof文件。这是因为在redis7中关于AOF持久化有一个更新操作。AOF持久化不再是以单独的文件存在,而是生成一个appendonlydir文件夹。在这个文件夹下存在三个以appendonly.aop为前缀的文件
如果aof文件出错,如在文件内手动输入几个字符,关闭redis容器后再次启动该容器,会发现报错启动不起来:
错误如下:
此时需要修复该文件,redis给我们提供了一个工具redis-check-aof --fix
,但在修复时也许会删除掉某些正确的值,这是难以避免的问题。
优缺点:
appendfsync everysec # 同步策略
everysec # 默认,每秒同步一次,缺点是可能会丢失这1s的数据
always # 每次修改都要sync,消耗性能
no # 不执行sync,这个时候操作系统自己同步数据,速度最快!
优点
-
每一次修改都会同步,文件的完整性会更加好
-
每秒同步一次,可能会丢失一秒的数据
-
从不同步,效率最高
缺点
-
相对于数据文件来说,aof占用内存远远大于rdb,修复速度比rdb慢!
-
Aof运行效率也要比rdb慢,所以我们redis默认的配置就是rdb持久化
3、总结︰
(1)RDB持久化方式能够在指定的时间间隔内对你的数据进行快照存储
(2)AOF持久化方式记录每次对服务器写的操作,当服务器重启的时候会重新执行这些命令来恢复原始的数据,AOF命令以Redis 协议追加保存每次写的操作到文件末尾,Redis还能对AOF文件进行后台重写,使得AOF文件的体积不至于过大。
(3)只做缓存,如果你只希望你的数据在服务器运行的时候存在,你也可以不使用任何持久化
(4)同时开启两种持久化方式
- 在这种情况下,当redis重启的时候会优先载入AOF文件来恢复原始的数据,因为在通常情况下AOF文件保存的数据集要比RDB文件保存的数据集要完整。
- RDB的数据不实时,同时使用两者时服务器重启也只会找AOF文件,那要不要只使用AOF呢?作者建议不要,因为RDB更适合用于备份数据库(AOF在不断变化不好备份),快速重启,而且不会有AOF可能潜在的Bug,留着作为一个万一的手段。
(5)性能建议
- 因为RDB文件只用作后备用途,建议只在Slave上持久化RDB文件,而且只要15分钟备份一次就够了,只保留
save 900 1
这条规则。 - 如果Enable AOF(开启AOF),好处是在最恶劣情况下也只会丢失不超过两秒数据,启动脚本较简单只load自己的AOF文件就可以了,代价一是带来了持续的lO,二是AOF rewrite 的最后将rewrite过程中产生的新数据写到新文件造成的阻塞几乎是不可避免的。只要硬盘许可,应该尽量减少AOF rewrite的频率,AOF重写的基础大小默认值64M太小了,可以设到5G以上,默认超过原大小100%大小重写可以改到适当的数值。
- 如果不Enable AOF,仅靠Master-Slave Repllcation实现高可用性也可以,能省掉一大笔IO,也减少了rewrite时带来的系统波动。代价是如果Master/Slave同时宕掉,会丢失十几分钟的数据,启动脚本也要比较两个MasterISlave 中的 RDB文件,载入较新的那个,微博就是这种架构。
十一、Redis发布与订阅
Redis 发布订阅(pub/sub)是一种消息通信模式:发送者(pub)发送消息,订阅者(sub)接收消息。
Redis客户端可以订阅任意数量的的频道。
订阅/发布消息图:
下图展示了频道 channel1 , 以及订阅这个频道的三个客户端 —— client2 、 client5 和 client1 之间的关系:
当有新消息通过 PUBLISH 命令发送给频道 channel1 时, 这个消息就会被发送给订阅它的三个客户端:
命令:
命令 | 描述 |
---|---|
PSUBSCRIBE pattern [pattern…] | 订阅一个或多个符合给定模式的频道。 |
PUNSUBSCRIBE pattern [pattern…] | 退订一个或多个符合给定模式的频道。 |
PUBSUB subcommand [argument[argument]] | 查看订阅与发布系统状态。 |
PUBLISH channel message | 向指定频道发布消息 |
SUBSCRIBE channel [channel…] | 订阅给定的一个或多个频道。 |
UNSUBSCRIBE channel [channel…] | 退订一个或多个频道 |
示例:
订阅端:
127.0.0.1:6379> subscribe lurenjia # 首先订阅lurenjia这个客户端
Reading messages... (press Ctrl-C to quit)
1) "subscribe"
2) "lurenjia"
3) (integer) 1
发送端:
127.0.0.1:6379> publish lurenjia "hello,hello" # 给lurenjia频道发送消息
(integer) 1
127.0.0.1:6379> publish lurenjia "hello,redis"
(integer) 1
订阅端收到消息:
127.0.0.1:6379> subscribe lurenjia
Reading messages... (press Ctrl-C to quit)
1) "subscribe"
2) "lurenjia"
3) (integer) 1
# 以下为推送的信息
1) "message" # 接收的类型:消息
2) "lurenjia" # 消息的来源频道
3) "hello,hello" # 消息的内容
1) "message"
2) "lurenjia"
3) "hello,redis"
原理:
Redis是使用C实现的,通过分析Redis源码里的pubsub.c文件,了解发布和订阅机制的底层实现,籍此加深对Redis的理解。Redis通过PUBLISH 、SUBSCRIBE和PSUBSCRIBE等命令实现发布和订阅功能。
通过SUBSCRIBE命令订阅某频道后,redis-server里维护了一个字典,字典的键就是一个个channel(频道),而字典的值则是一个链表,链表中保存了所有订阅这个channel的客户端。SUBSCRIBE命令的关键,就是将客户端添加到给定channel的订阅链表中。
通过PUBLISH命令向订阅者发送消息,redis-server会使用给定的频道作为键,在它所维护的channel字典中查找记录了订阅这个频道的所有客户端的链表,遍历这个链表,将消息发布给所有订阅者。
Pub/Sub从字面上理解就是发布( Publish)与订阅( Subscribe ),在Redis中,你可以设定对某一个key值进行消息发布及消息订阅,当一个key值上进行了消息发布后,所有订阅它的客户端都会收到相应的消息。这一功能最明显的用法就是用作实时消息系统,比如普通的即时聊天,群聊等功能。
缺点
-
如果一个客户端订阅了频道,但自己读取消息的速度却不够快的话,那么不断积压的消息会使redis输出缓冲区的体积变得越来越大,这可能使得redis本身的速度变慢,甚至直接崩溃。
-
这和数据传输可靠性有关,如果在订阅方断线,那么他将会丢失所有在短线期间发布者发布的消息
使用场景:
- 实时消息系统
- 实时聊天(频道当作聊天室,将消息回显给所有人即可)
- 订阅,关注系统也可以
实际开发中,稍微复杂的场景会使用消息中间件MQ。
十二、Redis主从复制
1、概念
主从复制,是指将一台Redis服务器的数据,复制到其他的Redis服务器。前者称为主节点(Master/Leader),后者称为从节点(Slave/Follower)。数据的复制是单向的,只能由主节点复制到从节点(主节点以写为主、从节点以读为主)。
默认情况下,每台Redis服务器都是主节点,一个主节点可以有0个或者多个从节点,但每个从节点只能由一个主节点。
2、作用
- 数据冗余:主从复制实现了数据的热备份,是持久化之外的一种数据冗余的方式。
- 故障恢复:当主节点故障时,从节点可以暂时替代主节点提供服务,是一种服务冗余的方式
- 负载均衡:在主从复制的基础上,配合读写分离,由主节点进行写操作,从节点进行读操作,分担服务器的负载;尤其是在多读少写的场景下,通过多个从节点分担负载,提高并发量。
- 高可用基石:主从复制还是哨兵和集群能够实施的基础。
3、使用集群原因
- 从结构上,单个Redis服务器会发生单点故障,并且一台服务器需要处理所有的请求负载,压力较大;
- 从容量上,单个Redis服务器内存容量有限,就算一台Redis服务器内存容量为256G,也不能将所有内存用作Redis存储内存,一般来说,单台Redis最大使用内存不应该超过20G。
电商网站上的商品,一般都是一次上传,无数次浏览的,说专业点也就是"多读少写"。
对于这种场景,我们可以使如下这种架构︰
主从复制,读写分离!80%的情况下都是在进行读操作!减缓服务器的压力!架构中经常使用!
一主二从!只要在公司中,主从复制就是必须要使用的,因为在真实的项目中不可能单机使用Redis !
4、环境配置
只需要配置从库,不需要配置主库,因为redis默认就是主库。
127.0.0.1:6379> info replication # 查看当前库的信息
# Replication
role:master # 角色:master
connected_slaves:0 # 从机:0
master_failover_state:no-failover
master_replid:798c2a7bb6fee2f79db133c42ca7fa5b4f6a606d
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
既然需要启动多个服务,就需要多个配置文件。每个配置文件对应修改以下信息:
- redis.conf名字
- 端口号
- pid文件名字
- 日志文件名字
- dump.rdb名字
修改完之后,docker启动进程:
启动命令:
docker run -it -p 9001:6379 --name myredis1 -v /home/config/redis/redis6379.conf:/etc/redis/redis6379.conf -v /home/appdata/redis/data6379:/data -d redis:7.0.4 redis-server /etc/redis/redis6379.conf
docker run -it -p 9002:6380 --name myredis2 -v /home/config/redis/redis6380.conf:/etc/redis/redis6380.conf -v /home/appdata/redis/data6380:/data -d redis:7.0.4 redis-server /etc/redis/redis6380.conf
docker run -it -p 9003:6381 --name myredis3 -v /home/config/redis/redis6381.conf:/etc/redis/redis6381.conf -v /home/appdata/redis/data6381:/data -d redis:7.0.4 redis-server /etc/redis/redis6381.conf
效果:
5、一主二丛
一般情况下只需要配置从机就可以了。
一主(79)二丛(80,81) docker版redis容器而非在linux中直接安装redis
方法一:命令式配置,未成功
首先按上述命令启动3个redis容器:
进入到三个容器中:
第一个容器:
[root@mycentos redis]# docker exec -it myredis1 /bin/bash
root@6267ca043b26:/data# cd ../usr/local/bin/
root@6267ca043b26:/usr/local/bin# redis-cli -p 6379
127.0.0.1:6379> info replication
# Replication
role:master # 主机
connected_slaves:0 # 没有从机
master_failover_state:no-failover
master_replid:a92030362994f3b6f7c72fffd26588c563d09438
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
第二个容器:
[root@mycentos redis]# docker exec -it myredis2 /bin/bash
root@3aa893951063:/data# cd ../usr/local/bin/
root@3aa893951063:/usr/local/bin# redis-cli -p 6380
127.0.0.1:6380> info replication
# Replication
role:master
connected_slaves:0
master_failover_state:no-failover
master_replid:2deec4e1e28b86207e2cee40039feec6de14f98c
master_replid2:d21c8ff9ed3ff198dcdc04f1bebb2226a48e5cc2
master_repl_offset:0
second_repl_offset:1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
# 设置从机尝试方法一:
127.0.0.1:6380> slaveof 127.0.0.1 6379 # 将该redis设为第一个容器的从机
OK
127.0.0.1:6380> info replication
# Replication
role:slave
master_host:127.0.0.1
master_port:6379
master_link_status:down # down:失败
master_last_io_seconds_ago:-1
master_sync_in_progress:0
slave_read_repl_offset:0
slave_repl_offset:0
master_link_down_since_seconds:-1
slave_priority:100
slave_read_only:1
replica_announced:1
connected_slaves:0
master_failover_state:no-failover
master_replid:2deec4e1e28b86207e2cee40039feec6de14f98c
master_replid2:d21c8ff9ed3ff198dcdc04f1bebb2226a48e5cc2
master_repl_offset:0
second_repl_offset:1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
# 设置从机尝试方法二:
[root@mycentos redis]# docker inspect 01197133ef90|grep IPAddress
"SecondaryIPAddresses": null,
"IPAddress": "172.17.0.3",
"IPAddress": "172.17.0.3",
127.0.0.1:6380> slaveof 172.17.0.3 6379 # 将该redis设为第一个容器的从机
OK
# 依然失败
# 设置从机尝试方法三:失败
127.0.0.1:6380> slaveof 172.17.0.3 9001 # 将该redis设为第一个容器的从机
OK
# 设置从机尝试方法四:失败
127.0.0.1:6380> slaveof 127.0.0.1 9001 # 将该redis设为第一个容器的从机
OK
如果直接在linux中安装redis,并按上述方法成功设置主从机,当从机或者主机宕机的时候,主从设置会直接消失,因为按照命令做的主从设置只是暂时的,想永久生效,必须在配置文件中增加设置!
方法二:命令式配置,成功
在很长的一段时间里,Redis一直使用SLAVEOF作为复制命令,但是从 5.0.0版本开始,Redis正式将SLAVEOF命令改名为REPLICAOF命令并逐渐废弃原来的SLAVEOF命令。因此,如果你使用的是Redis 5.0.0之前的版本,那么请使用SLAVEOF命令代替本章中的REPLICAOF命令,并使用slaveof配置选项代替本章中的replicaof配置选项。与此相反,如果你使用的是Redis 5.0.0或之后的版本,那么就应该使用REPLICAOF命令 而不是SLAVEOF命令,因为后者可能会在未来的某个时候被正式废弃。
127.0.0.1:6381> replicaof 127.0.0.1 6379 # 失败,docker启动的redis容器无法直接使用127.0.0.1连接
OK
127.0.0.1:6381> replicaof 172.17.0.3 6379 # 成功,使用方法一中查到的docker给容器内部分配的IPAddress
OK
方法二:直接修改配置文件,成功
redis6379.conf
#允许远程连接
bind 0.0.0.0 # 亲自测试,若为bind 127.0.0.1,则从机无法连接
#关闭保护模式
protected-mode no #若为yes,即便bind 0.0.0.0,从机也无法连接
redis6380.conf
bind 0.0.0.0 # 若为bind 127.0.0.1,可以连接上主机,但经过后续测试,若设置为127...,则哨兵模式无法成功
protected-mode no # 保护模式,从机设置为yes也能连接主机
replica-read-only yes # 设置从机是否是可读模式,默认yes,只读;no 支持写操作 旧版为slave-read-only
replicaof 172.17.0.3 6379 # 将服务器设置为从服务器,设置主机的host port
redis6381.conf
bind 0.0.0.0
protected-mode no
replica-read-only yes
replicaof 172.17.0.3 6379
细节
主机可以写,从机只能读不能写。主机中的所有信息和数据,都会被从机自动保存!
127.0.0.1:6379> set k1 vvvvvvv1 # 主机新建一个key
OK
127.0.0.1:6379> get k1
"vvvvvvv1"
# 从机 6380
127.0.0.1:6380> get k1 # 主机新建的key,从机也保存了
"vvvvvvv1"
127.0.0.1:6380> set k2 v2 # 从机无法进行写操作
(error) READONLY You can't write against a read only replica.
# 从机 6381
127.0.0.1:6381> get k1
"vvvvvvv1"
当主机宕机挂掉以后,从机依然存在,可读不可写。当主机恢复以后,再次写入数据,从机依然可以获取到主机内的数据信息,保证了集群的高可用性。
当从机宕掉以后,主机中写入了新的数据,从机恢复以后会立刻获得主机的全部信息。
复制原理
Slave启动成功连接到master后会发送一个sync同步命令
Master接到命令,启动后台的存盘进程,同时收集所有接收到的用于修改数据集命令,在后台进程执行完毕之后,master将传送整个数据文件到slave,并完成一次完全同步。
**全量复制:**而slave服务在接收到数据库文件数据后,将其存盘并加载到内存中。
**增量复制:**Master继续将新的所有收集到的修改命令依次传给slave,完成同步
但是只要是重新连接master,一次完全同步(全量复制)将被自动执行!我们的数据一定可以在从机中看到!
链路主从
如上图所示,80作为79的从节点,同时又作为81的主节点。当79内写入数据时,80,81也会保存相同的数据,只不过此时80无法进行写操作。
当主机宕掉以后,无法及时恢复,有两种方式方式可以产生新的主机:
- 手动执行命令
replicaof no one
(5.0版本之前是slave no one
)来让从机变成主机。 - 哨兵模式,会自动选举主机。
十三、哨兵模式
1、概述
主从切换技术的方法是︰当主服务器宕机后,需要手动把一台从服务器切换为主服务器,这就需要人工干预,费事费力,还会造成一段时间内服务不可用。这不是一种推荐的方式,更多时候,我们优先考虑哨兵模式。Redis从2.8开始正式提供了Sentinel (哨兵)架构来解决这个问题。
谋朝篡位的自动版,能够后台监控主机是否故障,如果故障了根据投票数自动将从库转换为主库。
哨兵模式是一种特殊的模式,首先Redis提供了哨兵的命令,哨兵是一个独立的进程,作为进程,它会独立运行。其原理是哨兵通过发送命令,等待Redis服务器响应,从而监控运行的多个Redis实例。
单哨兵模式:
哨兵的作用:
- 通过发送命令,让Redis服务器返回监控其运行状态,包括主服务器和从服务器。
- 当哨兵监测到master宕机,会自动将slave切换成master,然后通过发布订阅模式通知其他的从服务器,修改配置文件,让它们切换主机。
然而一个哨兵进程对Redis服务器进行监控,可能会出现问题,为此,我们可以使用多个哨兵进行监控。各个哨兵之间还会进行监控,这样就形成了多哨兵模式。
多哨兵模式:
假设主服务器宕机,哨兵1先检测到这个结果,系统并不会马上进行failover过程,仅仅是哨兵1主观的认为主服务器不可用,这个现象成为主观下线。当后面的哨兵也检测到主服务器不可用,并且数量达到一定值时,那么哨兵之间就会进行一次投票,投票的结果由一个哨兵发起,进行failover[故障转移]操作。切换成功后,就会通过发布订阅模式,让各个哨兵把自己监控的从服务器实现切换主机,这个过程称为客观下线。
2、示例
启动哨兵容器:
尝试一:失败
进入到redis容器中docker exec -it myredis1 /bin/bash
,在/usr/local/bin/
中,创建sentinel.conf,内容如下:
sentinel monitor mymaster 172.17.0.3 6379 2
#哨兵端口号 16379 16380 16381
port 16379
daemonize yes
注:容器中没有vim的话,需要先下载:
apt-get update # 更新依赖
apt-get install -y vim # 下载vim
启动哨兵:redis-sentinel sentinel.conf
另外两个同样操作,然后停掉redis主机,查看从机状态,结果并没有进行选举出新master。
尝试二:失败
配置:sentinel16379.conf,另外两个同理
# 不限制ip
bind 0.0.0.0 # 没有尝试127.0.0.1的话是否能监测
# 不让sentinel服务后台运⾏,设置为yes的话运行该容器时就不能再加`-d`
daemonize no
# 禁止保护模式
#protected-mode no
# sentinel monitor <master-redis-name> <master-redis-ip> <master-redis-port> <quorum>
# 配置监听的主服务器,master-redis-name代表服务器的名称,master-redis-ip代表监控的主redis服务,master-redis-port代表端⼝
# quorum是一个数字,指明当有多少个sentinel认为一个master失效时(值一般为:sentinel总数/2 + 1),master才算真正失效
sentinel monitor mymaster 172.17.0.3 6379 2
#哨兵端口号 16379 16380 16381
port 16379
# sentinel auth-pass定义服务的密码,mymaster是服务名称,123456是Redis服务器密码
# sentinel auth-pass mymaster 123456
#超过5秒master还没有连接上,则认为master已经停止
#sentinel down-after-milliseconds mymaster 5000
#如果该时间内没完成failover操作,则认为本次failover失败
#sentinel failover-timeout mymaster 180000
#sentinel parallel-syncs mymaster 1
logfile "sentinel16379.log"
启动sentinel容器:
docker run --name sentinel1 -p 16379:16379 --restart=always -v /home/appdata/redis/sentinel/data16379:/data -v /home/config/redis/sentinel16379.conf:/etc/sentinel16379.conf -d redis:7.0.4 redis-sentinel /etc/sentinel16379.conf
docker run --name sentinel2 -p 16380:16380 --restart=always -v /home/appdata/redis/sentinel/data16380:/data -v /home/config/redis/sentinel16380.conf:/etc/sentinel16380.conf -d redis:7.0.4 redis-sentinel /etc/sentinel16380.conf
docker run --name sentinel3 -p 16381:16381 --restart=always -v /home/appdata/redis/sentinel/data16381:/data -v /home/config/redis/sentinel16381.conf:/etc/sentinel16381.conf -d redis:7.0.4 redis-sentinel /etc/sentinel16381.conf
此时再次停掉master,仍然没有自动选举,失败
尝试三:失败
在尝试二基础上,进入到sentinel容器中:
docker exec -it sentinel1 /bin/bash
cd ../usr/local/bin/
redis-sentinel /etc/sentinel16379.conf
依然失败。
尝试四:成功
启动从机的redis.conf中,所有的bind设置必须为0.0.0.0,并且protected-mode设置为no,此时再按照尝试二,启动三个sentinel容器,然后停掉matser,然后过30s(默认时间),此时会其中的一台从机81变成了master,另一台从机80变成了原从机81的slave。
此时再次启动6379的redis容器,发现79角色同样变成了81的从机:
哨兵日志:
在master宕机后非发起投票的哨兵日志:
宕机后发起投票的哨兵日志:
3、哨兵模式优缺点
优点:
- 哨兵集群,基于主从复制模式,所有主从复制的优点,它都有
- 主从可以切换,故障可以转移,系统的可用性更好
- 哨兵模式是主从模式的升级,手动到自动,更加健壮
缺点:
- Redis不好在线扩容,集群容量一旦达到上限,在线扩容就十分麻烦
- 实现哨兵模式的配置其实是很麻烦的,里面有很多配置项
4、哨兵模式的全部配置
# Example sentinel.conf
# 哨兵sentinel实例运行的端口 默认26379
port 26379
# 哨兵sentinel的工作目录
dir /tmp
# 哨兵sentinel监控的redis主节点的 ip port
# master-name 可以自己命名的主节点名字 只能由字母A-z、数字0-9 、这三个字符".-_"组成。
# quorum 当这些quorum个数sentinel哨兵认为master主节点失联 那么这时 客观上认为主节点失联了
# sentinel monitor <master-name> <ip> <redis-port> <quorum>
sentinel monitor mymaster 127.0.0.1 6379 1
# 当在Redis实例中开启了requirepass foobared 授权密码 这样所有连接Redis实例的客户端都要提供密码
# 设置哨兵sentinel 连接主从的密码 注意必须为主从设置一样的验证密码
# sentinel auth-pass <master-name> <password>
sentinel auth-pass mymaster MySUPER--secret-0123passw0rd
# 指定多少毫秒之后 主节点没有应答哨兵sentinel 此时 哨兵主观上认为主节点下线 默认30秒
# sentinel down-after-milliseconds <master-name> <milliseconds>
sentinel down-after-milliseconds mymaster 30000
# 这个配置项指定了在发生failover主备切换时最多可以有多少个slave同时对新的master进行同步,这个数字越小,完成failover所需的时间就越长,但是如果这个数字越大,就意味着越 多的slave因为replication而不可用。可以通过将这个值设为 1 来保证每次只有一个slave 处于不能处理命令请求的状态。
# sentinel parallel-syncs <master-name> <numslaves>
sentinel parallel-syncs mymaster 1
# 故障转移的超时时间 failover-timeout 可以用在以下这些方面:
#1. 同一个sentinel对同一个master两次failover之间的间隔时间。
#2. 当一个slave从一个错误的master那里同步数据开始计算时间。直到slave被纠正为向正确的master那里同步数据时。
#3.当想要取消一个正在进行的failover所需要的时间。
#4.当进行failover时,配置所有slaves指向新的master所需的最大时间。不过,即使过了这个超时,slaves依然会被正确配置为指向master,但是就不按parallel-syncs所配置的规则来了
# 默认三分钟
# sentinel failover-timeout <master-name> <milliseconds>
sentinel failover-timeout mymaster 180000
# SCRIPTS EXECUTION
#配置当某一事件发生时所需要执行的脚本,可以通过脚本来通知管理员,例如当系统运行不正常时发邮件通知相关人员。
#对于脚本的运行结果有以下规则:
#若脚本执行后返回1,那么该脚本稍后将会被再次执行,重复次数目前默认为10
#若脚本执行后返回2,或者比2更高的一个返回值,脚本将不会重复执行。
#如果脚本在执行过程中由于收到系统中断信号被终止了,则同返回值为1时的行为相同。
#一个脚本的最大执行时间为60s,如果超过这个时间,脚本将会被一个SIGKILL信号终止,之后重新执行。
#通知型脚本:当sentinel有任何警告级别的事件发生时(比如说redis实例的主观失效和客观失效等等),将会去调用这个脚本,
#这时这个脚本应该通过邮件,SMS等方式去通知系统管理员关于系统不正常运行的信息。调用该脚本时,将传给脚本两个参数,
#一个是事件的类型,
#一个是事件的描述。
#如果sentinel.conf配置文件中配置了这个脚本路径,那么必须保证这个脚本存在于这个路径,并且是可执行的,否则sentinel无法正常启动成功。
#通知脚本
# sentinel notification-script <master-name> <script-path>
sentinel notification-script mymaster /var/redis/notify.sh
# 客户端重新配置主节点参数脚本
# 当一个master由于failover而发生改变时,这个脚本将会被调用,通知相关的客户端关于master地址已经发生改变的信息。
# 以下参数将会在调用脚本时传给脚本:
# <master-name> <role> <state> <from-ip> <from-port> <to-ip> <to-port>
# 目前<state>总是“failover”,
# <role>是“leader”或者“observer”中的一个。
# 参数 from-ip, from-port, to-ip, to-port是用来和旧的master和新的master(即旧的slave)通信的
# 这个脚本应该是通用的,能被多次调用,不是针对性的。
# sentinel client-reconfig-script <master-name> <script-path>
sentinel client-reconfig-script mymaster /var/redis/reconfig.sh
十四、缓存穿透与雪崩(面试高频,工作常用)
在这里我们不会详细的区分析解决方案的底层(专题)
Redis缓存的使用,极大的提升了应用程序的性能和效率,特别是数据查询方面。但同时,它也带来了一些问题。其中,最要害的问题,就是数据的一致性问题,从严格意义上讲,这个问题无解。如果对数据的一致性要求很高,那么就不能使用缓存。
另外的一些典型问题就是,缓存穿透、缓存雪崩和缓存击穿。目前,业界也都有比较流行的解决方案。
1、缓存穿透
1.1、概念
在默认情况下,用户请求数据时,会先在缓存(Redis)中查找,若没找到即缓存未命中,再在数据库中进行查找,数量少可能问题不大,可是一旦大量的请求数据(例如秒杀场景)缓存都没有命中的话,就会全部转移到数据库上,造成数据库极大的压力,就有可能导致数据库崩溃。网络安全中也有人恶意使用这种手段进行攻击被称为洪水攻击。
1.2、解决方案
1.2.1、布隆过滤器
布隆过滤器是一种数据结构,对所有可能查询的参数以hash形式存储,在控制层先进行校验,不符合则丢弃,从而避免了对底层存储系统的查询压力;
1.2.2、缓存空对象
当存储层不命中后,即使返回的空对象也将其缓存起来,同时会设置一个过期时间,之后再访问这个数据将会从缓存中获取,保护了后端数据源;
但是这种方法会存在两个问题:
- 如果空值能够被缓存起来,这就意味着缓存需要更多的空间存储更多的键,因为这当中可能会有很多的空值的键;
- 即使对空值设置了过期时间,还是会存在缓存层和存储层的数据会有一段时间窗口的不一致,这对于需要保持一致性的业务会有影响。
2、缓存击穿
2.1、概念
这里需要注意和缓存击穿的区别,缓存击穿,是指一个key非常热点,在不停的扛着大并发,大并发集中对这一个点进行访问,当这个key在失效的瞬间,持续的大并发就穿破缓存,直接请求数据库,就像在一个屏障上凿开了一个洞。
当某个key在过期的瞬间,有大量的请求并发访问,这类数据一般是热点数据,由于缓存过期,会同时访问数据库来查询最新数据,并且回写缓存,会导使数据库瞬间压力过大。
2.2、解决方案
设置热点数据永不过期
从缓存层面来看,没有设置过期时间,所以不会出现热点key过期后产生的问题。
加互斥锁
分布式锁:使用分布式锁,保证对于每个key同时只有一个线程去查询后端服务,其他线程没有获得分布式锁的权限,因此只需要等待即可。这种方式将高并发的压力转移到了分布式锁,因此对分布式锁的考验很大。
3、缓存雪崩
3.1、概念
缓存雪崩,是指在某一个时间段,缓存集中过期失效。Redis宕机!
产生雪崩的原因之一,比如在写本文的时候,马上就要到双十二零点,很快就会迎来一波抢购,这波商品时间比较集中的放入了缓存,假设缓存一个小时。那么到了凌晨一点钟的时候,这批商品的缓存就都过期了。而对这批商品的访问查询,都落到了数据库上,对于数据库而言,就会产生周期性的压力波峰。于是所有的请求都会达到存储层,存储层的调用量会暴增,造成存储层也会挂掉的情况。
其实集中过期,倒不是非常致命,比较致命的缓存雪崩,是缓存服务器某个节点宕机或断网。因为自然形成的缓存雪崩,一定是在某个时间段集中创建缓存,这个时候,数据库也是可以顶住压力的。无非就是对数据库产生周期性的压力而已。而缓存服务节点的宕机,对数据库服务器造成的压力是不可预知的,很有可能瞬间就把数据库压垮。
3.2、解决方案
redis高可用
这个思想的含义是,既然redis有可能挂掉,那我多增设几台redis,这样一台挂掉之后其他的还可以继续工作,其实就是搭建的集群。(异地多活!)
限流降级(在Springcloud讲解过!)
这个解决方案的思想是,在缓存失效后,通过加锁或者队列来控制读数据库写缓存的线程数量。比如对某个key只允许一个线程查询数据和写缓存,其他线程等待。
数据预热
数据加热的含义就是在正式部署之前,我先把可能的数据先预先访问一遍,这样部分可能大量访问的数据就会加载到缓存中。在即将发生大并发访问前手动触发加载缓存不同的key,设置不同的过期时间,让缓存失效的时间点尽量均匀。
docker run -d -v /home/ftpdata/data:/home/vsftpd -v /etc/localtime:/etc/localtime -v /var/log/vsftpd/:/var/log/vsftpd/ -p 20:20 -p 21:21 -p 21100-21110:21100-21110 -e FTP_USER=ftpuser -e FTP_PASS=ftpuser -e PASV_MIN_PORT=21100 -e PASV_MAX_PORT=21110 -e PASV_ADDRESS=43.143.140.141 -e LOG_STDOUT=1 --name myvsftpd --restart=always fauria/vsftpd:latest