注意InfluxDB的版本号,不同版本号的语法不同,我使用的是InfluxDB2.3.0版本
官网: InfluxDB OSS 2.3 Documentation (influxdata.com)
1.常用的查询命令
参考: InfluxDB v2：Flux 语言、快速参考指南和备忘单
2.生成配置文件
在InfluxDB中,配置文件默认是不存在的,需要执行以下命令进行生成
docker run --rm influxdb:2.3.0 influxd print-config > config.yml
具体内容如下:
assets-path: ""
bolt-path: /var/lib/influxdb2/influxd.bolt
e2e-testing: false
engine-path: /var/lib/influxdb2/engine
feature-flags: {}
flux-log-enabled: false
hardening-enabled: false
http-bind-address: :8086
http-idle-timeout: 3m0s
http-read-header-timeout: 10s
http-read-timeout: 0s
http-write-timeout: 0s
influxql-max-select-buckets: 0
influxql-max-select-point: 0
influxql-max-select-series: 0
instance-id: ""
key-name: ""
log-level: info
metrics-disabled: false
nats-max-payload-bytes: 0
nats-port: 4222
no-tasks: false
pprof-disabled: false
query-concurrency: 1024
query-initial-memory-bytes: 0
query-max-memory-bytes: 0
query-memory-bytes: 0
query-queue-size: 1024
reporting-disabled: false
secret-store: bolt
session-length: 60
session-renew-disabled: false
sqlite-path: ""
storage-cache-max-memory-size: 1073741824
storage-cache-snapshot-memory-size: 26214400
storage-cache-snapshot-write-cold-duration: 10m0s
storage-compact-full-write-cold-duration: 4h0m0s
storage-compact-throughput-burst: 50331648
storage-max-concurrent-compactions: 0
storage-max-index-log-file-size: 1048576
storage-no-validate-field-size: false
storage-retention-check-interval: 30m0s
storage-series-file-max-concurrent-snapshot-compactions: 0
storage-series-id-set-cache-size: 0
storage-shard-precreator-advance-period: 30m0s
storage-shard-precreator-check-interval: 10m0s
storage-tsm-use-madv-willneed: false
storage-validate-keys: false
storage-wal-fsync-delay: 0s
storage-wal-max-concurrent-writes: 0
storage-wal-max-write-delay: 10m0s
storage-write-timeout: 10s
store: disk
testing-always-allow-setup: false
tls-cert: ""
tls-key: ""
tls-min-version: "1.2"
tls-strict-ciphers: false
tracing-type: ""
ui-disabled: false
vault-addr: ""
vault-cacert: ""
vault-capath: ""
vault-client-cert: ""
vault-client-key: ""
vault-client-timeout: 0s
vault-max-retries: 0
vault-skip-verify: false
vault-tls-server-name: ""
vault-token: ""
3.调整Dockerfile文件
调整配置文件映射,并且设置环境变量,指定InfluxDB读取的配置文件夹,
Dockerfile模板如下:
# 时序性数据库
influxdb:
image: influxdb:2.3.0
container_name: influxdb
environment:
- DOCKER_INFLUXDB_INIT_MODE=setup
- DOCKER_INFLUXDB_INIT_USERNAME=xxxxx
- DOCKER_INFLUXDB_INIT_PASSWORD=xxxxx
- DOCKER_INFLUXDB_INIT_ORG=xxxxx
- DOCKER_INFLUXDB_INIT_BUCKET=xxxxx
- DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=xxxxx
- INFLUXD_CONFIG_PATH=/home/ # 设置环境变量
volumes:
- /data/zfzn/config.yml:/home/config.yml # 设置映射路径
ports:
- "18086:8086"
restart: always
可以通过api访问获取配置内容
http://{ip}:{port}/api/v2/config
4.调整配置,开始测试
首先,我们使用默认配置启动InfluxDB,执行一个大数据量查询
from(bucket: "zfzn_new")
|> range(start: -30d)
|> filter(fn: (r) => r["_measurement"] == "100" or r["_measurement"] == "103" or r["_measurement"] == "101" or r["_measurement"] == "102" or r["_measurement"] == "105" or r["_measurement"] == "108" or r["_measurement"] == "107" or r["_measurement"] == "106" or r["_measurement"] == "109" or r["_measurement"] == "110")
|> fill(usePrevious: true)
|> increase(columns: ["_value"])
|> last()
|> yield()
使用top查看Influxdb服务内存占用.
调整为最大查询内存1G之后:
优化常用字段:
具体说明参考官方文档: InfluxDB 配置选项
log-level: warn # 日志输出级别
query-concurrency: 5 # 允许并发执行的查询数。 默认10
query-initial-memory-bytes: 67108864 # 64 MB 为查询分配的初始内存字节数。
query-memory-bytes: 2147483648 # 2G 单个查询允许的最大内存字节数。必须大于或等于 query-initial-memory-bytes 。
query-max-memory-bytes: 8589934592 # 8G 允许查询的最大内存总字节数。 等于查询并发×单次查询内存字节数
http-read-header-timeout: 10s #服务器应尝试读取新请求的 HTTP 标头的最长持续时间。设置为 0 表示没有超时
http-read-timeout: 10s # 读超时
http-write-timeout: 10s # 写超时
query-queue-size: 10 # 执行队列中允许的最大查询数。当达到队列限制时，新查询将被拒绝。 默认10
storage-cache-max-memory-size: 2147483648 # 2G 控制存储引擎内存缓存的大小，以字节为单位。
storage-cache-snapshot-memory-size: 26214400 # 25M 控制存储引擎快照使用的内存大小，以字节为单位。
storage-cache-snapshot-write-cold-duration: 10m0s # 控制存储引擎冷数据快照写入的时间。
storage-compact-full-write-cold-duration: 4h0m0s # 控制存储引擎数据压缩的时间。
storage-compact-throughput-burst: 50331648
storage-max-concurrent-compactions: 0
storage-max-index-log-file-size: 1048576
storage-no-validate-field-size: false
storage-retention-check-interval: 30m0s
storage-series-file-max-concurrent-snapshot-compactions: 0
storage-series-id-set-cache-size: 0
storage-shard-precreator-advance-period: 30m0s
storage-shard-precreator-check-interval: 10m0s
storage-tsm-use-madv-willneed: false
storage-validate-keys: false
storage-wal-fsync-delay: 0s
storage-wal-max-concurrent-writes: 0
storage-wal-max-write-delay: 10m0s
storage-write-timeout: 10s
store: disk
5.另外一些优化方案
下面提出了内存释放问题解决方案
- GODEBUG=madvdontneed=1
- GOGC=80
