FastDFS+Nginx

  1. 云栖社区>
  2. 博客>
  3. 正文

FastDFS+Nginx

余二五 2017-11-14 14:40:00 浏览654
展开阅读全文

1.1 安装(两台一样的操作)

  github下载安装包

1
2
wget 
https://github.com/happyfish100/libfastcommon/archive/V1.0.35.tar.gz


解压安装包

1
2
3
4
5
6
tar xf V1.0.35.tar.gz
cd libfastcommon-1.0.35/
./make.sh
echo $?
./make.sh install
echo $?

编译安装完成

 

  github下载安装包

1
2
3
4
5
6
7
8
9
10
11
wget https://github.com/happyfish100/fastdfs/archive/V5.10.tar.gz
tar xf V5.10.tar.gz
cd fastdfs-5.10/
./make.sh
./make.sh install
echo $?
whereis fdfs_storaged
vim /etc/init.d/fdfs_storaged
whereis fdfs_storaged
vim /etc/init.d/fdfs_trackerd
whereis fdfs_trackerd

 

创建数据文件存放位置

1
2
3
4
5
6
mkdir -p /data/fdfs_tracker
mkdir -p /data/fdfs_storage
cd /etc/fdfs/
  
cp tracker.conf.sample tracker.conf
cp storage.conf.sample storage.conf

 

修改配置文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
mkdir /data/fdfs_tracker
先创建mkdir /data/fdfs_storage/base /data/fdfs_storage/store  /data/fdfs_tracker
  
vim tracker.conf
base_path=/data/fdfs_tracker   刚刚创建的目录
保存
  
vim storage.conf
base_path=/data/fdfs_storage/base      
store_path0=/data/fdfs_storage/store  
多节点
tracker_server=192.168.80.123:22122
tracker_server=192.168.80.201:22122
保存

 

 

查看配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
[root@lanyezi fdfs]# grep --color '^[a-z]' /etc/fdfs/tracker.conf
disabled=false
bind_addr=
port=22122
connect_timeout=30
network_timeout=60
base_path=/data/fdfs_tracker
max_connections=256
accept_threads=1
work_threads=4
min_buff_size = 8KB
max_buff_size = 128KB
store_lookup=2
store_group=group2
store_server=0
store_path=0
download_server=0
reserved_storage_space = 10%
log_level=info
run_by_group=
run_by_user=
allow_hosts=*
sync_log_buff_interval = 10
check_active_interval = 120
thread_stack_size = 64KB
storage_ip_changed_auto_adjust = true
storage_sync_file_max_delay = 86400
storage_sync_file_max_time = 300
use_trunk_file = false
slot_min_size = 256
slot_max_size = 16MB
trunk_file_size = 64MB
trunk_create_file_advance = false
trunk_create_file_time_base = 02:00
trunk_create_file_interval = 86400
trunk_create_file_space_threshold = 20G
trunk_init_check_occupying = false
trunk_init_reload_from_binlog = false
trunk_compress_binlog_min_interval = 0
use_storage_id = false
storage_ids_filename = storage_ids.conf
id_type_in_filename = ip
store_slave_file_use_link = false
rotate_error_log = false
error_log_rotate_time=00:00
rotate_error_log_size = 0
log_file_keep_days = 0
use_connection_pool = false
connection_pool_max_idle_time = 3600
http.server_port=8080
http.check_alive_interval=30
http.check_alive_type=tcp
http.check_alive_uri=/status.html

 

 

查看配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
[root@lanyezi fdfs]# grep --color '^[a-z]' /etc/fdfs/storage.conf
disabled=false
group_name=group1
bind_addr=
client_bind=true
port=23000
connect_timeout=30
network_timeout=60
heart_beat_interval=30
stat_report_interval=60
base_path=/data/fdfs_storage/base
max_connections=256
buff_size = 256KB
accept_threads=1
work_threads=4
disk_rw_separated = true
disk_reader_threads = 1
disk_writer_threads = 1
sync_wait_msec=50
sync_interval=0
sync_start_time=00:00
sync_end_time=23:59
write_mark_file_freq=500
store_path_count=1
store_path0=/data/fdfs_storage/store
subdir_count_per_path=256
tracker_server=192.168.80.123:22122
tracker_server=192.168.80.201:22122
log_level=info
run_by_group=
run_by_user=
allow_hosts=*
file_distribute_path_mode=0
file_distribute_rotate_count=100
fsync_after_written_bytes=0
sync_log_buff_interval=10
sync_binlog_buff_interval=10
sync_stat_file_interval=300
thread_stack_size=512KB
upload_priority=10
if_alias_prefix=
check_file_duplicate=0
file_signature_method=hash
key_namespace=FastDFS
keep_alive=0
use_access_log = false
rotate_access_log = false
access_log_rotate_time=00:00
rotate_error_log = false
error_log_rotate_time=00:00
rotate_access_log_size = 0
rotate_error_log_size = 0
log_file_keep_days = 0
file_sync_skip_invalid_record=false
use_connection_pool = false
connection_pool_max_idle_time = 3600
http.domain_name=
http.server_port=8888


 

 

1.1.1 启动服务

1
2
3
4
5
[root@lanyezi fdfs]# /etc/init.d/fdfs_trackerd start
Starting FastDFS tracker server:
[root@lanyezi fdfs]# /etc/init.d/fdfs_storaged start
Starting FastDFS storage server:
[root@lanyezi fdfs]# [2017-10-19 20:56:40] ERROR - file: process_ctrl.c, line: 276, "/data/fdfs_storage/base" can't be accessed, error info: No such file or directory

创建刚刚的文件

1
mkdir /data/fdfs_storage/base /data/fdfs_storage/store

 

查看有没有起来

1
2
3
[root@lanyezi fdfs_storage]# netstat -lntup |grep fdfs
tcp        0      0 0.0.0.0:22122               0.0.0.0:*                   LISTEN      29525/fdfs_trackerd
tcp        0      0 0.0.0.0:23000               0.0.0.0:*                   LISTEN      29567/fdfs_storaged

 

1.1.2 查看日志

1
 [root@master logs]# tailf /data/fdfs_tracker/logs/trackerd.log

1.1.3 配置客户端

1
2
3
4
5
[root@lanyezi fdfs]# cp client.conf.sample client.conf
vim client.conf
base_path=/tmp
tracker_server=192.168.80.123:22122
tracker_server=192.168.80.201:22122

 

 

 

 

1.2 测试操作(两台一样的操作)

测试上传一个文件

1
2
[root@lanyezi fdfs]# fdfs_upload_file /etc/fdfs/client.conf /etc/passwd
group1/M00/00/00/wKhQe1nopb6AAz7KAAAEyYPK5Fg0384170

下载这个文件

1
[root@lanyezi ~]# fdfs_download_file /etc/fdfs/client.conf group1/M00/00/00/wKhQe1nopb6AAz7KAAAEyYPK5Fg0384170

对比完整性

1
2
3
4
[root@lanyezi ~]# md5sum wKhQe1nopb6AAz7KAAAEyYPK5Fg0384170
d2daf23218e42ce37f799fa1316c8a7d  wKhQe1nopb6AAz7KAAAEyYPK5Fg0384170
[root@lanyezi ~]# md5sum /etc/passwd
d2daf23218e42ce37f799fa1316c8a7d  /etc/passwd

文件保存在这里 自动生成一个文件名

1
ls /data/fdfs_storage/store/data/00/00/wKhQe1nopb6AAz7KAAAEyYPK5Fg0384170

查看上传的文件的属性

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
[root@master data]# fdfs_file_info /etc/fdfs/client.conf group1/M00/00/00/wKhQe1nopb6AAz7KAAAEyYPK5Fg0384170source storage id: 0
source ip address: 192.168.80.123
file create timestamp: 2017-10-19 21:16:46
file size: 1225
file crc32: 2211112024 (0x83CAE458)
  
在上传的文件的末尾追加
[root@master data]# echo "hello" > append.txt
[root@master data]# echo "world" > append2.txt
[root@master tmp]# fdfs_upload_appender /etc/fdfs/client.conf append.txt
group1/M00/00/00/wKhQyVo7YHuEXDxFAAAAAHcc3SA805.txt
[root@master tmp]# fdfs_append_file /etc/fdfs/client.conf group1/M00/00/00/wKhQyVo7YHuEXDxFAAAAAHcc3SA805.txt
Usage: fdfs_append_file <config_file> <appender_file_id> <local_filename>
[root@master tmp]# fdfs_append_file /etc/fdfs/client.conf group1/M00/00/00/wKhQyVo7YHuEXDxFAAAAAHcc3SA805.txt append2.txt
[root@master tmp]# fdfs_download_file /etc/fdfs/client.conf group1/M00/00/00/wKhQyVo7YHuEXDxFAAAAAHcc3SA805.txt
[root@master tmp]#  cat wKhQyVo7YHuEXDxFAAAAAHcc3SA805.txt
hello
world

删除一个文件

先查看属性

1
2
3
4
5
6
7
8
9
10
11
[root@master tmp]# fdfs_file_info /etc/fdfs/client.conf group1/M00/00/00/wKhQyVo7YHuEXDxFAAAAAHcc3SA805.txt
source storage id: 0
source ip address: 192.168.80.201
file create timestamp: 2017-12-21 15:19:58
file size: 12
file crc32: 1998380320 (0x771CDD20)
[root@master tmp]# fdfs_delete_file /etc/fdfs/client.conf group1/M00/00/00/wKhQyVo7YHuEXDxFAAAAAHcc3SA805.txt   删除文件
[root@master tmp]# fdfs_file_info /etc/fdfs/client.conf group1/M00/00/00/wKhQyVo7YHuEXDxFAAAAAHcc3SA805.txt   再次查看已经没有
[2017-12-21 15:23:31] ERROR - file: tracker_proto.c, line: 48, server: 192.168.80.201:23000, response status 2 != 0
[2017-12-21 15:23:31] ERROR - file: ../client/storage_client.c, line: 372, fdfs_recv_response fail, result: 2
query file info fail, error no: 2, error info: No such file or directory

 

1.3 监控fdfs集群

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
[root@master tmp]# fdfs_monitor /etc/fdfs/client.conf
[2017-12-21 15:24:39] DEBUG - base_path=/tmp, connect_timeout=30, network_timeout=60, tracker_server_count=2, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0
  
server_count=2, server_index=1
  
tracker server is 192.168.80.201:22122
  
group count: 1
  
Group 1:
group name = group1
disk total space = 18356 MB
disk free space = 11706 MB
trunk free space = 0 MB
storage server count = 2
active server count = 2
storage server port = 23000
storage HTTP port = 8888
store path count = 1
subdir count per path = 256
current write server index = 0
current trunk file id = 0
  
    Storage 1:
        id = 192.168.80.123
        ip_addr = 192.168.80.123  ACTIVE
        http domain =
        version = 5.10
        join time = 2017-10-19 20:58:00
        up time = 2017-10-19 21:08:59
        total storage = 18356 MB
        free storage = 15355 MB
        upload priority = 10
        store_path_count = 1
        subdir_count_per_path = 256
        storage_port = 23000
        storage_http_port = 8888
        current_write_path = 0
        source storage id = 192.168.80.201
        if_trunk_server = 0
        connection.alloc_count = 256
        connection.current_count = 1
        connection.max_count = 2
        total_upload_count = 2
        success_upload_count = 2
        total_append_count = 0
        success_append_count = 0
        total_modify_count = 0
        success_modify_count = 0
        total_truncate_count = 0
        success_truncate_count = 0
        total_set_meta_count = 0
        success_set_meta_count = 0
        total_delete_count = 0
        success_delete_count = 0
        total_download_count = 2
        success_download_count = 2
        total_get_meta_count = 0
        success_get_meta_count = 0
        total_create_link_count = 0
        success_create_link_count = 0
        total_delete_link_count = 0
        success_delete_link_count = 0
        total_upload_bytes = 1231
        success_upload_bytes = 1231
        total_append_bytes = 0
        success_append_bytes = 0
        total_modify_bytes = 0
        success_modify_bytes = 0
        stotal_download_bytes = 1231
        success_download_bytes = 1231
        total_sync_in_bytes = 18
        success_sync_in_bytes = 18
        total_sync_out_bytes = 0
        success_sync_out_bytes = 0
        total_file_open_count = 7
        success_file_open_count = 7
        total_file_read_count = 2
        success_file_read_count = 2
        total_file_write_count = 5
        success_file_write_count = 5
        last_heart_beat_time = 2017-12-21 15:24:27
        last_source_update = 2017-10-19 21:29:37
        last_sync_update = 2017-10-19 21:37:31
        last_synced_timestamp = 2017-12-21 15:23:28 (0s delay)
    Storage 2:
        id = 192.168.80.201
        ip_addr = 192.168.80.201 (master.test.com)  ACTIVE
        http domain =
        version = 5.10
        join time = 2017-12-21 14:51:52
        up time = 2017-12-21 14:51:52
        total storage = 18356 MB
        free storage = 11706 MB
        upload priority = 10
        store_path_count = 1
        subdir_count_per_path = 256
        storage_port = 23000
        storage_http_port = 8888
        current_write_path = 0
        source storage id =
        if_trunk_server = 0
        connection.alloc_count = 256
        connection.current_count = 1
        connection.max_count = 2
        total_upload_count = 2
        success_upload_count = 2
        total_append_count = 1
        success_append_count = 1
        total_modify_count = 0
        success_modify_count = 0
        total_truncate_count = 0
        success_truncate_count = 0
        total_set_meta_count = 0
        success_set_meta_count = 0
        total_delete_count = 1
        success_delete_count = 1
        total_download_count = 3
        success_download_count = 3
        total_get_meta_count = 0
        success_get_meta_count = 0
        total_create_link_count = 0
        success_create_link_count = 0
        total_delete_link_count = 0
        success_delete_link_count = 0
        total_upload_bytes = 12
        success_upload_bytes = 12
        total_append_bytes = 6
        success_append_bytes = 6
        total_modify_bytes = 0
        success_modify_bytes = 0
        stotal_download_bytes = 1249
        success_download_bytes = 1249
        total_sync_in_bytes = 1231
        success_sync_in_bytes = 1231
        total_sync_out_bytes = 0
        success_sync_out_bytes = 0
        total_file_open_count = 8
        success_file_open_count = 8
        total_file_read_count = 3
        success_file_read_count = 3
        total_file_write_count = 5
        success_file_write_count = 5
        last_heart_beat_time = 2017-12-21 15:24:29
        last_source_update = 2017-12-21 15:23:28
        last_sync_update = 2017-12-21 15:15:36
        last_synced_timestamp = 2017-10-19 21:29:37 (0s delay)

 

1.4 php编译模块

ls /home/fastdfs-5.10/php_client

spacer.gif

 

第2章 安装nginx模块

2.1 编译安装nginx

1
2
3
4
5
6
7
8
9
10
[root@master src]# pwd       
/usr/local/src
[root@master src]# wget http://nginx.org/download/nginx-1.10.3.tar.gz
[root@master src]# tar xf nginx-1.10.3.tar.gz
l[root@master src]# cd nginx-1.10.3
[root@master nginx-1.10.3]# ls
auto  CHANGES  CHANGES.ru  conf  configure  contrib  html  LICENSE  man  README  src
[root@master nginx-1.10.3]#
  
https://github.com/happyfish100/fastdfs-nginx-module.git


 

1
2
3
4
5
6
7
8
9
10
11
12
[root@master nginx-1.10.3]# git clone https://github.com/happyfish100/fastdfs-nginx-module.git
Initialized empty Git repository in /usr/local/src/nginx-1.10.3/fastdfs-nginx-module/.git/
remote: Counting objects: 76, done.
remote: Total 76 (delta 0), reused 0 (delta 0), pack-reused 76
Unpacking objects: 100% (76/76), done.
[root@master nginx-1.10.3]# ls
auto  CHANGES  CHANGES.ru  conf  configure  contrib  fastdfs-nginx-module  html  LICENSE  man  README  src
  
[root@master nginx-1.10.3]# useradd  -s /sbin/nologin -M www
[root@master nginx-1.10.3]# ./configure --prefix=/usr/local/nginx-1.10.3 --user=www --group=www --with-http_ssl_module --with-http_stub_status_module --add-module=./fastdfs-nginx-module/src/
[root@master nginx-1.10.3]# make
[root@master nginx-1.10.3]# make install

编译的时候报错解决:将fastdfs升级到最新版本V5.11

1
2
3
4
5
6
7
8
9
10
11
12
./fastdfs-nginx-module/src//common.c: In function ‘fdfs_http_request_handler’:
./fastdfs-nginx-module/src//common.c:1245: error: ‘FDFSHTTPParams’ has no member named ‘support_multi_range’
make[1]: *** [objs/addon/src/ngx_http_fastdfs_module.o] Error 1
make[1]: Leaving directory `/usr/local/src/nginx-1.10.2'
make: *** [install] Error 2
  
cd ..
wget https://github.com/happyfish100/fastdfs/archive/V5.11.tar.gz
tar xf V5.11.tar.gz
cd fastdfs-5.11/
./make.sh
./make.sh install

重新编译

1
2
3
4
5
cd ..
cd nginx-1.10.2
[root@master nginx-1.10.3]# ./configure --prefix=/usr/local/nginx-1.10.3 --user=www --group=www --with-http_ssl_module --with-http_stub_status_module --add-module=./fastdfs-nginx-module/src/
[root@master nginx-1.10.3]# make
[root@master nginx-1.10.3]# make install

成功

1
2
3
4
5
6
7
8
9
[root@lanyezi nginx-1.10.3]# ln -s /usr/local/nginx-1.10.3/ /usr/local/nginx
[root@lanyezi src]# pwd
/usr/local/src/nginx-1.10.2/fastdfs-nginx-module/src
[root@lanyezi src]# cp mod_fastdfs.conf /etc/fdfs/   把模块的配置复制到/etc/fdf
  
复制后来最新安装的fastdfs(源码)的一些配置文件到/etc/fdfs/
[root@lanyezi conf]# cp http.conf anti-steal.jpg mime.types /etc/fdfs/
[root@lanyezi conf]# pwd
/usr/local/src/fastdfs-5.11/conf

 

配置NGINX

1
2
3
4
5
[root@lanyezi conf]# cd /usr/local/nginx/conf/
[root@lanyezi conf]# ls
fastcgi.conf          fastcgi_params.default  mime.types          nginx.conf.default   uwsgi_params
fastcgi.conf.default  koi-utf                 mime.types.default  scgi_params          uwsgi_params.default
fastcgi_params        koi-win                 nginx.conf          scgi_params.default  win-utf

配置文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
events {
    worker_connections  1024;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;
    server {
        listen       80;
        server_name  localhost;
        location /M00 ;  
#   添加的     这样的匹配要把url_have_group_name = true
  
            root   /data/fdfs_storage/store;   添加的
            ngx_fastdfs_module;   添加的
        location / {
            root   html;
            index  index.html index.htm;
        }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {

http://bbs.chinaunix.net/forum.php?mod=viewthread&tid=3598287  

解决后期报错400

1
2
3
4
5
6
vim /etc/fdfs/mod_fastdfs.conf
修改的地方
tracker_server=192.168.80.123:22122
tracker_server=192.168.80.201:22122
url_have_group_name = true(注意后边不要有分号)
store_path0=/data/fdfs_storage/store

 

启动NGINX

1
2
3
4
5
6
7
8
9
10
[root@lanyezi conf]# /usr/local/nginx/sbin/nginx
ngx_http_fastdfs_set pid=37613
[root@lanyezi conf]# netstat -lntup
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name  
tcp        0      0 0.0.0.0:22122               0.0.0.0:*                   LISTEN      37430/fdfs_trackerd
tcp        0      0 0.0.0.0:80                  0.0.0.0:*                   LISTEN      37582/nginx        
tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN      1093/sshd          
tcp        0      0 0.0.0.0:23000               0.0.0.0:*                   LISTEN      37444/fdfs_storaged
tcp        0      0 :::22                       :::*                        LISTEN      1093/sshd

 

 

 

测试上传一个文件

1
2
[root@lanyezi conf]# fdfs_upload_file /etc/fdfs/client.conf /etc/security/limits.conf 
group1/M00/00/00/wKhQe1no40WACD39AAAHK_yyDxI74.conf

2.2 测试浏览器访问

1
http://192.168.80.123/M00/00/00/wKhQe1no40WACD39AAAHK_yyDxI74.conf

可以下载

 

配置fastdfs集群nginx

 

配置多个group

        如果需要再单台服务器实例多个storage的解决办法


一、复制一份storage.conf并修改文件名storage_aaa.conf

 二、修改storage_aaa.conf相关配置

1、修改组名:group_name=group4

2、修改端口:port=23004

3、修改日志文件目录:base_path=/fastdfs/aaa/data-and-log

4、修改资源保存路径:store_path0=/fastdfs/aaa/aaa-data

三、修改mod_fastdfs.conf配置文件

1、修改组名:group_name=group1/group2/group3/group4  多个用/区分开

2、设置组个数:group_count = 4

3、设置每一组信息:

1
2
3
4
5
6
7
8
9
[group1]group_name=group1
storage_server_port=23000
store_path_count=1
store_path0=/fastdfs/storage/images-data
[group2]
group_name=group2
storage_server_port=23001
store_path_count=1
store_path0=/fastdfs/storage_voice/voice-data

等以此类推

1
nginx日志报错ERROR - file: ../common/fdfs_global.c, line: 52, the format of filename


解决办法:

1
2
3
4
5
vi /etc/fdfs/mod_fastdfs.conf
url_have_group_name=false
改为
url_have_group_name=true



四、重启相关服务,可以访问到相关图片、语音等信息


本人在码云上提交了关于阅读spring源码之后编写的一些基础模块的实现,https://git.oschina.net/73hulu/spring.git 望大家不吝给颗星star,谢谢大家

1
[root@iZ2ze66dst6erg4u4lszcsZ fdfs_storage]# /usr/bin/fdfs_storaged /etc/fdfs/storage_aaa.conf  启动单独的配置文件









本文转自 蓝叶子Sheep 51CTO博客,原文链接:http://blog.51cto.com/dellinger/2054687,如需转载请自行联系原作者

网友评论

登录后评论
0/500
评论
余二五
+ 关注