Building File Server Based on Distributed Learning

Keywords: Java ftp vsftpd iptables

1. Document upload in traditional environment

In traditional projects, you can add a folder to the web project to store the uploaded images. For example, create an image folder under the project's root directory, WebRoot. Storing pictures in this folder can be directly used in engineering reference.

Advantages: Easy to cite, easy to manage

Disadvantages:

1. If it is a distributed environment, there will be problems in image reference.

2. The download of pictures will add extra pressure to the server.

Problems of traditional projects in tomcat cluster environment:

 

2. How Documents are Processed in Distributed Environment

Distributed environments generally have a dedicated image server to store images.

We use virtual machines to build a dedicated server to store pictures. Install an nginx server to provide http services and an ftp server to provide image upload services.

Environment:

Linux: CentOS 6.5 64-bit

Nginx: 1.8.0 Refer to http://www.cnblogs.com/jalja/p/6104325.html

Vsftpd: It needs to be installed online.

Linux Installs ftp Components

1. Install vsftpd components

After installation, there is / etc/vsftpd/vsftpd.conf file, which is the configuration file of vsftp.

[root@bogon ~]# yum -y install vsftpd

2. Add an ftp user

This user is used to log on to the ftp server.

[root@bogon ~]# useradd ftpuser

When such a user is finished, he can use this login. Remember to use ordinary login instead of anonymity. The default path after login is / home/ftpuser.

3. Adding passwords to ftp users.

[root@bogon ~]# passwd ftpuser

Change the password after entering it twice.

4. Firewall Opens Port 21

Because the default port of ftp is 21 and centos is not open by default, the iptables file needs to be modified.

[root@bogon ~]# vim /etc/sysconfig/iptables

On the top of the line, there is another line under 22-j ACCEPT that has the same input as that line, just change 22 to 21, and then save wq.

To run, restart iptables

[root@bogon ~]# service iptables restart

5. Modify selinux

External network can be accessed, but found that the directory can not be returned (using ftp active mode, passive mode is still not accessible), and uploaded, because selinux is weird.

Modify selinux:

Execute the following command to view the status:

[root@bogon ~]# getsebool -a | grep ftp 

allow_ftpd_anon_write --> off

allow_ftpd_full_access --> off

allow_ftpd_use_cifs --> off

allow_ftpd_use_nfs --> off

ftp_home_dir --> off

ftpd_connect_db --> off

ftpd_use_passive_mode --> off

httpd_enable_ftp_server --> off

tftp_anon_write --> off

[root@bogon ~]#

Execute the above command, and return the result to see that both lines are off, on behalf of, no access to the external network.

[root@bogon ~]# setsebool -P allow_ftpd_full_access on

[root@bogon ~]# setsebool -P ftp_home_dir on

This should be no problem. (If, or not, see if it is accessed in passive mode with ftp client tool. If you prompt Entering Passive mode, it means passive mode. Default is not possible, because ftp passive mode is blocked by iptables. How to open it will be discussed below. If you are lazy, see if your client ftp has the choice of port mode. Item, or remove the option of passive mode. If the client still can't, check to see if the host computer on the client has a firewall. Shut it off.

FileZilla's active and passive mode modification:

Menu: Edit Settings

 

6. Turn off anonymous access

Modify the/etc/vsftpd/vsftpd.conf file:

# READ THIS: This example file is NOT an exhaustive list of vsftpd options.
# Please read the vsftpd.conf.5 manual page to get a full idea of vsftpd's
# capabilities.
#
# Allow anonymous FTP? (Beware - allowed by default if you comment this out).
anonymous_enable=NO  #Change YES to NO
#
# Uncomment this to allow local users to log in.
local_enable=YES
#
# Uncomment this to enable any form of FTP write command.
write_enable=YES
#
# Default umask for local users is 077. You may wish to change this to 022,
# if your users expect that (022 is used by most other ftpd's)
local_umask=022
#

Restart ftp service:

[root@bogon ~]# service vsftpd restart

7. Turn on passive mode

The default is open, but to specify a port range, open the vsftpd.conf file and add it later

pasv_min_port=30000

pasv_max_port=30999

The ports range from 30000 to 30999, which can be changed at will. Change and restart vsftpd

Since this port range is specified, iptables should also open this range accordingly, so open the iptables file as above.

It's also a new line on top and bottom of 21, more like that line, just change 21 to 30000:30999, then save wq and restart iptables. That's it.

8. Set up boot to start vsftpd ftp service

[root@bogon ~]# chkconfig vsftpd on

9. Use tools to test whether the environment has been successfully built

10. Modify nginx configuration file nginx.conf to point html root directory to FTP resource directory

    #gzip  on;

    server {
        listen       80;
        server_name  localhost;

        #charset koi8-r;

        #access_log  logs/host.access.log  main;

        location / {
            root    /home/ftpuser/www;
            index  index.html index.htm;
        }

 

 

java uploads files using FTP tools

 

Use the commons-net Toolkit

<dependency>
    <groupId>commons-net</groupId>
    <artifactId>commons-net</artifactId>
    <version>3.3</version>
</dependency>

Upload test code:

@Test
    public void testFtpClient() throws Exception {
        //Create a FtpClient object
        FTPClient ftpClient = new FTPClient();
        //Establish ftp Connect. The default is port 21
        ftpClient.connect("192.168.6.189", 21);
        //Sign in ftp Server, using username and password
        ftpClient.login("ftpuser", "ftpuser");
        //Upload files.
        //Read local files
        FileInputStream inputStream = new FileInputStream(new File("E:\\test\\2.png"));
        //Setting Upload Path
        ftpClient.changeWorkingDirectory("/home/ftpuser/www/images");
        //Modify the format of uploaded files to avoid losing data
        ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
        //First parameter: server-side document name
        //Second parameter: Upload document inputStream
        ftpClient.storeFile("hello1.png", inputStream);
        //Close connection
        ftpClient.logout();
        
    }

 

 

Tools:

package com.taotao.utils;

import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;

import org.apache.commons.net.ftp.FTP;
import org.apache.commons.net.ftp.FTPClient;
import org.apache.commons.net.ftp.FTPFile;
import org.apache.commons.net.ftp.FTPReply;

/**
 * ftp Upload and Download Tool Class
 * <p>Title: FtpUtil</p>
 * <p>Description: </p>
 * <p>Company: www.itcast.com</p> 
 * @author    Yun Long Yun
 * @date    2015 July 29, 2000, 8:11:51 p.m.
 * @version 1.0
 */
public class FtpUtil {

    /** 
     * Description: Upload files to FTP server 
     * @param host FTP Server hostname 
     * @param port FTP Server Port 
     * @param username FTP Login account 
     * @param password FTP Login password 
     * @param basePath FTP Server base directory
     * @param filePath FTP Server file storage path. For example, date storage: / 2015/01/01. The path of the file is basePath+filePath
     * @param filename File name uploaded to FTP server 
     * @param input Input stream 
     * @return Return true successfully or false otherwise 
     */  
    public static boolean uploadFile(String host, int port, String username, String password, String basePath,
            String filePath, String filename, InputStream input) {
        boolean result = false;
        FTPClient ftp = new FTPClient();
        try {
            int reply;
            ftp.connect(host, port);// Connect FTP The server
            // If the default port is used, you can use the ftp.connect(host)Ways to connect directly FTP The server
            ftp.login(username, password);// Sign in
            reply = ftp.getReplyCode();
            if (!FTPReply.isPositiveCompletion(reply)) {
                ftp.disconnect();
                return result;
            }
            //Switch to upload directory
            if (!ftp.changeWorkingDirectory(basePath+filePath)) {
                //Create a directory if it does not exist
                String[] dirs = filePath.split("/");
                String tempPath = basePath;
                for (String dir : dirs) {
                    if (null == dir || "".equals(dir)) continue;
                    tempPath += "/" + dir;
                    if (!ftp.changeWorkingDirectory(tempPath)) {
                        if (!ftp.makeDirectory(tempPath)) {
                            return result;
                        } else {
                            ftp.changeWorkingDirectory(tempPath);
                        }
                    }
                }
            }
            //Set the type of uploaded file to binary type
            ftp.setFileType(FTP.BINARY_FILE_TYPE);
            //Upload files
            if (!ftp.storeFile(filename, input)) {
                return result;
            }
            input.close();
            ftp.logout();
            result = true;
        } catch (IOException e) {
            e.printStackTrace();
        } finally {
            if (ftp.isConnected()) {
                try {
                    ftp.disconnect();
                } catch (IOException ioe) {
                }
            }
        }
        return result;
    }
    
    /** 
     * Description: Download files from FTP server 
     * @param host FTP Server hostname 
     * @param port FTP Server Port 
     * @param username FTP Login account 
     * @param password FTP Login password 
     * @param remotePath FTP Relative paths on servers 
     * @param fileName File name to download 
     * @param localPath The path saved locally after downloading 
     * @return 
     */  
    public static boolean downloadFile(String host, int port, String username, String password, String remotePath,
            String fileName, String localPath) {
        boolean result = false;
        FTPClient ftp = new FTPClient();
        try {
            int reply;
            ftp.connect(host, port);
            // If the default port is used, you can use the ftp.connect(host)Ways to connect directly FTP The server
            ftp.login(username, password);// Sign in
            reply = ftp.getReplyCode();
            if (!FTPReply.isPositiveCompletion(reply)) {
                ftp.disconnect();
                return result;
            }
            ftp.changeWorkingDirectory(remotePath);// Transfer to FTP Server directory
            FTPFile[] fs = ftp.listFiles();
            for (FTPFile ff : fs) {
                if (ff.getName().equals(fileName)) {
                    File localFile = new File(localPath + "/" + ff.getName());
                    OutputStream is = new FileOutputStream(localFile);
                    ftp.retrieveFile(ff.getName(), is);
                    is.close();
                }
            }

            ftp.logout();
            result = true;
        } catch (IOException e) {
            e.printStackTrace();
        } finally {
            if (ftp.isConnected()) {
                try {
                    ftp.disconnect();
                } catch (IOException ioe) {
                }
            }
        }
        return result;
    }
    
    public static void main(String[] args) {
        try {  
            FileInputStream in=new FileInputStream(new File("D:\\temp\\image\\gaigeming.jpg"));  
            boolean flag = uploadFile("192.168.25.133", 21, "ftpuser", "ftpuser", "/home/ftpuser/www/images","/2015/01/21", "gaigeming.jpg", in);  
            System.out.println(flag);  
        } catch (FileNotFoundException e) {  
            e.printStackTrace();  
        }  
    }
}

Posted by Mgccl on Tue, 09 Apr 2019 23:51:31 -0700