background

Physical isolation usually cuts off the physical and logical connection between the internal network and the external network by deploying a gateway, which only ferry the original data, and does not allow any connection or protocol to pass through the gateway. In this environment, the transmission of internal and external mail has become a problem. This paper attempts to provide a set of ideas and solutions to the email transmission in this scenario by exploring. The codes contained in this paper are all demo level codes, adhering to the exploration of new fields, but it is not sure whether this method is suitable for the solution of this scenario.

Environment and configuration

The environment used in this paper is as follows:

Apache James-2.3.2.1

Apache RocketMQ-4.2.0

The pom file is as follows:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.example</groupId>
    <artifactId>MailInPIE</artifactId>
    <version>1.0-SNAPSHOT</version>

    <dependencies>
        <dependency>
            <groupId>javax.mail</groupId>
            <artifactId>mail</artifactId>
            <version>1.4.7</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>1.7.30</version>
        </dependency>
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>5.1.8</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-nop</artifactId>
            <version>1.7.2</version>
        </dependency>
        <dependency>
            <groupId>org.apache.rocketmq</groupId>
            <artifactId>rocketmq-client</artifactId>
            <version>4.2.0</version>
        </dependency>
    </dependencies>

</project>

Architectural thinking

The general structure is as follows:

• The mailet program fragment mainly includes two programs: the divitiarea agreementmatcher and the divitiarea agreementmailet. The divitiarea agreementmatcher is mainly used to match the internal and external users. If it is an internal user, it will be released. If it is an external user, it will send the email to the divitiarea agreementmailet for processing. The divitiarea agreementmailet will extract the original content of the email, which is necessary The original content can be cut and sent to the message queue.
• The message queue mainly stores the messages to be ferried. The internal messages are taken out through the message API and handed over to the gateway for ferriing
• The gateway is responsible for the ferry of messages. Its essence is to realize the data transmission of internal and external networks in the physical and logical isolation environment by reading and writing a block of shared memory in time-sharing
• The message API at the other end of the gateway sends the intranet messages ferried by the gateway to the message queue
• The mail forwarder takes out the messages to be forwarded from the message queue, reorganizes the messages if necessary, and forwards them.

Because of many difficulties, this demo will skip the gateway part. After the message is sent to the message queue, the consumer will forward the message directly. Although it is not actually tested in the gateway and Intranet and Internet environment, but! I think it's feasible!

code implementation

mailet program fragment

The implementation idea of the mailet program segment is as follows:

Investitureagreementmatcher reference code:

package com.xxxxx.pie.mail.matcher;

import com.jlszkxa.pie.mail.db.DbOperation;
import org.apache.mailet.GenericRecipientMatcher;
import org.apache.mailet.MailAddress;
import java.sql.SQLException;

/**
 * @ClassName DivestitureAgreementMatcher
 * @Description Determine whether the receiver is an intranet user. If it is, match it with the Mailet to extract the original content. Otherwise, it will be released
 * @Author chenwj
 * @Date 2020/2/20 14:53
 * @Version 1.0
 **/
public class DivestitureAgreementMatcher extends GenericRecipientMatcher {

    @Override
    public boolean matchRecipient(MailAddress mailAddress) {

        DbOperation dbOperation = new DbOperation();

        try {
            dbOperation.connectDatabase();
            String userName = mailAddress.getUser();
            String host = mailAddress.getHost();
            System.out.printf("Intercept to send to%s@%s Mail\r\n", userName, host);
            return !dbOperation.isInnerUser(userName + "@" + host);
        } catch (Exception e) {
            System.out.printf("Exception information: %s\r\n", e.getMessage());
            e.printStackTrace();
            return true;
        } finally {
            try {
                dbOperation.closeConnection();
            } catch (SQLException e) {
                e.printStackTrace();
            }
        }
    }
}

dbOperation reference code:

package com.xxxxxx.pie.mail.db;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.sql.*;

/**
 * @ClassName DbOperation
 * @Description Database operations
 * @Author chenwj
 * @Date 2020/2/20 17:34
 * @Version 1.0
 **/

public class DbOperation {

    private static final Logger logger = LoggerFactory.getLogger(DbOperation.class);

    private Connection connection;

    /*
   Connect to database
    */
    public void connectDatabase() {

        String driver = "com.mysql.jdbc.Driver";
        String url = "jdbc:mysql://localhost:3306/mail?characterEncoding=UTF-8";
        String userName = "root";
        String password = "123456";
        logger.info("Start connecting to database");
        try {
            Class.forName(driver);
            connection = DriverManager.getConnection(url, userName, password);
            logger.info("Database connection successful");
        } catch (Exception e) {
            logger.warn("Exception message in database connection: {}", e.getMessage());
        }
    }

    /**
     * Judge whether the user is an intranet user
     *
     * @param userName
     * @return
     * @throws Exception
     */
    public boolean isInnerUser(String userName) throws Exception {
        String sql = "select USER_NAME from james_user where USER_NAME = \'" + userName + "\';";
        PreparedStatement pstmt = connection.prepareStatement(sql);
        ResultSet rs = pstmt.executeQuery();
        try {
            if (rs.next()) {
                return true;
            }
            return false;
        } finally {
            rs.close();
            pstmt.close();
        }
    }

    /*
    Close connection
     */
    public void closeConnection() throws SQLException {
        if (null != connection) {
            connection.close();
        }
    }

    public static void main(String[] args) throws Exception {
        DbOperation test = new DbOperation();
        test.connectDatabase();
        boolean innerUser = test.isInnerUser("97983398@qq.com");
        System.out.printf("The result is: %s\r\n", "true");
        test.closeConnection();
    }
}

The james e-mail server built in this example stores the user information in the database, so it can directly determine whether it is an intranet user by querying the database. In addition, the domain name can also be directly intercepted for judgment.

Investitureagreementmailet reference code:

package com.xxxxxx.pie.mail.mailet;


import com.alibaba.fastjson.JSONObject;
import com.jlszkxa.pie.mail.entity.ForwardMail;
import org.apache.mailet.GenericMailet;
import org.apache.mailet.Mail;
import org.apache.rocketmq.client.exception.MQBrokerException;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.client.producer.DefaultMQProducer;
import org.apache.rocketmq.common.message.Message;
import org.apache.rocketmq.remoting.exception.RemotingException;
import javax.mail.MessagingException;
import java.io.IOException;

/**
 * @author chenwj
 * @version 1.0
 * @className DivestitureAgreementMailet
 * @description Intercept the mail, split the protocol, and put it in the queue to wait for the Gate Ferry
 * @date 2020/2/20 14:53
 **/

public class DivestitureAgreementMailet extends GenericMailet {
    @Override
    public void service(Mail mail) throws MessagingException {
        String sender = mail.getSender().toString();
        String name = mail.getName();
        String subject = mail.getMessage().getSubject();
        String content = null;
        try {
            content = (String) mail.getMessage().getContent();
        } catch (IOException e) {
            e.printStackTrace();
        }
        System.out.printf("Intercept%s Mail name:%s subject:%s content:%s\r\n", sender, name, subject, content);
        System.out.println("Send intercepted mail to message queue...");
        DefaultMQProducer producer = new DefaultMQProducer("DivestitureAgreementGroup");
        producer.setNamesrvAddr("localhost:9876");
        producer.setInstanceName("rmq-instance");
        try {
            producer.start();
            System.out.println("Open message queue");
        } catch (MQClientException e) {
            e.printStackTrace();
        }
        try {
            ForwardMail forwardMail = ForwardMail.newBuilder()
                    .content(mail.getMessage().getContent())
                    .from(mail.getSender().getUser() + "@" + mail.getSender().getHost())
                    .hostName(mail.getRemoteHost())
                    .recipients(mail.getRecipients().iterator().next())
                    .subject(mail.getMessage().getSubject())
                    .build();
            Message message = new Message("demo-topic", "demo-tag", JSONObject.toJSONString(forwardMail).getBytes("UTF-8"));
            producer.send(message);
            System.out.println("Message successfully forwarded to message queue");
            System.out.printf("The forwarding content is as follows: %s\r\n", JSONObject.toJSONString(forwardMail));
        } catch (MQClientException e) {
            e.printStackTrace();
        } catch (RemotingException e) {
            e.printStackTrace();
        } catch (MQBrokerException e) {
            e.printStackTrace();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        } finally{
            producer.shutdown();
            System.out.println("Close message queuing");
        }
    }
}

The ForwardMail object is as follows:

package com.xxxxxx.pie.mail.entity;

/**
 * @ClassName Mail
 * @Description TODO
 * @Author chenwj
 * @Date 2020/2/21 13:42
 * @Version 1.0
 **/

public class ForwardMail {

    private String hostName;

    private String from;

    private String subject;

    private Object recipients;

    private Object content;

    public ForwardMail() {
    }

    private ForwardMail(Builder builder) {
        setHostName(builder.hostName);
        setFrom(builder.from);
        setSubject(builder.subject);
        setRecipients(builder.recipients);
        setContent(builder.content);
    }

    public static Builder newBuilder() {
        return new Builder();
    }


    public String getHostName() {
        return hostName;
    }

    public void setHostName(String hostName) {
        this.hostName = hostName;
    }

    public String getFrom() {
        return from;
    }

    public void setFrom(String from) {
        this.from = from;
    }

    public String getSubject() {
        return subject;
    }

    public void setSubject(String subject) {
        this.subject = subject;
    }

    public Object getRecipients() {
        return recipients;
    }

    public void setRecipients(Object recipients) {
        this.recipients = recipients;
    }

    public Object getContent() {
        return content;
    }

    public void setContent(Object content) {
        this.content = content;
    }

    public static final class Builder {
        private String hostName;
        private String from;
        private String subject;
        private Object recipients;
        private Object content;

        private Builder() {
        }

        public Builder hostName(String val) {
            hostName = val;
            return this;
        }

        public Builder from(String val) {
            from = val;
            return this;
        }

        public Builder subject(String val) {
            subject = val;
            return this;
        }

        public Builder recipients(Object val) {
            recipients = val;
            return this;
        }

        public Builder content(Object val) {
            content = val;
            return this;
        }

        public ForwardMail build() {
            return new ForwardMail(this);
        }
    }
}

Make the above-mentioned mailet program fragment into a jar package, paste and copy it to the James server.. \ james-2.3.2.1\apps\james\SAR-INF\lib. If there is no lib directory in the SAR-INF directory, add the directory manually. Add the following configuration in.. \ james-2.3.2.1\apps\james\SAR-INF\config.xml:

Restart the james server, and the mailet program fragment will take effect. Here is the forwarding of the consumer message.

Mail forwarder

In fact, I have been thinking about email forwarding for a long time. When the connection between the internal network and the external network cannot be established, the message should be forwarded intact, and the sender of the message still identifies the user of the internal network. This is what I expect to achieve, but I have not found a good solution. I tried to set the email from as an intranet user, or set the displayname as an intranet user, but it didn't seem to get a better result in the QQ mailbox. Therefore, at last, I take a shortcut to realize that in the external network, a single user forwards the message. When forwarding, I identify a user in the subject to forward the message from the internal network.

Consumer reference code:

package com.xxxxxx.pie.mail.mq;

import com.alibaba.fastjson.JSONObject;
import com.jlszkxa.pie.mail.entity.ForwardMail;
import com.jlszkxa.pie.mail.mail.MailSender;
import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyContext;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus;
import org.apache.rocketmq.client.consumer.listener.MessageListenerConcurrently;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.common.message.MessageExt;
import javax.mail.MessagingException;
import java.io.UnsupportedEncodingException;
import java.util.List;

public class Consumer {
    public static void main(String[] args) throws MQClientException {
        DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("my-group");

        consumer.setNamesrvAddr("localhost:9876");
        consumer.setInstanceName("rmq-instance");
        consumer.subscribe("demo-topic", "demo-tag");

        consumer.registerMessageListener(new MessageListenerConcurrently() {
            public ConsumeConcurrentlyStatus consumeMessage(
                    List<MessageExt> msgs, ConsumeConcurrentlyContext context) {
                for (MessageExt msg : msgs) {
                    ForwardMail forwardMail = JSONObject.parseObject(new String(msg.getBody()), ForwardMail.class);
                    JSONObject jsonObject = JSONObject.parseObject(JSONObject.toJSONString(forwardMail.getRecipients()));
                    String recipient = jsonObject.getString("user") + "@" + jsonObject.getString("host");
                    try {
                        System.out.printf("Successful proxy forwarding%s Mail\r\n", recipient);
                        MailSender.sendHtml(forwardMail.getFrom(), "979831398@qq.com", "xxxxx", "smtp.qq.com", recipient, "Forwarded from proxy by" + forwardMail.getFrom().split("@")[0] + "Outgoing mail:" + forwardMail.getSubject(), forwardMail.getContent());
                    } catch (MessagingException e) {
                        e.printStackTrace();
                    } catch (UnsupportedEncodingException e) {
                        e.printStackTrace();
                    }
                }
                return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
            }
        });
        consumer.start();
        System.out.println("Consumer Started.");
    }
}

MailSender reference code:

package com.xxxxxx.pie.mail.mail;

import javax.mail.MessagingException;
import javax.mail.internet.AddressException;
import java.io.UnsupportedEncodingException;

/**
 * @author 
 *
 */
public class MailSender {
    /**
     * Service mailbox
     */
    private static MailServer mailServer = null;
 
	//
	private static String userName;
 
	
	private static String password;
 
	
	private static String stmp;
 
	/**
	 * @param userName the userName to set
	 */
	public void setUserName(String userName) {
		if(MailSender.userName==null)
			MailSender.userName = userName;
	}
	/**
	 * @param password the password to set
	 */
	public void setPassword(String password) {
		if(MailSender.password==null)
			MailSender.password = password;
	}
	/**
	 * @param stmp the stmp to set
	 */
	public void setStmp(String stmp) {
		if(MailSender.stmp==null)
			MailSender.stmp = stmp;
	}
    /**
     * Send mail with default user name and password
     * @param recipient
     * @param subject
     * @param content
     * @throws MessagingException 
     * @throws AddressException 
     */
    public static void sendHtml(String recipient, String subject, Object content, String fromname) throws AddressException, MessagingException, UnsupportedEncodingException {
    	if (mailServer == null) 
        	mailServer = new MailServer(stmp,userName,password);
    	mailServer.send(recipient, subject, content, fromname);
    }
    /**
     * Send mail with the specified user name and password
     * @param server
     * @param password
     * @param recipient
     * @param subject
     * @param content
     * @throws MessagingException 
     * @throws AddressException 
     */
    public static void sendHtml(String fromname, String server,String password,String stmpIp, String recipient, String subject, Object content) throws AddressException, MessagingException, UnsupportedEncodingException {
    	 new MailServer(stmpIp,server,password).send(recipient, subject, content, fromname);
    }
    public static void main(String[] args) {
    	try {
    		String s = "This is a test email from the company's intranet. Please do not reply when you receive it!";
			sendHtml(null, "test@xxxxx.com","test","localhost", "979831398@qq.com", "Test mail", s);
		} catch (AddressException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (MessagingException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (UnsupportedEncodingException e) {
			e.printStackTrace();
		}
	}
}

MailServer reference code:

package com.xxxxxx.pie.mail.mail;

import org.apache.commons.lang3.StringUtils;

import javax.mail.*;
import javax.mail.internet.AddressException;
import javax.mail.internet.InternetAddress;
import javax.mail.internet.MimeMessage;
import java.io.UnsupportedEncodingException;
import java.util.List;
import java.util.Properties;

/**
 * Simple mail sender, can send single, group.
 * 
 * @author humingfeng
 * 
 */
public class MailServer {
 
	/**
	 * props file for sending mail
	 */
	private final transient Properties props = System.getProperties();
	/**
	 * Mail server login authentication
	 */
	private transient MailAuthenticator authenticator;
 
	/**
	 * Mailbox session
	 */
	private transient Session session;
 
	/**
	 * Initialize mail sender
	 * 
	 * @param smtpHostName
	 *            SMTP Mail server address
	 * @param username
	 *            User name (address) to send mail
	 * @param password
	 *            Password to send mail
	 */
	public MailServer(final String smtpHostName, final String username,
			final String password) {
		init(username, password, smtpHostName);
	}
 
	/**
	 * Initialize mail sender
	 * 
	 * @param username
	 *            Send the user name (address) of the mail and resolve the SMTP server address
	 * @param password
	 *            Password to send mail
	 */
	public MailServer(final String username, final String password) {
		// Resolve the smtp server through the mailbox address, which works for most mailboxes
		final String smtpHostName = "smtp." + username.split("@")[1];
		init(username, password, smtpHostName);
 
	}
 
	/**
	 * Initialization
	 * 
	 * @param username
	 *            User name (address) to send mail
	 * @param password
	 *            Password
	 * @param smtpHostName
	 *            SMTP Host address
	 */
	private void init(String username, String password, String smtpHostName) {
		// Initialize props
		props.put("mail.smtp.auth", "true");
		props.put("mail.smtp.host", smtpHostName);
		if(smtpHostName==null)props.put("mail.smtp.host", smtpHostName);
		// Verification
		authenticator = new MailAuthenticator(username, password);
		// Create session
		session = Session.getInstance(props, authenticator);
	}
 
	/**
	 * Send mail
	 * 
	 * @param recipient
	 *            Recipient email address
	 * @param subject
	 *            Mail theme
	 * @param content
	 *            Mail content
	 * @throws AddressException
	 * @throws MessagingException
	 */
	public void send(String recipient, String subject, Object content, String fromname)
			throws AddressException, MessagingException, UnsupportedEncodingException {
		// Create mime type mail
		final MimeMessage message = new MimeMessage(session);
		// Set sender
		if(StringUtils.isBlank(fromname)) {
			message.setFrom(new InternetAddress(authenticator.username, fromname));
		} else {
			message.setFrom(new InternetAddress(authenticator.username));
		}
		// Set up recipients
		if(recipient!=null&&recipient.indexOf(";")!=-1){
			//Multiple recipients
			String[] rec = recipient.split(";");
			int len = rec.length;
			InternetAddress[] iad = new InternetAddress[len];
			for(int i=0; i<len; i++){
				iad[i] =  new InternetAddress(rec[i]);
			}
			message.setRecipients(MimeMessage.RecipientType.TO, iad);
		}else{
			//Single recipient
			message.setRecipient(MimeMessage.RecipientType.TO, new InternetAddress(recipient));
		}
		// set up themes
		message.setSubject(subject);
		// Set message content
		message.setContent(content.toString(), "text/html;charset=utf-8");
//		message.setText(content.toString(), "GBK");
		// Send out
		Transport.send(message);
	}
 
	/**
	 * Mass mail
	 * 
	 * @param recipients
	 *            Recipient
	 * @param subject
	 *            theme
	 * @param content
	 *            content
	 * @throws AddressException
	 * @throws MessagingException
	 */
	public void send(List<String> recipients, String subject, Object content, String fromname)
			throws AddressException, MessagingException, UnsupportedEncodingException {
		// Create mime type mail
		final MimeMessage message = new MimeMessage(session);
		// Set sender
		if(StringUtils.isBlank(fromname)) {
			message.setFrom(new InternetAddress(authenticator.username, fromname));
		} else {
			message.setFrom(new InternetAddress(authenticator.username));
		}
		// Set recipients
		final int num = recipients.size();
		InternetAddress[] addresses = new InternetAddress[num];
		for (int i = 0; i < num; i++) {
			addresses[i] = new InternetAddress(recipients.get(i));
		}
		message.setRecipients(MimeMessage.RecipientType.TO, addresses);
		// set up themes
		message.setSubject(subject);
		// Set message content
		message.setContent(content.toString(), "text/html;charset=utf-8");
		// Send out
		Transport.send(message);
	}
 
	/**
	 * Server mailbox login authentication
	 * 
	 * @author MZULE
	 * 
	 */
	public class MailAuthenticator extends Authenticator {
 
		/**
		 * User name (login mailbox)
		 */
		private String username;
		/**
		 * Password
		 */
		private String password;
 
		/**
		 * Initialize mailbox and password
		 * 
		 * @param username
		 *            mailbox
		 * @param password
		 *            Password
		 */
		public MailAuthenticator(String username, String password) {
			this.username = username;
			this.password = password;
		}
 
		String getPassword() {
			return password;
		}
 
		@Override
		protected PasswordAuthentication getPasswordAuthentication() {
			return new PasswordAuthentication(username, password);
		}
 
		String getUsername() {
			return username;
		}
 
		public void setPassword(String password) {
			this.password = password;
		}
 
		public void setUsername(String username) {
			this.username = username;
		}
 
	}
}

Finally, start the james service and the Comsumer service, and the test mail forwarding results are as follows:

There is a problem in the way of forwarding to the Internet by a certain external user, repudiation. A user sent an email, but denied that he had never sent it. In this regard, I think it can be solved by digital signature. A user generates a public-private key pair when creating the intranet. By signing the content of the mail, the mail can be tampered and non repudiated.

Finally, all the above codes have been uploaded to github warehouse.

The above idea and implementation is just my exploration in this new field, which may be far from the actual landing plan, but it is also a weak attack on this new field. In the future, I hope to broaden my vision in the field of data exchange, meet more hardworking tycoons in this field, and guide my many confused directions.

Reference resources

Secure exchange of e-mail in physically isolated environment

If you are interested in my article, please pay attention to my official account.