target

• Realize the change from ordinary website to short website

  reference resources:
  Design short URL system

  How are short links designed? Get you started

  A short link service implemented by go language, including generating and parsing short links

  java request microblog short link API

  The principle and method of Url short connection

  Research on the principle of short link of web page

  How to convert a long URL to a short URL? Recommended reference

Business scenario

In general, in short message promotion, long links need to be converted into short links to reduce the length of short message characters and save the cost of short messages. Optimize the user experience for easy replication and dissemination. In general, it will push activity links, Download app links, query information links, etc. in SMS content.

Practice mode

After online inquiry and reference, there are roughly two ways.

The first way: call the short connection generation api provided by Baidu, Tencent and Weibo or the short connection service provided by the third party.

The second way: self realization in the system.

The first way

Baidu's short website can only be used by enterprise level users. For details, please refer to( http://dwz.cn/ ), but there is no anti blocking effect in Tencent app s.

Weibo stopped calling the official short URL api in 2019. For details, please refer to( https://open.weibo.com/wiki/2/short_ URL / shortcut), short connection: t.cn

Tencent has a long link to the short link interface in the official account of WeChat. https://developers.weixin.qq.com/doc/offiaccount/Account_ Management/URL_ Shortener.html ), short connection: W url.cn/s/

The second way

The specific implementation needs to see the business scenario.

Idea: when users visit short links, the background finds the corresponding long links according to the short links, and redirects (301302) to the corresponding URL of the long links.

It is recommended to refer to this article: How to convert a long URL to a short URL? See this article for details.

Or look directly: https://www.zhihu.com/question/29270034/answer/46446911

code implementation

Specific code reference: https://github.com/gengzi/codecopy.git/ Medium fun.gengzi.codecopy.business.shorturl Code below

Database: resources/db/shorturl.sql

jmeter test script: resources/shorturltest.jmx

To sum up, there are two interfaces: one is the interface from long link to short link, and the other is the interface from short link to corresponding long link.

public interface ShortUrlGeneratorService {
 
    /**
     * Back to short link
     *
     * @param longUrl Common links
     * @return Short link
     */
    String generatorShortUrl(String longUrl);
 
    /**
     * Back to long link
     * @param shortUrl Short link
     * @return Long link
     */
    String getLongUrl(String shortUrl);
}

Long link to short link interface:

/**
     * Back to short link
     * // Determine whether the current long connection can be found in redis, find the short link directly returned, and update the key value with an expiration time of 1 hour
     * // No, call the redis logical sender
     * // Return the number, as the primary key of the database, to detect whether the primary key conflicts. In case of conflict, try to get a new number again (you can also not verify whether the primary key conflicts, as long as you can ensure that the number sent by the sender is unique)
     * // Bind the long connection with the number, and convert the decimal number to 62
     * // Group short link, set timeout
     * // Store in database
     * // In the form of redis and key value, the key 62 base id corresponds to a long connection. If there are too many connections, you can set an expiration time (such as three days) to prevent too much cache in redis
     * // Save the form of reids and key value again, long connection, corresponding to 62 base of a short link, and set the expiration time to 1 hour. When the same long link comes again, it can be directly returned from redis
     * // return
     *
     * @param longUrl Common links
     * @return
     */
    @Transactional
    @Override
    public String generatorShortUrl(String longUrl) {
        logger.info("-- longurl to shorturl start --");
        logger.info("param longurl : {}", longUrl);
        // Judge that the current connection cannot be null or ""
        if (StringUtils.isNoneBlank(longUrl)) {
            boolean isExist = redisUtil.hasKey(longUrl);
            if (isExist) {
                String shortUrl = (String) redisUtil.get(longUrl);
                redisUtil.expire(longUrl, ShortUrlConstant.UPDATETIMEHOUR, TimeUnit.HOURS);
                logger.info("redis get shorturl success, return shorturl : {}", linkUrlPre + shortUrl);
                return linkUrlPre + shortUrl;
            } else {
                long number = redisUtil.getRedisSequence();
                String str62 = BaseConversionUtils.to62RadixString(number);
                String genShortUrl = linkUrlPre + str62;
 
                Shorturl shorturl = new Shorturl();
                shorturl.setId(number);
                shorturl.setLongurl(longUrl);
                shorturl.setShorturl(genShortUrl);
                shorturl.setIsoverdue(Integer.valueOf(ShortUrlConstant.ISOVERDUE));
                shorturl.setTermtime(new Date());
                shorturl.setCreatetime(new Date());
                shorturl.setUpdatetime(new Date());
 
                shortUrlGeneratorDao.save(shorturl);
                // Save the 62 base long link to the session. There is no need to save the short link because the prefix does not need to be cached
                redisUtil.set(str62, longUrl);
                redisUtil.set(longUrl, str62, 1, TimeUnit.HOURS);
                logger.info("insert shorturl success , return shorturl : {} ", genShortUrl);
                return genShortUrl;
            }
        }
 
        return "";
    }

Short link to long link:

controller:

    @ApiOperation(value = "Short link Jump Service", notes = "Short link Jump Service")
    @ApiImplicitParams({
            @ApiImplicitParam(name = "shorturl", value = "Short link", required = true)})
    @GetMapping("/u/{shorturl}")
    public String redirectUrl(@PathVariable("shorturl") String shorturl) {
        logger.info("redirectUrl start {} ", System.currentTimeMillis());
        String longUrl = shortUrlGeneratorService.getLongUrl(shorturl);
        return "redirect:" + longUrl;
    }

service:

/**
     * Back to long link
     * // Determine whether the current connection can be found in redis. If it is found, it will directly return to the long connection
     * // Convert 62 base to 10 base
     * // Judge whether the return long connection is none, then search in the database
     *
     * @param shortUrl Short link
     * @return Long link
     */
    @Override
    public String getLongUrl(String shortUrl) {
        String longUrl = (String) redisUtil.get(shortUrl);
        if (StringUtils.isNoneBlank(longUrl)) {
            return longUrl;
        }
        long shortUrlId = BaseConversionUtils.radixString(shortUrl);
        Shorturl shorturl = shortUrlGeneratorDao.getOne(shortUrlId);
        if (shorturl != null) {
            return shorturl.getLongurl();
        }
        return "";
    }

Function extension

Add cache layer and use nosql database to store long links and short links that are frequently converted, and set expiration time, and update time every time. (yes)

Add the verification of link timeliness. If it is judged to be overdue, it will jump to the prompt page and prompt that the activity has expired

Add interface authentication. Only authorized users can call the long link to short link service, record the number of calls, record the ip address, limit the number of calls, and prevent malicious users from swiping the interface

The short link redirection 302 records the behavior data, user's ip, used terminals, regions, etc., which are used to optimize the future business scenarios

Short link business differentiation, such as http: / / domain name / s/uA3x, a specific scenario identified by s

For the shared link, you can desensitize the information of the sharer, and add the short link to make some business statistics.

Distributed high availability

In the previous signer (that is, the strategy of generating database id), we used a single redis auto increment sequence signer and a segmented redis auto increment sequence signer. These are not suitable for distributed and high parallel distribution, so you can use distributed transceivers to generate IDs. You can refer to snowflake algorithm or open source global id generator.

For the storage of long links and short links, if the data volume is large, the query and update of a single table are slow. Separate reading and writing, separate database and table, or store in other ways

//todo, when I get here, I'll see what I need

other

• Conversion between 10 base and 62 base

  public class BaseConversionUtils {
  
      static final char[] DIGITS =
              {'0', '1', '2', '3', '4', '5', '6', '7', '8', '9',
                      'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j',
                      'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't',
                      'u', 'v', 'w', 'x', 'y', 'z', 'A', 'B', 'C', 'D',
                      'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N',
                      'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z'};
  
      // Convert to 62 base
      public static String to62RadixString(long seq) {
          StringBuilder sBuilder = new StringBuilder();
          while (true) {
              int remainder = (int) (seq % 62);
              sBuilder.append(DIGITS[remainder]);
              seq = seq / 62;
              if (seq == 0) {
                  break;
              }
          }
          return sBuilder.reverse().toString();
      }
  	// Convert to 10 base
      public static long radixString(String str) {
          long sum = 0L;
          int len = str.length();
          for (int i = 0; i < len; i++) {
              sum += indexDigits(str.charAt(len - i - 1)) * Math.pow((double) 62, (double) i);
  
          }
          return sum;
      }
  
      private static int indexDigits(char ch) {
          for (int i = 0; i < DIGITS.length; i++) {
              if (ch == DIGITS[i]) {
                  return i;
              }
          }
          return -1;
      }
  
  }