Preface
What is short URL? As the name implies, it is a relatively short web address in form. At present, it is mainly to replace the original lengthy web address with the help of short web address to facilitate transmission and sharing. Short web service is the service that transforms long web address into short web address, which not only facilitates the Internet users, but also brings some security risks.
The Tencent blade team has studied the security of short URLs and shared some results on kcon 2018. This paper is also an interpretation and supplement of the topic "attack and defense of short URLs".
Special thanks: lake2, Wester, martinzhou
1、 Short URL basis
The short URL service can provide a very short URL to replace the original possibly longer URL and shorten the long URL address. When users access the shortened URL, they will usually be redirected to the original URL. Short URL service mainly originated from some micro blog services with word limit, but now it is widely used in SMS, email and so on.
Many security issues are related to security scenarios. With the constant changes of scenarios, security issues are also changing. The original intention of short web address is to use it on the public platform of microblog, which limits the number of words. That is to say, it is basically public, but in the following personal SMS and email, some of them are already private. This directly leads to the first relatively large potential risk of a short URL.
Before we understand the risks and vulnerabilities of short URLs, we should first understand what short URLs are and how they work.
The basic process of short URL service: users submit the long URL to the short URL service. After the short URL service is processed by URL, the long URL is converted by using the conversion algorithm. Finally, the long URL and the short URL are stored in the database respectively. Some short URL services will visit the long URL in order to prevent continuous conversion of short address or provide some functions to display the long URL title.
In fact, the core of short URL service is the conversion algorithm of short URL. So what are the commonly used short URL algorithms? We analyze the algorithms corresponding to ten short URL services with the largest number of stars on GitHub, which can be roughly divided into three categories: base algorithm, random number algorithm and hash algorithm.
Next, I use three simple examples to introduce the corresponding algorithm:
(1) Base algorithm:
Algorithm description: an algorithm with 62 characters in numbers and upper and lower case letters.
The ID in the database is increasing. When the ID is 233, the calculation process of the corresponding short URL is as follows:
① Set the sequence to:
“0123456789abcdefghijklmnopqrstuvwxyz”
(2) 233/36=6
③ 233%36= 17
④ Take 6 bits of the above characters in sequence, 17 bits, then 6H
The generated short URL is xx.xx/6h
(2) Random number algorithm:
Algorithm description: select random digits of candidate characters every time, and check whether they are repeated after splicing
If the required number of digits is 2, the corresponding short address is calculated as follows:
① Set character sequence:
“0123456789abcdefghijklmnopqrstuvwxyz”
② According to the number of characters, set the maximum value to 35 and the minimum value to 0, and take the random number of 2 times as 6,17
③ Take 6 and 17 bits of the above characters in sequence, then 6H
The generated short URL is xx.xx/6h
(3) Hash algorithm:
Algorithm description: hash ID (optional: add salt with random number), and check whether it is repeated
Set ID auto increment. If id = 233, the corresponding short address is calculated as follows:
① Take random number as salt
② SHA1 encryption of 233 is as follows:
aaccb8bb2b4c442a7c16a9b209c9ff448c6c5f35:2
③ It is required that the number of bits is 7, and the first 7 bits of the above encryption result are taken directly as aaccb8
The generated short website is xx.xx/2e8c027
After learning about the process of converting a long URL to a short URL, we will briefly explain the process of converting a short URL to a long URL. When users visit a short URL, the short URL service will return a response of 302 or 301, so as to jump to a long URL. This place, almost all short URL services will choose 302, because 302 is convenient for statistics and analysis of user attributes and other data.
2、 Short URL service risk
Short URL service has its own design defects, especially the general short URL using 6 or 7 letters and numbers, which can be well predicted and targeted.
The most important step in the blasting is how to detect the algorithm used by the current short URL, so as to generate the dictionary corresponding to the algorithm. Below we give some common algorithm detection processes:
1. Base algorithm
(1) Third party short URL service
For the short URL service of the third party, you can input the URL multiple times to check whether the returned short URL is continuous. If it is continuous, it is a binary algorithm, as follows:
In addition, due to the fact that some of them are distributed short URL services, the ID is not single increasing, and there will be regular changes of multiple characters, such as 87bnwj, 87bo82, 87boqw, 87bogz, 87bppd
(2) Self operated short website service:
The following two steps can be taken for the self operated short website service:
① If there are suffixes such as xx.xxx/1 and xx.xxx/2, it can be judged that the radix algorithm is used for conversion.
② Try to increase or decrease the suffix of existing records. If there are records or regular interval records, it is basically considered that the base algorithm is used.
That is, if there is http://xxx.xx/abzc4 in a short URL, 62 changes will be made to the last single character {0-z} in abzc4. If there are records or regular intervals such as a, C and E, it can also be considered that the base algorithm is used.
2. Hash algorithm & random number algorithm
(1) Third party short URL service:
For the third party, you can input the URL multiple times to check whether the returned short URL is continuous or not. For the discontinuous and irregular URL, hash algorithm & random number algorithm is used.
The following picture:
(2) Self operated short website service
① Directly access the suffixes such as xx.xxx/1 and xx.xxx/2. If none of them exist, proceed to step 2.
② We try to increase or decrease the suffixes of the existing records. If there are records with non-uniform intervals, we think that the base algorithm is used.
That is, if there is http://xxx.xx/abzc4 in a short URL, 62 changes will be made to the last single character {0-z} in abzc4. If there is no obvious law, it is considered as hash random number algorithm.
Next, let's share two attack scenarios of short URLs. The first one is that some short URLs use long URLs containing sensitive permissions and information in the transmission process, resulting in a large number of personal information leaks. The second one is that short URLs are predictable and explosive, and sometimes may produce some unexpected effects.
Case 1: blasting short website service to obtain a large number of services and system sensitive information:
1. Access to personal information:
http://xx.xx/auth?contractId=d57f17139247036b72******b5554a830305ec139d
2. Contract acquisition:
https://xx.xx/get.action?transaction_id=290414****03784&msg_digest=RUQ2MUQ5NjcxQzc5MjcxQ*******4QTExNTZFNjgzQTJENEExQjc5Nw==
3. Reset password:
https://xx.xx/resetPassword?emailType=RESET_PASSWORD&encryptionEmail=***GHOsR%2FMfiNEv8xOC29.&countersign=eyJhbGciOiJIUzUxMiJ9.eyJBQlNPTFVURV9FWFBJUkVfVElNRV9NSUxMUyI6IjE1M******zA1OTMxNjU4OTQiLCJORVdfRU1BSUwiOiJ5YW54aXUwNjE0QGdtYWlsLmNvbSIsIlRPS0VOX1RZUEUiOiJSRVNFVF9QQVNTV09SRCIsIkVNQUlMIjoieWFueGl1MDYxNEBnbWFpbC5jb20ifQ
In fact, it's very difficult to guess from the above exploded links, but it's because of the use of short URL that some very difficult to guess high-dimensional information is reduced to very easy to predict low-dimensional information, just like you build a very solid door, but others have a spare key.
Case 2: business security attack chain
1. The invitation link is directly sent to the inviting person, who can click to complete the registration;
2. The invitation link is sent as a short URL;
3. Batch invitation, blasting short web site, batch click registration, you can complete the collection of wool;
An app has a reward making activity in which old users invite new users. The invitation link is sent to new users in the form of a short URL. After new users click the link, the reward will be placed in the old user's account. In this activity, attacker user a can randomly select two mobile phone numbers. We use user B and user C to replace the two users respectively. Then attacker user a invites the two mobile phone numbers randomly selected, and then directly blasts the short URL for confirmation, and completes the collection of reward without the knowledge of B and C.
3、 Short URL Service Vulnerability
In fact, when the short URL appears to be solved by guessing and exploding, will there be other problems? So we have carried out other security tests on it.
1. SSRF safety issues
The remote access function will cause SSRF if the filtering is not rigorous. During the test, the user-defined domain name is used to bind an intranet address for access. The short URL service shows the title of the long URL, and the intranet address is successfully accessed as follows:
2. Get Title Function and show long URL page, cause XSS when filtering is not rigorous.
Some short URL services provide the display function of long URL title and the function of displaying long URL on the current page. XSS can also be caused when the filtering is not rigorous.
As shown in the figure above, CVE causes XSS problems in displaying long URL pages. While taking title and displaying it on the page at the same time will also cause XSS. For example, you can construct a payload: "< title > < script > alert (1) < script > < title >".
3. SQL injection problem
SQL injection will be caused during splicing query. In the test, we first tested and 1 = 1, and found that it can be read normally, as shown in the following figure:
After that, perform the database version joint injection, as shown in the following figure:
4、 Short website defense practice
For short web services, the following measures are recommended to improve security:
1. Increase the limit of the single IP access frequency and the total amount of single IP access, and block the access beyond the threshold.
2. Expire the short URL containing permission and sensitive information.
3. The second authentication is added to the long URL containing permission and sensitive information.
5、 Scope of influence
Adhering to the "responsible vulnerability disclosure process", the short URL security problems we found during the test process have been notified to the relevant manufacturers through the corresponding SRC, and the manufacturers have completed the repair. As follows:
Limited energy, failed to test one by one, please self test and repair.