Preparing the Bug Bounty Program
Hello. I am MJ, a LINE security engineer.
In this post I would like to share the results of the 2015 LINE Bug Bounty (August 24-September 23). As our first ever bug bounty program, we are very pleased with how well the program came together.
The core objective of the program was to discover and fix potential vulnerabilities in our services so that our users can enjoy them safely. Several departments lent us a hand in preparing the program and we could not have done it without them.
What we needed the most during preparation was support from management and enough members to prepare the program. From what I have heard, most companies have difficulty securing a budget to have their own bug bounty programs. Luckily, management gave us their blessing through a large budget expenditure. It was fortunate that they understood the value of long-term stability over the possible losses that could result in the event of a security incident. The prize money for each vulnerability was calculated by their severity, so that participants who found the more critical vulnerabilities would receive higher rewards. The maximum amount that could be received was $20,000 USD.
Also, the development and customer service departments worked closely together to successfully run the program. The development department helped create the LINE Bug Bounty website where participants would submit their entries and the customer service department was in charge of handling any questions regarding the program.
Without all the cooperation from the various departments, the program would not have been a success.
Submissions
First, here are the details about the submissions we received from LINE Bug Bounty.
Submissions by date
The number of submissions during the 1-month period between August 24 and September 23 was 194. The first day of the program was the day with the most submissions. We suspect this is because we announced the website in advance on August 5 giving the participants time to research. The number of submissions gradually decreased over time but there was a slight increase at the end.
Figure 1. Submissions by date and category
Participants by country
Of the 194 submissions, 89 were sent from Japan. There were ten participants who sent in more than five submissions, with one participant sending in 16. The figure below shows which countries accessed the website the most. Several countries including Japan showed great interest in the program.
Figure 2. Access by country
Evaluation and Results
Evaluation process
Submissions were put through preliminary and final evaluation processes.
The preliminary evaluation process involved judging reports in terms of relevance, detail, and whether it was reported in English or Japanese. "I have updated LINE and now the app will not launch" for example does not count as a vulnerability. Screenshots of supposed vulnerabilities without an explanation were also counted out.
The final evaluation process involved judging the submissions that passed the preliminary process. We checked the submissions to see if they were within the boundaries of LINE services and domains and if the reported exploit codes were replicable. Submissions that met the criteria were approved as vulnerabilities.
There were five judges in total. The members were selected based on their expertise in application security, code security, web security. Several of the judges also had experience participating in other bug bounties. These five judges scrutinized the submissions through a thorough screening process to determine if they met all the criteria of the LINE Bug Bounty.
Submissions that passed the final evaluation will be approved as vulnerabilities and rewarded. The names of the submitters and vulnerabilities were registered on the Hall of Fame.
Results
14 submissions including exploits using XSS, CSRF were approved as vulnerabilities. 10 participants who had submissions that did not meet the evaluation criteria but still managed to provide valuable information to LINE will be named and rewarded as Special Contributors. All of the results can be seen in the Hall of Fame website.
Participant Interview
We interviewed a participant for opinions on the LINE Bug Bounty.
Q: What made you participate in the program?
A: Finding vulnerabilities on mobile apps is a hobby of mine. I decided to participate as soon as I saw the news of a bug bounty program hosted by LINE because bug bounty programs for messaging apps are not very common.
Q: Any trouble during analysis?
A: I needed various ways to analyze the app since I had to work without the source code. LINE proved to be difficult to debug compared to other apps.
Q: Any thoughts on the bug bounty program?
A: Finding previously unknown messaging app attack vectors, bugs, and exploits was a fun experience. Watching my own attack codes work was very fulfilling. In my experience, informing companies of their vulnerabilities is something you rarely get thanked for. In fact, most companies would give you negative feedback and take a long time to patch the vulnerabilities. I think it says a lot that LINE is hosting their own bug bounty. It shows that LINE is very much interested in and willing to strengthen their security. The staff was also very helpful when I asked about the steps required for submitting a vulnerability.
Q: What were some things that needed improvement?
A: Nothing in particular, but I wish we could also submit web vulnerabilities alongside app vulnerabilities next time.
Future Bug Bounties
Thanks to the support you have shown us during the LINE Bug Bounty, we were able to find many vulnerabilities. While the bug bounty program was only open for a month, we learned how effective it can be in such a short timespan. There were also lessons learned on how we should run the next bug bounty program. Some have complained that the submission period was not long enough, the evaluation process took too long, or that they were unsatisfied with the results in general. For those that had their submissions turned down, we can understand why they may be upset. However, as stated in the rules, we had to rule out any vulnerabilities that we were already aware of or were reported by other participants first.
The reason we set such a short submission period was because we anticipated a lot of trial and error as this was our first attempt. All of these issues will be improved in the future so that we can have a better program. Please look forward to the next LINE Bug Bounty!
Although this year's LINE Bug Bounty has already come to an end, you can still send information on any LINE security vulnerabilities to dl_bugbounty@linecorp.com. You can also check the LINE Bug Bounty website for more information.