In this article, we’re going to discuss how LINE goes about measuring and improving the level of quality offered by our products from a user perspective, as we continue to expand globally. It does bear mentioning that the processes outlined in this article only cover a portion of what actually goes on here. Every one of our developers, designers, and business representatives all contribute to the process in their own unique way.
The LINE Fellowship
Since beginning service in Japan during June of 2011, LINE has exploded into a messenger app with over 400 million users worldwide. We are experiencing a rate of growth faster than any other messenger service out there, and that has brought along with it a large number of quality issues related to our user experience that we have had to face full on. In the same way that people who leave their hometown to move to a new country must change and adapt their way of talking and thinking in order to live, LINE was forced to shed its familiar Japanese skin and modify countless facets of its services in order to operate and thrive in other countries. This is exactly the part of the process we want to share with you today.
When WhatsApp was rumored to charge for its services back in December 2012, it triggered a mass exodus in which hordes of the app’s users in Spain abandoned the app in outrage and started using LINE instead. The number of LINE users in Spain exploded almost overnight. Unfortunately, during this period LINE users would sometimes experience temporary delays in message delivery, push notifications would arrive late, and the app would use up too much battery power on the user’s device. In order to fix these problems, a team of three of our developers made an emergency trip to Spain, got their hands on some local SIM cards, and worked as fast as they could analyze the problem. The LINE developers back at home in Japan used the data from this analysis report to make significant improvements targeted at users on the other side of the world, and the changes went live in April 2013. This was only the beginning of many trips made to improve the quality of LINE.
Through this trip to Spain, LINE learned the incomparable value of experiencing the problems local users face first hand and working to find ways to fix issues at the source of the problem. Since then, LINE developers have travelled directly to Southeast Asia, Central and South America, Europe, Russia and any other place where our users are having trouble in order to understand the problem from the local user’s perspective. Sometimes this means sending developers to countries where no other LINE employees have ever been before. These brave engineers struggle with unfamiliar languages to purchase SIM cards for local carriers, and then test the devices on streets, inside hotels, on subway trains, and inside restaurants buried deep beneath department stores. Due in part to parallels between what the members of this expeditionary force do and what Frodo’s band of warriors did in the “The Lord of the Rings” trilogy, somewhere along the way, this fearless band of developers came to be known as the “LINE Fellowship”.
Next, I would like to answer a few questions about the LINE Fellowship.
- Who is involved in the LINE Fellowship?
- The group is made up of two to six developers from the client app, server application and infrastructure sections of our development team.
- How long are these excursions?
- This varies according to how big the country is, how many major cities it has, and what the purpose of the test is, but the team usually spends anywhere from 3 to 5 days conducting tests locally.
- What kind of things do you test for?
- We look for issues unique to the country’s network system and check the quality of internet service available there. When we’re about to release a new feature, we check it in the actual environment it is going to be used in order to make sure there are no problems or if there are any ways the feature can be improved.
- Where do you conduct your tests during your trip?
-We test every version of the LINE app we can get our hands on during the trip. In order to get good comparative data, we mostly test in stable network environments like hotel lobbies. We also test the new features on the go on buses and subways, at restaurants, inside shopping malls, and around popular tourist destinations in order to see how the app reacts in real life situations that local users experience every day.
- Do you ask locals for their opinions?
- Yes. When we are using the local transportation system to move around, we gather data indirectly by paying careful attention to which messenger apps are being used. One time in Malaysia, we got a taxi driver to tell us all about “Zello” he was using, and some girls in a coffee shop in Turkey told us what they thought of our stickers. When we made a trip to Argentina and Columbia, we booked appointments in advance with local IT engineers in order to ask them about their country’s internet infrastructure and their opinions on LINE. Asking local users for their insight is an important part of what the LINE Fellowship does.
- Is there a particular trip that really stands out to you?
- The trip we took testing in-flight internet on airlines around the world was pretty interesting. We conducted a series of tests on a number of airlines around the world in order to see how well LINE worked with the various in-flight Wi-Fi networks that have become surprisingly prevalent in recent years. The LINE Fellowship flew on Lufthansa, JAL, AeroFlot, and United Air in order to ensure that LINE users can use every one of the services offered by LINE on each airline’s Wi-Fi network. We also made improvements to our services based on our findings. Next time you’re on a plane, make sure you fire up LINE and give it a try on the flight!
As a result of one year of LINE Fellowship trips around the world, we set up Point of Presence (PoP) in many countries around the world in order to get closer to our users. We also created a system that analyzes user feedback from the client app, and now run a daily analysis using our quantified Performance Index (PI). The LINE Fellowship takes action when the need arises. One might even call these trips a sporadic type of feature improvement. However, these periodic trips have led to a systemic process of daily analysis dedicated to making our services better.
The Global Infrastructure of LINE
LINE is based a mobile client-server application based on TCP/IP which is standard for most messenger apps. The most important facet of this system in regards to performance is network latency, or in other words, the distance between the user and the server. In order to keep this distance as short as possible, LINE used the data gathered from the LINE Fellowship tests to establish and operate global PoP on every continent with a high concentration of LINE users in 2013. Since then, every LINE user is automatically connected to the nearest PoP, and the client app itself was also modified to take advantage of the improvements offered by these additional PoP locations. LINE currently runs a network of 7 PoP of various sizes strategically positioned around the world. Our service now boasts a global network of dedicated lines with redundant servers in order to ensure the safety of the data being sent back and forth on our systems.
This does not mean that all the services offered by LINE are being handled by our own internet data centers (IDC). A good example is the way we handle data transfers for which users are potentially painfully aware of the performance factor, such as downloads of large files, various event pages, and sticker downloads. For these types of transfers, we make use of content delivery networks (CDN) in order to get a little closer to the user if at all possible. Cloud infrastructure such as CDN has played an extremely effective role in improving app performance for our users located in countries far away from our PoPs.
In order to accommodate for variations across different internet systems, we implemented new functionality on our servers that allows us to deploy different configuration settings in order to meet the needs of each system, much like how people use different versions of the LINE app. For example, to ensure that no problems arise when using a single protocol for messages exchange, our system allows for negotiations using a variety of different protocols. In order to solve a problem we were experiencing on some networks where file download speeds were slowing down, we modified our system so that LINE can use many different download methods, allowing the app to choose the most effective method based on the user’s network. Our development team is constantly thinking of ways we can get the LINE app to automatically adapt and connect to each user’s network without having the user configure any settings themselves.
From setting up a global PoP network to taking advantage of optional cloud infrastructure in order to enhance app performance, LINE has been able to significantly improve the quality of the user experience available to our customers around the world by modifying both the client app and our server system.
The LINE Fellowship has journeyed to many far-off lands, but in order to find problems users are experiencing in countries we have not been to directly, LINE started collecting user review data concerning the LINE app for all available platforms. When the LINE Fellowship plans to visit another country, the team need to decide whether the problem is unique to the country before they set off to conduct testing. If the problem does indeed appear to be a local issue, a thorough analysis of the situation is conducted in order to determine for absolute certainty that it is not just a general issue common to other places as well. The App Review system provides data analysis based on country, language, version, keywords, the number of app reviews, as well as score, and allows for keyword-based searching and even includes an automatic translation tool.
The most unique part of our system is that not only do we analyze the data for the apps released by LINE, but we also run the same analysis on apps released by our competitors. If one of our rivals offers a good feature that LINE doesn’t have, we often run a series of comparisons in order to create features that actually meet our users’ needs. Another positive aspect of this system is that if an issue arises in a particular region, this data proves absolutely invaluable in determining if the problem is a LINE issue or the fault of the local network environment. This app review data is analyzed weekly by the Service Operation Team, who then delivers the results to the developers to be improved in the next release of LINE app.
The most interesting feature we use may be the “Tag Cloud”. Even if a version update spends a long time in the development phase and goes through the most stringent quality assurance (QA) testing we can muster, it is impossible to catch all the issues that will arise for users around the world once the app has been released globally. However, within five to six hours after the new update has been released, keywords repeatedly found in the data collected from the app review system are displayed in the Tag Cloud, making it easy to find if any problems have popped up.
LINE Performance Index
The process of looking at the numbers when making decisions is ingrained in the corporate culture of LINE. The quality of LINE products and services, methods of improvement, and the process of checking the results after an update is made are all measured using “the numbers”, and even further improvements are implemented after that. When there are several test cases, we create internal test codes used for selecting and confirming the quantifiable results of the improvements that have been made. This process is repeated as many times as is necessary.
The effort made to reduce the amount of battery power used by LINE Android version is an excellent example of this, and the improvement process itself was conducted continuously throughout all of 2013. The LINE app contains far more features than the messenger apps of its competitors. Unfortunately, our app was also receiving criticism for using up too much battery power on Android devices, a fault due to insufficient optimization efforts stemming from the explosive growth of LINE. The development team had a valid excuse – it was extremely difficult to implement both the instant push notification required for LINE Free Call and lower battery consumption at the same time. In order to solve both problems at once, an internal improvement process spanning all of last year was conducted, resulting in three major updates. Now that we are nearing completion of another important improvement, a fourth update dealing with this issue will soon be released. After the features currently being improved by our development team are released on our end-of-April target date, we will check the user feedback on this latest update and use it to make even further improvements for the next version of the app.
The story of how we went about improving the battery usage rate for our app is just one of over 100 different issue indicators currently being tracked and managed by LINE. For a long time, we dreamed of developing our own system similar to Google Analytics where every part of our user experience related to performance would be turned into quantifiable numbers. At the same time, we knew we couldn’t just run our LINE user indicators through Google Analytics and analyze them that way. We put our heads together and at long last came up with a plan. It took a long time to get everything ready, but we finally did it – and that’s how LINE Analytics was born!
LINE Analytics uses Hadoop for analyzing big data. It’s been almost a month since we first started operating the new LINE Analytics system. We are already rendering a lot of number data, but we are working to convert even more app info into quantifiable numbers. We are constantly searching for features that should be improved in our next version update by looking at these numbers.
Numbers are making LINE better.
In this blog article, we discussed how LINE goes about measuring the quality of its products and services. We also went into detail on how we make them better, and the process used to check these indicators of improvement. At first, we were a bit concerned about making an article like this open to the public when our competitors have never published something like this. However, the quality of the user experience is an integral part of LINE, and we wanted you to know just how hard we work to improve our services. We focus more developers, financial assets, and time resources on this vital step than any other.
There are still many features of LINE that call for improvement. Our developers actually enjoy searching out these issues and then tweaking them to make them better. There may be some issues with LINE that we haven’t even realized yet. If you’re reading this today and you happen to run across a problem, please take a few minutes to enter the details into your app review along with the #LINEXP tag. We’d love to fix the problem.