Blog - LINE ENGINEERING

TAG ARCHIVES: Spark

Troubleshooting Data Engineering Software
Keiji Yoshida2019-03-26
I am a data engineer at LINE Data Labs.
Hello, I'm Keiji Yoshida. I currently work as a data engineer at LINE Data Labs. In this posting, I'd like to share a few cases on data engineering software troubleshooting: Apache Hadoop YARN ResourceManager failover Apache Hadoop HDFS NameNode failover Apache Zeppelin Notebook scheduler issue Apache Zeppelin deadlock issue Apache Spark SQL performance issue Troubleshooting failover issue in Apache Hadoop YARN ResourceManager Let's start with the failover issue in
- Hadoop
- Spark
- Zeppelin
Analyzing Large Amounts of Security Data With Spark, Mesos, Zeppelin, and HDFS
OH WANG JIN2015-06-02
He is a data engineer for the LINE Ads Platform. His main development areas are games, security and search and is currently working on the big data technology.
Hello, we are WJ and KH and we are in charge of game security development at LINE. A massive number of players play LINE Games at any given time, and that is why quick issue analysis and response can be difficult. With the large increase in LINE Game players and users from abroad, many instances of abuse (profiting through unauthorized actions) are being observed. Abusing the system affects other players that are playing the game by the rules, while also directly affecting the game itself. Respo
- Security
- Spark
- Zeppelin

1