Woo-yeong Jeong
2023-09-26A data engineer at LINE.
Hello. I'm Woo-yeong Jeong, a data engineer from the LINE Plus Messaging Data Eng Dev (NP) team. Recently, I've been working on a task to migrate HiveQL (Hive SQL) jobs to Spark SQL. I primarily used the SQL statement INSERT OVERWRITE TABLE to load data. However, depending on the Spark settings, several issues occurred that did not happen in Hive. In this article, I will introduce what I learned in the process of finding the cause and solution.
This article will first introduce the backgrou
