Reading multiple json files from Spark -


i have list of json files load in parallel.

i can't use read.json("*") cause files not in same folder , there no specific pattern can implement.

i've tried sc.parallelize(filelist).select(hivecontext.read.json) hive context, expected, doesn't exists in executor.

any ideas?

looks found solution:

val text sc.textfile("file1,file2....") val df = sqlcontext.read.json(text) 

Comments

Popular posts from this blog

php - Webix Data Loading from Laravel Link -

libGdx unable to find files in android application data directory -

python - Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>] error -