Reading multiple json files from Spark -


i have list of json files load in parallel.

i can't use read.json("*") cause files not in same folder , there no specific pattern can implement.

i've tried sc.parallelize(filelist).select(hivecontext.read.json) hive context, expected, doesn't exists in executor.

any ideas?

looks found solution:

val text sc.textfile("file1,file2....") val df = sqlcontext.read.json(text) 

Comments

Popular posts from this blog

Load Balancing in Bluemix using custom domain and DNS SRV records -

oracle - pls-00402 alias required in select list of cursor to avoid duplicate column names -

python - Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>] error -