Reading multiple json files from Spark -

-

i have list of json files load in parallel.

i can't use read.json("*") cause files not in same folder , there no specific pattern can implement.

i've tried sc.parallelize(filelist).select(hivecontext.read.json) hive context, expected, doesn't exists in executor.

any ideas?

looks found solution:

val text sc.textfile("file1,file2....") val df = sqlcontext.read.json(text)

Comments

Post a Comment

Popular posts from this blog

python - Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>] error -

-
Image
i getting same error when python manage.py runserver. there no such error when have ubuntu 15.10. got started when upgraded ubuntu 16.04. question might duplicate have tried solution provided question have applied command dpkg --configure -a, have done apt-get update,upgrade,dist-upgrade, clean, -f install. have reinstalled python2.7 , python3 no success though. here screenshot while trying run django application i got after re-installing python please me? bashrc file # ~/.bashrc: executed bash(1) non-login shells. # see /usr/share/doc/bash/examples/startup-files (in package bash-doc) # examples # if not running interactively, don't case $- in *i*) ;; *) return;; esac export pythonhome="/usr/bin/python" echo $virtualenvwrapper_python echo 'tushant' # don't put duplicate lines or lines starting space in history. # see bash(1) more options histcontrol=ignoreboth # append history file, don't overwrite shopt -s histappend # setting h...
Read more

php - Prepared Statement Inner Join Cannot Pass Parameter by Reference -

-
i've been trying figure out while , i've searched similar problems, can't find solution. i'm working on game setup application , i'm having issue prepared statement. if ($stmt = $connection->prepare("select pack.id, count(race.name) pack inner join race on pack.id = race.pack pack.id = ? group race.pack")) { $stmt->bind_param("s", $_session["id"]); $stmt->execute(); $stmt->bind_result($rsltid, rsltraces); while ($stmt->fetch()) { $racespack[$rsltid] = $rsltraces; } $stmt->close(); } this returns error: "fatal error: cannot pass parameter 2 reference". any help/advice appreciated! your $stmt->bind_result($rsltid, rsltraces); wrong. should $stmt->bind_result($rsltid, $rsltraces);
Read more

c# - Value cannot be null or empty.\r\nParameter name: name -

-
Image
when go cshtml page error occur: an exception of type 'system.argumentexception' occurred in system.web.mvc.dll not handled in user code in cshtml code : <div class="form-group"> @html.label("model", new { @class = "control-label col-md-4" }) <div class="col-md-8"> @html.editor("model") </div> </div> but on other hand if change code in @html.editor("propertymodel") it working no error occur . update: in model: public class propertyassign { public int propertyassignid { get; set; } public string propertyname { get; set; } public string model { get; set; } public datetime assigndate { get; set; } } in controller: public actionresult create() { return view(); } the reason exception thrown htmlhelpers inspect viewdata values bind to. viewdata typeof viewdatadictionary contains property mod...
Read more