python - Detect header delimiters importing a csv file using pandas read_csv -

-

here go again.

hi, i'm trying detect error in csv file.

the file should follows

    goodfile.csv      "col_a","col_b","col_c","col_d"     "row1cola","row1colb","row1colc","row1cold"     "row2cola","row2colb","row2colc","row2cold"     "row3cola","row3colb","row3colc","row3cold"     "row4cola","row4colb","row4colc","row4cold"     "row5cola","row5colb","row5colc","row5cold"     "row6cola","row6colb","row6colc","row6cold"     "row7cola","row7colb","row7colc","row7cold"

but file have actually

    brokenfile.csv      "col_a","col_b",col c,"col_d"     "row1cola","row1colb","row1colc","row1cold"     "row2cola","row2colb","row2colc","row2cold"     "row3cola","row3colb","row3colc","row3cold"     "row4cola","row4colb","row4colc","row4cold"     "row5cola","row5colb","row5colc","row5cold"     "row6cola","row6colb","row6colc","row6cold"     "row7cola","row7colb","row7colc","row7cold"

when import 2 files pandas

    data = pd.read_csv('goodfile.csv')     data = pd.read_csv('brokenfile.csv')

i same result

    data            col_a     col_b     col_c     col_d     0  row1cola  row1colb  row1colc  row1cold     1  row2cola  row2colb  row2colc  row2cold     2  row3cola  row3colb  row3colc  row3cold     3  row4cola  row4colb  row4colc  row4cold     4  row5cola  row5colb  row5colc  row5cold     5  row6cola  row6colb  row6colc  row6cold     6  row7cola  row7colb  row7colc  row7cold

anyway, want detect error in second file "brokenfile.csv" lacks "" between header col_c

i think can detect missing " in columns of dataframe str.contains , boolean indexing inverted boolean array ~:

import pandas pd import io  temp=u'''"col_a","col_b",col c,"col_d" "row1cola","row1colb","row1colc","row1cold" "row2cola","row2colb","row2colc","row2cold" "row3cola","row3colb","row3colc","row3cold" "row4cola","row4colb","row4colc","row4cold" "row5cola","row5colb","row5colc","row5cold" "row6cola","row6colb","row6colc","row6cold" "row7cola","row7colb","row7colc","row7cold"''' #after testing replace io.stringio(temp) filename df = pd.read_csv(io.stringio(temp), quoting = 3) print df       "col_a"     "col_b"       col c     "col_d" 0  "row1cola"  "row1colb"  "row1colc"  "row1cold" 1  "row2cola"  "row2colb"  "row2colc"  "row2cold" 2  "row3cola"  "row3colb"  "row3colc"  "row3cold" 3  "row4cola"  "row4colb"  "row4colc"  "row4cold" 4  "row5cola"  "row5colb"  "row5colc"  "row5cold" 5  "row6cola"  "row6colb"  "row6colc"  "row6cold" 6  "row7cola"  "row7colb"  "row7colc"  "row7cold"  print df.columns index([u'"col_a"', u'"col_b"', u'col c', u'"col_d"'], dtype='object')  print df.columns.str.contains('"') [ true  true false  true]  print ~df.columns.str.contains('"') [false false  true false]  print df.columns[~df.columns.str.contains('"')] index([u'col c'], dtype='object')

Comments

Post a Comment

Popular posts from this blog

Load Balancing in Bluemix using custom domain and DNS SRV records -

-
i'm trying load balancing of application in bluemix using custom domain , srv record in dns manager. created own doamin ccbluemix.dynu.com , added custom domain bluemix. created application in region , created route custom domain myapp.ccbluemix.dynu.com deployed same application in uk region , created route custom domain myapp.ccbluemix.dynu.com there well. now whenever user hits route want load balancing between , uk region alternate requests served , uk regions. for created srv records like _http._tcp.ccbluemix.dynu.com srv eu-gb.mybluemix.net [priority: 0] [weight: 50] 10 _http._tcp.ccbluemix.dynu.com srv mybluemix.net [priority: 0] [weight: 50] 10 but nothing happening , i'm not able access application using custom route @ all. if create 'a' record cc-load-bal-app.ccbluemix.dynu.com 75.126.81.68 i can access app using custom route ends hitting server. how can achieve load balancing using srv records? wrong creating srv records here? i read ne...
Read more

oracle - pls-00402 alias required in select list of cursor to avoid duplicate column names -

-
my cursor : select xzis.msisdn msisdn, xzis.in_account_number in_account_number xxobw_zte_in_sync xzis not exists (select 1 hz_contact_points hcp, hz_cust_accounts hca hcp.owner_table_id = hca.party_id , hcp.owner_table_name = 'hz_parties' , hcp.phone_number = substr (xzis.msisdn, -8) , nvl (hca.attribute12, 'nil') = xzis.in_account_number); i have given alias columns, still facing issue. suggestions? it's cursor. 1 has no such problems. problem is, when you're joining 2 tables, or apply wrong aliases in subselect, cursor may @ end return 2 columns same name/alias. the problem not need alias, no 2 columns can have same alias (the name counting alias, when no alias given). apparently has happened, far can tell, it's ...
Read more

python - Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>] error -

-
Image
i getting same error when python manage.py runserver. there no such error when have ubuntu 15.10. got started when upgraded ubuntu 16.04. question might duplicate have tried solution provided question have applied command dpkg --configure -a, have done apt-get update,upgrade,dist-upgrade, clean, -f install. have reinstalled python2.7 , python3 no success though. here screenshot while trying run django application i got after re-installing python please me? bashrc file # ~/.bashrc: executed bash(1) non-login shells. # see /usr/share/doc/bash/examples/startup-files (in package bash-doc) # examples # if not running interactively, don't case $- in *i*) ;; *) return;; esac export pythonhome="/usr/bin/python" echo $virtualenvwrapper_python echo 'tushant' # don't put duplicate lines or lines starting space in history. # see bash(1) more options histcontrol=ignoreboth # append history file, don't overwrite shopt -s histappend # setting h...
Read more