Friday 15 April 2011

python - BigQuery API:: Able to insert CSV data but data reflection in BigQuery is untimely -



python - BigQuery API:: Able to insert CSV data but data reflection in BigQuery is untimely -

using python script google app engine upload csv data bigquery. coded using pydev perspective of eclipse on windows 7.

the insert successful within bigquery info gets inserted , takes hours reflect.

j={ 'kind': 'bigquery#insertrequest', 'jobreference': {'projectid': '#######'}, 'configuration': { 'load': { 'sourceformat': 'csv', 'destinationtable': {'projectid': '############', 'tableid': '###########', 'datasetid': '##########'}, 'allowjaggedrows': true, 'sourceuris': ['gs://bucket_naem/file_name'], 'skipleadingrows': 1, 'schema': {'fields': [ {'type':'data_type','name':'col1_name'}, {'type':'data_type','name':'col2_name'} ] }, }, }, } response = service.jobs().insert(projectid = "##########",body = j).execute()

a bigquery load job asynchronous, why returns immediately. after insert() phone call returns, give job id. can utilize job id state of job. 1 time job completes successfully, info should available.

if import takes hours, unexpected (unless importing massive amount of data); if case, please provide job id , (bigquery engineers) can happened in logs.

python google-app-engine csv google-bigquery

No comments:

Post a Comment