I have a custom pythonoperator in an airflow instance that supposed to pull a list of records from bigquery that i can reference in a API look up.
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "/home/airflow/airflow/keys/the_JSON.json"
client = bigquery.Client()
geoclient = GeocodioClient('apikey')
bq_sql = "SELECT ATTOM_ID, ADDRESS, CITY, STATE, ZIP FROM `project.dataset.tbl` Where GEOCODE_DT < DATE_SUB(CURRENT_DATE(), INTERVAL 2 month) and ATTOM_ID IN (1, 2)"
query_job = client.query(bq_sql)
print(f'query job: {query_job}')
results = query_job.result() # Waits for job to complete.
ERROR - 400 Syntax error: Expected end of input but got identifier "Sedgwick" at [4:37]
"Sedgwick" is part of a value in a record.
When I attempt to run this code in the python interpreter my airflow server is at, it works fine. but in a dag pythonoperator, I get the error.
What can I do? thanks!
from PythonOperator fails where normal python code succeeds for google Bigquery
No comments:
Post a Comment