Using Python to run BigQuery job with project id
source link: https://donghao.org/2022/11/18/using-python-to-run-bigquery-job-with-project-id/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Using Python to run BigQuery job with project id
Here is the code for me to query a table of BigQuery:
from google.cloud import bigquery
from google.cloud.bigquery_storage import BigQueryReadClient
client = bigquery.Client()
storage_client = BigQueryReadClient()
df = client.query("select * from my_table1").to_dataframe(bqstorage_client=storage_client)from google.cloud import bigquery
from google.cloud.bigquery_storage import BigQueryReadClient
client = bigquery.Client()
storage_client = BigQueryReadClient()
df = client.query("select * from my_table1").to_dataframe(bqstorage_client=storage_client)Then it reported the error:
“Access Denied: Project PRJ_B: User does not have bigquery.jobs.create permission in project PRJ_B.”
But actually, I want to launch a job in project PRJ_A. So I add a shell command “gcloud config set project PRJ_A” before running this python script. But the errors continued.
After searching the API doc of Python BigQuery, I found out that the “bigquery.Client()” function could add an argument:
client = bigquery.Client(project="PRJ_A")
client = bigquery.Client(project="PRJ_A")
Now the script works well.
Related Posts
- Migrate Spark job to BigQuery
I have just finished a work about migrating Spark job to BigQuery, or more precisely:…
- Recover truncated table in BigQuery
If you accidentally truncate a table in BigQuery, you can try this article to recover…
- A few notes for Pandas and BigQuery
Get the memory size of a DataFrame of Pandas df.memory_usage(deep=True).sum() 2. Upload a large DataFrame…
November 18, 2022 - 2:17
RobinDong
bigdata
BigQuery
Leave a comment
Leave a Reply Cancel reply
Your email address will not be published. Required fields are marked *
Comment *
Name *
Email *
Website
Save my name, email, and website in this browser for the next time I comment.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK