Skip to content

get_data_from_apify

get_data_from_apify(actor, input, api_token)

Description

Get data using an apify actor.

Requirements

You must create an account on apify and rent an actor to be able to get data from it.

Usage

Call or Deploy get_data_from_apify ?
Call get_data_from_apify directly

The easiest way to use bigfunctions

  • get_data_from_apify function is deployed in 39 public datasets for all of the 39 BigQuery regions.
  • It can be called by anyone. Just copy / paste examples below in your BigQuery console. It just works!
  • (You need to use the dataset in the same region as your datasets otherwise you may have a function not found error)

Public BigFunctions Datasets

Region Dataset
eu bigfunctions.eu
us bigfunctions.us
europe-west1 bigfunctions.europe_west1
asia-east1 bigfunctions.asia_east1
... ...
Deploy get_data_from_apify in your project

Why deploy?

  • You may prefer to deploy get_data_from_apify in your own project to build and manage your own catalog of functions.
  • This is particularly useful if you want to create private functions (for example calling your internal APIs).
  • Get started by reading the framework page

Deployment

get_data_from_apify function can be deployed with:

pip install bigfunctions
bigfun get get_data_from_apify
bigfun deploy get_data_from_apify
Keep the secrets safe!

Do NOT write secrets in plain text in your SQL queries!

Otherwise, anyone with access to your BigQuery logs can read and use them.

Instead, generate an encrypted version that you can safely share:

  1. Enter a secret value below along with the emails of the users who are authorized to use it (separated by commas).
  2. Click on Encrypt Secret.
  3. The browser (no server is called) will generate an encrypted version and copy it in the clipboard
  4. Paste the encrypted secret into the arguments of your function exactly like if you passed the plain text version.
  5. The bigfunction will decrypt it and check that the calling user is authorized.
More on secret encryption

Technically, this encryption system uses the same encryption mechanism used to transfer data over the internet. It uses a pair of a public and private keys.

The public key (contained in this web page) is used to encrypt a text. The corresponding private key is the only one who is able to decrypt the text. The private key is stored in a secret manager and is only accessible to this function. Thus, this function (and this function only) can decrypt it.

Moreover, the function will check that the caller of the function belong to the list of authorized users that you gave at encryption time.

Thanks to this:

  • Nobody but this function will be able to decrypt it.
  • Nobody but authorized users can use the encrypted version in a function.
  • No function but the function get_data_from_apify can decrypt it.

Examples

Get PSG followers on instagram

select bigfunctions.eu.get_data_from_apify("apify/instagram-followers-count-scraper", json '{ "usernames": ["psg"] }', "ENCRYPTED_SECRET(Eu5XjDuxO1IBPOAN0U...==)")

select bigfunctions.us.get_data_from_apify("apify/instagram-followers-count-scraper", json '{ "usernames": ["psg"] }', "ENCRYPTED_SECRET(Eu5XjDuxO1IBPOAN0U...==)")

select bigfunctions.europe_west1.get_data_from_apify("apify/instagram-followers-count-scraper", json '{ "usernames": ["psg"] }', "ENCRYPTED_SECRET(Eu5XjDuxO1IBPOAN0U...==)")

+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| data                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| [
  {
    "followersCount": 62841046,
    "followsCount": 168,
    "profilePic": "https://instagram.fmel7-1.fna.fbcdn.net/v/t51.2885-19/449700979_1506857413562380_7088351830712502452_n.jpg?stp=dst-jpg_e0_s150x150_tt6&_nc_ht=instagram.fmel7-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2AF9oFgyWnsA4x8tu8w5Rr1J_mI9FoabJNg7fHkelZ6jSaMwUSor_QZNs9IQ-iRs_iw&_nc_ohc=gUrxGWpcWb4Q7kNvgEJE0ih&_nc_gid=cb1d2f086b814adca4020f9401c84a32&edm=AOQ1c0wBAAAA&ccb=7-5&oh=00_AYBbHG5DjV1qxsQhy0IaD0_d51XzY0LAcRhK2Fx1lnjRKA&oe=67B4E610&_nc_sid=8b3546",
    "timestamp": "2025-02-14 - 11:22",
    "userFullName": "Paris Saint-Germain",
    "userId": "232024162",
    "userName": "psg",
    "userUrl": "https://www.instagram.com/psg"
  }
]
 |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Need help or Found a bug?
Get help using get_data_from_apify

The community can help! Engage the conversation on Slack

We also provide professional suppport.

Report a bug about get_data_from_apify

If the function does not work as expected, please

  • report a bug so that it can be improved.
  • or open the discussion with the community on Slack.

We also provide professional suppport.

Show your ❤ by adding a ⭐ on