BigFunctions¶
BigFunctions are open-source BigQuery routines that give you SQL-superpowers in BigQuery 💪.
✅ You can call ANY of the following public BigFunctions from your Google Cloud Project (no install).
- The functions are deployed in
bigfunctions
GCP project in 39 datasets for all of the 39 BigQuery regions. - They are public, so they can be called by anyone.
- For any question or difficulties, please read Getting Started.
- If you prefer to deploy the BigFunction in your own project, read Getting Started.
- Found a bug? Please raise an issue here
All BigFunctions Datasets >
Region | Dataset |
---|---|
eu |
bigfunctions.eu |
us |
bigfunctions.us |
europe-west1 |
bigfunctions.europe_west1 |
asia-east1 |
bigfunctions.asia_east1 |
asia-east2 |
bigfunctions.asia_east2 |
asia-northeast1 |
bigfunctions.asia_northeast1 |
asia-northeast2 |
bigfunctions.asia_northeast2 |
asia-northeast3 |
bigfunctions.asia_northeast3 |
asia-south1 |
bigfunctions.asia_south1 |
asia-south2 |
bigfunctions.asia_south2 |
asia-southeast1 |
bigfunctions.asia_southeast1 |
asia-southeast2 |
bigfunctions.asia_southeast2 |
australia-southeast1 |
bigfunctions.australia_southeast1 |
australia-southeast2 |
bigfunctions.australia_southeast2 |
europe-central2 |
bigfunctions.europe_central2 |
europe-north1 |
bigfunctions.europe_north1 |
europe-southwest1 |
bigfunctions.europe_southwest1 |
europe-west2 |
bigfunctions.europe_west2 |
europe-west3 |
bigfunctions.europe_west3 |
europe-west4 |
bigfunctions.europe_west4 |
europe-west6 |
bigfunctions.europe_west6 |
europe-west8 |
bigfunctions.europe_west8 |
europe-west9 |
bigfunctions.europe_west9 |
europe-west12 |
bigfunctions.europe_west12 |
me-central1 |
bigfunctions.me_central1 |
me-west1 |
bigfunctions.me_west1 |
northamerica-northeast1 |
bigfunctions.northamerica_northeast1 |
northamerica-northeast2 |
bigfunctions.northamerica_northeast2 |
southamerica-east1 |
bigfunctions.southamerica_east1 |
southamerica-west1 |
bigfunctions.southamerica_west1 |
us-central1 |
bigfunctions.us_central1 |
us-east1 |
bigfunctions.us_east1 |
us-east4 |
bigfunctions.us_east4 |
us-east5 |
bigfunctions.us_east5 |
us-south1 |
bigfunctions.us_south1 |
us-west1 |
bigfunctions.us_west1 |
us-west2 |
bigfunctions.us_west2 |
us-west3 |
bigfunctions.us_west3 |
us-west4 |
bigfunctions.us_west4 |
👀 Explore¶
explore_table(fully_qualified_table)
: Show table infos and column statisticssankey_chart(data)
: Return html with a Sankey Google chartexplore_dataset(fully_qualified_dataset)
: Show infos about dataset tableschart(data, chart_type, options)
: Return html with a chartjs chartexplore_column(fully_qualified_column)
: Show column statistics
🧠 AI¶
ask_appstore_reviews(prompt, app_url_in_appstore)
: Ask AI what your app users think.generate_sql(question, fully_qualified_table)
: Transformquestion
to a SQL query.ask_my_data(question, fully_qualified_table)
: Ask your data anyquestion
in natural language.ask_ai(prompt, model)
: Ask Anything!
🛢 Get data¶
get_webpage_metadata(url)
: Get webpage metadataget_appstore_reviews(url)
: GET Apple App Store Reviews of an appget_meteo(latitude, longitude, date)
: Getmeteo
get_webpage_structured_data(url)
: Get webpage Structured Dataget_json(url, headers)
: GET jsondata
fromurl
exchange_rate(base, to)
: Getexchange_rate
get(url, headers)
: Requesturl
💬 Notify¶
send_mail_with_excel(to, subject, content, excel_filename, table_or_view_or_query)
: Sends an email withtable_or_view_or_query
data attached as excel filesend_sms(message, phone_number)
: Sendsmessage
via SMS tophone_number
send_teams_message(message, webhook_url)
: Sendsmessage
to a Microsoft Teams channel.send_slack_message(message, webhook_url)
: Sendsmessage
to a slack channel.send_google_chat_message(message, webhook_url)
: Sendsmessage
to google chat spacesend_mail(to, subject, content, attachment_filename, attachment_content)
: Sends an email
🚀 Export¶
upload_table_to_gsheet(table_or_view_or_query, max_rows, spreadsheet_url, worksheet_name, write_mode)
: Upload data fromtable_or_view_or_query
to Google Sheetpost(url, data, headers)
: POSTdata
tourl
.export_to_pubsub(project, topic, data, attributes)
: Exportsdata
andattributes
to Pub/Subtopic
.export_to_datastore(project, namespace, kind, key, data)
: Exportsdata
to Datastoreupload_to_gsheet(data, spreadsheet_url, worksheet_name, write_mode)
: Uploaddata
(a json array of objects) to a Google Sheet
1️⃣ Transform numeric¶
format_percentage(first_number, second_number, nb_decimals)
: Returnfirst_number / second_number
as a formatted percentagequantize_into_fixed_width_bins(value, min_bound, max_bound, nb_bins)
: Get thebin_range
in which belongsvalue
quantize_into_bins(value, bin_bounds)
: Get thebin_range
in which belongsvalue
✨ Transform string¶
phone_number_info(phone_number, options)
: Getphone_number
inforemove_strings(string, strings_to_remove)
: Remove any string ofstrings_to_remove
fromstring
translate(text, target_language)
: Translatetext
intotarget_language
deidentify(text, info_types)
: Masks sensitive information of typeinfo_types
intext
parse_url(url)
: Returnurl
partslevenshtein(string1, string2)
: Compute levenshtein distance betweenstring1
andstring2
convert_non_ascii_characters_to_unicode_escape_sequences(text)
: Replace all non ASCII characters with escape unicodefaker(what, locale)
: Generates fake dataremove_words(string, words_to_remove)
: Remove any word ofwords_to_remove
fromstring
is_phone_number_valid(phone_number, options)
: Return ifphone_number
is validip2country_name(ip)
: Getcountry_name
ofip
xml_extract(xml, x_path)
: Returns content extracted from XML from given XPATHrender_template(template, context)
: Render template with context using nunjucks.js templating libraryreplace_special_characters(string, replacement)
: Replace most common special characters in astring
withreplacement
remove_accents(str)
: Remove accentsparse_user_agent(user_agent_string)
: Parses User Agent strings into several componentsip_range2ip_networks(first_ip, last_ip)
: Convert an IP range into a json list of IP networks in CIDR notationurl_decode(url_encoded_string)
: Decodeurl_encoded_string
detect_sensitive_info(text)
: Detect sensitive information intext
ip2asn(ip)
: Getasn
ofip
is_email_valid(email)
: Return true ifemail
is validip2continent_name(ip)
: Getcontinent
ofip
ip2country(ip)
: Getcountry_code
ofip
render_string(template, context)
: Render template with context using nunjucks.js templating libraryip2continent(ip)
: Getcontinent_code
ofip
remove_extra_whitespaces(str)
: Remove unwanted whitespaces
🌐 Transform geo data¶
validate_address(address)
: Validateaddress
using Google Mapsreverse_geocode(latitude, longitude)
: Get address details atlatitude
,longitude
h3(function_name, arguments)
: Wrapper around Uber H3geocode(address)
: Getaddress
details from Google Maps
📆 Transform date¶
gregorian2hijri(gregorian_date)
: Convert Gregorian Date to Hijri Date (taken from here)is_public_holiday(date, country_code)
: Return true ifdate
corresponds to a public holiday incountry_code
date_sub_isoyear(date, years)
: Returns same dayyears
beforetranslated_weekday_name(date, language)
: Gettranslated_weekday_name
translated_month_name(date, language)
: Gettranslated_month_name
generate_dates(start_date, end_date)
: Generate a table of datesparse_date(date_string)
: Parse date with automatic format detection
{...} Transform json¶
items2json(key_value_items)
: Returnsjson
object from array ofkey_value_items
json_values(json_string)
: Extractvalues
fromjson_string
json_schema(json_string)
: Return the schema of a json string as[{path, type}]
json_keys(json_string)
: Extractkeys
fromjson_string
json_query(json_string, query)
: Extract data fromjson_string
using advanced json queryingjson_merge(json_string1, json_string2)
: Mergejson_string1
andjson_string2
json_items(json_string)
: Extractkey_value_items
fromjson_string
[...] Transform array¶
remove_value(arr, value)
: Return an array with all values exceptvalue
.distinct_values(arr)
: Return distinct valuesmedian_value(arr)
: Return median value of arraypercentile_value(arr, percentile)
: Returns percentile of an array with percentile a float in range [0, 1].sort_values_desc(arr)
: Return sorted array (descending)last_value(arr)
: Return last value of arrayfind_value(arr, value)
: Return the firstoffset
(zero-based index) ofvalue
in arrayarr
find_lower_value(arr, x)
: Return theoffset
(zero-based index) of the firstvalue
inarr
wherevalue <= x
get_value(key_value_items, search_key)
: Return the firstvalue
with a keysearch_key
fromkey_value_items
min_value(arr)
: Return min value of arrayarray_intersect(array1, array2)
: Returns the intersection of two arrays.sum_values(arr)
: Return the sum of array valuesarray_union(array1, array2)
: Returns the union of two arrays.find_greater_value(arr, x)
: Return theoffset
(zero-based index) of the firstvalue
inarr
wherevalue >= x
sort_values(arr)
: Return sorted array (ascending)max_value(arr)
: Return max value of arrayare_arrays_equal(array1, array2)
: Return true ifarray1
=array2
🧠 Machine learning¶
roc_auc(predictions)
: Returns the Area Under the Receiver Operating Characteristic Curve (a.k.a. ROC AUC)precision_recall_auc(predictions)
: Returns the Area Under the Precision Recall Curve (a.k.a. AUC PR)prophet(records, periods, kwargs)
: Forecast time-series using prophetroc_curve(predictions)
: Returns the Receiver Operating Characteristic Curve (a.k.a. ROC Curve)sentiment_score(content)
: Compute sentiment score ofcontent
precision_recall_curve(predictions)
: Returns the Precision-Recall Curve
🌐 Graph¶
connected_components(fully_qualified_table)
: Compute the connected components of a non-directed graph.
🔨 Convert data format¶
json2excel(data)
: Dump data to excel filejson2xml(json)
: Returns XML for given JSON stringhtml2pdf(html)
: Converthtml
topdf
xml2json(xml)
: Returns JSON as a string for given XML string
🔨 Utils¶
deduplicate_rows(query_or_table_or_view)
: Returns the deduplicated rows ofquery_or_table_or_view
get_table_columns(fully_qualified_table)
: Get the column information of the given table fromINFORMATION_SCHEMA.COLUMNS
get_view_history(fully_qualified_view)
: Get BigQuery View historysleep(seconds)
: Sleep duringseconds
secondsget_latest_partition_timestamp(fully_qualified_table)
: Return the maximum of the partition column offully_qualified_table
upsert(query_or_table_or_view, destination_table, insertion_mode, primary_keys, recency_field)
: Mergesquery_or_table_or_view
into thedestination_table
.run_python(python_code, requirements, kwargs)
: Run anypython_code
.timestamp_to_unix_date_time(timestamp_expression, date_time_part)
: Returns the number ofdate_time_part
since1970-01-01 00:00:00 UTC
.timestamp_from_unix_date_time(unix_date_time, date_time_part)
: Interpretsunix_date_time
as the number ofdate_time_part
since1970-01-01 00:00:00 UTC
.