Skip to content

remove_strings

remove_strings(string, strings_to_remove)

Description

Remove any string of strings_to_remove from string

Usage

Call or Deploy remove_strings ?
Call remove_strings directly

The easiest way to use bigfunctions

  • remove_strings function is deployed in 39 public datasets for all of the 39 BigQuery regions.
  • It can be called by anyone. Just copy / paste examples below in your BigQuery console. It just works!
  • (You need to use the dataset in the same region as your datasets otherwise you may have a function not found error)

Public BigFunctions Datasets

Region Dataset
eu bigfunctions.eu
us bigfunctions.us
europe-west1 bigfunctions.europe_west1
asia-east1 bigfunctions.asia_east1
... ...
Deploy remove_strings in your project

Why deploy?

  • You may prefer to deploy remove_strings in your own project to build and manage your own catalog of functions.
  • This is particularly useful if you want to create private functions (for example calling your internal APIs).
  • Get started by reading the framework page

Deployment

remove_strings function can be deployed with:

pip install bigfunctions
bigfun get remove_strings
bigfun deploy remove_strings

Examples

select bigfunctions.eu.remove_strings("I can eat candies", ['can', 'eat'])

select bigfunctions.us.remove_strings("I can eat candies", ['can', 'eat'])

select bigfunctions.europe_west1.remove_strings("I can eat candies", ['can', 'eat'])

+----------------+
| cleaned_string |
+----------------+
| I  dies        |
+----------------+

Use cases

Let's say you have a dataset of product descriptions that are cluttered with promotional phrases like "Free Shipping!", "Limited Time Offer!", or "New Arrival!". You want to clean these descriptions to improve text analysis or create a more uniform presentation.

Here's how remove_strings could be used:

SELECT product_id, bigfunctions.us.remove_strings(description, ['Free Shipping!', 'Limited Time Offer!', 'New Arrival!']) AS cleaned_description
FROM product_descriptions;

This query would process each row in the product_descriptions table. For each product, the remove_strings function would remove any occurrences of "Free Shipping!", "Limited Time Offer!", or "New Arrival!" from the description field. The result would be stored in a new column called cleaned_description.

Another example: Imagine you have user-generated comments and want to remove common spam words or phrases.

SELECT comment_id, bigfunctions.us.remove_strings(comment_text, ['[link removed]', 'click here', 'make money fast']) AS cleaned_comment
FROM user_comments;

This would remove instances of "[link removed]", "click here", and "make money fast" from the comment_text, resulting in a cleaner cleaned_comment field.

In essence, remove_strings is helpful anytime you need to remove a specific set of strings from a larger body of text for cleaning, pre-processing, or standardization purposes.

Need help or Found a bug?
Get help using remove_strings

The community can help! Engage the conversation on Slack

We also provide professional suppport.

Report a bug about remove_strings

If the function does not work as expected, please

  • report a bug so that it can be improved.
  • or open the discussion with the community on Slack.

We also provide professional suppport.

Show your ❤ by adding a ⭐ on