Product ConceptsLast Updated: 2022-02-02
Welcome to the Grouparoo Dictionary! This is the collection of terms we use throughout the Grouparoo application to help you sync customer data between your source(s) of truth and business applications.
- Aggregation Methods
- API Key
- App Refresh Query
- Application Database
- Batch Generation
- Columnar Source
- High Watermark
- Primary Key
- Record Property
- Query Source
- Sync Mode
- Table Source
Aggregation methods define how we combine the data. When adding multiple properties from a Source, the default aggregation is exact. This means that the imported Grouparoo Record will retain the exact value of each row for that property.
When adding properties individually, one of the required options to choose is the aggregation method.
Not all source types are compatible with all aggregation methods, but most are compatible with the following standard list:
exact: use the value directly
average: take the average
count: count the occurrences
sum: add it up
min: find the smallest value
max: find the largest value
most recent value: use the value of the newest record
least recent value: use the value of the oldest record
Grouparoo uses API Keys to allow for programmatic access to API provided by our Grouparoo instance. Each API Key can have different read and write permissions to various sections of thr Grouparoo API.
You can learn more about how to use and generate API Keys from the REST API documentation.
An App Refresh Query can be used to direct Grouparoo to a particular place to check for data. This may looking for new data in a column of one of your sources, or it may refer to a table that you do not use as a source. If the Query you configure returns a value different from what you had stored, Grouparoo will enqueue all Schedules(#schedule) on all Sources(#source) that have refresh enabled.
The Application Database, sometimes referred to as the Grouparoo database, is the database created and used by the Grouparoo application. This is not to be confused with any database source that in which you store customer data.
Some versions of the
generate command support creating multiple config files from a single command. This is what we call Batch Generation.
A Columnar Source is a specific type of Source that imports data from a column-based mechanism, like a database. Sources of this type share most of their behavior in common.
Destinations are where you want Grouparoo Records and Groups to be sent. Often Destinations are customer marketing tools such as email marketing or push notification services, though databases and data warehouses can also function at Destinations.
When you set up a Destination in Grouparoo, you will choose which Groups as well as which Record Properties you will send to each Destination. You will have control and visibility over what data gets sent to each Destination.
Grouparoo will keep the Record and Group data in your Destinations up-to-date in real time.
Filters are a series of rules used to filter data into some desired subset. Sometimes you may want to specifically include or exclued rows that meet a criteria. This is where Filters come in.
Each filter rule has a key and operation. The key is a property name, and the operation is a comparison operator.
The following operations are available to use as Grouparoo filters. Each operation has a shortened version that is used in the configuration files:
does not equal:
does not exist:
greater than or equal to:
less than or equal to:
does not contain:
in a given set:
Groups are a collection of Records that you can define based on rules. These rules (or filters) are built to be easy-to-understand by non-technical users.
Let's say we wanted to make a Group of high-value customers who recently abandoned the checkout process. Here is an example of the rules to define this Group:
[Total Customer Value] [is greater than] [$50]
[Abandoned Cart] [in the last] [3 days]
With these rules, you have a Group consisting of every user who has purchased more than $50 and recently abandoned their cart. You can then synchronize that Group in real-time to all of your Destinations to then communicate with them.
A High Watermark is a value used for comparison during Runs that are scheduled. Grouparoo will only import items where the High Watermark column is greater than the largest previously imported High Watermark value. For example, let's say you use
updated_at as your High Watermark. Grouparoo will run an import and Record the highest value from that column (presumably from the most recently updated Record). The next time the import runs, Grouparoo will only consider records where
updated_at is greater than the previous High Watermark value. And it doesn't have to be a date, it just has to be comparable. Another example would be a numeric field, like
version, in which the value is incremented when changes are made to the Record in the Source database.
When importing data from a Source, Grouparoo needs to know how to relate the data its importing to the Records already in Grouparoo. This is what we call a Mapping. It operates like a foreign key in a database.
Consider that you have a Property called
emailAddress, which is a unique field in that Source. You could map
emailAddress from your source to
Every Source needs a Mapping, as Grouparoo must know how to associate imported information.
A Model defines the shape of your Records. Each Model has its own Sources, Records, and Properties. Together, this allows you to do things like create generate different types of Records such as Leads, Customers, Users, and Visitors that contain different Properties. For example, a Visitor Model may have a
number_of_sessions Property, while a Customer may have a
ltv Property. Models allow you to only add the Properties you need to keep your Records clean.
The core Grouparoo application is the engine that drives syncing data from Sources to Destinations. The application relies on plugins to provide you with only the functionality you require to sync data between your preferred services.
For example, in a
User Model with a Postgres table import as the Primary Source, the Primary Key may be the
userId Property mapped to the
id column in the Postgres
Records represent individual people. For your specific company or organization, that may mean:
- Visitors to your site or App
Record Properties are data associated with a Record. You can define Record Properties based on data that exists in your data Sources.
Record Properties are given a specific type, letting us know if they are a
string. Having a type helps Grouparoo know how to display this data and send it to Destinations.
Record Properties can also be Unique or Not Unique. Defining Uniqueness is needed in order to create and merge Records from various sources without creating duplicate Records.
Common examples of Unique Record Properties are:
Common examples of Not Unique Record Properties are:
Date of Last Purchase
For example, you might define a Property called
first_name which you are pulling from your product database via the
A Query Source is an implementation of a Columnar Source that enables you to write custom queries to extract information from your database. This is a more flexible version when compared with a Table Source.
Schedules define a frequency that Grouparoo should check your Sources for new data, creating a Run. Whenever possible, Schedules use High watermarks to scan for only new and updated data in your Data Warehouse. Schedules can also be triggered manually or via the API.
Though technically separate objects, Schedules are often used in conjunction with Sources.
Sources are the objects within Apps that Grouparoo will connect with to pull in customer data. A Source can be anything that contains customer data such as data warehouses, databases, and CRMs.
Sources belong to Models, and each Model has a Primary Source and, optionally, multiple Secondary Sources. The Primary Source is the data source where the bulk of the Record data will be imported for the Model, and Secondary Sources serve additional data. The Mapping for the Primary Source is set via the Model's Primary Key.
When creating a Grouparoo Destination, you will be asked to choose a Sync Mode. A Sync Mode tells the Grouparoo application how to create and delete Records on this Destination. The types of Sync Modes are:
- Sync: Add, update and remove records from the Destination as needed.
- Additive: Add new records and update those that already exist in the Destination, but never remove anyone.
- Enrich: Update records that already exist in the Destination, enriching them with additional data and Group membership information. No records are added or removed.
A Query Source is an implementation of a Columnar Source that makes it easy to import data from a database by using aggregation methods to extract data from a column in your database. This is a more rigid version when compared with a Query Source, but often enables you to move faster when you either want to import data directly or perform minor operations or transformations of that data.