Properties (Code Config)

Last Updated: 2021-10-12

Now that you have a Source or some Sources (see here if you don't have a Source), you can begin to define additional Properties. Properties are data values associated to the Records you have in Grouparoo. See this doc to learn more about the core concepts in Grouparoo.

Generating a New Property

You are likely to have more Properties than any other type of config object in your Grouparoo application. Because of this, creating them with the CLI can be a tedious venture. While we still recommend using the CLI, you can also generate a single Property and the duplicate the code inside one file or among multiple files.

To generate a new Property for your Grouparoo application, you can run the generate command.

For example, let's say you have configured a Postgres Table Source with the ID users. And let's say you want to add a Property that calculates the lifetime value of each customer. You can generate a new Record for that Source like this:

grouparoo generate postgres:table:property ltv --parent users_table

Note here that ltv is the id for the Property. An id is always required when generating a config object.

This command will generate a file in your application directory at config/properties/ltv.js. Fill that file out with the appropriate values. The file will look something like this:

exports.default = async function buildConfig() {
  return [
    {
      id: "ltv",
      name: "ltv",
      class: "property",
      sourceId: "...",
      type: "string",
      unique: true,
      identifying: true,
      isArray: false,
      options: {
        column: "...",
        aggregationMethod: "exact",
        sort: null,
      },
      filters: [],
    },
  ];
};

The shape of the object(s) returned by the function will look different depending on the type of Property generated. More on this below.

List of Available Property Types

A Postgres table Property is just one type of Property you can create. To see a full list of available types, use the --list option, filtered by "Property":

grouparoo generate property --list

This will give you something like the following:

bigquery:query:property (id, parent) - Config for a bigquery Query Property
bigquery:table:property (id, parent) - Config for a bigquery Table Property
csv:property (id, parent) - Config for a CSV Property
google-sheets:property (id, parent) - Config for a Google Sheets Property
mailchimp:property (id, parent) - Config for a Mailchimp Property
manual:property (id, parent) - Config for a Grouparoo Property based on a Manual Source
mysql:query:property (id, parent) - Config for a mysql Query Property
mysql:table:property (id, parent) - Config for a mysql Table Property
postgres:query:property (id, parent) - Config for a postgres Query Property
postgres:table:property (id, parent) - Config for a postgres Table Property
redshift:query:property (id, parent) - Config for a redshift Query Property
redshift:table:property (id, parent) - Config for a redshift Table Property
snowflake:query:property (id, parent) - Config for a snowflake Query Property
snowflake:table:property (id, parent) - Config for a snowflake Table Property

This list is determined by the plugins that you have installed. Learn more about Plugins.

Configuring Your Property

Each Property has a common set of options, while other options are specific to the type of Property. Here are the common options:

  • class: (required) Should be set to "property".
  • id: (required) The unique value that defines this Property.
  • identifying: (required) Consider this option to be the primary key for your Record. It is how Grouparoo identifies the Property in the UI.
  • isArray: (required) Is the Property an array?
  • name: (required) A display name for the Property. Will be set to the id by default.
  • sourceId: (required) The id of the Source that the Property is created from. This will be the value of the --parent option you passed to the generate command.
  • type: (required) The data type of the Property. The options available will be generated as a comment in the generated config file.
  • unique: (required) If true Records will be forced to have unique records for this Property.

The Generator attempts to make a reasonable guess at the values in the config file it generated (in the config/properties directory). There are ellipses ("...") in the places in which it couldn't make a reasonable guess. In the example above, after generating a Property for a Postgres Source, all but sourceId and options.column were pre-populated.

Other sections may be commented out but require some action by you. The best practice when configuring an Property is to read the comments, keys, and values within the generated file and fill in the appropriate values for your Property.

Aggregation Method and Filters

Your Property config file will also contain aggregationMethod and Filter options. Options for these may vary depending on your source type. All available options for your source will be listed in your config file template created when you run grouparoo generate. Some common examples are:

aggregationMethods

  • "exact"
  • "sum"
  • "average"
  • "count"
  • "min"
  • "max"
  • "most recent value"
  • "least recent value"

filters

  • "equals"
  • "does not equal"
  • "greater than"
  • "greater than or equal to"
  • "less than"
  • "less than or equal to"
  • "contains"
  • "does not contain"
  • "in"

These can be combined to create more complex properties such as:

exports.default = async function buildConfig() {
  return [
    {
      id: "active_paid_subscriptions",
      name: "Active Paid Subscriptions",
      class: "property",
      sourceId: "users_table",
      type: "integer",
      unique: false,
      identifying: false,
      isArray: false,
      options: {
        column: "subscription_id",
        aggregationMethod: "count",
        sort: null,
      },

      filters: [
        {
          key: "status",
          op: "equals",
          match: "active",
        },
        {
          key: "subscription_type",
          op: "does not equal",
          match: "trial",
        },
      ],
    },
  ];
};

Which would count every subscription per Record, but leave out any trial subscriptions. Or:

exports.default = async function buildConfig() {
  return [
    {
      id: "annual_purhcases_2020",
      name: "Annual Purchases 2020",
      class: "property",
      sourceId: "purhcases_table",
      type: "float",
      unique: false,
      identifying: false,
      isArray: false,
      options: {
        column: "purhcase_amount",
        aggregationMethod: "sum",
        sort: null,
      },

      filters: [
        {
          key: "created_at",
          op: "greater than",
          match: "2019-12-31T11:59:59.999+00.00",
        },
        {
          key: "created_at",
          op: "lt",
          match: "2021-01-01T00:00:00.000+00:00",
        },
      ],
    },
  ];
};

which would return the sum of a Record's purchases in 2020.

Validating & Applying Your Config

You can validate your config at any time using the validate command:

$ grouparoo validate

And you can apply that config (save it to your Grouparoo application's database) using the apply command:

$ grouparoo apply