Code Config

Last Updated: 2021-06-23

To use Grouparoo to import and export your data, you must configure Apps, Sources, Destinations, Properties, Groups, and more. You can learn more about Grouparoo's product concepts here. With Code Config, you will make use of the Grouparoo CLI to create and validate your config files.

The Grouparoo CLI is built to generate an individual file for each object that you wish to configure. These config files are either JSON or Javscript which allow you the flexibility to manipulate data or load configuration from external sources.

Generating Config Files

You can generate a new config file for any type of object. For general info on the generate command, check out the CLI docs. Or, for info on generating a specific type of object, check out one of our config guides:

Config Object Shapes

Every type of object brings its own options. However, every object you define needs at least the following properties:

  • id
  • class
  • name

id values have some restrictions:

  • Lowercase letters, numbers, and underscores only
  • No spaces
  • Fewer than 38 characters
  • Must be unique among other objects of the same typee.

Grouparoo will format the id values provided to the grouparoo generate command to match these rules.

Config Files Location

Config files are generated at the root of your application directory, in a directory called config. A typical application structure looks like this:

my-application/
├── package.json
├── package-lock.json
├── .env
└── config/
    └── ... (config files go here)

You can optionally choose to use another directory for your configuration by setting the GROUPAROO_CONFIG_DIR environment variable, for example GROUPAROO_CONFIG_DIR=/path/to/config.

Using Dynamic Values

There are two methods for loading configuration information dynamically:

  • Environment Variables
  • JavaScript Functions

Environment Variables

If you want to set a variable called production_hubspot_api_key for use only in Hubspot Apps, you would set GROUPAROO_OPTION__APP__production_hubspot_api_key=abc123.

When setting an Environment Variable to use with Grouparoo, you'll be using the pattern:

export GROUPAROO_OPTION__{topic}__{name}={value}

See the Secrets page for more information about using environment variables for config.

JavaScript Functions

Code config files can be use Javascript functions that can export an async method, which can await while loading data from an database or API. (See the Examples section below.)

Example Code Configs

Let's say you wanted to define a Postgres App called "Data Warehouse" in code. You can use either JSON or JavaScript to define the application.

JSON:

// file: config/apps/data_warehouse.json

[
  {
    "name": "Data Warehouse",
    "id": "data_warehouse",
    "class": "app",
    "type": "postgres",
    "options": {
      "host": "127.0.0.1",
      "port": "5432",
      "username": "person",
      "password": "pass",
      "database": "data_warehouse"
    }
  }
]

Grouparoo parses JSON config files with JSON5. This enables you to use comments and multi-line strings.

JavaScript:

// file: config/apps/data_warehouse.js

exports.default = [
  {
    name: "Data Warehouse",
    id: "data_warehouse",
    class: "app",
    type: "postgres",
    options: {
      host: "127.0.0.1",
      port: "5432",
      username: "person",
      password: "pass",
      database: "data_warehouse",
    },
  },
];

JavaScript with Async Loading:

// file: config/apps/data_warehouse.js

exports.default = async function GetDataWarehouseConfig() {
  const { databasePassword } = await fetch(`/api/company/secrets`);

  return [
    {
      name: "Data Warehouse",
      class: "app",
      id: "data_warehouse",
      type: "postgres",
      options: {
        host: "127.0.0.1",
        port: "5432",
        username: "person",
        password: databasePassword,
        database: "data_warehouse",
      },
    },
  ];
};

Defining Multiple Object in one File

No matter which type of file you create for your code config, you can export more than one object per file. Note how in the examples above we always return/export an array.

// file: config/apps/databases.js

exports.default = [
  {
    name: "Data Warehouse",
    id: "data_warehouse",
    class: "app",
    type: "postgres",
    options: {
      host: "datawarehouse.db.internal",
      port: "5432",
      username: "person",
      password: "pass",
      database: "data_warehouse",
    },
  },
  {
    name: "Product DB Replica",
    id: "replica_db",
    class: "app",
    type: "postgres",
    options: {
      host: "replica.production.db.internal",
      port: "5432",
      username: "person",
      password: "pass",
      database: "product_db",
    },
  },
];

Referencing other Config Objects

To fully define your infrastructure and Profile Properties, you will need to reference one config from another. We do this by referencing the id of the related object.

For example, to define an App → Source → Profile Property, you could do:

// file: config/grouparoo.js

exports.default = [
  // App
  {
    name: "Data Warehouse",
    id: "data_warehouse",
    class: "App",
    type: "postgres",
    options: {
      host: "127.0.0.1",
      port: "5432",
      username: "person",
      password: "pass",
      database: "grouparoo_dev",
    },
  },
  // Source
  {
    name: "Users Table",
    id: "users_table",
    class: "Source",
    type: "postgres-table-import",
    appId: "data_warehouse", // <-- reference id
    options: {
      table: "users",
    },
    mapping: {
      id: "User Id",
    },
  },
  // Property
  {
    name: "User Id",
    class: "ProfilePropertyRule",
    type: "integer",
    unique: true,
    isArray: false,
    sourceId: "users_table", // <-- reference id
    options: {
      column: "id",
      aggregationMethod: "exact",
      sort: null,
    },
    filters: [],
  },
];

Validating & Applying Your Config

You can validate your config at any time using the validate command:

$ grouparoo validate

And you can apply that config (save it to your Grouparoo application's database) using the apply command:

$ grouparoo apply