DevelopmentLast Updated: 2021-01-12
First, 🎉THANK YOU🎉.
Grouparoo relies on the contributions of many people just like you to create the best marketing infrastructure & automation tools. Thank you for taking the time to join this open source community, and create something we can all use! By contributing to Grouparoo, you agree to our Community Guidelines and Code of Conduct
When developing Grouparoo, you'll be running code within the
grouparoo monorepo. This git repository contains a number of related bu separate projects. There are
Plugins, along with a number of tools and utilities. We use
pnpm to install packages to allow us to run and test all of these packages together without needing to publish them to NPM. We use
lerna to publish packages.
- Apps: These are example applications like a Grouparoo customer would run - they require
@grouparoo/coreand any number of plugins and may have toggled certain settings
- Core: The
@grouparoo/coreproject is the main part of any Grouparoo application and includes the API (both web and workers).
- CLI: The Grouparoo CLI tool, ie:
grouparoo init .
- Plugins: There are opt-in add-ons for Grouparoo customers, and contain the logic to connect to new sources & destinations, or otherwise add new functionality to Grouparoo.
- UI: The grouparoo UI plugins,
ui-enterpriseand shared component libraries.
See below for the installation guide for your operating system.
# Install NVM to manage node versions # From: https://github.com/nvm-sh/nvm curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.34.0/install.sh | bash # Install the homebrew package manager # From: https://brew.sh/ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" # Install Node.js v14 nvm install v14 # Depending on your shell (bash, zsh, etc) you may need to ensure that nvm is loaded into your environment via .bashrc, .bash_profile, etc # On OSX/ZSH, we recommend installing Oh My ZSH (https://github.com/ohmyzsh/ohmyzsh) and enabling the nvm plugin # Install & run postgresql (v9.x) brew install postgresql brew services start postgresql # Install & run Redis brew install redis brew services start redis # Clone this repository git clone firstname.lastname@example.org:grouparoo/grouparoo.git cd grouparoo # Install with pnpm (If you don't have pnpm, install it via "npm i -g pnpm") pnpm install # Configure your local environment variables in .env to be able to run the "staging-enterprise" app cp apps/staging-enterprise/.env.example apps/staging-enterprise/.env open apps/staging-enterprise/.env # Create your local development database (Postgres) createdb "grouparoo_development" # or in `psql` CREATE DATABASE grouparoo_development; GRANT ALL PRIVILEGES ON DATABASE grouparoo_development TO postgres; # Run the staging-enterprise app example project cd apps/staging-enterprise # If you aren't already here npm run dev
npm run dev (development mode), which will handle the compilation on the fly. Every sub-project in the Grouparoo Monorepo has an
npm run prepare command which handles the compilation process.
npm run prepare is also an NPM lifecycle hook, which will automatically run after every
npm install as well.
If you are developing a plugin, you will need to
npm run prepare it before seeing your changes. In many cases, the
prepare step only calls out to
tsc, and in these cases, you may be able to
tsc --watch to automatically source changes and rebuild your plugin.
Grouparoo self-documents its Typescript interface to docs.grouparoo.com via Typedoc. The docs from the
main branch are automatically published to this site. If you want to read the docs for an older version of Grouparoo or a different branch, you can generate the docs locally by running
cd core && npm run docs in this repository.
Learn about the Grouparoo architecture and models from the other pages in this documentation section.
Grouparoo is built on top of many wonderful open source technologies. An exhaustive list of our dependencies can be found within our package.json files. This below list provides links to the core packages we rely on.
All dependencies Grouparoo uses must be licensed under a non-viral open-source licence that allows for commercial use. We audit every dependency in every sub-project of this monorepo via license-checker. The list of licenses we accept can be found here.
@grouparoo/demo plugin makes it easy to generate sample data! You can use the
staging-community app to try things out:
cd apps/staging-community # populate the system with 1000 profiles, properties, 1000 purchases, groups, and some events grouparoo demo-data-purchases # including the --scale param allows you to control how many profiles you make. e.g. --scale 10 makes 10,000 extra profiles. grouparoo demo-data-purchases --scale 10 # populate the system with events grouparoo demo-event-stream
Note: This will clear all existing data, replacing it with the sample data.
You can then use the following credentials to log in:
Grouparoo uses Jest as our testing framework. To run tests, first make sure you've installed all dependencies in the root of the project.
pnpm install # we use pnpm to manage the monorepo
You will need
chromedriver installed to run the automated browser tests as well:
brew cask install chromedriver
- Windows: TODO
- Linux: TODO
Next, create the test databases:
cd core ./bin/create_test_databases
npm test from the top-level of this monorepo will run the
npm test command in every sub-package of the monorepo via pnpm. You can also test an individual package by entering that directory and running
npm test. If you want to run a single test file you can
./node_modules/.bin/jest /path/to/test and you can watch files with jest's
Jest will automatically test the latest version of your typescript files, and you do not need to compile your code to test. However, the test suite should test the compile step as well. We usually run
npm run prepare as a
pretest NPM hook.
A note on chromedriver for OSX Users: As Chrome updates, you may need to
brew cask reinstall chromedriver to keep up to date as well. Every time you update chromedriver, you will need to re-authorize the application to run in system preferences, as it is unsigned.
- If you create a new example app, you will need to
pnpm installagain to link up the dependencies in the monorepo.
- Be sure that your
namefield matches the folder name.
- Be sure that
grouparoo.grouparoo_monorepo_appmatches the folder name.
On the server:
- Ensure you have the proper environment variables set
pnpm installat the root of this project, which will run the build steps too
cd apps/staging-public && npm run start
Procfile for an example of how to run a web and worker process independently.
If you are deploying to Heroku or a similar PaaS, they might prune node_modules that aren't explicitly in "dependencies". However, due to the way lerna works, this might delete all of your modules. Check with your provider for how to disable this (ex: https://devcenter.heroku.com/articles/nodejs-support#build-behavior)