modular_srm_server ================== # Overview The Modular SRM Server project is a Sinatra server which serves as the backend for the modular_srm UI project. The majority of the Sinatra/Ruby code for the project is in the app/ and the lib/ folders. The UI code for the project is primarily written in javascript using reactjs. The UI code can be compiled, minified, and packaged for the Sinatra server using the Gulp task runner and the provided gulp commands listed in this Readme # Getting Started (MAC OSX local development) This guide assumes you are running things on your local mac and your machine has Homebrew, Node, NPM, & Ruby installed ### Prerequisites #### Postgres brew install postgres initdb /usr/local/var/postgres -E utf8 cp /usr/local/Cellar/postgresql/9.2.3/homebrew.mxcl.postgresql.plist ~/Library/LaunchAgents/ launchctl load -w ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist createuser -s -r postgres If you already have Postgres but don't have 'postgres' user and command above doesn't help try this: psql template1 CREATE ROLE postgres LOGIN SUPERUSER INHERIT CREATEDB CREATEROLE REPLICATION; \q ### Step 1) Clone the Repository ``` git clone git@github.com:vitrue/modular_srm_server.git ``` cd modular_srm_server ### Step 2) bundle install to get the gem dependencies ``` bundle install --path=.bundle ``` ### Step 3) install the node js, and gem dependencies with npm and bundler ``` npm install bundle install ``` ### Step 4) Preparing foreman: 1) Copy the .env.example in the base directory to .env. ```bash cp .env.example .env ``` After copying, you will need to **replace the values** in the new .env file for all keys related to sessions, tokens, and secrets as we do not check those values into source code. Please contact the Modular team if you have problems locating valid values. If you are running locally, replace redis://redis:6379 with redis://**localhost**:6379 ### Step 5) 2) Copy over all relevant config files. You will need to copy over the database.yml.example file separately. ```bash bundle exec rake app:configure ``` ### Step 6) Drop the DB (Just in case you have a old copy lying around) ```bash rake db:drop ``` ### Step 7) Create the databases ```bash rake db:create ``` ### Step 8) Run the db setup script ```bash rake db:schema:load ``` ### Step 9) Populate the asset packs in the database ```bash rake db:assetpacks ``` ### Step 10) Run the server ```bash bundle exec foreman start ``` ### Step 11) Setting up the Testing Environment's database ```bash bundle exec rake db:test:clone ``` # Developing For The Project (External Module Development) One of the main purposes of the modular_srm_server architecture was to provide a Module Ecosystem for adding features built externally by disparate teams. These external projects are called "Asset Packs". In the most simple terms, an asset-pack is simply a collection of urls, available at a url, which is registered into the AssetPack Model's DB table in the Modular Database. ## server-side inclusion of asset packs Asset Packs are registered into the system by adding them to the project's AssetPack database table. Currently, all asset packs entries are created via a database seed file located at db/seeds/all.rb In the future, we will develop an admin panel UI for OPs to use, and for development purposes ### Requirements of an asset pack An npm asset pack should distribute a lib of static asset files alongside a main.js manifest file. An example asset pack properties: ```ruby { name => "modular-engage", style_url => "css/bundle-cbe.css", script_url => "js/bundle-cbe.js", locale_base_url => "js/i18n/" } ``` #### explanation of AssetPack properties ##### name (String - required) the module-name. Should match the github repo name ##### local (Boolean) **NOTE: This is a temporary STOP GAP:** it should only be entered into the modular system by a modular admin. A boolean value which indicates whether the asset_pack asset files should be copied into the modular_srm_server dist folder and served internally by the app While we are building out the system, we are allowing an asset-pack's static files to be served from the modular_srm_server. When local is true, the asset pack urls will be relative urls, and will be added to the Modular server at build time and served by the Modular app server ##### script_url (String - required) The url to reach a single, concatenated and minified javascript file built from all of your app's js ##### style_url (String - required) The url to reach a single, concatenated and minified css file built from all of your app's css ##### locale_urls (Array of Strings - required) An array of all the js paths of locale culture files for translations needed by i18n to translate your UI's strings # Developing For The Project (Internal) This guide assumes you are running things on your local mac and your machine has Homebrew, Node, NPM, & Ruby installed We have added a few helpful features for developers working on client code in the project. The project has a strict build process which lints all javascript and scss files. In order to facilitate faster development we have included a ##Git Pre-commit Hooks We register pre-commit hooks in Git during the npm install process. Any hooks are pulled in which exist in config/git-hooks Currently, we run our lint script in a hook to lint all js and scss files before allowing a git commit. ## Gulp Build process You can build the project using the gulp task ```bash gulp build ``` The process will compile react jsx files, scss stylesheets, minify images, and move tranlation files and font files into a dist/ folder ready which will be served via the sinatra server Note that the process will fail if you have any linting errors or other compilation problems. The console output provides a lot of debugging info in these cases. ## Gulp watch process ###Live JSHint linting and SCSS linting with gulp watch Running the command (from the main project directory) ```bash gulp watch ``` in a terminal window will enable a watch process which will run jshint and gulp-scss-lint on your files and inform you of any liniting errors in real-time inside the console. It is recommended that you always keep a terminal window open with this task running while you are developing client code ###Running Javascript Tests We are using the React team's Jest testing framework to write client side tests Running the command ```bash npm test ``` will run your test suite in a terminal window and report any errors to your console Alternatively, you can run a single test file or directory with the following: ```bash $(npm bin)/jest {file or directory name} ``` where {file or directory name} is replaced with a valid path in your project directory The Jest convention is to name files with the src file name, appending *-spec*, prior to the file extenion Test files should be stored in a directory named __tests__ in the same parent directory as the file you are testing. ex tree: ```bash /components __tests__/ my-component-spec.js my-component.js ``` **Gulp - The Nitty Gritty** Gulp is a javascript task-runner/build-management tool which is powered by node-js under the hood. It has a unix-like piping syntax that makes it easy to run tasks across many files. . For notes on installing gulp see: [https://github.com/gulpjs/gulp/blob/master/docs/getting-started.md#getting-started](Gulp Getting Started) We have set up a simple **gulpfile.js** in the main project directory that sets up a watch task. It will look for any javascript and scss stylesheet changes in the assets directory. When it detects a change, it will run jshint and scss-lint with the configuration options specified in **.jshintrc** and **.scsslintrc** respectively. If you see errors, fix them, and get your builds passing! # Getting Started (Ubuntu VM) This guide assumes you are running things on your local mac and your machine has Homebrew, Node, NPM, & Ruby installed ### Prerequisites ####Redis You need to have redis installed and running on the VM ```bash sudo apt-get update sudo apt-get upgrade sudo apt-get install redis-server sudo redis-server & #start redis-server ``` #### Postgres The project uses postgres as its database: To intall postgres on Ubuntu: ```bash sudo su add-apt-repository ppa:pitti/postgresql apt-get update apt-get install postgresql-9.1 postgresql-contrib-9.1 libpq-dev su postgres -c "createuser -s vitrue" ``` Make the following edits to the file: /etc/postgresql/9.1/main/postgresql.conf Uncomment the line ```bash listen_addresses='localhost' ``` Make the following edits to the file: /etc/postgresql/9.1/main/pg_hba.conf change the line ```bash local all all peer ``` to ```bash local all all trust ``` and change the line ```bash # IPv4 local connections: host all all 127.0.0.1/32 md5 ``` to ```bash # IPv4 local connections: host all all 127.0.0.1/32 trust ``` Restart Postgres ```bash service postgresql restart ``` Note: You can access the postgres client with the following command: ```bash sudo su - postgres -c psql postgres ``` ### Step 1) Instal libcurl openssl ``` sudo apt-get install libcurl4-openssl-dev ``` ### Step 2) Clone the Repository (we typically install at ~/projects/) ``` git clone git@github.com:vitrue/modular_srm_server.git ``` cd modular_srm_server ### Step 3) install the bundler gem & intall bundle ``` gem install bundler bundle install ``` ### Step 4) Install nodejs & npm ```bash sudo apt-add-repository ppa:chris-lea/node.js sudo apt-get update sudo apt-get install nodejs sudo apt-get install npm sudo npm install ``` ### Step 5) Install the gulp task manager globally from npm ```bash sudo npm install gulp -g ``` ### Step 6) Preparing foreman: 1) Copy the .env.example in the base directory to .env ```bash cp .env.example .env ``` After copying, you will need to **replace the values** in the new .env file for all keys related to sessions, tokens, and secrets as we do not check those values into source code. Please contact the Modular team if you have problems locating valid values. ### Step 7) 2) Copy over all relevant config files ```bash bundle exec rake app:confgure ``` ### Step 8) Drop the DB (Just in case you have a old copy lying around) ```bash bundle exec rake db:drop ``` ### Step 9) Create the databases ```bash bundle exec rake db:create ``` ### Step 10) Run the db setup script ```bash bundle exec rake db:schema:load ``` ### Step 11) seed the db ```bash bundle exec rake db:seed ``` ### Step 12) Setting up the Testing Environment's database ```bash bundle exec rake db:test:clone ``` ### Step 13) Build the javascript, css, font, and image asset directories with gulp ```bash gulp build ``` or, alternatively with npm ```bash npm run-script build ``` ### Step 14) The server should assign a random session token for added security. This is obtained via the `RACK_SESSION_SECRET` environment variable. You can set this in your `.env` file or export it directly. For example: ```bash export RACK_SESSION_SECRET=`openssl rand -base64 32` ``` Run the server ```bash bundle exec foreman start ``` # Getting Started (Docker) There is a Modular_SRM docker container available which points to the standard SRMA Staging environment. ### Prerequisites You'll need a running Docker environment configured with docker-compose ~~(ODC VM's work great for this)~~. Docker only runs on linux, so under OS X or Windows, you'll need Docker Machine (formally boot2docker) ```bash brew install docker-machine docker-compose docker-machine create dev docker-machine start dev eval $(docker-machine env dev) ``` ## How To (container only) Obtain the `modular-srm.tgz` container tarball from someone on the dev team and run `docker load -i modular-srm.tgz` to install the container image. _(only until we have a secure docker repository)_ Create a `docker-compose.yml` file containing the following: ```yaml web: image: modular-srm ports: - 8000:8000 links: - db - redis db: image: postgres ports: - 5432 redis: image: redis ports: - 6379 ``` #### Start the environment 1. Start the environment: `docker-compose up -d` 1. Create and seed the database _(only necessary when you create a new environment)_ ```bash docker-compose run web rake db:create docker-compose run web db:schema:load docker-compose run web db:seed ``` #### Connect to your running environment Your container will be running a web server, listening on port 8000. If you're running on OS X or Windows, you'll need to obtain the IP address of your docker VM with `docker-machine ip dev`. Connect to either localhost:8000 or http://YOUR-VM-IP:8000 ## How To (development environment) Clone the repo, if you don't already have a local copy `git clone https://github.com/vitrue/modular_srm_server.git` We've created a couple of helper scripts to build and run a complete development environment. - `script/docker-build` will build the environment - `script/docker-start` will start it and seed the database if necessary. Or, if you prefer... ##### Manually Build 1. Package the gems into `vender/cache` so that they can be accessed inside the container (which has no github credentials): `bundle install && bundle package --all` 1. Do the same for node modules: `npm install && npm shrinkwrap —dev` 1. Finally, build the environment: `docker-compose build` Start the environment either with the helper script or follow the instructions under `Start the environment` in the container-only section. ### Note on Development Environment The `docker-compose.yml` file included within the repo is slightly different from the one we created in the container-only section above. When started, it will mount the current directory into the container at `/data/modular_srm`. Any changes within your local source directory will be reflected within the running container, so you can develop normally and see your changes reflected within the running Docker environment. ## Troubleshooting ###OSX OSX users may run into a browserify related error on OSX due to a low operating system limit on the number of files which can be required. The errors tend to surface in the form of ```node Error: EMFILE, open '{path_on_your_system}' ``` You can work around this problem by setting the ulimit in OSX by typing the following to set the number to a higher limit (2560 is usually sufficient) ```bash ulimit -n 2560 ``` Please reference the issue here: [OSX Error:EMFILE browserify issue #431](https://github.com/substack/node-browserify/issues/431)