· 6 years ago · Oct 15, 2019, 06:56 PM
1[**Home**](Home) > Local Development
2
3This guide will walk you through the steps for setting up your local environment to develop a ShopStorm Shopify app. App-specific instructions can be found in their repository README or wiki.
4
5These instructions mostly assume that you're using macOS with Homebrew as package manager, and rbenv for managing ruby versions. However if you have a different preferred dev environment, please feel free to add to this guide!
6
7# Setup
8
9## Ingredient List
10
11* macOS: Xcode Command Line Tools
12* macOS: [Homebrew](http://brew.sh) or other package manager like MacPorts.
13* [Rbenv](http://rbenv.org/) or other ruby version manager like RVM
14* [bundler](http://bundler.io/) (to manage required gems and dependencies)
15* [ForwardHQ](https://forwardhq.com) to expose your local server to the interwebs with a url like `john-pc-skyverge.fwd.wf`
16 * [ngrok](https://ngrok.com) alternative for when ForwardHQ is down
17* PostgreSQL - [postgresapp](http://postgresapp.com/)
18* [Heroku Tool Belt](https://toolbelt.heroku.com/) for simulating the Heroku environment locally
19* [Redis](http://redis.io/) backing for Sidekiq and other short term storage
20* Optional: [Setup a test Shopify shop](https://github.com/skyverge/shopstorm-app/wiki/Shopify-Dev-Store) (named like `skyverge-dev-john.myshopify.com`)
21* A little bit of luck
22
23## Setup your environment
24
25Follow these steps when starting development for your first ShopStorm app. If you've already followed these instructions, jump to the [ShopStorm app setup instructions](#create-a-new-development-shopify-app-or-use-an-existing-one)
26
27### 1. macOS: Install Xcode Command Line Tools
28
29First verify whether you already have them installed on your machine:
30
31``` sh
32$ xcode-select -p
33```
34
35If you see something like:
36
37``` sh
38xcode-select: error: unable to get active developer directory, use `xcode-select --switch` to set one (or see `man xcode-select`)
39
40# for automation purposes:
41$ echo $?
422
43```
44
45Then install the tools:
46
47``` sh
48$ xcode-select --install
49```
50
51And click "Install" in the resulting dialog box:
52
53
54
55_Note:_ It seems like it might not be possible to _upgrade_ an existing xcode CLI tools. If someone knows how to do this, please update this wiki page here!
56
57### 2. macOS: Install a package manager
58
59_Note:_ [Homebrew](http://brew.sh/) is a great choice if you're setting up your dev environment; skip this step if you already use another package manager like MacPorts.
60
61``` sh
62$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
63$ echo 'export PATH="/usr/local/bin:$PATH"' >> ~/.bash_profile
64$ source ~/.bash_profile
65$ brew doctor
66Your system is ready to brew.
67```
68
69### 3. Install a ruby version manager
70
71_Note:_ [`rbenv` and `ruby-build`](https://github.com/sstephenson/rbenv#homebrew-on-mac-os-x) are a great choice if you're setting up your dev environment; skip this step if you already use another version manager like RVM.
72
73**rbenv-default-gems**: This rbenv plugin hooks into the `rbenv install` command to automatically install gems every time you install a new version of Ruby
74
75``` sh
76$ brew update
77$ brew install rbenv ruby-build rbenv-default-gems
78$ echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bash_profile
79$ echo 'eval "$(rbenv init -)"' >> ~/.bash_profile
80$ source ~/.bash_profile
81$ echo 'gem: --no-rdoc --no-ri' >> ~/.gemrc
82```
83
84### 4. Install ruby with rbenv, and the Bundler gem
85
86```sh
87$ rbenv install 2.2.0
88$ rbenv rehash
89$ rbenv global 2.2.0
90$ gem install bundler
91$ echo 'bundler' >> "$(brew --prefix rbenv)/default-gems"
92```
93
94For additional details on managing ruby with rbenv, check out our [rbenv wiki page](Rbenv)
95
96### 5. Install [ForwardHQ](https://forwardhq.com) gem
97
98Because apps use the Shopify API, your local environment needs to have a public hostname that matches the app setup within our Shopify partners account. We use [ForwardHQ](https://forwardhq.com) which provides a dead-simple gem to setup forwarding.
99
100If you've never used forward on this machine, run the following in your home directory:
101
102``` sh
103$ gem install rack -v 1.6.4
104$ gem install forward
105$ echo 'forward' >> "$(brew --prefix rbenv)/default-gems"
106$ source ~/.bash_profile
107```
108
109then
110
111``` sh
112$ forward 3000
113```
114
115You'll be prompted to enter a username and password: ask Justin or Max for an invite to the SkyVerge ForwardHQ team, if you have not already received one. After this initial setup, you won't need to enter a username/password again.
116
117For additional details on using/troubleshooting ForwardHQ, check out our [ForwardHQ wiki page](ForwardHQ)
118
119#### Alternative to ForwardHQ
120
121Sometimes ForwardHQ doesn't work, especially on holidays when their support team isn't around. A quick and dirty alternative is to use [ngrok](https://ngrok.com), which has a functional free option. To do so, first download and unzip the ngrok binary from [ngrok.com](https://ngrok.com) if you have not already done so, then follow these steps:
122
1231. Start forwarding with `./ngrok http 5000` where `5000` is the port that your local application serves over, as configured in your `.env` file
124 * You'll be presented with a screen containing a line like `Forwarding https://4b8ecd47.ngrok.io -> localhost:5000`
1251. Copy that `*.ngrok.io` address and enter it into the following spots:
126 * Your `.env` file as the `ROOT_URL`
127 * In the Shopify Partners Account settings for your app, replacing the existing values for **Application URL** and **Redirection URL** (Ask Emily, Justin, or Max to do this)
1281. Comment out the `forwarding:` line from your `Procfile.dev`
1291. Start up your local app server as usual
130
131Note that each time you launch ngrok you'll be assigned a new random subdomain, so you'll need to update the environment file and Partner Account app settings each time, making this most appropriate as a short-term solution while ForwardHQ is offline, rather than a longterm solution.
132
133### 6. Install [Postgres.app](http://postgresapp.com/)
134
135``` sh
136$ brew install wget
137$ wget https://github.com/PostgresApp/PostgresApp/releases/download/9.4.0.1/Postgres-9.4.0.1.zip
138$ unzip Postgres-9.4.0.1.zip
139$ mv Postgres.app/ /Applications/
140$ echo 'export PATH="/Applications/Postgres.app/Contents/Versions/latest/bin:$PATH"' >> ~/.bash_profile
141$ source ~/.bash_profile
142$ open -a Postgres.app
143```
144
145Click the Postgres elephant menu bar icon > "Preferences..." > Make sure to check **Start Postgres automatically after login**.
146
147If Rails has trouble opening a PG connection (may happen after `rails`/`pg` gem upgrade):
148```
149could not connect to server: No such file or directory
150 Is the server running locally and accepting
151 connections on Unix domain socket "/var/pgsql_socket/.s.PGSQL.5432"?
152```
153
154Solution: uninstall and reinstall `pg` gem.
155```sh
156$ gem uninstall pg
157$ bundle install
158```
159
160For Linux, see [**Postgresql setup on Ubuntu**](https://github.com/skyverge/shopify-app/wiki/Postgresql-setup-on-Ubuntu)
161
162### 7. Install [heroku toolbelt](https://toolbelt.heroku.com/)
163
164``` sh
165$ brew install heroku-toolbelt
166$ heroku update
167$ heroku login
168Enter your Heroku credentials.
169Email: you@example.com
170Password (typing will be hidden):
171Authentication successful.
172```
173
174Reference: http://sourabhbajaj.com/mac-setup/Heroku/README.html
175
176### 8. Install Git
177
178macOS ships with an outdated version of Git
179
180``` sh
181$ brew install git
182```
183
184For an initial environment setup you'll need to add/create your public key, otherwise skip this part:
185
186``` sh
187$ cd ~/.ssh
188$ ssh-keygen -t rsa -C "<your-email>@example.com"
189Generating public/private rsa key pair.
190Enter file in which to save the key (~/<username>/.ssh/id_rsa):
191Enter passphrase (empty for no passphrase):
192```
193
194Next, add that key to your github profile:
195
1961. Log into Github
1971. Click settings
1981. Click "SSH Keys"
1991. Click "Add SSH key" and enter the newly generated public key
200
201### 9. Redis
202
203We use [sidekiq](https://github.com/mperham/sidekiq) to provide background job processing for the app. Common background jobs include: webhook processing, app review scraping, etc. Each Sidekiq task is implemented by a different worker object. Sidekiq uses Redis as it's datastore:
204
205```sh
206$ brew install redis
207
208# To have launchd start redis at login
209$ ln -sfv /usr/local/opt/redis/*.plist ~/Library/LaunchAgents
210
211# and start redis
212$ launchctl load ~/Library/LaunchAgents/homebrew.mxcl.redis.plist
213
214# verify redis is running
215$ redis-cli ping
216PONG
217```
218
219Ubuntu instructions for installing redis are available [here](https://github.com/skyverge/shopify-app/wiki/Redis-setup-on-Ubuntu).
220
221## Create a Shopify Development Store
222
223Instructions found [here](Shopify-Dev-Store).
224
225## Create a new development Shopify app or use an existing one
226
227Message Emily, Justin, or Max on Slack for this step :smile:
228
229Add a new development app to our Shopify partner account with the following configuration:
230
231**Name of app (required):** `<App Name> Dev - <Your First Name>`
232**Embedded settings:** `Enabled`
233**Application URL (required):** `https://<your-first-name>-<app-id>-skyverge.fwd.wf`
234**Preferences URL:** `https://<your-first-name>-<app-id>-skyverge.fwd.wf/`
235**Support URL:** `http://shopstorm.com/support/`
236**Redirection URL (required):** `https://<your-first-name>-<app-id>-skyverge.fwd.wf/auth/shopify/callback`
237**Shop admin links:**
238* See individual app setup instructions
239
240You will be provided with the app API Key and Secret Token, which will be used in your `.env` file described below.
241
242**Note:** `app-id` is typically a two character representation of the app name, e.g. "pc" for "product-customizer", "bf" for "blogfeeder", etc. Due to a ForwardHQ constraint, `<your-first-name>-<app-id>` must be less than 24 characters long, so do what you have to.
243
244## Optional Services
245
246### Create a new GitHub application for authorization purposes
247
248This step is technically optional, but necessary if you want to access/develop the Sidekiq admin or app Active Admin locally.
249
2501. Log into your GitHub account, open the profile dropdown in the top-right and click [Settings](https://github.com/settings/profile)
2511. Under "Developer Settings" click [OAuth applications](https://github.com/settings/developers)
2521. Click [Register new application](https://github.com/settings/applications/new)
2531. Configure the new application with:
254 **Application name:** *something descriptive like* `Shopify <app name> Dev`
255 **Homepage URL:** `https://github.com/skyverge/<app repo>`
256 **Authorization callback url:** `https://<your-first-name>-<app-id>-skyverge.fwd.wf`
257
258Once you've registered the OAuth application, note the Client ID and Client Secret, which will be used in your `.env` file described below.
259
260For the `WARDEN_GITHUB_VERIFIER_SECRET` configuration var required in `.env`, generate a suitable random md5 secret — you can use something like `$ openssl rand -base64 8 | md5`
261
262### Help Scout Custom App
263
264See [Help Scout Integration](https://github.com/skyverge/shopstorm-app/wiki/Help-Scout-Integration#development) for details on testing a Help Scout app in development mode.
265
266### Help Scout API
267
268If developing the active admin Help Scout panel, log into Help Scout, edit your profile, click API Keys and generate a new key. Or, message Justin or Max on Slack for an existing key.
269
270This secret key will be used in your `.env` file described below
271
272### Customer.io Integration
273
274If developing the customer.io integration, log into the customer.io account and use or create an existing environment, or message Justin or Max on Slack for the ID/Key for an existing environment.
275
276This environment ID/Key which will be used in your `.env` file described below
277
278### Sentry Exception reporting
279
280If developing/testing the Raven exception reporting provided by Sentry, log into getsentry.com to find the API key for an existing project, or message Justin or Max on Slack.
281
282This API key will be used in your `.env` described below
283
284### Slack Webhook
285
286If developing/testing the Sidekiq health monitoring script, you'll need the Slack incoming webhook URL. Either log into the Slack team settings, click "Incoming Webhook" under [integrations](https://skyverge.slack.com/services), and find the Webhook URL, or message Justin or Max on Slack.
287
288Use this Webhook URL in your `.env` described below
289
290### Heroku API
291
292If developing/testing the Sidekiq health monitoring script, you'll need Heroku API credentials. Instructions: *TBD*
293
294This app id/token will be used in your `.env` described below
295
296## Setup for Development of an App With Existing Codebase
297
2981. Clone the relevant repository :)
299
3002. Create an `.env` file in the root directory of the repository. This file will simulate [config variables](https://devcenter.heroku.com/articles/config-vars) that Heroku provides when the app runs in production.
301
302 ```
303 RACK_ENV=development
304 SHOPIFY_APP_ENV=development
305 SHOPIFY_APP_WHITELIST_DOMAINS=skyverge-development.myshopify.com[,<your-shop-name>.myshopify.com[,<some-other-shop-name>.myshopify.com...]]
306 SHOPIFY_APP_API_KEY=<shopify api key for the development app>
307 SHOPIFY_APP_SECRET=<shopify api secret for said app>
308 # note: root url should NOT end with slash ('/')
309 ROOT_URL=https://<your-first-name>-<app-id>-skyverge.fwd.wf
310 # note: useful for bypassing port forwarding when dealing with slow requests in development mode
311 # ASSET_HOST=http://localhost:5000
312 SIDEKIQ_CONCURRENCY=5
313 MIN_THREADS=5
314 MAX_THREADS=5
315 # note: it's handy to increment this for each new app so you can have multiples running simultaneously
316 PORT=5000
317 # defaults to 15
318 RACK_TIMEOUT=<in seconds>
319
320 # the app name will be used for the namespace by default, or you can set a
321 # name here when running multiple instances in the same environment
322 REDIS_NAMESPACE=<some unique name>
323
324 # for GitHub auth to active admin/sidekiq admin
325 WARDEN_GITHUB_VERIFIER_SECRET=<random hex used for sidekiq's sessions>
326 GITHUB_AUTH_KEY=<client ID for the GitHub app, used for authentication>
327 GITHUB_AUTH_SECRET=<client Secret for the GitHub app, used for authentication>
328
329 # if developing the Help Scout custom app
330 HELPSCOUT_APP_SECRET=<help scout app secret key>
331 DEVELOPMENT_FORCE_SSL=false
332
333 # if developing the active admin Help Scout panel
334 HELPSCOUT_API_KEY=<help scout user API key>
335 HELPSCOUT_API_PASS=<help scout user password>
336
337 # if developing the in-app support widget (submit support requests, and join mailing list)
338 HELPSCOUT_MAILBOX=<help scout receiving mailbox id>
339 MAILCHIMP_API_KEY=<mailchimp api key>
340 MAILCHIMP_GROUP_ID=<mailchimp interest group id, e.g. for "Which app(s) would you like to hear about?". this is found by going to List - Manage Subscribers - Groups and inspecting the group name to get the group name id. Note that it is not the id shown in the URL or the "edit" link next to the group name>
341 MAILCHIMP_GROUPS=<comma separated list of mailchimp group names within the given interest group, e.g. "BlogFeeder,ShopStorm News">
342 MAILCHIMP_LIST_ID=<mailchimp list id, e.g. for the ShopStorm list, this is found in the list Settings page>
343
344 # if developing customer.io integration
345 CIO_SITE_ID=<customer.io test site id>
346 CIO_API_KEY=<customer.io test api key>
347
348 # if development kissmetrics integration
349 KMTS_API_KEY=<kissmetrics api key>
350
351 # if testing raven exception reporting
352 RAVEN_DSN=<raven test dsn>
353 RAVEN_ENVIRONMENTS=development
354
355 # if developing the sidekiq health monitoring rake task
356 SLACK_INCOMING_WEBHOOK_URL=https://hooks.slack.com/services/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
357 HEROKU_APP_ID=<heroku app id>
358 HEROKU_APP_TOKEN=<heroku app token>
359
360 # any app specific configurations (see individual app repos for details)
361 ```
362
363 **Rails env specific env vars** To add development-specific env vars (e.g. those that you want excluded from the text environment) create a second file named `.env.development`. This file will be loaded after the main `.env` file, overwriting any config vars set there.
364
365 **Connecting to the database** - Our standard configuration above assumes that you are using the default psql user (your username) with no password. If this is not the case, and you have a user with a password (_other than the SUPER ADMIN PG user_) you can add `PGUSER`, and/or `PGPASSWORD` to the `.env` file.
366
3673. To maintain dev/prod parity, you should start the app using [`heroku local`](https://devcenter.heroku.com/articles/heroku-local) which simulates how the app is run on Heroku. `heroku local` works by using a [Procfile](https://devcenter.heroku.com/articles/procfile) to create and manage processes that run the app. In development, we only use three processes, `web`, `forward`, and `worker`. You should create a local Procfile named `Procfile.dev` in the root directory:
368
369 ```
370 web: bundle exec puma -C config/puma.rb
371 forwarding: forward <HOST>:<PORT> <your-first-name>-<app-id>
372 worker: bundle exec rerun --background --dir app,db,lib --pattern '{**/*.rb}' -- env DB_POOL=$SIDEKIQ_CONCURRENCY bundle exec sidekiq --verbose -c $SIDEKIQ_CONCURRENCY
373 ```
374 Note: Your Procfile __must__ have the `web` process type before `forwarding` in order to work correctly.
375
376 Note: HOST is whatever the web worker is listening on, as reported during startup, e.g. `0.0.0.0` with `11:09:20 web.1 | [79348] * Listening on tcp://0.0.0.0:5003`
377
378 Sample `Procfile.dev` for the BlogFeeder app:
379
380 ```
381 web: bundle exec puma -C config/puma.rb
382 forwarding: forward 0.0.0.0:5000 justin-bf
383 worker: bundle exec rerun --background --dir app,db,lib --pattern '{**/*.rb}' -- env DB_POOL=$SIDEKIQ_CONCURRENCY bundle exec sidekiq --verbose -c $SIDEKIQ_CONCURRENCY
384 # Only for development: useful for showing the rails app logs directly in the heroku local logs
385 # log: tail -f log/development.log
386 ```
387
3884. Next install the required gems and dependencies:
389
390 From the project directory:
391
392 ``` sh
393 $ bundle install
394 ```
395
396 1. On OS X, it's possible the install will fail on the `pg` (postgres) gem with an error like:
397
398 ```sh
399 An error occurred while installing pg (0.20.0), and Bundler cannot continue.
400 Make sure that `gem install pg -v '0.20.0'` succeeds before bundling.
401 ```
402
403 or:
404
405 ``` sh
406 Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
407 ...
408 You have to install development tools first.
409 ```
410
411 It's possible that this is the result of the Apple Developer Tools license agreement not being accepted. Check this by running:
412
413 ``` sh
414 $ sudo gcc
415
416 You have not agreed to the Xcode license agreements. You must agree to both license agreements below in order to use Xcode.
417 Hit the Enter key to view the license agreements at '/Applications/Xcode.app/Contents/Resources/English.lproj/License.rtf'
418 ```
419
420 After agreeing, attempt `bundle install` again. If it _still_ fails, try installing manually, e.g:
421
422 ```sh
423 $ gem install pg -v '0.20.0' -- --with-pg-config=/Applications/Postgres.app/Contents/Versions/latest/bin/pg_config
424 ```
425
426 You'll need to re-run `bundle install` to ensure everything was installed correctly.
427
428 1. On OS X it's possible the install will fail on the `capybara-webkit` gem with something like `Command 'qmake -spec macx-g++ ' not available`. If so, simply install `qt` with brew:
429
430 ```sh
431 $ brew update
432 $ brew install qt
433 ```
434
435 You'll need to re-run `bundle install` to ensure everything was installed correctly.
436
437 1. Note that on Ubuntu you'll need a couple extra packages in order to successfully install all of the gems:
438 ```sh
439 $ sudo apt-get install libqt4-dev libcurl4-openssl-dev
440 ```
441
442 You'll need to re-run `bundle install` to ensure everything was installed correctly.
443
4445. From the command line run the following to create, migrate, and seed the database:
445
446 ``` sh
447 $ bundle exec rake db:create
448 $ bundle exec rake db:migrate
449 $ bundle exec rake db:seed
450 ```
451
452 **Important:** You must run these `rake db` commands individually, if you chain them like `bundle exec rake db:create db:migrate db:seed` then the ShopStormApp common migrations will not be run!
453
4546. Perform a quick test to make sure everything was installed properly:
455
456 ``` sh
457 $ heroku local -f Procfile.dev
458 ```
459
460 Open the app URL `https://<your-first-name>-<app-id>-skyverge.fwd.wf/login` in your browser to make sure it's rendering correctly
461
4627. You're finished!
463
464## Install Your Dev App Into Your Test Store
465
466To install the application on your dev-shop go to:
467
468 https://<your-first-name>-<app-id>-skyverge.fwd.wf/login?shop=<your-shop-name>.myshopify.com
469
470After installing, if you see an error like "This Connection is Untrusted" when your app tries to load, you may need to confirm a secuirty exception for your app url within your browser.
471
472### Firefox SSL Exception
473
474* Go to Settings > Advanced > Certificates
475* Location: `https://<your-first-name>-<app-id>-skyverge.fwd.wf`
476 Click "Get Certificate"
477 [x] Permanently store this exception
478 Click "Confirm Security Exception"
479
480### Chrome SSL Exception
481
482If you see the following when loading the local embedded app in Chrome:
483
484
485
486Visit `https://<your-first-name>-<app-id>-skyverge.fwd.wf/login` and you should see something like the following:
487
488
489
490Click "Show Advanced" then click "Proceed..." to accept the security exception, the page should load, and the embedded app should now be functional.
491
492## Optional Bonus Steps!
493
494In some instances, depending on where in the development lifecycle the app happens to be, you may need to use a version of the ShopStormApp gem other than the current release. If this is the case (ask the current dev lead for the project, unless you _are_ the dev lead, in which case you should probably already know this) then you'll need to clone the ShopStormApp branch required by the app and edit the Gemfile of the app to point to the local clone, e.g.:
495
496**Gemfile**
497
498```
499...
500
501# Our custom Shopify App (standard library)
502source 'https://Rz3outjzzgqCeSUJmgEe@gem.fury.io/skyverge/' do
503- gem 'shop_storm_app', '0.6.4'
504+ gem 'shop_storm_app', path: '../shopstorm-app' # Clone here and use this for development
505end
506
507...
508```
509
510And don't forget to run `$ bundle install` to update the `Gemfile.lock` to use the local gem.
511
512### Speeding up local development
513
514If you have less than awesome interwebs or the cats are clogging your pipes you can do the following:
515
5161. Change the following in your .env file:
517 * Set `DEVELOPMENT_FORCE_SSL=false` to stop forcing ssl for assets
518 * Set `ASSET_HOST=http://localhost:5000` or whatever your url is in your Procfile
5192. Restart your server
5203. Enable mixed content loading in your browser of choice
5214. Profit!
522
523### Redis namespacing
524
5251. If you need multiple versions of the app running simultaneously you can specify `REDIS_NAMESPACE` in your .env file. By default the app will use the app name along with the environment to determine the namespace.
526
527## Testing App Email
528
529* Preview at: `https://<your-first-name>-jilt-skyverge.fwd.wf/rails/mailers`
530* Rails logger (tbd more info)
531* in tests use a debugger to view the raw emails (tbd more info)