Couchbase Cluster and NodeJS app with Docker on CoreOS

I have been working on more samples for hipops, and since couchbase is my new favorite no-sql storage, I wanted to build a POC hipops demo with nodeJS. In this demo, we build a 3-cluster couchbase server on CoreOS using docker containers, and then we deploy our beer nodejs app to the fourth server pointing it back to the cluster. Before we get started, you can find the finished code here. Let's get started.


hipops is a Docker Orchestration based on a JSON Configuration. The idea behind hipops configuration is to define a series of apps and re-use them in playbooks, so you just focus on the orchestration of your containers, whether across different physical hosts or the same host.

Now that you know what hipops is, let's take a look at the config.json for this demo.

{
  "id": "cbs-n",
  "description": "couchbase-server + nodejs app",
  "env": "dev",
  "dest": "/data",
  "oses": [{
    "user": "core",
    "pythonInterpreter": "PATH=/home/core/bin:$PATH python"
  }],
  "apps": [{
    "name": "couchbase-server",
    "type": "db",
    "image": "aminjam/couchbase:latest",
    "cred": {
      "username": "Administrator",
      "password": "password"
    }
  }, {
    "name": "couchbasenode-app",
    "type": "nodejs",
    "host": "cbnodejs-demo.com",
    "repository": {
      "branch": "master",
      "sshUrl": "github.com/aminjam/beersample-node.git"
    },
    "customizations": [{
      "src": "https://gist.githubusercontent.com/aminjam/ffc6c8e9c7a709a05a34/raw/d8a6531985fc36720b6ff251544e274d967ddd8b/nodejs-start-app-with-make_g++_node-gyp.sh",
      "dest": "custom-start-app.sh",
      "mode": 744
    }],
    "image": "aminjam/nodejs:latest"
  }],
  "playbooks": [{
    "inventory": "tag_App-Role_CB-SERVER-00",
    "apps": ["{{index .Apps 0}}"],
    "containers": [{
      "params": "--net=host -v {{.App.Dest}}:/home/app -e COUCHBASE_USER={{.App.Cred.Username}} -e COUCHBASE_PASS={{.App.Cred.Password}} -e COUCHBASE_DATA=/home/app -d {{.App.Image}} cluster-init"
    }, {
      "params": "--name beer-sample --rm {{.App.Image}} cbdocloader -n $COUCHBASE_SERVER:8091 -u {{.App.Cred.Username}} -p {{.App.Cred.Password}} -b beer-sample /opt/couchbase/samples/beer-sample.zip"
    }]
  }, {
    "inventory": "tag_App-Role_CB-SERVER",
    "apps": ["{{index .Apps 0}}"],
    "containers": [{
      "params": "--net=host -e COUCHBASE_USER={{.App.Cred.Username}} -e COUCHBASE_PASS={{.App.Cred.Password}} -e COUCHBASE_SERVER=$COUCHBASE_SERVER -d {{.App.Image}} rebalance"
    }]
  }, {
    "inventory": "tag_App-Role_CBnodejs-app",
    "apps": ["{{index .Apps 1}}"],
    "state": "deploying",
    "containers": [{
      "params": "-v {{.App.Dest}}:/home/app -v {{.App.Dest}}/custom-start-app.sh:/start-app.sh -e VIRTUAL_HOST={{.App.Host}} -e NODE_ENV=development -e COUCHBASE_SERVER=http://$COUCHBASE_SERVER:8091 -d {{.App.Image}}"
    }]
  }, {
    "inventory": "tag_App-Role_CBnodejs-app",
    "containers": [{
      "params": "--name nginx-proxy -v /var/run/docker.sock:/tmp/docker.sock -p 80:80 -d aminjam/nginx-proxy:tiny"
    }]
  }]
}

Let's walk through what this configuration means. Starting with the apps, we are defining two apps for this demo. couchbase-server and couchbasenode-app are fairly straight forward. The only thing worth noting is that couchbase npm module uses node-gyp, NodeJS app will also have a customized script when running the docker ensuring g++, make, and node-gyp installation on the ubuntu container. We can do that with hippops under customizations section to pull in the custom scripts from github gist, and attaching it to the runtime docker container. Also it’s worth noting that we are using the attached beer-sample.zip bucket with couchbase installation, but if you have your own content, you can use the customizations section to attach the bucket data .zip file at the runtime instead.

Let’s move to the playbooks section. The first playbook will use the first server to initialize the couchbase cluster and then load the beer-sample.zip bucket data into the first server. The second playbook will deploy a couchbase server to the rest of the cluster with the IP address of the first server, so the new servers can rebalance the load. It’s worth noting that couchbase has a built-in master-master replication with consistent hashing built in! That’s rad!

Now that we have deployed our couchbase cluster, let’s move to our nodejs app. The third playbook will be attached to the second app (couchbasenode-app), retrieves the source code, and deploys the docker container passing in the IP address of the couchbase cluster. Finally, the fourth playbook will run a nginx-proxy docker container to pass through the traffic for host cbnodejs-demo.com to the nodejs container. Let’s run this configuration and see how it works.

Setup

Before we continue with the setup, let’s make sure you have hippos setup and running properly, and for this demo we are using the CoreOS vagrant machines, so make sure that the vagrant machines are running.

Now that we have setup hippos, let’s run this scenario.

cd scenarios/cbs-n

COUCHBASE_SERVER=172.17.8.101 hipops exec -plugin ansible -private-key ~/.vagrant.d/insecure_private_key -playbook-path /PATH/TO/hipops-playbooks/ansible  

That’s it. Let’s make sure your hosts file is setup properly with 172.17.8.104 cbnodejs-demo.com. If you browse to

http://172.17.8.101:8091 is pointing to the first machine in the couchbase cluster
http://172.17.8.102:8091 is pointing to the second machine in the couchbase cluster
http://172.17.8.103:8091 is pointing to the third machine in the couchbase cluster
http://cbnodejs-demo.com: will bring your nodejs beer-sample app connecting to the cluster.

Integrating hipops with your current CI engine (e.g. Jenkins) is very easy, checkout hipops wiki for more information on how to get started. If you are happy with deployment, you could easily deploy the same CoreOS cluster to AWS by switching the hosts file, and making sure App-Role tags are setup properly.

COUCHBASE_SERVER=IP.ADDR.OF.CB-SERVER-00 hipops exec -plugin ansible -private-key /PATH/TO/AWS.pem -playbook-path /PATH/TO/hipops-playbooks/ansible -inventory /PATH/TO/EC2.py  

If you just want to try out hipops without installation, you can do so by getting the docker image instead.

comments powered by Disqus