How to Setup Sitecore 8 on Windows Azure IaaS: Part One Mongo DB

Vasiliy Fomichev

In Azure, How To, Installation Posted January 07, 2015

Microsoft has been putting a lot of effort into making Windows Azure its flagship, and, to be honest – rightly so! Recently, during the Sitecore 8 Technical Preview program, and in the light of the upcoming Los Angeles User Group, at which I was planning on speaking,  – I decided to see if I could put Sitecore on Azure. This is a two-part series where we will look at the exact steps of first setting up the MongoDB for the Sitecore xDB, and then pushing Sitecore 8 with all of its SQL databases up to Windows Azure.

Why Choose Windows Azure for Sitecore?

There is currently no other cloud provider that can match Azure in hosting .NET solutions (this may actually change with a recent news release of .NET going open-source, but challenging Azure at this point is like tackling Jupiter, you just won’t get far with that).

When Azure first came out – what separated it from other providers is the offering of Worker and Web Roles, the .NET and IIS environments optimized for .NET code. Although starting off slowly, Azure continued to grow steadily into what now rivals most of the existing on-premise setups and data-centers. Where before many companies hesitated to switch to Azure because of lack of support for certain platforms or services, now with virtually all most common technologies and services being supported – watch out Sys Amins – Azure is coming for ya!

There is one flag that people seem to raise – Azure is expensive. Is it? In relation to what?  All too often managers seem to forget the total cost of ownership of any infrastructure. Just think about the real estate space, the hardware, the software licenses, the maintenance, electricity (oh yes – servers get hungry!), disaster recovery, backup systems, technician and system admin salaries…etc. As you can see, this can add up quickly. So is $12 per day for a distributed four-server (two-Windows and two-Linux), and five-database scaled environment really that expensive?

If you eventually decide to switch to Azure, make sure to check out my other blog post on evaluating which Azure Environment is right for you.

How to Setup Sitecore 8 xDB on Azure IaaS

Introduction

Azure has been attracting a lot of excitement in the development world, and the Sitecore community is no exception. When I registered to speak at the Fall LASUG, my original plan was to speak on Azure and Sitecore 7.2, but then the Sitecore North America Symposium 2014 happened… Everyone seemed to have forgotten about Azure being in awe from the new amazing featured and UI of Sitecore 8.

Of course I always want to stay relevant, so speaking strictly about Azure felt a bit outdated to me. To be relevant – it had to be related to Sitecore 8. So I started kicking around a few topic ideas. Sitecore 8 was definitely cool, I thought, but then it dawned on me – but Sitecore 8 on Azure is even cooler!

That was it – I knew that was what I wanted to speak about, the problem was to actually trying to make it work. The Sitecore xDB Cloud Edition that is supposed to support the xDB is not out as of the time of this writing, and just like with any release dates it’s TBD.

I had under a week left to come up with a solution and present it at the LASUG. Although I always found times like this exciting, this time there was a lot of unknown – Sitecore 8, xDB, Mongo – it was time to get to work!

My plan was to first tackle the biggest unknown – Mongo DB. I had very little experience with the “new popular kid”, so it was time to get to know it. After doing some quick research, I found plenty of information on Mongo DB, however, for the purpose of Sitecore enthusiasts, I would suggest to start with mongodb.org and move on further from there, if needed.

After some trial and error (yep, tutorials are rarely accurate)  I conquered the Mongo milestone, and focused on setting up Sitecore 8. This is where I hit possibly the biggest snag with pushing the Sessions SQL database to SQL Azure, but with a little wizardry, I got the entire setup up and running in a little under two days.

The following tutorial will cover a step-by-step setup of Sitecore xDB on Azure IaaS. The first article will cover setting up the MongoDB and the second –Sitecore. So let’s get going!

Tools Used for the Tutorial

In this tutorial we will be using the following tools:

• Putty  – used for accessing the RHEL servers remotely using SSH
• WinSCP – although not required, it may come in handy for file transfers using SFTP from your local machine to the remote servers, or vice versa.

Guide on Setting Up Mongo DB on Azure Infrastructure as a Service (IaaS)

The Mongo DB setup can be broken into the following stages:

1. Linux Virtual Machine Setup on Azure
2. Disk Attachment to Linux Virtual Machine
3. Mongo Installation on Azure Virtual Machine
4. Replica Set Setup for Mongo Databases
5. Mongo Database Arbiter Setup on a Virtual Machine

For the purpose of the tutorial we will be setting up a simple two-server Mongo environment configured for replication with one arbiter.

Why is Mongo Arbiter Important?

There is a bit of confusion about the function of an arbiter instance. First, it is not a “Mongo traffic router” or “Mongo load balancer”, its only purpose is to ensure that when one of the primary Mongo instance goes down – the election of the next one always succeeds via ensuring the majority of votes.  For instance, if we had a three server setup, where one is primary and the other two -secondary, and if the primary one were to go down, the other two may not be able to vote out a primary instance by majority of vote, because of the even number of participants. In this exact situation an arbiter would step in and give an extra vote to ensure the election succeeds.

That makes sense, right? However, why do we need an arbiter with a two server setup? Shouldn’t the second server become primary when the original primary fails, since it would be the only one left? – it turns out that this is one of the exceptional situations where when a secondary server cannot elect itself to be the primary, if it’s the only one left in the replica set, therefore, we would need an arbiter to save the day. This is all part of Mongo’s failover process.

I cannot stress enough how minimal the resource use is by the arbiter; therefore, if possible, install it on a server with very low resources, or even on a server whose primary function is something else. Don’t worry, it will likely not affect the server performance at all (remember what its function is – it only steps in when a primary server goes out to cast a vote – that’s it!)

For more information on Mongo Arbiter check out article on Mongo Replica Sets.

Linux Virtual Machine Setup on Azure

At this point you should already be logged into Azure, so let’s create a new Linux virtual machine. I personally prefer CentOS, as it is the closest one to Red Hat Linux Enterprise, which is near and dear to me. In this tutorial we will use with the latest version of CentOS offered on Azure – version 7.0.

In the Azure portal click on +NEW  –> COMPUTE –> VIRTUAL MACHINE –> FROM GALLERY

The OS selection should appear. In the left navigation click on CENTOS-BASED and in the list of operating systems select OpenLogic 7.0.

Sitecore xDB on Azure CentOS

On the next screen give the virtual machine a name, for the purpose of the tutorial I will call this machine AzureMongo1. Also for the purpose of the tutorial I chose the Basic type A2 for replica members and A1 for the arbiter (please check with the Azure Virtual Machine Comparison table to choose the option that’s right for you). I would also suggest to modify the default username (never use default usernames anywhere for security purposes), and give it a secure password (uncheck the Upload Compatible SSH Key for Authentication checkbox).

Azure Virtual Machine Setup

On the next screen – page three of the Virtual Machine Configuration, configure the cloud service according to your needs, and add an extra endpoint under ENDPOINTS with the following settings –

Sitecore xDB on Azure CentOS Settings

Notice we are not using the default MongoDB port – 27017, this is also for security purposes, although, it’s not far from the original port and is rather security by obscurity, I believe that the devil is always in the details.

After getting the service configured, click next and confirm the machine creation. At this time you have about 10-15 minutes you could use to get a cup of coffee.

Once the virtual machine has been created, go ahead and connect to it using SSH. My tool of choice for this is putty.

After logging in, first things first – let’s make sure our virtual machine is “up2date” (does anyone miss this command?)

sudo yum update

That’s it for the machine setup, let’s now attach an empty disk.

Disk Attachment to Linux Virtual Machine

Although we could install Mongo on the same disk as the OS, it’s always best to put mission critical applications on a separate drive for security, availability, and maintenance purposes, and since in Azure it’s a piece of cake – we will definitely follow this practice.

So let’s navigate to the Virtual Machines view of the Azure portal and click on the row of AzureMongo1 to highlight it and click Attach Disk in the bottom command menu. The Virtual Machine Name, Storage Location and File Name should be generated automatically, so the only things that we need to provide are the Size and Host Cache Preference, so let’s set the first to 10 GB (minimum suggested) and READ/WRITE for caching and submit. The disk attachment should only take about a minute – a few minutes at the most.

Once the disk has been attached, go ahead and SSH into the virtual machine. What we now need to do is format the newly added disk and mount it to a folder (yes, Linux is very particular 🙂 ).

First, let’s find the identifier on the last attached disk by running the following command –

sudo grep SCSI /var/log/messages

We can see from the screenshot below that the last attached disk identifier is sdc, because it’s the last line in the log; so let’s continue with creating a partition on it:

 sudo fdisk /dev/sdc

Creating Filesystem in Newly Attached Disk

Type n to create a new partition.

Type p to see the details about the disk that is being partitioned. (or sudo  fdisk -l to find the name of the new partition)

Type 1 to make it the first partition

Hit Enter twice to accept the first and last sector default settings

Type w to write the changes to the disk.

Successful Filesystem Creation

Windows users are already likely rolling their eyes, but hold your horses, we are almost done. Let’s format our new partition with ext4 file system –

 sudo mkfs -t ext4 /dev/sdc1

Next let’s make a new directory to which we will be mounting our new drive –

sudo mkdir /mongodrive

and let’s mount it –

sudo mount /dev/sdc1 /mongodrive

To ensure the drive is re-mounted automatically after each reboot it must be added to the /etc/fstab file. In addition, it is highly recommended that the UUID (Universally Unique IDentifier) is used in /etc/fstab to refer to the drive rather than just the device name (i.e. /dev/sdc1). To find the UUID of the new drive you can use the blkid utility:

sudo -i blkid

The output will look similar to the following:

Fstab Content

NOTE:

Improperly editing the /etc/fstab file could result in an unbootable system. If unsure, please refer to the distribution’s documentation for information on how to properly edit this file. It is also recommended that a backup of the /etc/fstab file is created before editing.

Next, open the /etc/fstab file in a text editor.

sudo vi /etc/fstab

In this example we will use the UUID value for the new /dev/sdc1 device that was created in the previous steps, and the mountpoint /mongodrive. Add the following line to the end of the /etc/fstab file:

UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx   /mongodrive ext4   defaults   1   2

(make sure to use the proper UUID given by the blkid command)

You can now test that the file system is mounted properly by simply unmounting and then re-mounting the file system, i.e. using the example mount point /mongodrive created in the earlier steps:

sudo umount /mongodrive
sudo mount /mongodrive

If the mount command produces an error, check the /etc/fstab file for correct syntax. If additional data drives or partitions are created you will need to enter them into /etc/fstab separately as well.

That’s it for the disk it should be ready to, next let’s install MongoDB.

How to Install Mongo on an Azure Virtual Machine

Good news! The database installation could not have been made any easier! There are many ways to install Mongo, but I would suggest pulling the latest from the mongodb.org repo. Let’s see how we can do that –

We should still be SSH’ed into the virtual machine. Let’s add a new repository definition file –

sudo vi /etc/yum.repos.d/mongodb.repo

and populate it with the folling configuration  –

[mongodb]
name=MongoDB Repository
baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/x86_64
gpgcheck=0
enabled=1

After saving the new .repo file, issue the following command to update the local package database –

sudo yum update

Lastly, run the following command (as root or with sudo) to install the latest stable version of MongoDB and the associated tools –

sudo yum install mongo-10gen mongo-10gen-server

Successful Mongo Installation

Suring the installation you will have to hit “y” to confirm the packages about to be installed, so don’t walk away just yet to get another cup of coffee. Once CentOS finishes running through the setup, Mongo should be installed. Next, we need configure it.

You should still be logged into the Linux box using SSH. As sudo, edit /etc/mongod.conf to set the following parameters:

port = 27018
dbpath = /mnt/mongodrive/data
logpath = /mnt/mongodrive/mongod.log
replSet = mongors

and comment out the bind_ip=127.0.0.1 setting to allow access from remote machines. My editor of choice is vi, only because there isn’t much choice (note the replica set setting; after starting mongo, you may see some logs related to the replica set not being configured; these will go away after the set is properly configured).

Using VI: vi is the grandfather of text editors. It stands for Visual Editor. Just think how bad things had to be for them to call this “visual”.  So in the midst of those hard times, a Unix developer travelled to a parallel universe and when came back to create the key combinations for the editor. According to the direction from the parallel universe vi opens files in read-only mode. To enter the editing mode hit “i”. Now you should be able to use editing controls like left arrow, right arrow, delete, backspace… don’t get too carried away there though, as many controls you are used to don’t work. Once finished, you want to hit Esc to return to the read-only mode, then hit type colon (: ) to enter the command mode and then type in “wq”, which stands for write and quit and hit enter. Wasn’t that intuitive? By the way, to quit without saving changes you cannot simply type in “q”, you would actually have to get into the command mode and type “q!”, a forced quit.

Next we need to make sure that the user that Mongo will be running under has proper rights to the dbpath folder –

sudo chown –u username /mongodrive

Let’s now create a folder for the database –

mkdir /mongodrive/data

And again ensure that our user is the owner –

sudo chown –u username /mongodrive/data

Now, let’s ensure that our user can also create the .pid file under the path specified in mongo.conf in the pidfilepath setting (/var/run/mongodb/mongod.pid is default) –

chmod  -u username 777 /var/run/mongodb

And finally let’s make sure that the CentOS firewall allows traffic via the port 27018 –

firewall-cmd --add-port=27018/tcp --permanent
firewall-cmd –reload

At this point we should be good to go, so let’s start our MongoInstance for the first time –

mongod --config /etc/mongod.conf

Once the server started successfully, you should return back to the command promt.

It may take a few minutes the first time around; however, if you don’t get it after say five minutes, Ctrl+X out of the command and check the log file for any errors (I ran into some permission issues in one the past installs) –

tail --f /mnt/mongodrive/mongod.log

The “waiting for connections” message in the log file indicates mongod is up and running and waiting for client connections.

 ***At this point before moving to the replica set setup, we should repeat all the above installation steps to setup the second Mongo DB and the arbiter instance. Only after all the instances are successfully setup and have Mongo DB running – proceed to configuring the replica set and selecting the arbiter instance.

Replica Set Setup for Mongo Databases

The replica set setup needs just one final polish. We have already defined a replica set in /etc/mongo.conf and the Mongo log file should show messages related to connection attempts to that set. To make this set function properly, we need to select the primary instance.

Let’s SSH into the first Linux instance and connect to Mongo –

mongo --port 27018

We should now be given a Mongo prompt “>”. Remember that Mongo syntax is JavaScript-based, so keep that in mind for the next couple of steps.

Let’s first define the variable that will carry our replica set configuration –

conf = {
_id : "mongors",
members : [
           {_id:0, host:"sc8mongo1.cloudapp.net:27018"},
           {_id:1, host:"sc8mongo2.cloudapp.net:27019"}
          ]
}

and now we need to simply load it  as follows –

rs.initiate(conf)

This command should initialize the replica set, which can be checked by running –

rs.status()

Make sure that one of the instance shows up as primary and the other – secondary.

Successful Mongo Replica Set Configuration Status

 Mongo Database Arbiter Setup on a Virtual Machine

Adding an arbiter as many things in Mongo is much easier than it sounds (it’s usually the other way around isn’t it?). We should still be connected to the primary instance of the Mongo replica set, so to add the arbiter, simply run the following command –

rs.addArb("m1.example.net:30000")

That’s it! Yep! We now have a two-instance Mongo replica set running with an arbiter. There are many more settings that are available in Mongo, as we have obviously only scratched the surface, so I encourage developers to do their due diligence and look through the documentation on mongodb.org for more information. For the purpose of the tutorial, this setup is sufficient and is ready to be connected to Sitecore.

In the second part, we will look at the steps of setting up Sitecore 8 on Azure IaaS with SQL Azure, the challenges and ways to overcome them. Cheers!