Author: Mihai Moldovanu

Creating a TFM Vagrant Box from Scratch

tfm logoCreating a Vagrant box from a TFM/GNU Linux image tutorial. It’s an easy 10 step tutorial that will alow you to have a development box based on TFM/GNU Linux

1) Create a new Virtual Machine in VirtualBox

Type: Linux
Version: Linux (64-bit)
Memory Size: 2048MB
New Virtual Disk:
Type: VMDK (Dynamically allocated)
Max Size: 40GB

2) Modify Virtual Machine settings:

* Disable audio
* Disable USB
* Ensure Network Adapter 1 is set to NAT
* Mount the TFM Linux ISO

3) Add the vagrant group and user and set password to vagrant

groupadd vagrant
useradd -m -g vagrant
passwd vagrant

4) Modify /etc/sudoers

Defaults:vagrant !requiretty
#Defaults !visiblepw
Defaults env_keep = “SSH_AUTH_SOCK”
vagrant ALL=NOPASSWD: ALL

5) Add vagrant ssh key to allow logins

mkdir -p /home/vagrant/.ssh
chmod 0700 /home/vagrant/.ssh
wget –no-check-certificate \
https://raw.github.com/mitchellh/vagrant/master/keys/vagrant.pub \
-O /home/vagrant/.ssh/authorized_keys
chmod 0600 /home/vagrant/.ssh/authorized_keys
chown -R vagrant /home/vagrant/.ssh

6) modify /etc/ssd/sshd_config

Port 22
PubKeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys
PermitEmptyPasswords no
PasswordAuthentication no

7) Install the Guest Tools for our VM package.
* From Virtualbox Install Guest Additions…
* from linux prompt:

mkdir /mnt/cdrom
mount /dev/cdrom /mnt/cdrom
/mnt/cdrom/VBoxLinuxAdditions.run
umount /mnt/cdrom

8) Adjust the base settings of the configuration

echo “dhclient eth0” > /etc/init.d/local
echo “nameserver 8.8.8.8” >/etc/resolv.conf
ln -s /etc/init.d/local /etc/rc.d/rc3.d/S99local
ln -s /etc/init.d/vboxadd /etc/rc.d/rc3.d/S25vboxadd

9) Shutdown the TFM guest OS .

shutdown -h now

10) Pack and distribute the image

vagrant package –base vagrant-tfm

vagrant box add dev-new http://hq.tfm.ro/vagrant/vagrant-tfm.box
vagrant init dev-new
vagrant up

So … Happy developing .

Some thoughts ( depending on what you want that machine for ):
– You will probably want to mount a local directory into VM
– You probably want to start nginx / mysql / php-fpm
– Maybe the you want to add the logs on the local machine …

Ubuntu 14.04 adding HP proliant support pack (hpacucli problem solved)

tfm_logoIn the begining there was nothing but few baremetal. After a while someone delivers a whole bunch of baremetal on your doorstep and say : “I need them installed by tomorrow”. Same configuration … With the harddisks in raid and ubuntu on all of them . What do you do ?

It’s a big problem. Like most big problems you split it in lots of little problems that can be managed easier.
What you need to do for one baremetal:

  1. Update the ILO firmware and Bios (if necessary) . This will come handy: https://play.google.com/store/apps/details?id=com.hp.essn.iss.ilo.iec.spa&feature=search_result . I’m not going into details about it in this post
  2. Create the disk arrays
  3. Install the operating system on it
  4.  Configure it and deploy it.

For one server let’s say you can do it in few hours , few beers and some pizza’s. But … wait a minute .. There are a LOT of baremetals to be installed. One option is to call some friends and do that while you watch a movie.

OR you can be smart and automate the tasks. How ? What i need ?

You need a baremetal installer server or a laptop or a virtual something (virtual box / vmware / you choose )  image that will do the job for you while you sit back and relax.

The ideea is simple:

  • Baremetal will boot from network
  • tftp server will deliver the boot image, boot it , get an ip addres from the dhcp server and register the new server into the baremetal installer and will fill the hardware configuration there.
  • Then you can ( using ansible  ) to actually do the raid configuration ,  bios updates ,  firmware updates, and operating system install.
  • Add the necessary configurations.
  • Once complete the system will boot from raid and you have a system up and and running ready to be deployed.

Now … Back to the post subject .

How you can configure HP raid from inside  ubuntu ? In order to have in the network bootable image the proper tools to do actually do the configurations…

First we install hpacucli :


sudo echo "deb http://downloads.linux.hp.com/SDR/downloads/MCP/ubuntu precise/current non-free" >>/etc/apt/sources.list
wget http://downloads.linux.hp.com/SDR/downloads/MCP/GPG-KEY-mcp
sudo apt-key add GPG-KEY-mcp
sudo apt-get update
apt-get install cpqacuxe hp-ams hp-health hpacucli hponcfg
service hpsmhd stop
update-rc.d hpsmhd disable
hpasmcli -s "show server"

Then when we boot the new baremetal to be installed we can gather the informations about the raids:


hpacucli ctrl all show config

That will produce an output like ( in this case i already configured the raid:


Smart Array E200i in Slot 0 (Embedded) (sn: VX9AMP1927 )

array A (SAS, Unused Space: 0 MB)

logicaldrive 1 (136.7 GB, RAID 1, OK)

physicaldrive 1I:1:1 (port 1I:box 1:bay 1, SAS, 146 GB, OK)
physicaldrive 1I:1:2 (port 1I:box 1:bay 2, SAS, 146 GB, OK)

At this point you can create some scripts that will create the partitions in the way that the shareholder wants them.

for example:


hpacucli ctrl slot=9 create type=logicaldrive drives=1I:1:3,1I:1:4 raid=1

Custom version of nginx in Ubuntu

Building a custom ( adding a custom plugin )  version of nginx in debian is not a very complicated job:

Create a directory  ( ex test2 ) and change directory to it.

After that do:

apt-get source nginx

in debian directory you should modify rules file , then if necessary  source/include-binaries , add your module in modules directory and modify README.Modules-versions to reflect your changes .

Change back to test2/nginx-1.4.1 directory and run

dpkg-buildpackage -b

If everything works ok and module is configured correctly you should have it ready to be run:

dpkg -i nginx-full_1.4.1-3ubuntu1.3_amd64.deb nginx-common_1.4.1-3ubuntu1.3_all.deb

 

Enjoy.

Cloudera CDH5 adding mysql drivers to sqoop

I installed a Cloudera hadoop CDH5 cluster ( 4 mchines )  and started to play with it .

However soon i ran into a weird problem. When i tried to import some data from mysql using sqoop i realized that the mysql drivers weren’t installed .

Installation was done using parcels . So i after reading a lot of documentation i realized that the directory where i was supposed to put the mysql jar is:

/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/sqoop2/webapps/sqoop/WEB-INF/lib

I used mysql-connector-java-5.1.30-bin.jar .

If the file is not there … It wont be deployed .