Tuesday, June 14, 2016

Running your own help desk platform with docker and OTRS


At work we use OTRS for our help desk platform. We chose it because it's open source and very flexible, and we could install it on our premises to have more control. So, I went ahead and made a set of docker containers that we have been running multiple OTRS  4 .0.x installations for small companies for more than a year now without issues. Now I've had some time to upgrade the containers to OTRS 5.

The first thing to know is that this is an unofficial OTRS docker container.

For setting up an OTRS system you need several services:
  • A web server with the OTRS installation.
  • A database server.
  • An SMTP server.
  • A proxy server (optional).
This container setup is designed that way. It uses:

The docker-compose configuration files include all of those services, and upon container start a fresh OTRS install will be started, ready to to be configured by an OTRS administrator.

There are some environment variables you can use to control the container startup and initial state. For example, the container can be started in three ways, controlled by the OTRS_INSTALL environment variable:
  • OTRS_INSTALL=no when the container is run, it will load a default vanilla OTRS installation that is ready to be configured as you need. This is the default. 
  • OTRS_INSTALL=yes will launch the OTRS install web interface at http://localhost/otrs/install.pl.
  • OTRS_INSTALL=restore Will restore the backup specified by the OTRS_BACKUP_DATE environment variable. OTRS_BACKUP_DATE is the backup name to restore, in the same date_time format that the OTRS backup script uses, for example OTRS_BACKUP_DATE="2015-05-26_00-32". Backups must be inside the /var/otrs/backups directory (you should host mount it).
You need to mount that backups volume from somewhere, it can be from another volume (using --volumes-from) or mounting a host volume which contains the backup files.

For testing the containers you can bring them up with docker-compose:

    sudo docker-compose build
    sudo docker-compose up

This command will build all containers and pull missing images, bring up all needed containers, link them and mount volumes according to the docker-compose.yml configuration file:

version: '2'
services:
  otrs:
    build:
      context: otrs
      args:
        OTRS_VERSION: 5.0.10-01
    ports:
    - "80:80"
  # If running behind a proxy container, expose the ports instead
  # and link the proxy container to this one.
  #  expose:
  #  - "80"
    links:
    - mariadb:mariadb
    - postfix:postfix
    volumes_from:
    - data

  mariadb:
    build:
      context: mariadb
    expose:
    - "3306"
    volumes_from:
    - data
    environment:
        MYSQL_ROOT_PASSWORD: changeme
  postfix:
     image: juanluisbaptiste/postfix:latest
     expose:
     - "25"
     env_file: credentials-smtp.env
  data:
    image: centos/mariadb:latest
    volumes:
    - /var/lib/mysql
    - "./otrs/backup:/var/otrs/backups"

    command: /bin/true

The default database password is changeme, to change it, edit the docker-compose.yml file and change the MYSQL_ROOT_PASSWORD environment variable on the mariadb image definition before running docker-compose.

To start the containers in production mode use this docker-compose.yml file that points to the latest version of all images to be pulled and run instead of Dockerfiles to be built:

sudo docker-compose -f docker-compose-prod.yml pull


sudo docker-compose -f docker-compose-prod.yml -p company_otrs up -d  

After the containers finish starting up you can access the administration and customer interfaces can be accessed at following addresses:

Administration Interface

    http://$OTRS_HOSTNAME/otrs/index.pl

Customer Interface

    http://$OTRS_HOSTNAME/otrs/customer.pl

There are also some other environment variables that can be set to customize the default install, like root and database passwords, language, theme, ticket counter start, number generator, etc, check the github page for more info. OTRS 4 sources are still available in otrs-4_0_x branch.


Monday, May 16, 2016

How to mount VirtualBox shared folders at boot

I'm setting up a network install server on a VirtualBox VM, and I didn't want to copy the contents of the iso images of the distros going to be available through PXE to avoid having a huge VM. Worse if I also included other repos like epel or centosplus, or updates, so I created a VirtualBox shared folder pointing to the directory containing the iso images. After having created the shared folder, you can mount it like this:

sudo mount -t vboxsf -o uid=$UID,gid=$(id -g) share ~/host

To avoid having to manually mount the share each time the VM boots , the shared mount needs to be added to /etc/fstab, but there's a catch: the vboxsf kernel module, needed to mount the shared folder, isn't available when mounting all filesystems during the boot process. So, to fix this we need to make sure the vboxsf module is loaded before the filesystems mount at boot.

On CentOS 7, create a file on /etc/sysconfig/modules directory ending in .modules and add this to load VirtualBox kernel module before filesystems are mounted:

#!/bin/sh

    lsmod |grep vboxsf >/dev/null 2>&1
    if [ $? -gt 0 ] ; then
      exec /sbin/modprobe vboxsf >/dev/null 2>&1
    fi

On Ubuntu/Debian, add the module name to /etc/modules. Now we need to add the shared mount to /etc/fstab. In my case, my shared folder is called isos, so I added the following line:

isos    /isos   vboxsf  defaults        0 0

After adding this line you can reboot the server/vm and see if it mounted correctly at boot.

If you want to mount the iso images too at boot, add a line like this one to /etc/fstab, for each iso to mount:

/isos/CentOS/CentOS-7-x86_64-DVD-1511.iso /distros/centos7 iso9660 loop 0 0

Remember to adjust loopback device limits if you plan to mount more than 8 or 10 images (don't remember right now the limit).


Thursday, December 31, 2015

Using a portable SMTP relay server with docker

I have had a very busy year and had the chance to work on a lot of new and useful docker containers. Taking advantage of these holidays, I finally started to catch up here with my latest work on them :D

There's one thing that it's a pretty common requirement of a website: a SMTP email server or relay.
Every web app needs one for different tasks, like sending out notifications, registration emails, password resets, etc. I made a Postfix SMTP relay container that is easy to use with other containers.

Before running the container, first you need to set the following environment variables to configure the SMTP relay host:
  • SMTP_SERVER: Server address of the SMTP server that will send email from our postfix container.
  • SMTP_USERNAME: Username to authenticate with.
  • SMTP_PASSWORD: Password of the SMTP user.
  • SERVER_HOSTNAME: Server hostname for the Postfix container. Emails will appear to come from this hostname domain.
To use it you need to first pull the image:
docker pull juanluisbaptiste/postfix
and then fire it up with the previous variables defined:
docker run -d --name postfix -P \
       -e SMTP_SERVER=smtp.bar.com \
       -e SMTP_USERNAME=foo@bar.com \
       -e SMTP_PASSWORD=XXXXXXXX \
        -e SERVER_HOSTNAME=helpdesk.mycompany.com \        
        juanluisbaptiste/postfix
Lastly, link your container against it:
docker run --name mycontainer --link "postfix:postfix" myimage

Using docker-compose

Or, you could use docker-compose to start your application containers linked against the postfix container with one command. Suppose you have a web application that links against a database and this postfix container. Download and install docker-compose for your platform, and then on your website's docker project, create a new file called docker-compose.yml, and put the following contents there:
myapp:
  build: myapp
  ports:
  - "80:80"
# If running behind a proxy container, expose the ports instead
# and link the proxy container to this one.
#  expose:
#  - "80"
  links:
  - mariadb:mariadb
  - postfix:postfix
  volumes_from:
  - data
mariadb:
  image: centos/mariadb:latest
  expose:
  - "3306"
  volumes_from:
  - data
  environment:
      MYSQL_ROOT_PASSWORD: changeme
postfix:
   image: juanluisbaptiste/postfix:latest
   expose:
   - "25"
  environment:
      SMTP_SERVER: smtp.mycompany.com
      SMTP_USERNAME: user@mycompany.com
      SMTP_PASSWORD: changeme
      SERVER_HOSTNAME: helpdesk.mycompany.com
data:
  image: centos:latest
  volumes:
  - /var/lib/mysql
  - /var/www/webapp

  command: /bin/true
Then, you can launch your webapp, the database and the postfix container with this command:
docker-compose up
All containers will be started in the right order and the webapp will be linked against the mariadb and postfix containers. Also, the webapp and the mariadb database container will share the same data volume container (unrelated to the postfix container but a good practice).

On thing to note, this container doesn't enable client SMTP authentication, the idea is to expose the port 25 to containers, and then link the containers that need a SMTP service against it so the relay isn't publicly exposed.

A note about using gmail as a relay

Since last year, Gmail by default does not allow email clients that don't use OAUTH 2
for authentication (like Thunderbird or Outlook). First you need to enable access to "Less secure apps" on your google settings.

Also take into account that email From: header will contain the email address of the account being used to authenticate against the Gmail SMTP server (SMTP_USERNAME), the one on the email will be ignored by Gmail.

Friday, February 13, 2015

Setting up a BigBlueButton 0.81 docker container: Part 2

I have made some improvements on my BigBlueButton docker images since I last posted about that. Now, the container can be accessed externally and not only through the private IP address docker assigns to it (by default in 172.17.0.x range) as before. For this to work, the SERVER_NAME env variable must be set pointing to the hostname that is going to be used to access your BigBlueButton container. Now, the container can be started like this:

sudo docker run -d --name bbb -p 80:80 -p 9123:9123 -p 1935:1935 -e SERVER_NAME=meeting.somedomain.com bbb_0.81

Then you can access the container externally (provided SERVER_NAME resolves to a public IP address) using $SERVER_NAME. The hostname set in SERVER_NAME must point to the docker host machine. If the container can't use the same host ports (ie: there's already a web server running on port 80) you can start the container using other ports:

sudo docker run -d --name bbb -p 80:8080 -p 9123:91230 -p 1935:19350 -e SERVER_NAME=meeting.somedomain.com bbb_0.81
And configure a reverse proxy server (like nginx) to go to the BigBlueButton's container private IP address and the new http port in the docker run command when accessing SERVER_NAME, and port forward ports 1935 and 9123 on the docker host machine to the container. Or even easier, use a nginx container and link it to the BigBlueButton container but this deserves another post.

More detailed instructions in the github project page.

Monday, October 6, 2014

Official mageia docker images available

We now have official docker images for mageia !!

After some weeks working with the docker team we managed to get mageia as an official docker image (the ones that have the blue whale icon). You can find them at the docker hub, and if you want to contribute to them you can go to mageia's docker brew project.

There are three images available:

  • Mageia 3
  • Mageia 4 (latest)
  • cauldron

Currently the cauldron image is outdated (probably more than a month), but I plan to automate the docker image update process so we can have an updated version at least once a week.


How to use these images


You can pull them on the command line (as root):

          # docker pull mageia:latest
          Pulling repository mageia
          147b6e8a8cbd: Download complete 
          511136ea3c5a: Download complete 
          e65cc271e617: Download complete 
          
          # docker start -ti --name mymageia_4 mageia:latest


Or create a Dockerfile file to build your own custom mageia-based image:
FROM mageia:4
MAINTAINER  "Foo Bar" 
CMD [ "bash" ]
All mageia docker images install the following packages:
  • basesystem-minimal
  • urpmi
  • locales
  • locales-en

Please test these images, and if you find any issues or have suggestions don't forget to report them here. Also I'm thinking of adding some other custom images for specific applications and uses, like:

Ready to run server application-oriented containers


We could have several application oriented containers: mariaDB, nginx, wordpress, Apache+php/{cakephp,zend,codeigniter}, Apache+python/{django,codegears,flask}, tomcat preconfigured to use an apache container as front end, etc, the possibilities are endless. All these containers could be linked, packaged and orchestrated using fig for an easier application control and management.

Another example could be FPS game servers (Urban Terror,  OpenArena, Warsow, World of Padman, Smokin' Guns), with their server package, some license-redistributable maps, a web admin panel, mumblebigbrotherbot (already working on a package) and anything else needed to have a kinda of "one click" game server setup. This could be very useful for example, to quickly launch game servers at a LAN party, or to provision game servers at a game hosting company.

Docker for distribution development


At the very least I see a couple of uses for docker within mageia development. First, as a quick and easy way to use iurt for local package building. We could have a custom docker image for package development that comes with a preconfigured iurt binary, package build tools like bm, rpmbuild, rpmlint, mgarepo, etc, all preinstalled, this could be a build/packaging environment with one command:

          # docker pull mageia:devenv
          Pulling image...
          # docker run --rm -ti --name mageia_dev -v /home/juancho/iurt:/opt/iurt/ mageia:devenv iurt SRPMS/foo-1.0-1mga5.src.rpm

That command would launch a docker container using our custom development image, launch iurt to build a source package, leave the binary packages in /home/juancho/iurt/RPMS/{i586,x86_64,noarch} and delete it self when it finishes. This is a clean way to locally build packages in a fresh environment. Remove the --rm parameter if you want to use the container later, for example to work on package version updates:

          # docker run -ti --name mageia_dev -v /home/juancho/.ssh:/home/juancho/.ssh -v /home/juancho/iurt:/opt/iurt/ mageia:devenv bash
       
Also by mapping your .ssh directory to a docker volume, mgarepo can be used within the container.

The other important use for docker within mageia could be to help with QA testing. The reproducible nature of docker makes it very interesting from a QA point of view, the repeatability of tests could be of great help for application testing and bug triaging.

We could teach bug reporters how to create their own images or write their own Dockerfiles with the needed packages and configuration changes to reproduce a bug. The reporter would point QA back to an image that they can download and test (for example, from our own docker repository). The creation of those containers could ease and speed the testing process. As these custom images would be based on our official images, there wouldn't be the need for QA to setup the same test case to reproduce the bug in another environment, the reporter image should be enough for them to test and validate it. In some way, we could be making the bug reporters also contribute the test case.

Docker application containers


What about preconfigured docker containers for software development environments, like images that have Netbeans/Eclipse for python/java/php, git/mercurial/svn/bazaar, any development libs and tools needed depending on the platform, etc, all preinstalled and preconfigured. This could be a good idea as sometimes these tools are difficult to install and update, having these ready to use containers could be cool. Probably it also could be used to package nonfree applications or 32bits applications on x86_64.

I don't know, there are many ideas that come to my mind about stuff that can be done with docker in different areas, like these ones on linux distribution development and such.


Tuesday, July 22, 2014

Setting up a BigBlueButton 0.81 docker container: Part 1

Because of my current job, the last few years I have become more interested on virtualization and cloud computing technologies like OpenStack, but during the last month I have been playing with a new and disrupting technology called docker. So, as an exercise to learn about it I started working on a docker container for BigBlueButton online web conferencing platform. The focus of this article is about setting up the docker image for this software, but lets talk first about what's docker is and what it can do.

What's docker and what's so disruptive about it ? 


docker leverages LxC (linux containers) and kernel cgroups to create application containers that are much more efficient than running separate virtual machines in a cloud computing environment. The media is calling this type of technology "lightweight virtualization". Why this terminology ? unlike virtual machines that run a complete operating system and applications on top of it, docker containers are composed of just a base linux operating system, your application and its dependencies, and reuses the host's kernel. This make docker containers much more efficient, fast and with little overhead, a container can be started in a few seconds, or less.

The advantages of docker compared with current virtualization technologies are many, these are some of them:

  • Lightweight: as a docker container only includes your application and needed dependencies and reuses the host's kernel, they have a very small footprint and boot in seconds with little overhead.
  • Isolation: a docker container running on the same host than others will not be able to negatively affect other containers.
  •  Management of applications with conflicting dependencies: You can have different docker containers that may have conflicting dependencies for your application: two applications that rely on different versions of the same package, no problem, create two different docker images from the same base image and add the different dependencies versions with your app.
  • Portability: Use the same container for the complete development pipeline of your application: from the developers laptop, to the QA server and to production, goodbye to developers saying: "it works on my machine" when a problem arises on one of the other environments. 
  • Repeatability of deployments: create exact docker containers from the same docker image every time from an already exiting image, or build them from a Dockerfile.

It's important to mention that docker isn't a good fit for all cases. The use of linux only technologies like lxc and cgroups, means that docker can only run on Linux hosts, and can only run linux based containers. So, if you need to run another operating system like for example MS Windows, you will need to use traditional virtualization instead.

There are already tons of docker images at docker Hub, you can find more than 15,000 already "dockerized" applications and base operating systems, like CentOS, Ubuntu, Debian, OpenSuSE (and soon Mageia), and applications and services like wordpress, MySQL, PosgreSQL, nginx, MongoDB. etc, the catalog is huge.

We have included docker in mageia cauldron (the development version), and it will be available in mageia 5. You can install mageia 5 alpha1 and follow the wiki instructions to change the repositories to point to cauldron if you want to try it out. Soon we will also have mageia 3, 4 and cauldron base images available on docker hub, I will be posting when they're available.

Ok, enough introduction, lets get down to business.


What is BigBlueButton ?


BigBlueButton is an open source web conferencing system for online e-learning with many of the features of commercial propietary products like Citrix Gotomeeting. You can visit BigBlueButton website to learn more about it.

NOTE: this is an unofficial BigBlueButton 0.81 docker image. On my github account you can find the Dockerfile and all other files needed to build it.


About this image and the Dockerfile


This image is based on Ubuntu 10.04 x86_64, which is the officially supported O.S. for BigBlueButton 0.81. The Dockerfile follows the official installation instructions found on BigblueButton's documentation, plus some fixes needed to successfully boot the container (see the scripts folder at github). To run docker you need to do it as root or use sudo.

You can find a prebuilt docker image from Docker Hub. To be able to use it, first it has to be pulled off from the Hub:

# docker pull juanluisbaptiste/bigbluebutton:latest

And then you can run a container from it, see instructions below on how to do it.

This is still an alpha version use it at your own risk. There is still some stuff about how to handle the different services that compose the BigBlueButton app inside the docker container that I need to improve.


Build Instructions


After you clone this repository you need to build the image with the docker command like this:

# cd docker-bigbluebutton 
# docker build -t bbb_0.81 .

How to launch the container


This docker command will launch a new BigBlueButton container:

# docker run -d --name bbb bbb_0.81

You can attach to the container while it starts and wait for it to finish, then take the IP address from the end of the output. To attach to the container run the following docker command:

# docker attach --sig-proxy=false bbb

How to access the container


For now it's only possible to access the BigBlueButton container using the private IP address docker has assigned to it. after you attach to the container you will see an output like the following one telling you the IP address:

*******************************************
Use this IP address to locally access your
BigBlueButton container: 
 
172.17.0.2 
*******************************************
Access that address from your browser and you will get to the demo page like this one:



Then to test BigBlueButton enter your name on the bottom of the screen where it says "Join a Demo Meeting" to see the e-learning platform in action:


NOTE: If you try to use the exposed ports, the bundled nginx server will show the default page instead of BigBlueButton's demo page. I'm working on this.

In a second part I will describe how to link this container to a Wordpress container with the BigBlueButton plugin already installed and configured, and a MySQL container for the Wordpress installation, stay tuned.

Go to Part2.

Monday, October 22, 2012

Tora packages with Oracle support for Mageia 2

In Mageia we include a really good database administration client called TOra, you can learn more about it on its website, but for those of you that work with databases, to give you an idea of its features (PL/SQL debugger, SQL worksheet with syntax highlighting, DB browser and DBA tools), I can tell you that it's very similar to the commercial client, TOAD. It's developed with the Qt framework, here's an screenshot so you can have an idea of how it looks:



The version we include of TOra in Mageia has Oracle database support disabled because we can't include the Oracle instant client packages due to licensing issues, so we have to do it in the same way I did it before at Mandriva: have on Mageia a package named tora with support enabled for MySQL and PostgreSQL only, and another one at TOra's download page called tora-oracle, which has additionally Oracle databases support enabled.

I had forgotten to compile and upload these Oracle enabled packages to TOra's SF download page until today. Now you can find here these packages, but remember that Oracle instant client must be installed before running this version of TOra. If you don't need Oracle support then install the packages that come with the distro. Maybe we can add a get-tora-oracle similar to the get-skype package to download tora-oracle from TOra's download page, but the Oracle instant client still has to be manually downloaded and installed by the user.

You can download Oracle instant client packages from here. Both oracle-instantclient11.2-basic and oracle-instantclient11.2-basiclite rpm packages work, but if you want to rebuild tora-oracle src package then you need to have oracle-instantclient11.2-devel too with the basic package.