Monday, May 16, 2016

How to mount VirtualBox shared folders at boot

I'm setting up a network install server on a VirtualBox VM, and I didn't want to copy the contents of the iso images of the distros going to be available through PXE to avoid having a huge VM. Worse if I also included other repos like epel or centosplus, or updates, so I created a VirtualBox shared folder pointing to the directory containing the iso images. After having created the shared folder, you can mount it like this:

sudo mount -t vboxsf -o uid=$UID,gid=$(id -g) share ~/host

To avoid having to manually mount the share each time the VM boots , the shared mount needs to be added to /etc/fstab, but there's a catch: the vboxsf kernel module, needed to mount the shared folder, isn't available when mounting all filesystems during the boot process. So, to fix this we need to make sure the vboxsf module is loaded before the filesystems mount at boot.

On CentOS 7, create a file on /etc/sysconfig/modules directory ending in .modules and add this to load VirtualBox kernel module before filesystems are mounted:


    lsmod |grep vboxsf >/dev/null 2>&1
    if [ $? -gt 0 ] ; then
      exec /sbin/modprobe vboxsf >/dev/null 2>&1

On Ubuntu/Debian, add the module name to /etc/modules. Now we need to add the shared mount to /etc/fstab. In my case, my shared folder is called isos, so I added the following line:

isos    /isos   vboxsf  defaults        0 0

After adding this line you can reboot the server/vm and see if it mounted correctly at boot.

If you want to mount the iso images too at boot, add a line like this one to /etc/fstab, for each iso to mount:

/isos/CentOS/CentOS-7-x86_64-DVD-1511.iso /distros/centos7 iso9660 loop 0 0

Remember to adjust loopback device limits if you plan to mount more than 8 or 10 images (don't remember right now the limit).

Thursday, December 31, 2015

Using a portable SMTP relay server with docker

I have had a very busy year and had the chance to work on a lot of new and useful docker containers. Taking advantage of these holidays, I finally started to catch up here with my latest work on them :D

There's one thing that it's a pretty common requirement of a website: a SMTP email server or relay.
Every web app needs one for different tasks, like sending out notifications, registration emails, password resets, etc. I made a Postfix SMTP relay container that is easy to use with other containers.

Before running the container, first you need to set the following environment variables to configure the SMTP relay host:
  • SMTP_SERVER: Server address of the SMTP server that will send email from our postfix container.
  • SMTP_USERNAME: Username to authenticate with.
  • SMTP_PASSWORD: Password of the SMTP user.
  • SERVER_HOSTNAME: Server hostname for the Postfix container. Emails will appear to come from this hostname domain.
To use it you need to first pull the image:
docker pull juanluisbaptiste/postfix
and then fire it up with the previous variables defined:
docker run -d --name postfix -P \
       -e \
       -e \
        -e \        
Lastly, link your container against it:
docker run --name mycontainer --link "postfix:postfix" myimage

Using docker-compose

Or, you could use docker-compose to start your application containers linked against the postfix container with one command. Suppose you have a web application that links against a database and this postfix container. Download and install docker-compose for your platform, and then on your website's docker project, create a new file called docker-compose.yml, and put the following contents there:
  build: myapp
  - "80:80"
# If running behind a proxy container, expose the ports instead
# and link the proxy container to this one.
#  expose:
#  - "80"
  - mariadb:mariadb
  - postfix:postfix
  - data
  image: centos/mariadb:latest
  - "3306"
  - data
      MYSQL_ROOT_PASSWORD: changeme
   image: juanluisbaptiste/postfix:latest
   - "25"
      SMTP_PASSWORD: changeme
  image: centos:latest
  - /var/lib/mysql
  - /var/www/webapp

  command: /bin/true
Then, you can launch your webapp, the database and the postfix container with this command:
docker-compose up
All containers will be started in the right order and the webapp will be linked against the mariadb and postfix containers. Also, the webapp and the mariadb database container will share the same data volume container (unrelated to the postfix container but a good practice).

On thing to note, this container doesn't enable client SMTP authentication, the idea is to expose the port 25 to containers, and then link the containers that need a SMTP service against it so the relay isn't publicly exposed.

A note about using gmail as a relay

Since last year, Gmail by default does not allow email clients that don't use OAUTH 2
for authentication (like Thunderbird or Outlook). First you need to enable access to "Less secure apps" on your google settings.

Also take into account that email From: header will contain the email address of the account being used to authenticate against the Gmail SMTP server (SMTP_USERNAME), the one on the email will be ignored by Gmail.

Friday, February 13, 2015

Setting up a BigBlueButton 0.81 docker container: Part 2

I have made some improvements on my BigBlueButton docker images since I last posted about that. Now, the container can be accessed externally and not only through the private IP address docker assigns to it (by default in 172.17.0.x range) as before. For this to work, the SERVER_NAME env variable must be set pointing to the hostname that is going to be used to access your BigBlueButton container. Now, the container can be started like this:

sudo docker run -d --name bbb -p 80:80 -p 9123:9123 -p 1935:1935 -e bbb_0.81

Then you can access the container externally (provided SERVER_NAME resolves to a public IP address) using $SERVER_NAME. The hostname set in SERVER_NAME must point to the docker host machine. If the container can't use the same host ports (ie: there's already a web server running on port 80) you can start the container using other ports:

sudo docker run -d --name bbb -p 80:8080 -p 9123:91230 -p 1935:19350 -e bbb_0.81
And configure a reverse proxy server (like nginx) to go to the BigBlueButton's container private IP address and the new http port in the docker run command when accessing SERVER_NAME, and port forward ports 1935 and 9123 on the docker host machine to the container. Or even easier, use a nginx container and link it to the BigBlueButton container but this deserves another post.

More detailed instructions in the github project page.

Monday, October 6, 2014

Official mageia docker images available

We now have official docker images for mageia !!

After some weeks working with the docker team we managed to get mageia as an official docker image (the ones that have the blue whale icon). You can find them at the docker hub, and if you want to contribute to them you can go to mageia's docker brew project.

There are three images available:

  • Mageia 3
  • Mageia 4 (latest)
  • cauldron

Currently the cauldron image is outdated (probably more than a month), but I plan to automate the docker image update process so we can have an updated version at least once a week.

How to use these images

You can pull them on the command line (as root):

          # docker pull mageia:latest
          Pulling repository mageia
          147b6e8a8cbd: Download complete 
          511136ea3c5a: Download complete 
          e65cc271e617: Download complete 
          # docker start -ti --name mymageia_4 mageia:latest

Or create a Dockerfile file to build your own custom mageia-based image:
FROM mageia:4
CMD [ "bash" ]
All mageia docker images install the following packages:
  • basesystem-minimal
  • urpmi
  • locales
  • locales-en

Please test these images, and if you find any issues or have suggestions don't forget to report them here. Also I'm thinking of adding some other custom images for specific applications and uses, like:

Ready to run server application-oriented containers

We could have several application oriented containers: mariaDB, nginx, wordpress, Apache+php/{cakephp,zend,codeigniter}, Apache+python/{django,codegears,flask}, tomcat preconfigured to use an apache container as front end, etc, the possibilities are endless. All these containers could be linked, packaged and orchestrated using fig for an easier application control and management.

Another example could be FPS game servers (Urban Terror,  OpenArena, Warsow, World of Padman, Smokin' Guns), with their server package, some license-redistributable maps, a web admin panel, mumblebigbrotherbot (already working on a package) and anything else needed to have a kinda of "one click" game server setup. This could be very useful for example, to quickly launch game servers at a LAN party, or to provision game servers at a game hosting company.

Docker for distribution development

At the very least I see a couple of uses for docker within mageia development. First, as a quick and easy way to use iurt for local package building. We could have a custom docker image for package development that comes with a preconfigured iurt binary, package build tools like bm, rpmbuild, rpmlint, mgarepo, etc, all preinstalled, this could be a build/packaging environment with one command:

          # docker pull mageia:devenv
          Pulling image...
          # docker run --rm -ti --name mageia_dev -v /home/juancho/iurt:/opt/iurt/ mageia:devenv iurt SRPMS/foo-1.0-1mga5.src.rpm

That command would launch a docker container using our custom development image, launch iurt to build a source package, leave the binary packages in /home/juancho/iurt/RPMS/{i586,x86_64,noarch} and delete it self when it finishes. This is a clean way to locally build packages in a fresh environment. Remove the --rm parameter if you want to use the container later, for example to work on package version updates:

          # docker run -ti --name mageia_dev -v /home/juancho/.ssh:/home/juancho/.ssh -v /home/juancho/iurt:/opt/iurt/ mageia:devenv bash
Also by mapping your .ssh directory to a docker volume, mgarepo can be used within the container.

The other important use for docker within mageia could be to help with QA testing. The reproducible nature of docker makes it very interesting from a QA point of view, the repeatability of tests could be of great help for application testing and bug triaging.

We could teach bug reporters how to create their own images or write their own Dockerfiles with the needed packages and configuration changes to reproduce a bug. The reporter would point QA back to an image that they can download and test (for example, from our own docker repository). The creation of those containers could ease and speed the testing process. As these custom images would be based on our official images, there wouldn't be the need for QA to setup the same test case to reproduce the bug in another environment, the reporter image should be enough for them to test and validate it. In some way, we could be making the bug reporters also contribute the test case.

Docker application containers

What about preconfigured docker containers for software development environments, like images that have Netbeans/Eclipse for python/java/php, git/mercurial/svn/bazaar, any development libs and tools needed depending on the platform, etc, all preinstalled and preconfigured. This could be a good idea as sometimes these tools are difficult to install and update, having these ready to use containers could be cool. Probably it also could be used to package nonfree applications or 32bits applications on x86_64.

I don't know, there are many ideas that come to my mind about stuff that can be done with docker in different areas, like these ones on linux distribution development and such.

Tuesday, July 22, 2014

Setting up a BigBlueButton 0.81 docker container

Because of my current job, the last few years I have become more interested on virtualization and cloud computing technologies like OpenStack, but during the last month I have been playing with a new and disrupting technology called docker. So, as an exercise to learn about it I started working on a docker container for BigBlueButton online web conferencing platform. The focus of this article is about setting up the docker image for this software, but lets talk first about what's docker is and what it can do.

What's docker and what's so disruptive about it ? 

docker leverages LxC (linux containers) and kernel cgroups to create application containers that are much more efficient than running separate virtual machines in a cloud computing environment. The media is calling this type of technology "lightweight virtualization". Why this terminology ? unlike virtual machines that run a complete operating system and applications on top of it, docker containers are composed of just a base linux operating system, your application and its dependencies, and reuses the host's kernel. This make docker containers much more efficient, fast and with little overhead, a container can be started in a few seconds, or less.

The advantages of docker compared with current virtualization technologies are many, these are some of them:

  • Lightweight: as a docker container only includes your application and needed dependencies and reuses the host's kernel, they have a very small footprint and boot in seconds with little overhead.
  • Isolation: a docker container running on the same host than others will not be able to negatively affect other containers.
  •  Management of applications with conflicting dependencies: You can have different docker containers that may have conflicting dependencies for your application: two applications that rely on different versions of the same package, no problem, create two different docker images from the same base image and add the different dependencies versions with your app.
  • Portability: Use the same container for the complete development pipeline of your application: from the developers laptop, to the QA server and to production, goodbye to developers saying: "it works on my machine" when a problem arises on one of the other environments. 
  • Repeatability of deployments: create exact docker containers from the same docker image every time from an already exiting image, or build them from a Dockerfile.

It's important to mention that docker isn't a good fit for all cases. The use of linux only technologies like lxc and cgroups, means that docker can only run on Linux hosts, and can only run linux based containers. So, if you need to run another operating system like for example MS Windows, you will need to use traditional virtualization instead.

There are already tons of docker images at docker Hub, you can find more than 15,000 already "dockerized" applications and base operating systems, like CentOS, Ubuntu, Debian, OpenSuSE (and soon Mageia), and applications and services like wordpress, MySQL, PosgreSQL, nginx, MongoDB. etc, the catalog is huge.

We have included docker in mageia cauldron (the development version), and it will be available in mageia 5. You can install mageia 5 alpha1 and follow the wiki instructions to change the repositories to point to cauldron if you want to try it out. Soon we will also have mageia 3, 4 and cauldron base images available on docker hub, I will be posting when they're available.

Ok, enough introduction, lets get down to business.

What is BigBlueButton ?

BigBlueButton is an open source web conferencing system for online e-learning with many of the features of commercial propietary products like Citrix Gotomeeting. You can visit BigBlueButton website to learn more about it.

NOTE: this is an unofficial BigBlueButton 0.81 docker image. On my github account you can find the Dockerfile and all other files needed to build it.

About this image and the Dockerfile

This image is based on Ubuntu 10.04 x86_64, which is the officially supported O.S. for BigBlueButton 0.81. The Dockerfile follows the official installation instructions found on BigblueButton's documentation, plus some fixes needed to successfully boot the container (see the scripts folder at github). To run docker you need to do it as root or use sudo.

You can find a prebuilt docker image from Docker Hub. To be able to use it, first it has to be pulled off from the Hub:

# docker pull juanluisbaptiste/bigbluebutton:latest

And then you can run a container from it, see instructions below on how to do it.

This is still an alpha version use it at your own risk. There is still some stuff about how to handle the different services that compose the BigBlueButton app inside the docker container that I need to improve.

Build Instructions

After you clone this repository you need to build the image with the docker command like this:

# cd docker-bigbluebutton 
# docker build -t bbb_0.81 .

How to launch the container

This docker command will launch a new BigBlueButton container:

# docker run -d --name bbb bbb_0.81

You can attach to the container while it starts and wait for it to finish, then take the IP address from the end of the output. To attach to the container run the following docker command:

# docker attach --sig-proxy=false bbb

How to access the container

For now it's only possible to access the BigBlueButton container using the private IP address docker has assigned to it. after you attach to the container you will see an output like the following one telling you the IP address:

Use this IP address to locally access your
BigBlueButton container: 
Access that address from your browser and you will get to the demo page like this one:

Then to test BigBlueButton enter your name on the bottom of the screen where it says "Join a Demo Meeting" to see the e-learning platform in action:

NOTE: If you try to use the exposed ports, the bundled nginx server will show the default page instead of BigBlueButton's demo page. I'm working on this.

In a second part I will describe how to link this container to a Wordpress container with the BigBlueButton plugin already installed and configured, and a MySQL container for the Wordpress installation, stay tuned.

Go to Part2.

Monday, October 22, 2012

Tora packages with Oracle support for Mageia 2

In Mageia we include a really good database administration client called TOra, you can learn more about it on its website, but for those of you that work with databases, to give you an idea of its features (PL/SQL debugger, SQL worksheet with syntax highlighting, DB browser and DBA tools), I can tell you that it's very similar to the commercial client, TOAD. It's developed with the Qt framework, here's an screenshot so you can have an idea of how it looks:

The version we include of TOra in Mageia has Oracle database support disabled because we can't include the Oracle instant client packages due to licensing issues, so we have to do it in the same way I did it before at Mandriva: have on Mageia a package named tora with support enabled for MySQL and PostgreSQL only, and another one at TOra's download page called tora-oracle, which has additionally Oracle databases support enabled.

I had forgotten to compile and upload these Oracle enabled packages to TOra's SF download page until today. Now you can find here these packages, but remember that Oracle instant client must be installed before running this version of TOra. If you don't need Oracle support then install the packages that come with the distro. Maybe we can add a get-tora-oracle similar to the get-skype package to download tora-oracle from TOra's download page, but the Oracle instant client still has to be manually downloaded and installed by the user.

You can download Oracle instant client packages from here. Both oracle-instantclient11.2-basic and oracle-instantclient11.2-basiclite rpm packages work, but if you want to rebuild tora-oracle src package then you need to have oracle-instantclient11.2-devel too with the basic package.

Thursday, February 9, 2012

It's all about the games !!

UPDATE (15/02/2012): Warsow was updated to 0.62 and World of Padman to 1.6.

In the past months I have been working on many things for Mageia, like mentoring some novice packagers, fixing bugs, preparing updates for Mageia 1, and the usual packaging of new applications and updating existing ones. On this last activity I've been working really hard to get the biggest amount of games packaged for Mageia. I always have loved FPS games, so that kind of games have been my main objective.

Of course, all are open source games, but unfortunately, for some of them the data files have non-free licenses, so those have been put on the non-free repository, or are using a nifty feature to autodownload the data files at first run of the game. How this autodownload feature works ? using a program developed by Fedora called autodownloader. Here is how it works:

The user install one of the games that has non-free licensed data files using urpmi, Mageia Control Center or drakrpm as usual. Then, when the user runs the game for the first time, he/she will be prompted with the following screen describing the game and the space requirements for the download that is about to be done:

If the user accepts, then another screen with the game license will be displayed so the user can agree to it or refuse it:

Upon the user agreeing the game license, the download of the data files will begin from the first configured mirror:

You can have multiple mirrors configured in case that the first mirror is down, or the files aren't available anymore on that server. After the files finish downloading, they will be verified against a md5 hash to be sure that download was ok. Then, the user will see a screen telling him that the download completed successfully and asks the user if he wants to launch the game !! :D :

And voila ! the game starts running :D

The autodownloader program uses a very simple configuration file where you specify the text of all of these screens and the mirror list from where to download the files with it's corresponding md5 hash. What I like the most of this autodownload technique is that we can use it for other programs that can't be included on the distro, not even on the non-free repository, like skype or flash plugin perhaps ?

Well enough about technicalities !! lets see which new games we will have in the upcoming Mageia 2 in May :D

The following games are all based on ioquake3 engine, thus they use it as their game engine, which has been patched to better support these games. Also most of these (autodownloader) packages have been taken from Fedora and adapted for Mageia. I'll start with my preferred one :D

Urban Terror

Version: 4.1.1
Media: core/release (uses autodownloader)
Description from the game's website:
Urban Terror is a free multiplayer first person shooter developed by FrozenSand, that (thanks to the ioquake3-code) does not require Quake III Arena anymore. It is available for Windows, Linux and Macintosh. Urban Terror can be described as a Hollywood tactical shooter; somewhat realism based, but the motto is "fun over realism". This results in a very unique, enjoyable and addictive game.

There's an excellent video that shows the gameplay of the game on youtube. You can find more screenshots here.

World Of Padman

Version: 1.6
Media: core/release (uses autodownloader)

Description from the game's website:
World of Padman (WoP) is an open source first-person shooter computer game available in both English and German. Originally it was a modificationfor the game Quake III Arena titled PadMod created in the year 2004. After the source code for Quake III Arena was released, the game became standalone.[1] The idea is based on the Padman comic strip for the magazine PlayStation Games created by the professional cartoon artist Andreas 'ENTE' (German for "Duck") Endres, who is also the man who made many of the maps included with the game in 1998.

You can find more screenshots here.

Version: 1.0
Media: core/release (uses autodownloader)
Description from the game's website:
Smokin' Guns is intended to be a semi-realistic simulation of the "Old West's" great atmosphere & was developed on Id Software's Quake III Arena Engine. Gameplay as well as locations are inspired by Western movies, particularly those from the Spaghetti Western genre.

You can find more screenshots here.


Version: 0.5.3
Media: core/release

Description from the game's website:
Turtle Arena (working title) is a free and open source cross-platform third-person action game using a modified version of the ioquake3 engine.
Turtle Arena is currently focused on multiplayer (with multiple game modes) and can be played with human players over a network, splitscreen, or with AI players. In the future there will also be a single player / cooperative reach the end of the level mode with AI enemies.

This game has it's own engine called ioq3ztm, which it's a modified ioquake3 engine to support additional features like the four player split screen.


Version: 2010_07_28 Justice Edition
Media: nonfree/release

Description from the game's website:
Cube 2: Sauerbraten is a free multiplayer/singleplayer first person shooter, built as a major redesign of theCube FPS.
Much like the original Cube, the aim of this game is not necessarily to produce the most features & eyecandy possible, but rather to allow map/geometry editing to be done dynamically in-game, to create fun gameplay and an elegant engine.

You can find more screenshots here.
Media: nonfree/release

Description from the game's website:
AssaultCube is a FREE, multiplayer, first-person shooter game, based on the CUBE engine. Taking place in realistic environments, with fast, arcade gameplay, it's addictive and fun!

You can find more screenshots here.

Version: 0.62
Media: nonfree/release
Description from the game's website:
Warsow is set in a futuristic cartoonish world where rocketlauncher-wielding pigs and lasergun-carrying cyberpunks roam the streets. It is a completely free, fast-paced first-person shooter for Windows, Linux, and Mac OS X.

You can find more screenshots here.

There are other excellent FPS games that already can be fouond in Mageia:
If you would like to see any other games packaged please post a comment and open a feature request in Bugzilla. If you play Urban Terror or Nexuiz it would be nice to meet other Mageia users and share some frags. Leave a comment with your nick and usual server you play on and maybe we can meet there :)

Have fun !!