Analyzing awareness of MidoNet globally

One thing that is hard to measure in open source projects is just how aware people are of your community or its software. It's even harder to find out where in the world people have heard about you so far. You might have a rough feeling, probably based on some facts like the country-specific TLDs of email addresses on your mailing lists. But that's very vague and only includes people actively participating (and there's more and less vocal people, too).

One approach is to check the hits your repository or website is getting. What most people hearing about something for the first time are doing nowadays is quickly check out that something's website. So it's a perfect way to measure where people have at least heard of you!

There's a couple of ways to do this. One way is tracking the IPs that hit your webserver and turn them into a geographical location. Another way is leveraging tools like Google Analytics that you can build into your website. Today, we'll have a brief look at the results for those analytics for, from November 1 until August 31. The data used below is showing numbers of sessions, where a session is defined as starting with the first page impression by a user and ends 30 minute after the user's last page impression. The total duration doesn't matter as much as the fact that any page impressions in between the first and last in a single session are still counting to the same, single session.

Sessions per Country
If you would have given me a map of the world and asked me to color all the countries where I think people have heard of MidoNet, I would have colored the USA and Canada, Europe (minus some South-Eastern parts), East Asia (minus North Korea and Mongolia), Russia and Australia. Again, that's mostly based on where I see activity coming from, plus where I know MidoNet has been presented at some conference or meetup.

However, to my surprise, we've actually seen visitors from many more countries! People are aware of MidoNet almost globally! As you can see on the image below, the map is mostly blue. You can also see, that my expectations weren't totally off the mark as most of the countries I would have colored have a darker hue, indicating more sessions.

Looking at full list of countries reveals that we've had visitors from a total of 122 countries! Admittedly, we've only seen a single session from many of those but it still looks promising. Looking at the top 20 countries by number of sessions, my gut feelings are once more by and large confirmed:
  1. United States
  2. Japan
  3. Russia
  4. China
  5. Spain
  1. South Korea
  2. India
  3. France
  4. Germany
  5. Canada
  1. United Kingdom
  2. Australia
  3. Taiwan
  4. Italy
  5. Brazil
  1. Netherlands
  2. Switzerland
  3. Vietnam
  4. Finland
  5. Norway
Of those, only just Brazil and Vietnam surprised me a little bit, but not enough to go dig for the reasons.

Sessions by City
Well, some countries are just huge while others are smaller than some cities so the above doesn't tell us much about the actual awareness of MidoNet in e.g. the United States, Russia or China. Let's briefly look at cities, but be prepared to see more white than blue on the global map this time. Again, given a map and asked to color it accordingly...well, this one is seriously tough. Obvious high on the list must be Barcelona, Tokyo and San Francisco just because MidoNet's principal sponsor, Midokura, has major offices in those cities. Other than that, I would probably color the economic or technical capital of the countries I would have colored above. Afterwards, I'd just make lots of random dots all over Europe, the US East and West Coats (plus Texas) and that's it. I'd make the bubbles at the Midokura offices as well as those over Seoul, Beijing, Paris and Vancouver fairly bigger and the rest about equally small, again based on where I see activity from, plus we've been promoting MidoNet quite much at the OpenStack Summits in Paris and Vancouver.

Looking at the actual data, I'm not too far off this time. The habitable parts of Europe are covered with dots, and so are the US East and West Coasts. Furthermore, the economical or technical capitals also got dots...and then some.
As I mentioned before, the first map doesn't tell us much about huge countries, but now we see a more detailed image of those countries and can see that actual awareness is much more clustered in only some regions, compared to e.g. Europe or East Asia. But the dots are hard to interpret, let's look at the top 20 cities by number of sessions:
  1. Tokyo
  2. Samara
  3. Barcelona
  4. Seoul
  5. San Francisco
  1. Beijing
  2. New York
  3. Kawasaki
  4. Paris
  5. Bengaluru
  1. Fukuoka
  2. San Jose
  3. Yokohama
  4. Moscow
  5. Santa Clara
  1. Toronto
  2. London
  3. Shanghai
  4. Kansas City
  5. Osaka
No major surprise here, but I'll share some thoughts and facts. First of all, Google Analytics actually lists the single cities that Tokyo is made of - in case you didn't know, Tokyo is not actually a city (anymore) - but I added them up and listed Tokyo instead as otherwise, 6 out of the 20 top ranks would go to Tokyo cities! Also, if you take a train from Tokyo to Yokohama, through Kawasaki (all three listed), you couldn't possibly tell where one ends and the other begins and all of them are home to major tech companies, so no surprise here. Similarly, it's not surprising to see San Jose and Santa Clara listed shortly after San Francisco.

Two smaller surprises came in the form of Samara and Kansas City. Now, when I said I see interested in MidoNet from Russia, I didn't know where in Russia it's coming from. It seems Samara is a major tech city, so seeing Samara and Moscow listed seems plausible but I still wouldn't have expected it on rank #2. Kansas City surprised me a bit more - I know that a local company there is one of our major users, but it's hard to believe that a single company propelled the city into the top 20...

In just 10 months, we have raised awareness for MidoNet all over the globe! Alright, the more detailed analysis by cities draws a bit of a different image, but still a very successful one. All strategically important (i.e. major IT) cities are highlighted, and more. The map also shows that our efforts to promote MidoNet in key areas is very successful and often spreads well outside of the city we were actually visiting.

If I had to predict the coming 10 months, I would first and foremost expect even more interest from the same cities. I don't think we're going to target the regions that are virtually blank on the map as those are mostly regions where cloud technologies have yet to gain traction and SDN/NFV is usually not the first thing people look at. We're trying to put a bit more of a focus on China and India than previously, and continue promoting MidoNet at OpenStack Summits (Tokyo and Austin TX) and Dockercon (Barcelona and ???) but we already see interest from all these places so it's mostly about growth.

Of course, growing interest is only the first step. We also need to convert that interest into users - and the number of users is really the hardest to measure - and finally into contributors, something we're only just at the very beginning of. So continuing to focus on those key areas does make sense, and is still important.

Want to give BPG / libbpg a try? Here's a repo for you.

By now, most people will have heard about libbpg. Or maybe rather about the new BPG (better portable graphics) image format that was created by Fabrice Bellard, creator of qemu and ffmpeg, to possibly hopefully replace JPEG (and maybe even PNG) image formats one day.

Right now, there's no support in any tool but only just, two binaries (command line decoder and encoder for conversion to/from other image formats), a static library and a javascript decoder (to use BPG format images on websites while browsers don't yet support it). Therefore, it's likely way too early for most people to do anything with it.

But since I was playing around with the library anyway, I created a RPM package, just for fun, and created a Fedora COPR repository from it for everyone's profit. Feel free to use this repo/package/software at your own risk and get it by following the instructions at

Patches to the spec file welcome, but don't expect any kind of support or such.

In case you've been missing

If you're running Fedora rawhide and would like to install e.g. Google Chrome, you probably already noticed there's no anymore due to a soname bump some time ago.

Facing the same issue, I finally gave in and took the last pre-bump srpm and turned it into a compat package. Thinking others might share the pain and would like to receive some relief, I turned that into a Copr.

Enjoy at your own risk:

There won't be any updates (that's what the proper libgcrypt package is there for), fixes or warranties. If anything at all, all other warranties will be voided if you install the compat-libgcrypt package.

Install OpenStack Havana-2 from RDO on Fedora 19 and avoid the pitfalls

If you'd like to try out OpenStack Havana on Fedora 19, there's a few pitfalls that need to be avoided. Let me be your navigator and show you a way around them. But first, let me warn you of a few things:
  • Below, I might sometimes expect you are familiar with Linux, Fedora and OpenStack.
  • Havana is not yet finished, stable or even released. Havana-2 is only just a milestone and development will continue for two whole months so expect things to be broken. Please do report bugs upstream.
  • RDO and the Packstack tool are not supported products, except for a community forum and mailing list. Red Hat offers a commercial offering called RHOS if you need professional support.
  • The installation I describe is very simple and not suited for production even if the software was. It's a single-node setup without any high-availability mechanics. It does not include Neutron, Ceilometer, Heat or any of the even newer components.
  • I don't use \ to continue commands on the next line. Instead, the # simply marks a new command line, everything after it before normal text or another # belongs to the same line.

Fedora 19 Host Preparation

  1. Install Fedora 19 on your host. Yes, it can be done in a VM (with or without nested virtualization) if you wish so. Without nested virtualization, guests will obviously run very slow when run within a VM, though.
    Personally, I do Minimal Installs but you're probably also fine if you go with an Infrastructure Server installation with the Virtualization Add-On. The setup routine will install the missing parts either way. Obviously if you want to save some space, starting with a Minimal Install will help to keep the total installation size smaller.
  2. Make sure the latest updates are installed.
    # yum -y update
  3. Some parts might run into some trouble eventually, if your hostname is set to localhost. So if that's the case for you, change it to something different. You should be good by giving your loopback address another hostname but you can also add the name to your routable IP.
    # sed 's/localhost/openstack/' -i /etc/hostname
    # sed 's/^*$/& openstack openstack.localdomain/' -i /etc/hosts
  4. Fedora 19 does have some SELinux issues around Havana-2, most particularly with Swift. Since you're only doing this for testing purposes you shall exceptionally be allowed to set SELinux into permissive mode. Never do this in production, you're seriously tampering with your system's security.
    Bugs: rhbz#995779 and rhbz#995780
    # sed 's/^\(SELINUX=\).*$/\1permissive/' -i /etc/selinux/config
  5. The new firewall daemon is not (yet?) compatible with OpenStack so we better switch back to plain old iptables.
    Bugs: rhbz#981652 causing rhbz#981583
    # yum -y install iptables-services
    # systemctl disable firewalld
    # systemctl enable iptables
  6. To make all of the above changes effective (including a possible kernel upgrade), go ahead and
    # reboot

Install OpenStack Havana-2 from RDO

  1. Activate the RDO Havana-2 yum repository.
    # yum -y install
  2. Install Packstack, a simple tool to install OpenStack on Fedora or RHEL and derivatives. Using Python scripting and some Puppet-foo, it can turn answers, command-line switches or an answer file into a fully installed and configured OpenStack setup to make your life easier.
    # yum -y install openstack-packstack
  3. Unfortunately, one dependency will be missing from the installation, so we better install it up front.
    Bug: rhbz#995751
    # yum -y install fprintd-pam
  4. That's what you're here for. Now, we'll install a very simple OpenStack setup and it could hardly be any easier. You will be asked for your root password once, so Packstack can connect to the host over SSH and deploy a SSH key for future use. Packstack connects several times to all hosts it sets up in order to install and configure everything. In our case that's just a connection to localhost and SSH would not be required, but there's no separate routine for that use case. This will take a while as it will download and install a number of packages. Important: don't ever reboot after this step before you've performed the next one as well (I mention all the necessary reboots anyway).
    # packstack --allinone --os-neutron-install=n
    Note, if your installation should fail and you want to try again, there should be an answer file in either the folder you executed the above command in or in /root. Now, in this case, you must issue the following command instead of the one above or things might go very wrong (since passwords for services, databases, etc. are randomly generated at each run but saved in the answer file).
    # packstack --anwer-file packstack-answers-<date-time>.txt
  5. Due to a recent regression in LVM in Fedora 19, physical volumes on a loopback device won't be available after a reboot anymore. Packstack set up such a file for Cinder so we better change the boot script to first look for physical volumes using the cache before activating logical volumes.
    Bug: rhbz#997897
    # sed 's/vgchange/pvscan --cache \&\& &/' -i /etc/rc.d/rc.local
  6. Last but not least, Django, which is used by the Horizon web dashboard, introduced a security feature which is not set up properly yet so we've got to do that manually. We'll allow all hosts in this example but you could just insert your local hostname instead of the * in the command below.
    Bug: rhbz#988316
    # sed 's/^TEMPLATE_DEBUG.*$/&\n\nALLOWED_HOSTS = [\"*\"]/' -i /etc/openstack-dashboard/local_settings
  7. Restart Apache for the last config change to take effect.
    # systemctl restart httpd
That's it. If all went well (and it should, I tested it a couple of times and so did others) you now have a running OpenStack installation. Simply point your web browser at your installation and you should find a login page. The login details can be found in /root/keystonerc_admin (os_username and os_password). Sourcing that very file, you'll also be able to use the API tools.

Should you require community support or dive in deeper, RDO lives at

Flock 2013 Review

As many before me have blogged, Flock 2013 has taken place between August 9 and 12, 2013. Thanks to generous sponsors my travel and accommodation have been funded this time so I could make the trip and participate. Thanks to all who made this possible!

While many other bloggers found Flock to be very productive I personally found it less productive than some of the previous FUDCons in North America but still more productive than FUDCons in Europe. But that might well have two reasons. First, it's been the very first Flock and I'm sure lessons were learnt and the feedback showed the big gaps already. Secondly, my personal focus has shifted quite some and into a field that is only a smart part of Fedora: the Cloud (cloud images and OpenStack) so naturally less would be going on in that area than in many other areas of interest where Fedora is working on for a longer time already and more focused on anyway (like being an Linux OS).

So I attended a couple of sessions and most of them were interesting. But I don't have much to add to most of them, yet I'd like to comment shortly on two.

OpenStack Test Event

It was great to see that more people were interested in OpenStack than expected. We were an interesting mix of (upstream) developers, testers, (RDO) community folks and users. There's also been a few people who were looking for an introduction to OpenStack by this way so we begun with a quick overview by Russell Bryant. Afterwards we went on trying to install Havana-2 (i.e. the second milestone of the upcoming release) on Fedora 19 which Kashyap Charmathy prepared for us. Obviously, some did it to learn how OpenStack works first-hand and others like me were there to find and document existing bugs (we already knew it wouldn't run out of the box). Personally, I did quite some bug hunting and also found work around for all issues plus I tried my best to help new users with problems.

So in the end I had a number of bugs with workarounds plus one issue I didn't have the time to debug yet which I did later back home. From all this, I'm going to write a blog post so others can profit from my experience.

Last but not least, I must say it was great to finally meet some people I've been working with over IRC, mailing lists and forums for a while already. It's been a great pleasure with you all!

QA Meeting

Since Monday just before lunch time (EDT) is the regular QA IRC meeting time, we decided to do it LIVE at Flock. Agenda was mostly how to test ARM stuff (since it's now a primary architecture) and the cloud image (which is getting increasingly popular and important). Since many Fedora QA, Fedora ARM and Fedora Cloud SIG people were around it only made sense it sit together and do it this way. We also broadcasted (and archived) the show live to Youtube (as most sessions were) and had the usual IRC meeting running in parallel so remotees could still participate as usual. Well, almost (the live video feed lagged quite a bit behind and we were not very good and getting everything into IRC).

So again, it's been nice to see those people but this time I already knew most of them. Still, always good to see you all! Also, we quickly came to the conclusion that sometimes a face-to-face meeting like this would be helpful. Maybe we'll do it again over Hangout or such soon, or at least as a VoIP meeting.