Ubuntu Server Trenches, The Big Picture

Posted by kim0 // October 6th, 2011 // Uncategorized

Ubuntu Oneiric 11.10 is shaping up to be a fantabulous release from the server team! I’m starting early celebrations for Oneiric by pushing out a series of Ubuntu Server from the trenches articles. Get to know all the hot new features landing in Oneiric server, as well as the people behind them! I start by interviewing Robbie the Ubuntu server team manager. Robbie is just great, always fun to be around! Ahmed Kamal (AK) will be asking Robbie to introduce the newest features of 11.10 as well as shed some light on the way forward. Let’s get started

AK: Hi Robbie, can you please introduce yourself, your background (professional and personal), your hobbies, kids

My name is Robert “Robbie” Williamson, and I currently manage the Ubuntu Server engineering team.  I’ve been working for Canonical for just over 3 years.  Previous to this role, I did a stint as Ubuntu Release Manager (10.10), Canonical CTO (briefly covering for Matt Zimmerman), and managed the Ubuntu Foundations engineering team.  I’ve been using Ubuntu for about 5 years, and Linux in general for around 11.  Before joining Canonical, I worked for 10 years in IBM’s Linux Technology Center, where I held various positions starting from a QA engineer up to manager.  One of my “claims to fame” is that I maintained and grew the Linux Test Project while at IBM, even going so far as to create the project logo :-).

I currently live in Austin, Texas (born and raised).  I’m married to my wife, Courtney, of 7 years this November:

and have two boys: Kalen (5) and Bryce (2)

When I’m not trying to make Ubuntu Server the best OS for the cloud, or doing the bazillion things a husband and parent of 2 young boys has to do daily, I’m usually either sleeping or eating….sound fun, right?  Heh, actually in terms of hobbies, or things I enjoy, I like playing basketball, watching sports, enjoying music (primarily HipHop, Rap, and R&B), and I’m known to occasionally “bust a move” every once and awhile when the mood….and alcohol strike me

AK: As the Ubuntu Server Engineering manager, what are the most important changes arriving in Ubuntu Server Oneiric (11.10)

There are loads of new features, fixes, and applications arriving in 11.10, but I’d say we have four main changes people should be aware of:

  1. Ubuntu Cloud – Ubuntu Cloud is the name of the portfolio of cloud technologies that we offer. It is composed of two halves:
    1. Ubuntu Cloud Infrastructure, which is the successor of Ubuntu Enterprise Cloud, is a ready to deploy Infrastructure-as-a-Service (IaaS) based on OpenStack Diablo. It is making it’s official entry into Ubuntu 11.10
    2. Ubuntu Cloud Guest, which used to be called Jeos or UEC-image, is the Ubuntu Server image specially tailored for use in public or private cloud infrastructure.
  2. Orchestra – A collection of what we think are the best free software services for provisioning, deploying, hosting, managing, and orchestrating enterprise data center infrastructure services, by, with, and for the Ubuntu Server. Orchestra, enables users to quickly deploy a solution in the datacenter. Instead of manually setting up a complex network installation environment, users can leverage Orchestra to rapidly deploy new servers in production using the best open-source tools. The process is standardised and fully automated, minimising manual intervention and ensuring consistency.  This is a direct response to all of our users who’ve been asking, and even begging, us to help them make doing multiple installs and deployments easier.
  3. juju – Juju is a service deployment and orchestration framework developed by Canonical and used to deploy and manage services both on bare-metal and in the cloud. Through the use of what we call charms,  juju provides you with shareable, re-usable, and repeatable expressions of DevOps best practices. You can use them unmodified, or easily change and connect them to fit your needs. Deploying a charm is similar to installing a package on Ubuntu: ask for it and it’s there, remove it and it’s completely gone. I’m pretty excited about this one because I feel we’re doing some real original thinking and innovation around solving real problems for users of the cloud.
  4. ARM – Ubuntu Server 11.10 will be the first release with support for the ARM architecture.  Usually thought of as something used in embedded or mobile devices, the ARM architecture has advanced tremendously over recent years.  We’re now at a point where ARM processors are able to handle workloads traditionally done by Intel and AMD based servers.  Canonical has a long history with ARM and the various partners around enabling Ubuntu on ARM devices.  We were also one of the first members of the Linaro project,  a not-for-profit software engineering company investing in core Linux software and tools for ARM SoCs.  Over this past cycle, we worked closely with the Ubuntu ARM team, as well as ARM partners interested in the ARM server market, to produce a tech preview of Ubuntu Server for ARM.  Our goal is to deliver a full blown server tuned OS for ARM in Ubuntu Server 12.04LTS, ready in time for the first official, production-ready ARM servers due to roll out in 2012.

AK: Ubuntu Server has been very successful in the cloud. Why do you think this is so? Is the Ubuntu server team focusing only on cloud technologies? What about legacy workloads?

I wish I could say our success occurred through careful planning on our part, but I’d be lying.  Sure, we are based on one of the most rock solid Linux distributions available, Debian.  We were one of the early adopters of cloud technology, in terms of private cloud deployments.  And we were the first Linux operating system to ship an AWS compatible private cloud deployment solution (based on Eucalyptus).  I’m not saying none of this didn’t help, but truth be told, I suspect a lot of the early users of the cloud used Ubuntu because that’s what they had on their desktops and laptops.  So in many ways, we owe our early success in the cloud to our success on the desktop.  I also feel that some of the core philosophies of how we deliver Ubuntu helps us in the cloud.  We strive to ship the latest and greatest versions of applications and technologies, so you can deploy for tomorrow, instead of yesterday.  Canonical doesn’t charge licensing fees for Ubuntu Server, no matter if you are deploying 1 server or 1000…into test/dev or production.  You only pay support for the installations you want support for.  Finally, Ubuntu’s long-standing commitment to delivering on a predictable and regular bi-annual cadence, down to the month, allows users to plan their solution roll outs without having to worry about a release date slip to ruin their plans…and possibly affect their revenue stream.

As for the Ubuntu Server team focusing only on cloud technologies, and possibly leaving legacy workloads behind, that’s just not true.  What is true, is that from a Canonical investment and strategic business perspective, the cloud is our top priority for Ubuntu Server.  This means Canonical-employed engineers working on features for Ubuntu Server will be focusing primarily on cloud technologies, both guest and host related.  However, as members of the Ubuntu Server team, I expect each and every one of my Ubuntu Server engineers to contribute and care for Ubuntu Server just like any other member of the wider community team.  This means mentoring new contributors, fixing bugs, syncing packages from upstream, and helping users who need it.  The last thing I’ll say about this is that, “legacy workloads” run in the cloud..and are ran in the cloud today, so abandoning them would be counterproductive to Canonical’s interests.

AK: The server team has been working hard to bring juju to life. Juju is often positioned as a paradigm shift, and different mindset to think about deploying and managing your infrastructure. How much do you think that is true? Why should someone be interested in playing with juju in the 11.10 release

Just as Debian’s apt/dpkg and archive concept was a huge evolutionary step forward in how applications were installed, configured, and shared amongst server admins, I see juju, charms, and the charm collection as being revolutionizing to the way multi-node services are deployed, controlled, and shared amongst the DevOps community, in the cloud or on bare-metal.  Juju aims to remove the hurdles of deploying and controlling complex, multi-node services, so users can focus on the solutions they need to deploy.  For example, if I need to create a presentation or document with LibreOffice, I don’t want to waste time downloading source tarballs, resolving libraries, flipping various compilation flags, and sorting out install paths…I just want to `apt-get install`, start it, and go.  The same can be said regarding someone needing to deploy OpenStack on bare-metal or Hadoop into the cloud.  They don’t want to waste time learning to configure and deploy these services.  It’s sometimes hard enough just to learn how to use them…they need them deployed fast, so they can focus on rolling out whatever solution or task their boss or CIO asked them to do. Through the use of charms and the charm collection, juju will allow users to harness, adapt, and easily share the experience of people skilled in deploying services they need, so they can just `juju deploy` to get their job done.

AK: Ubuntu Lucid had a well integrated private cloud solution built on top of Eucalyptus, Ubuntu server 11.10 made a switch to OpenStack. What’s your take on that?

Our switch to OpenStack as the core of our cloud infrastructure technology was based on three key points:

  1. The Code: We new early on that ARM server support was something we wanted to pursue.  If we were to elevate ARM support to being first class, it means doing everything we can to ensure that the default technologies and solutions we choose execute and perform within reason on ARM.  Eucalyptus is written in Java.  While completely functional, OpenJDK on ARM is still very slow performance wise, which means any cloud deployment based on it would be suboptimal.  Oracle sells a better performing Java implementation for ARM, but we didn’t want to force our users to use that just to have a cloud.
  2. The Process: OpenStack adopted many of the same tools and processes we use in Ubuntu. For example, they have an open design summit, where contributors can voice their opinions, introduce features, and directly influence the direction of the project.  Feature development is publicly tracked in, so you know where things are at any given time.  They publish a release schedule and maintain a cadence, so consumers of their project are better able to plan their own releases around it.  All of these things made it easier for us to integrate the project into our own.
  3. The Community: I’d be lying if I said we didn’t notice and take into consideration the wide-scale adoption and momentum that occurred within the OpenStack project early on.  Not to say that every opensource project that garners a lot of early attention and support is a success, but the types of companies joining and the contributions they were making gave us some comfort in knowing we were not alone.  Similar to the Linux kernel itself, we knew that if there was a critical issue raised, because so many people and companies were betting their futures on this project, because of its’ open nature, the issue would be fixed….and fixed quickly.

As I stated at the Oneiric 11.10 UDS, Eucalyptus is still a supported cloud solution with Ubuntu.  However, for this cycle, while I intended to keep the packages in the Ubuntu Main archive, they were moved to the Universe archive at the request of Eucalyptus.  This means that Eucalyptus will take over the support burden, from Canonical…which is fine, as I feel they are best positioned to do this anyway.

AK: How can an interested enterprise get help from Canonical implementing this cloud solution?

That’s an easy one :-)… just start here: http://www.ubuntu.com/business/services/cloud

AK: Oneiric leads to the next LTS release (12.04), can you let us know more information about plans for 12.04, as well as any long term strategy for Ubuntu Server

12.04 plans are pretty simple: harden the quality of everything.  We realize many of our users prefer to deploy the LTS because of its extended 5 year length of support.  With that in mind, I’ve put an emphasis on completing previously deferred/postponed 11.10 work items and fixing bugs this cycle, over adding new features. I’m not saying there won’t be anything new, but the scope and complexity of any feature we add will be reduced compared to non-LTS releases.  For example, in this cycle we introduced Orchestra (big), next cycle we might add improved client auto-registration, or in this cycle we introduced Ubuntu Cloud, next cycle we will try to make it even easier to deploy from the initial first server install, etc.

As for long term strategy, it’s difficult to say with any great detail because of the community-oriented and inclusive planning we do within the Ubuntu project itself. I can say that with cloud providers now rolling out their own Linux OS images, Canonical recognizes that Ubuntu could easily be pushed out, and thus our technological investment in Ubuntu Server over the next few years will remain cloud focused to help resist that.

For the record, I’m not saying a closely coupled OS is a bad idea, but I’m not sure it’s best for users long term.  From a short-term perspective, as a cloud provider, locking in users to your own OS makes sense, as it makes it harder for them to leave you for another provider, or even to transition to using their own private cloud.  In the long term, I believe this strategy will play out just as it did when Linux went up against the proprietary operating systems years ago, i.e. HPUX, AIX, etc.  Initially, having a tightly coupled OS to the hardware seemed like a benefit, until users wanted to migrate workloads and solutions between systems provided by different vendors.  Having an OS that you can run across all your hardware, that is opensourced for you to analyze and modify to suit your needs, and that’s developed in such a way that you can influence the design direction proved to be a much stronger benefit.

I know Ubuntu is the OS for the cloud, and I will do whatever it takes to stay on top.  To me, this means that from an engineering perspective, Ubuntu Server must constantly innovate and evaluate around three key decisions:

  1. What technologies we integrate and architectures we support to deliver the best Ubuntu Cloud Infrastructure
  2. How we improve the user’s experience running workloads within Ubuntu Cloud Guest
  3. How we best enable the orchestration of service applications, primarily focusing on Cloud and Big Data related deployments

If we can continue pushing the envelope around these areas, I’m confident Ubuntu Server will continue to succeed as the best operating system for the cloud.

AK: Thanks Robbie, we got a bunch of questions from our readers in a previous article. Robbie will answer a bunch of those. If you’d like to talk more, you can always join #ubuntu-server on freenode IRC

Reader Questions

Q: Any chance for a feature like a “ubuntu firewall” – like smoothwall but ubuntuwall in a way? -t94xr
A: We have a firewall solution called UFW – Uncomplicated FireWall, please see https://help.ubuntu.com/community/UFW

Q: Will Ubuntu be targetting/planning a drop in replacement for Windows Active Directory Server/Exchange mail services, or Windows Small Business Server? -James Carthew
A: No, not currently. However, I’d be more than willing to work with and help anyone interested in contributing this feature for 12.04LTS or beyond.

Q: Whitepaper on how to replicate a windows DOMAIN enviroment via UBUNTU and I am all yours. -Alexander Ashton
A: I don’t know of a white paper, but the website below has all you need for client-side integration, including links to information about applications we have in our Partner archive for integrating into an Active Directory environment.   https://help.ubuntu.com/community/ActiveDirectoryHowto If you need server-side, then you’ll need to use Samba4, which is packaged and installable from our Universe archive: http://packages.ubuntu.com/oneiric/samba4 The Samba project itself has a good HowTo on configuration: http://wiki.samba.org/index.php/Samba4/HOWTO

Q:any easy guide to create a ldap server with Ubuntu????thanks! -richardsith
A: https://help.ubuntu.com/10.04/serverguide/C/openldap-server.html
I also should mention that we are also strongly considering the elevation of SSSD (System Security Services Daemon) to the Main archive for 12.04LTS: https://fedorahosted.org/sssd/

Q:can we have ADS on ubuntu and client pc’s on windows, as my team is comfortable and do not want to switch to ubuntu. if yes how can maintain the ADS on ubuntu
A: Yes.  See my Samba4 response above.


AK: I hope you enjoyed this article, I’m personally looking forward to the next one. Let me know your thoughts and comments, shoot me comments

11 Responses to “Ubuntu Server Trenches, The Big Picture”

  1. jhansonxi says:

    “A: We have a firewall solution called UFW – Uncomplicated FireWall, please see https://help.ubuntu.com/community/UFW

    Would be nice if bug 659619 was solved and the profiles added.

  2. Jason Brooks says:

    Great post! I’ve been testing out the various components of Ubuntu Server 11.10, but I’ve had a hard time coming up with good docs on Ubuntu Cloud Infrastructure — the only one of the important changes list above w/o a link to more info.

    With the release of 11.10 coming next week, are there docs available on this feature? The best I’ve found so far, dealing specifically with Oneiric, is this http://zulcss.wordpress.com/2011/09/01/el-diablo-running-nova-on-oneiric, but it doesn’t quite cover all the bases.

  3. ckempo says:

    This was a good read.

    I think the question that prompted the UFW response was more geared towards a new *buntu distro that functions much like PFSense/m0n0wall/Smoothwall/Untangle do. Not that it’s a bad answer, but I’m not convinced that it’s an answer fully geared towards the question.

    And the samba4/AD stuff could have been asked by me – I now know I’m not alone, will be reading those links shortly. Thanks!

  4. kim0 says:

    Hey Jason,
    Yes ubuntu cloud infrastructure is under documented atm. However since it’s openstack based, upstream docs apply to it. So http://docs.openstack.org/ is a good start

    Apart from that, I know the dev team has been super busy integrating and testing the stack. Docs are definitely lagging, however they’re planned. I know a couple of devs will be writing such docs very soon. I’ll probably be helping as well. You can count on such docs being featured on this site cloud.ubuntu.com once they’re ready. Thanks for your interest

  5. kim0 says:

    If that was the intended question, then it sounds like an interesting project. If any member of the community team is interested to step up and work on that, ping me and I’ll be sure to help make it all happen.
    Regarding the second point, I now understand there’s a lot of interest regarding AD/samba4 with ubuntu server users. It would be very helpful if you or any other interested community member would test what’s available today, and start filing bugs against documentation or code to get things rolling faster

    As usual, ping me if you need anything and I’ll certainly help

  6. [...] of the most rock solid Linux distributions available, Debian. We were one of the early adopters of Cloud technology, in terms of private Cloud deployments. Just as Debians apt/dpkg [...]

  7. Jef Spaleta says:

    Two questions:
    1) Is the Ubuntu Image Store going to be revived in the transition to Openstack? Or is that offering being retired?

    2) Why has Canonical chosen not to use an utility pricing model for its Cloud guest support? A flat cost for support contract for up to 100 guests is not the sort of pricing model that has come to exemplify the cost effectiveness of cloud deployments. Why not price support with some aspect of per hour of use? For example Rackspace is offering managed cloud support which mixes a per hour per server utility fee plus a flat per month account fee. I quite frankly do not understand how Canonical plans to compete for support on public clouds (even when those public clouds run Ubuntu has the host and the guest) if the providers like RackSpace can offer utility-priced Ubuntu OS support for the same guest instances. How is Canonical’s flat support fee a better value?



  8. Gad-tech.com says:

    Ubuntu 11.10 to feature Arm support, cloud orchestration…

    By Joab Jackson | IDG News Service The next version of Canonical’s Ubuntu Linux distribution, to be released next week, will be the first to run on the Arm architecture, as well as the first edition to offer……

  9. [...] level overview of 11.10 features, as well as future cloud and server directions  Linux Read the full post on LXer… Share [...]

  10. [...] Ubuntu Server Trenches, The Big Picture Ubuntu Oneiric 11.10 is shaping up to be a fantabulous release from the server team! I’m starting early celebrations for Oneiric by pushing out a series of Ubuntu Server from the trenches articles. Get to know all the hot new features landing in Oneiric server, as well as the people behind them! I start by interviewing Robbie the Ubuntu server team manager. Robbie is just great, always fun to be around! Ahmed Kamal (AK) will be asking Robbie to introduce the newest features of 11.10 as well as shed some light on the way forward. Let’s get started [...]

  11. [...] #split {}#single {}#splitalign {margin-left: 0px; margin-right: auto;}#singlealign {margin-left: 0px; margin-right: auto;}#splittitlebox {text-align: center;}#singletitlebox {text-align: center;}.linkboxtext {line-height: 1.4em;}.linkboxcontainer {padding: 7px 7px 7px 7px;background-color:#eeeeee;border-color:#000000;border-width:0px; border-style:solid;}.linkboxdisplay {padding: 7px 7px 7px 7px;}.linkboxdisplay td {text-align: center;}.linkboxdisplay a:link {text-decoration: none;}.linkboxdisplay a:hover {text-decoration: underline;} function opensplitdropdown() { document.getElementById('splittablelinks').style.display = ''; document.getElementById('splitmouse').style.display = 'none'; var titleincell = document.getElementById('titleincell').value; if (titleincell == 'yes') {document.getElementById('splittitletext').style.display = 'none';} } function closesplitdropdown() { document.getElementById('splittablelinks').style.display = 'none'; document.getElementById('splitmouse').style.display = ''; var titleincell = document.getElementById('titleincell').value; if (titleincell == 'yes') {document.getElementById('splittitletext').style.display = '';} } The World Welcomes Oneiric Ocelot: Ubuntu 11.10Linux, a second class system?Ubuntu Server Trenches, The Big Picture [...]