Posts Tagged ‘uec’

Zero to Ensemble in 5 mins

// June 14th, 2011 // 11 Comments » // Uncategorized

So you’ve heard great things about Ensemble but didn’t have enough time to check it out yet? Fear no more! Check out this short video tutorial where I go through setting up ensemble for the very first time, and launching your first cloud wordpress deployment. This video is a step-by-step getting started with Ensemble, it only scratches the surface of Ensemble

If you can’t see the embedded player, here is a direct link

So, did this video help get you started with Ensemble ? Between this video and our previous blog about What Ensemble is, Do you feel it’s clear what Ensemble is and what it does? Let me know your thoughts

Creating Customized UEC and EC2 Images

// April 1st, 2011 // Comments Off // Uncategorized

Want to create a customized Ubuntu cloud image that you can deploy to EC2 or UEC ? No problem! Check out this 5 min video where I demo everything needed to download, customize, rebundle and publish the new image to EC2! Isn't this just great :)



If you can't see the embedded player, here's a direct link: http://www.youtube.com/watch?v=K1IwTGLWbu0

Creating Customized UEC and EC2 Images

// April 1st, 2011 // Comments Off // Uncategorized

Want to create a customized Ubuntu cloud image that you can deploy to EC2 or UEC ? No problem! Check out this 5 min video where I demo everything needed to download, customize, rebundle and publish the new image to EC2! Isn't this just great :)



If you can't see the embedded player, here's a direct link: http://www.youtube.com/watch?v=K1IwTGLWbu0

Ubuntu, the cloud OS

// March 2nd, 2011 // Comments Off // Uncategorized

We made a small flurry of announcements last week, all of which were related to cloud computing. I think it is worthwhile to put some context around Ubuntu and the cloud and explain a little more about where we are with this critical strategic strand for our beloved OS.

First of all, the announcements. We announced the release of Ubuntu Enterprise Cloud on Dell servers. This is a hugely significant advance in the realm of internal cloud provision. It’s essentially formalising a lot of the bespoke work that Dell has done in huge data centres (based on a variety of OSes) and making similar technology available for smaller deployments. We attended the Dell sales summit in Las Vegas and we were very encouraged to meet with many of the Dell salespeople whose job it will be to deliver this to their customers. This is a big company, backing a leading technology and encouraging businesses to start their investigations of cloud computing in a very real way.

More or less simultaneously, we announced our formal support for the OpenStack project and the inclusion of their Bexar release in our next version of Ubuntu, 11.04. This will be in addition to Eucalyptus, it is worth stating. Eucalyptus is the technology at the core of UEC – and will be in Ubuntu 11.04 – as it has been since 9.04. Including two stacks has caused some raised eyebrows but it is not an unusual position for Ubuntu. While we look to pick one technology for integration into the platform in order to deliver the best user experience possible, we also want to make sure that users have access to the best and most up to date free and open-source software. The increasing speed of innovation that cloud computing is driving has meant that Ubuntu, with its 6 month release cadence, is able to deliver the tools and programs that developers and admins want before any other operating system.

Users will ultimately decide what deployment scenarios each stack best suits. Eucalyptus certainly has the advantage of maturity right now, especially for internal cloud deployments. OpenStack, meanwhile, continue to focus on rapid feature development and, given its heritage, has appeal to service providers looking to stand up their own public clouds. Wherever the technology is deployed, be it in the enterprise or for public clouds, we want Ubuntu to be the underlying infrastructure for all the scenarios and will continue to direct our platform team to deliver the most tightly integrated solution possible.

Finally we saw our partner Autonomic Resources announce UEC is now available for purchase by Federal US government buyers. This is the first step on a long road the federal deployment, as anyone familiar with the governmental buying cycles will realise. But it is a good example of the built-to-purpose cloud environments that we will see more of – with the common denominator of Ubuntu at the core of it.

Which actually raises an interesting question – why is it that Ubuntu is at the heart of cloud computing? Perhaps we ought to look at more evidence before the theory. In addition to being the OS at the heart of new cloud infrastructures, we are seeing enormous usage of Ubuntu as the guest OS on the big public clouds, such as AWS and Rackspace, for instance. It is probably the most popular OS on those environments and others – contact your vendor to confirm :-)

So why is this OS that most incumbent vendors would dismiss as fringe, seeing such popularity in this new(ish) wave of computing? Well there are a host of technical reasons to do with modularity, footprint, image maintenance etc. But they are better expressed by others.

I think the reason for Ubuntu’s prominence is because it is innovation made easy. Getting on and doing things on Ubuntu is a friction-free experience. We meet more and more tech entrepreneurs who tell us how they have built more than one business on Ubuntu on the cloud. Removing licence costs and restrictions allows people to get to the market quickly.

But beyond speed, it is also about reducing risk. With open-source now firmly established in the IT industry, and with the term open used so promiscuously, it is easy to forget that the economic benefits of truly free, open-source software. The combination of cloud computing, where scale matters, and open source is a natural one and this is why Ubuntu is the answer for those who need the reassurance that they can both scale quickly but also avoid vendor lock-in in the long-term.

More specifically, and this brings us back to the announcements, there are now clear scenarios where users can reach a point where even the economics of a licence-free software on a public cloud start to break down. At a certain stage it is simply cheaper to make the hardware investment to run your own cloud infrastructure. Or there might be regulatory, cultural or a host of other reasons for wanting cloud-like efficiencies built on internal servers.

The work we have done with OpenStack and with Eucalyptus means Ubuntu is an ideal infrastructure on which to build a cloud. This will typically be for the internal provision of a cloud environment but equally could be the basis or a new public cloud. It is entirely open as to the type of guest OS and in all cases continues to support the dominant API of Amazon EC2, ensuring portability for those writing applications.

And as we have seen, Ubuntu is the ultimate OS to deploy in a cloud and with which to build a cloud. No-one provides more up-to-date images on the most popular public cloud platforms. Our work to ensure compatibility to the most popular standards means that those guests will run just as well on a UEC cloud however that is deployed – either internally or for cloud provision externally.

So technology moves markets. Economics does too, only more so. Ubuntu has come at the right point in our short IT history to ride both waves. The scale is there, the standards are emerging and the ability to provide an answer to the choice between running a cloud or running on a cloud is more fully realised on Ubuntu than on any other OS – open source or not.

Dell provides Ubuntu-powered IaaS-in-a-box

// February 3rd, 2011 // Comments Off // Uncategorized

Yesterday, the announcement went out that the Dell | Canonical Enterprise Cloud, Standard Edition was out and ready for consumption.  What this cloud-in-a-box allows folks to do is to set-up affordable Infrastructure-as-a-Service (Iaas)-style private clouds in their computer labs or data centers.  The cool thing is that, because the Ubuntu Enterprise Cloud (UEC) software  is compatible with Amazon Web Services EC2 and S3 services, it enables IT admins and developers to move workloads between public and private clouds.

Who cares?

Application developers and IT service providers and admins who are setting up cloud POC’s are perfect candidates for this pre-configured testing and development environment.  With regards to industries, areas where there is a lot of software development work like Hosters, Telco & Communications, Media & Entertainment and Web 2.0 businesses are prime markets for the Dell UEC solution.

So what’s in it?

The solutions’ basic components are Dell PowerEdge C systems plus a Dell-specific download of the Ubuntu Enterprise Cloud (made up of the Ubuntu operating system and the Eucalyptus platform for private cloud computing).  To simplify getting the whole shebang up and running Dell and Canonical are providing the following:

Here’s a peak at the hardware that supports it:

The Dell UEC cloud solution pod.

  • Cloud Compute Server – PowerEdge C6100 that embeds four discrete compute nodes in a single enclosure
  • Cloud Front-end Server – PowerEdge C2100 server that acts as an all-in-controller and runs all shared UEC-related services
  • Infrastructure Server – PowerEdge C2100 that runs two components of the cloud infrastructure:
    • Cloud Deployment and Landscape Management
    • Cloud Storage
  • Network switch – PowerConnect 6248

And on the software side…

The architecture looks something like this:

The Dell UEC cloud solution architecture

The software components are:

  • Cloud Controller (CLC) – the cloud portal
  • Walrus Controller (W) – the cloud’s storage repository
  • Cluster Controller (CC) – the controller for a up to 1024 compute cores grouped together as a cluster
  • Storage Controller (SC) – the controller for cluster’s storage repository
  • Compute Node (CN) – cloud’s compute node

And on the support side…

If you’re looking for systems management and support services with your order, you are in luck.  Canonical, the company behind Ubuntu, has put together UEC Assist, a support service built specifically for Dell customers deploying SE Edition and which is delivered by Canonical’s Global Services and Support team.

Its all about efficiency

From a Dell DCS (the group at Dell behind this) point of view, this offering fits in well with our strategy of bringing total solutions to market that optimize efficiency at every layer, from code to servers to storage.  The open source Dell UEC solution is tailor made to deliver a ready to go IaaS solution.

Extra-credit reading:

Pau for now..


Cloud Computing 101, p2

// November 18th, 2010 // Comments Off // Uncategorized

Continuing my part-1 post about cloud computing basics, this second post should continue to define different types of "clouds" as well as what you gain and loose by using them

If you look at a cloud solution, it's really a bunch of software layers stacked on top of each other. You have the hardware (servers, disks, switches, routers), you have bare metal operating systems, hypervisors, virtual servers and inside those you have programming languages (python, java, php), development frameworks, database servers, and your own business logic code living on top! Clouds are categorized as either IaaS, PaaS or SaaS. The type of cloud is basically defined by which layers of the stack the cloud abstracts away from you, and which layers you "own" and control. Another categorization scheme is private, public and hybrid clouds. Let's take a quick tour on what each of those cloud types mean

IaaS is Infrastructure as a Service. The cloud abstracts away as little as possible from you. Basically the cloud provides you with virtual servers, networking and storage and that's it. You use those building blocks, just as you would use them in any physical datacenter to build your own compute infrastructure. The only difference is that you don't worry about how the servers are powered, cooled, what brand of disk or SAN is used ..etc. All you care about is your provider's SLA as mentioned in part-1. Other than that, it's business as usual

PaaS is Platform as a Service. The cloud is abstracting away the infrastructure and some more. The cloud in this case is no longer composed of virtual servers and disks, it is however a "development framework". When you write code, you are coding against the platform, against the cloud itself. A PaaS cloud, assuming you're creating a web application, would tell you how to route requests to your handlers, how to write code to handle specific requests, would provide an API for storage, would perhaps provide an API for database (SQL, or noSQL doesn't matter here). Your application code is written against the API of the cloud. As such, you have no idea about "low level" details such as networking, IP addresses, failed servers or even the number of virtual servers running your code! So essentially you upload your code archive and it just runs on the cloud, no questions asked

SaaS is Software as a Service. In this case, you're only using software running on the cloud that someone else had written. If you've used facebook, Gmail, linkedin, Google docs, SalesForce ...etc that's it. In essence the service you're getting is the actual final "application" you need. This is the highest level of abstraction. You do not concern yourself with infrastructure, nor with code to build an application with. You pay to use the application itself and the SLAs you get are for the application availability and your data availability

What type of cloud suits you best, is a question that needs some thought and that depends on one's set of requirements. IaaS clouds provide the least abstraction and the most control! They are a good first step to migrating off-the-shelf software to the cloud and benefiting from cheap, on-demand, elastic infrastructure. Since they provide the least abstraction, if you'd like a scalable infrastructure, you would have to do all the work yourself. It is generally not so painful to migrate from an IaaS cloud to another. PaaS clouds however, since they provide higher levels of abstraction, are much easier to manage and scale. It essentially auto-scales delivering cloud computing holy grail. However the big price is that you generally have to rewrite your application to the particular cloud platform. Not only is that painful, but it also may lock you in to the cloud vendor making it extremely hard to change vendors afterwards. Which is why I think the open-source world needs great open-source PaaS cloud frameworks (Have a favorite? drop me a line in the comments section). If a SaaS application meets your needs at a good price point, then the only potential disadvantage would be data lock-in, as well as the (in)ability to mashup the SaaS application with other tools. A good piece of advice here is to choose SaaS applications that provide full API access to your data such that you can easily pull off all data and meta-data should you need to.

A different categorization of clouds is private vs public. Private simply means that the cloud infrastructure is built in-house behind the firewall. For example you could turn your corporate datacenter into a private cloud. The benefits being, you gain better efficiency and datacenter utilization across different departments as well as being able to provide an elastic and fast response to your enterprise's departmental IT needs. Should you want to start playing with a private cloud solution, Ubuntu Enterprise Cloud is a good start. Public clouds pertain to a cloud run by a third party service provider. A public cloud is either Iaas, PaaS or SaaS or even a mix of some. Why you would want to migrate some workloads to a public cloud, is simply because public cloud vendors due to their economies of scale are able to provide equivalent if not better service, at a significantly lower price point coupled by the ability to instantly grow. A hybrid cloud on the other hand is a private cloud that can "burst" to a public cloud when its resources are exhausted. The goal is to bring the best of both worlds, the control, data-security of private clouds with the elasticity and economies of large scale public clouds. More and more work-loads are being migrated to the cloud and it's all just starting.

Has your organization migrated some workloads to the cloud already, are you planning on that? Are you planning on building your own private cloud? Please let me know in the comments, let me know the motivations and the challenges you faced. If you have any questions in general, let me know as well

Chatting with Eucalyptus Co-founder about OpenStack

// November 12th, 2010 // Comments Off // Uncategorized

Continuing in my series of videos from the OpenStack design summit this week in San Antonio, here is an interview I did yesterday with Eucalyptus systems co-founder Graziano Obertelli.

Eucalyptus allows enterprises to set up open source infrastructure-as-a-service private clouds.  Eucalyptus is also one of the key ingredients in the Ubuntu Enterprise Cloud that is being certified to run on Dell’s PowerEdge C systems as part of our cloud ISV program.

Here is what Graziano had  to say:

Some of the ground Graziano covers:

  • What goals do the Eucalyptus team have for the summit
  • They’ve recently hired a community manager – Mark Atwood
  • What are their goals for their next release

Extra-credit reading:

Pau for now…


Cloud on Cloud, UEC on EC2

// October 22nd, 2010 // Comments Off // Uncategorized

So you wanted to play with Ubuntu Enterprise Cloud (UEC), but didn't have a couple of machines to play with ? Want to start a UEC instance right now, no problem. You can use an Amazon EC2 server instance as your base server to install and run UEC on! Of course the EC2 instance is itself a virtual machine, thus running a VM inside that would require nested virtualization which AFAIK wouldn't work over EC2. The trick here is that we switch UEC's hypervisor temporarily to be qemu. Of course this won't win any performance competitions, in fact it'd be quite slow in production, but for playing, it fits the bill just fine.

If you're thinking doing all that is gonna be complex, you have a point, however it won't! In fact it'll be very easy due to efforts of Ubuntu's always awesome Scott Moser. Scott has written a script that automates all needed steps. But wait, it gets better, we're not even going to run this script ourselves, we're passing it as a parameter to the EC2 instance invocation and due to Ubuntu's cloud-init technology, that script is going to be run upon instance boot-up, doing its work automagically. Now let's get started

On your local machine, let's install bzr and get the needed script

cd /tmp
sudo apt-get install bzr -y
bzr branch lp:~smoser/+junk/uec-on-ec2
cd uec-on-ec2/

The file "maverick-commands.txt" contains the script needed to turn the generic Ubuntu image on ec2 into a fully operational single-node UEC install. If you don't have "ssh keys" (seriously?) generate some

ssh-keygen -t rsa

Now let's do a neat trick to import the keys into EC2, marking them with the name "default"

for r in us-east-1 us-west-1 ap-southeast-1 eu-west-1; do ec2-import-keypair --region $r default --public-key-file ~/.ssh/id_rsa.pub ; done

Let's open a few needed ports in EC2's default security group

for port in 22 80 8443 8773; do ec2-authorize default -p $port ; done

Very well .. We now launch our EC2 instance, passing in the "maverick commands" file. What happens is, the server instance is created, booted, Ubuntu's cloud-init reads up the maverick commands script we passed to it, and executes it, it starts downloading, installing and configuring UEC in the background while you ssh into your new EC2 instance

ec2-run-instances ami-688c7801 --instance-type m1.large -k default --user-data-file=maverick-commands.txt

Give it a minute to boot, then try to ssh in
ec2-describe-instances
ssh ubuntu@ec2-a-b-c-d.compute-1.amazonaws.com

Replace the DNS name for the EC2 instance, with the proper one you get from ec2-describe-instances. Once logged into the EC2 instance, I would start byobu and tail the log file to monitor progress. The whole thing takes less than 5 minutes

byobu
tailf uec-setup.log

The script once finished configuring UEC, actually downloads a tiny linux distro and registers its image in UEC, so that you can start your own instances! You know the script has finished when you see a line that looks like
emi="emi-FDC21818"; eri="eri-53721963"; eki="eki-740D19EC";

UEC is now up and running, let's check the web interface! You login with the default credentials admin/admin

uec-on-ec2-1

Let's navigate to the Images tab, to get the EMI ID (Equivalent of an AMI ID)

uec-on-ec2-2

Is that cool or what, Hell Yes! Now let's start our own VM inside UEC that's inside EC2. Remeber to replace emi-FE03181A with the EMI ID you got from the web interface

euca-run-instances --key mykey --addressing private emi-FE03181A

You can use "euca-describe-instances" to get the new instance internal IP address and ssh to that

ubuntu@domU-12-31-38-01-85-91:~$ ssh -i euca/mykey.pem ubuntu@172.19.1.2
The authenticity of host '172.19.1.2 (172.19.1.2)' can't be established.
RSA key fingerprint is db:9b:47:a4:06:81:26:d7:cf:38:a4:0e:6c:05:54:0d.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.19.1.2' (RSA) to the list of known hosts.

Chop wood, carry water.

$ uname -r
2.6.35-16-virtual
$ df -h
Filesystem Size Used Available Use% Mounted on
/dev/sda1 23.2M 14.1M 7.9M 64% /
tmpfs 24.0K 0 24.0K 0% /dev/shm

et voila, you're ssh'ed into a ttylinux instance running inside qemu managed by UEC running over EC2 :) If you do find that cool, what about contributing back. How about you start hacking on that script to make it even more awesome, such as maybe installing over multiple nodes or whatever crazy idea you can think of! If you're interested to start hacking, drop me a hi in #ubuntu-cloud at Freenode

Seven Reasons to Deploy Your Enterprise Cloud on Ubuntu OS (Webcast I’m giving at Intel.com)

// September 28th, 2010 // Comments Off // Uncategorized

I'm giving a live webcast on Thursday, September 30, 2010 at 7:30am US Pacific time on Intel.com, through Intel's CloudBuilder program, where Canonical and Intel partnered to produce a whitepaper about the Ubuntu Enterprise Cloud.


The title and abstract:
Seven Reasons to Deploy Your Enterprise Cloud on Ubuntu OS

Join Canonical and Intel for a lively discussion on Canonical's Ubuntu OS, a key enabling technology for Enterprise Clouds. Learn how Ubuntu Enterprise Cloud implementations address Amazon EC2 compatibility, public/private cloud interoperability, Intel Virtualization Technology and more. Ask a question of our experts and gain insights that will guide your own cloud deployment.
To attend, go to:
Hope to see you there!

:-Dustin

Cloud in your Pocket — UEC LiveISO!

// June 15th, 2010 // Comments Off // Uncategorized

Background

At UDS in Belgium, I gave a 5-minute Lightning Talk during Friday's plenary. In that talk I gave a demonstration of a UEC LiveISO, based on Ubuntu 10.04 LTS. You can hear the 5 minute talk starting at the 29m30s mark of this audio file.

And you can find the 823MB ISO I used for my demonstration here:
I took the 10.04 LTS Desktop, and modified that ISO according to the instructions at:
I removed a ton of unnecessary packages for cloud computing, such as OpenOffice, Translations, and the Games, Graphics, Sound & Video applications. I also added the necessary Eucalyptus applications, and provided some initial configuration. I also pre-loaded a small, ttylinux image in the ISO itself (username=root, password=linux).

Motivation

Why would anyone want to do this? Well, for many of the same reasons Linux Desktops landed on LiveCDs. It's useful for testing, prototyping, and learning about the environment, before deploying to real hardware.

It used to be amazing that you could carry around a complete, bootable desktop operating system in your pocket. Now, you can carry a whole cloud! What's next? :-)

Caveats

So this LiveISO is completely unsupported right now. This was just a weekend hack that I put together because I thought it might be interesting, and because some people said it couldn't be done ;-)

This LiveISO is not meant to replace the Ubuntu 10.04 LTS Server ISO, from which I recommend you install UEC, if that's your goal.

How to Use this LiveISO

Download the ISO.

If you wan to try this out on real hardware, fire up Startup Disk Creator. Insert USB flash disk, at least 2GB in size (the bigger, the better). In Startup Disk Creator, create a very large persistence file with the slider toward the bottom.

You can now boot this on a system that has a CPU with VT extensions (try running kvm-ok to find out if your hardware supports VT), and with at least 4GB of memory. Sorry about the memory requirements -- Java is a memory hog.

Alternatively, you can actually be able to boot this ISO entirely inside of a KVM, if you can give it enough memory.

Basically, I gave this KVM all of my free memory (2300MB), and 2 CPUs, using this command:
kvm -m 2300 -smp 2 -cdrom ubuntu-10.04-ueclive-amd64-custom.iso
A few minutes after booting (~4 minutes), I can see within Byobu that all of the Eucalyptus services are running: CLC,WC,CC,SC,NC.



Once your CLC is running, you can grab your credentials:

sudo euca_conf --get-credentials mycreds.zip
unzip mycreds.zip
ln -s eucarc .eucarc

Now, register the node.

sudo euca_conf --register-nodes 127.0.0.1

And check your cluster's availability.

euca-describe-availability-zones verbose

With a little luck, you should see some free VMs!

Next, register an image. We included a tiny, ttylinux one on the ISO. You can try to run bigger images, but note that it will be rather slow (either you're reading/writing from slow flash media, or you're running in KVM, which means your VM will be running in non-accelerated QEMU).

uec-publish-image -K vmlinuz i386 tty-linux.img foo

And check your image's registration.

euca-describe-images

Now, run your image!

euca-run-instances emi-DEADBEEF --addressing private
watch -n 5 euca-describe-instances

You should that go to "running" eventually. If you're already running inside of KVM, this QEMU emulated virtual machine will be very slow. You should be able to ping it, and you might even eventually be able to ssh to it.

euca-authorize default -P tcp -p 22 -s 0.0.0.0/0
ping 172.19.1.2
ssh root@172.19.1.2

You can also explore the UEC administrative web interface. The LiveISO still has Firefox. You can point it to http://localhost, and login with admin/admin credentials.


So this is all fine and dandy if you want to try out UEC on your own hardware. But that's not very cloud of you... Why don't you just try out UEC in EC2? Do what, huh? Yup, totally doable too. The work described above inspired Scott Moser to publish his own how-to on the matter:


So What's Next?

Well, I'm actually trying to improve this a bit during the 10.10 cycle. You can follow our plans here. Basically, I'd like to create a UEC Live ISO seed, and have Ubuntu's cdimage publisher crank one out on a weekly basis. There are a few hiccups around auto-registration (which you might encounter in your testing of the current ISO) -- these should be fixed.

Hopefully you found this article useful, and might use this ISO in your own demonstration, education, and edification around Ubuntu's Enterprise Cloud!

:-Dustin