Distributed Thinking

This seven minute report is mostly about the Large Hadron Collider at CERN, but any discussion of the LHC must mention the Grid. From our point of view, the novelty is not the technology - that's what we do - but seeing it explained on the national news.

I'm very pleased to report that OGF20 will be held in the UK: to be precise, at the Manchester International Convention Centre from May 7-11 next year. It will be co-located with the 2nd EGEE User Forum. Grid Computing Now! are organising a two-day industry outreach track.

This is something I've been working on for almost a year. At first my main interest was to get the event in the UK and run the GCN! part of the programme. This got me involved in choosing the venue and general planning; I am now programme chair of the whole shebang and devoting quite a bit of time to planning the sessions.

We have just announced a Call For Proposals for presentations, panels and workshops. This call covers all aspects of industry and academic use of Grids, including the GCN! track. Full details are available on the OGF web site. I would encourage everyone to submit suggestions; the deadline for submission is February 9th. Obviously I'd be happy to discuss this with anyone who is interested…

Power consumption is already on the minds of IT managers, having replaced space as the main (non-staff) cost of running IT centres. The recent publication of the Stern report on the economic effects of climate change [full report; BBC online coverage] gives extra incentive to look at ways of reducing power use.

Virtualisation and Grid seem obvious tools to investigate for this end. Case studies have shown they can more than double the utilisation rates of existing servers. The Grid Computing Now! team are looking into this approach and we're keen to hear your stories. Please get in touch if you have experience of increasing server utilisation - or if you need to control your power bills!

Last week we recorded and broadcast another Grid Computing Now! webinar. This featured Steve Wallage of The 451 Group and Shahid Mohammed of Marsh talking about the business case for Grid. Steve presented a distillation of the 451 Group's findings from talking to Grid users across a wide range of sectors. Shahid then gave us the benefit of his experience applying Grid in an e-commerce business. Both are good presenters and I recommend their talks.

You can see the webinar at http://brighttalk.com/comm/gridcomputingnow/3b03d97eee-2473-632-2319. This uses Flash and so should be compatible with most browsers.

Viewers submitted a good range of questions. A couple of people asked about the type of businesses - small, medium or large; finance, pharma or other - that can benefit from Grid. Shahid's talk demonstrated that reasonably small firms in a general commercial sector can benefit. Another question focussed on security. Both Steve and Shahid explained that Grid can be dep…

I've already mentioned last week's workshop on Virtual Organisations and Grids. You can now read my review of the event on the Grid Computing Now! web site.

I am gratified to see that GRIDtoday's summary of SC06 gave prominence to two UK projects - the social science simulation of population trends and the Grid monitor based on Google Earth. (I've mentioned both of these on this blog). The UK e-Science stand was only one of many in a large exhibition, so it's good to see that we made an impression.

We just held an interesting and I think successful workshop on Virtual Organisations and Grids, in which we looked at the limitations of the current state of the art and what is needed to support industrial requirements. A short summary is in the works and will appear on the Grid Computing Now! web site in a couple of days, with a longer report to follow in due course.

What I want to raise here is a more speculative notion that arose during the discussions - what would be the Web 2.0 implementation of a Virtual Organisation (VO)? Could users easily create their own VO, invite their colleagues to join, actually manage distributed access in a distributed fashion, etc.? Obviously, being Web 2.0, the legal and contractual issues would have to be minimal, so this would apply more to open-source projects.

The nearest model I can think of is more the SourceForge approach, in which anyone can create a project and invite others to join. The SourceForge (or equivalent) software provides core…

One last comment on SC06. We sent out a press release about Imperial College's new visual monitor of the EGEE Grid, which uses Google Earth to show the state of jobs submitted around the world and made a rather eye-catching display on the UK e-Science stand. This news even found its way into a paragraph on the science page of the Metro (the freesheet that is read by commuters on public transport across the UK).

This is how they rendered it:

GOOD GRID

The first map of the world's most powerful computer grid is to be unveiled this week. Nine of the largest grids are featured in the display at the Supercomputing festival in Tampa, Florida. The map uses Google Earth to pinpoint more than 300 sites on six continents. Grids are made up of hundreds or thousands of PCs, linked together to create a supercomputer. They are vital for scientists who need extra computing power to process large amounts of data.

As you can see, that's not quite right - Grids in general and EGEE in pa…

I'm back from SC06 and almost recovered from jetlag. We're in the middle of a workshop on Virtual Organisations, which I'll report on in due course. Right now I want to highlight another of the projects we demonstrated on the UK e-Science stand at SC06.

ClimatePrediction.net uses home enthusiasts to run climate models on their PCs, inspired by SETI@Home and similar projects. This has the advantage that it can run multiple versions of models with slightly varying parameters and look at the most likely results.

It's good and worthwhile science but what makes the project particularly worth noting is the work they've done with education and the media. They have created a schools pack so that the system can be used in school teaching (that's pre-college school in USA terms). As a result they have several schools running models on their IT labs.

The project has also been fortunate to attract the interest of the BBC, who included it in a documentary on climate chang…

One of the events on our SC06 stand was a vendors' round-table discussion on Grid licensing. Representatives from Platform, NAG, Visual Numerics, Allinea, among others, contributed their thoughts on how we can produce more flexible licensing schemes for Grid use cases. There are two levels of concern: technical and business.

At the technical level, some vendors expressed frustration that the licensing systems that are currently available are too restrictive. So even though they want to offer more flexible business models, the underlying technology prevents them from doing so. There was broad agreement that a usage-based monitoring technology would provide a more flexible substrate.

There were several scenarios offerred at the business level. One of the simplest is to charge directly based on use. Ohio State University want to do this for local SME's who are just setting out down the road to Grid adoption; these SMEs don't want to commit to a large upfront cost when the…

Kim Cameron's Identity weblog is worth keeping an RSS eye on. In this article he reports an Australian politician who is seeking to limit the uses of medical identity cards by law. I wish our UK politicians showed this level of understanding.

I'm particularly taken by one of the projects we're demonstrating on the UK e-Science Stand at SC06. "e-Social Science in action: a prototype geo-simulation portal" is, as the name suggests, applying Grid computing to the social sciences. It is simulating the interactions of individual households across an entire city to produce estimates of various factors such as health, housing, and car ownership. The simulations are based on UK census data. Users can then model the effects of various policy changes on these factors.

Under the hood, this is using standard Grid systems such as a compute cluster for running the simulations and SRB for managing the data. The system is easily scalable, so we might see more detailed simulations or see the system applied to larger cities such as London. There are some interesting research challenges left - I'm especially interested in the secure management of sensitive data - but the potential is clear.

The UK is at the leading edge o…

This week I'm at the SC06 Conference in Tampa, Florida, where I'm organising the content for the UK e-Science booth in the exhibition. This exhibition is large by the standards of most conferences, with booths from all the major vendors (IBM, Sun, HP, Microsoft, etc.) and research labs. It's a good opportunity to catch up with developments around the world.

Our booth has 10 demonstrations running continously. Researchers will explain to visitors what their science is all about and how the computing infrastructure helps them to achieve it. The demos range from physics (astronomy & particle physics) through to the social sciences, taking in the life sciences and computer science too.

We also have some interesting talks, including panel sessions on topics such as licensing and volunteer computing. We will also show a BBC horizon programme about climate change, which features one of our demos (climateprediction.net).

For more info, see http://www.nesc.ac.uk/events/sc06/.

ETSI recently announced they were entering the Grid standardisation space. This week I got the chance to learn more about this, thanks to Mike Fisher of BT, who leads the relevant ETSI Technical Committee. He explained that ETSI has a protocol and testing competence centre, along with the methods and experience to take specifications written in English and write conformance test suites that allow precise checks of interoperability. This is certainly a facility that the OGF does not have; indeed, the OGSA working group has carefully framed its work to avoid making any claim about conformance. So if the ETSI initiative can agree use cases and find sufficient consensus on standards, perhaps building on OGF work, it could add a valuable facility to the Grids standards community.

This week I attended a meeting at the ITU in Geneva on the topic of Next Generation Networks (NGNs) and Grids. NGNs are developed by the telco industry to provide a range of services over IP networks. The key to their design is the separation of service provision from transport mechanisms. Traffic may be routed over different types of network, each of which provides a common interface. The same traffic may in turn implement all sorts of services, including voice, television, videoconferencing, data transfer, multiplayer games, and whatever application designers come up with in the future.

So NGNs have a virtualised infrastructure providing a range of user services. They are using Web Services technology and are tackling issues such as security (authentication, authorisation, audit), accounting & billing, service description and deployment. Which all sounds familiar from the Grid world. That's what this workshop was about. In particular, it was to investigate opportunite…

This week I chaired a web seminar on Grid Markets. We had two excellent speakers: Dennis Kehoe from the AIMES Centre in Liverpool and John Darlington from the Imperial College Internet Centre. Between them they covered what is possible for businesses now and what we might expect in the future. After the presentations we had a fruitful discussion and answered some questions from the viewers.

This was the first time I've taken part in a web seminar and I know I made some mistakes - such as not introducing myself! A little more briefing by the hosting company would have helped but at least I'll know better for next time. Perhaps I'll even dare to watch myself on the stored video. In the meantime, if you would like to watch the seminar, click to http://brighttalk.com/event/gridcomputingnow/9b8619251a-130-intro-2-1.

In the last two weeks, several people have suggested to me that the UK could take more advantage of data collected here. Public agencies such as the Met. Office, Ordnance Survey, Transport for London, the British Crime Survey, the Land Registry and others hold data on various aspects of our lives. Private companies such as Experian and major retailers collect much more. If these were available as web services (in the broadest sense), inventive developers could mix and match them in new ways, displaying the results in meaningful ways (such as in map form).

Such activities are already popular in the USA, where more similar data sets are available. A widely-quoted example is Chicago Crime and there are many more. The question is, how can the UK catch up?

A few example projects are already underway. The DEWS project is combining Met Office data with health information and (separately) the coastguard. The Ordnance Survey is holding a Mashups day. Paul Longley's team at UCL caug…

This year's e-Science All-Hands Meeting has had a really good buzz going on. The conference is the same size as last week's OGF/GridWorld meeting in Washington, in a friendlier venue and with lots going on. This year's talks & demos seem more about applications than technology, which suggests that e-Science is maturing and becoming viable for more widespread use.

I'm here mainly wearing my Grid Computing Now! hat. We have a booth in which we present the DTI-funded R&D projects alongside our industry case studies. We've had quite a good interest from the delegates. From my point of view it's been good to see what the R&D projects are doing now that they are up and running. We had excellent demos from BROADEN, DEWS and Healthcare@Home.

Healthcare@Home is developing a system whereby diabetes patients can monitor key indicators (such as glucose level in the blood) at home, with readings transmitted to a server via mobile phone. This enables clinici…

GGF18 included a series of sessions that were intended to capture requirements from the Pharmaceutical and Electronic Design Automation industries, with the intended output of guiding and prioritising the standards work in OGF. These started well. A session on EDA produced a small set of priotised requirements. Then the Pharma folks had their turn and produced a similar set, with some overlaps.

Unfortunately things went wrong after that. These requirements were added to a veritable soup of other requirements, some of which were reasonable and some of which were high-level and vague. Then people made an attempt to order all these, which ran out of steam because the list was too long. The original focus from the industry speakers was lost.

The goal of these sessions was highly desirable but this time it wasn't achieved. Afterwards I heard industry representatives bemoaning the outcome. They received the impression that the OGF is rather an academic organisation that is not …

UK e-Science won an award in the first GridToday Readers' and Editors' Choice awards - see http://www.gridtoday.com/grid/888759.html.

I'm very pleased to say that the Grid Computing Now! KTN (see sidebar) will run our first web seminar on October 4. The title is The next-generation internet: Business opportunities and challenges for Grid markets.

This webinar will describe new mechanisms for business enabled by the next generation internet and computing infrastructures. Prof. Dennis Kehoe, Director of AiMeS (Advanced Internet Methods and Emergent Systems) will talk about current opportunities and challenges for utility computing and software services. He will be followed by Prof. John Darlington, Imperial College London, with a look at a future for Grid markets.

We expect this to be of interest to MDs of SMEs, CIOs, CTOs, business analysts and consultants. For more details, please contact tara.kelly@intellectuk.org.

What does the merger of the Global Grid Forum and the Enterprise Grid Alliance mean for the people actually developing Grid standards? This has been addressed in a couple of sessions here at GGF18. The first indications is that their efforts mesh quite well - or at least they avoid much conflict.

The EGA have produced a reference model, a security model and use cases, concentrating on the management of data centres. Their focus on the provisioning of data servers should nicely complement the existing OGSA work, which addresses provisioning of compute servers rather than data and considers only higher-level data services. Similarly the EGA reference model describes the components of a data centre at a higher level than current CIM work, which is where OGSA is currently concentrating its efforts. And of course the Enterprise requirements have a major focus on SLAs, QoS, policy management, billing and chargeback - long recognised by OGSA but not something they've got around to addres…

This morning's sessions at GGF18 in Washington showed the interesting contrast between the academic and commercial ideas of what "Grid computing" means. The academic view came from Ian Foster's classic paper, The Anatomy of the Grid, which was referenced by Dan Atkins of the USA National Science Foundation in his keynote speech. This view stresses the importance of a Virtual Organisation - a group of people from different organisations working together on a project or task. The term "Virtual Organisation" comes from business economics, so is not unknown to the commercial world - classic examples are the supply chain of a manufacturing process or the various organisations working together on a civil engineering project - but in the Grid world, VOs are more commonly seen in science Grids.

The commercial view was presented during the session on Enterprise Grid Requirements. Paul Strong of Ebay put it clearly: the focus is on removing silos between sub-organisat…

One of my many hats is as co-chair of the OGSA Data working group of the Open Grid Forum. We work on the data-oriented aspects of the Open Grid Services Architecture (see also here), covering the description, movement, access, replication, federation and storage of data in a Grid environment. Our aim is to provide a framework that links existing standards with those in development, and to show gaps that are not currently covered by standards work.

We had a fruitful face-to-face meeting here in Edinburgh last week, in which we concentrated on the descriptions of the interfaces in the architecture. The architecture is really a toolbox of data services that can be composed in various ways to address a range of use cases. As I expected, this focus helped us to to describe in greater detail how the various components will work together.

I believe we should have a draft of the architecture ready for public comment soon. I'm looking forward to actually completing this work. It's…

There is a key concept that links Utility Computing, Grid, Service-Oriented Architecture, Virtualisation and several more slippery terms as well. That concept is of computing as a commodity. This is not the same as computers being a commodity - we're all used to that, whether we're buying a home PC by mail order or building even mid-range servers from similar machines. Commodity computing lets us use computing resources or services when we need them and only when we need them. It takes several forms.

The most obvious example is Utility Computing, where a vendor (such as Sun or Amazon) provide you with CPU cycles or storage space and you pay for what you use. Web hosting companies operate on a similar basis, just slightly higher up the software stack. But other examples show that the same basic idea can operate within a company.

Trader Media are a classic example of how computing can be a virtual commodity within a company. They started with the classic setup of separate …

Welcome to Distributed Thinking. In this blog I will present my thoughts and experiences in the world of distributed computing, Grid, e-Science, utility computing, service-oriented architectures, virtualisation, Web 2.0 and other similar technologies. Via my work at the National e-Science Centre, I'm involved in several projects and other activities which have snippets of interest to a broader community. Some of these appear in more formal forums, including www.gridcomputingnow.org; this blog gives me the chance to mention smaller items, to discuss things of interest and to express the occasional personal opinion. I hope that you enjoy reading it.