Distributed Thinking

I've already mentioned last week's workshop on Virtual Organisations and Grids . You can now read my review of the event on the Grid Computing Now! web site.

I am gratified to see that GRIDtoday's summary of SC06 gave prominence to two UK projects - the social science simulation of population trends and the Grid monitor based on Google Earth . (I've mentioned both of these on this blog). The UK e-Science stand was only one of many in a large exhibition, so it's good to see that we made an impression.

We just held an interesting and I think successful workshop on Virtual Organisations and Grids , in which we looked at the limitations of the current state of the art and what is needed to support industrial requirements. A short summary is in the works and will appear on the Grid Computing Now! web site in a couple of days, with a longer report to follow in due course. What I want to raise here is a more speculative notion that arose during the discussions - what would be the Web 2.0 implementation of a Virtual Organisation (VO)? Could users easily create their own VO, invite their colleagues to join, actually manage distributed access in a distributed fashion, etc.? Obviously, being Web 2.0, the legal and contractual issues would have to be minimal, so this would apply more to open-source projects. The nearest model I can think of is more the SourceForge approach, in which anyone can create a project and invite others to join. The SourceForge (or equivalent) software provides cor...

One last comment on SC06. We sent out a press release about Imperial College's new visual monitor of the EGEE Grid, which uses Google Earth to show the state of jobs submitted around the world and made a rather eye-catching display on the UK e-Science stand. This news even found its way into a paragraph on the science page of the Metro (the freesheet that is read by commuters on public transport across the UK). This is how they rendered it: GOOD GRID The first map of the world's most powerful computer grid is to be unveiled this week. Nine of the largest grids are featured in the display at the Supercomputing festival in Tampa, Florida. The map uses Google Earth to pinpoint more than 300 sites on six continents. Grids are made up of hundreds or thousands of PCs, linked together to create a supercomputer. They are vital for scientists who need extra computing power to process large amounts of data. As you can see, that's not quite right - Grids in general and EGEE in ...

I'm back from SC06 and almost recovered from jetlag. We're in the middle of a workshop on Virtual Organisations, which I'll report on in due course. Right now I want to highlight another of the projects we demonstrated on the UK e-Science stand at SC06. ClimatePrediction.net uses home enthusiasts to run climate models on their PCs, inspired by SETI@Home and similar projects. This has the advantage that it can run multiple versions of models with slightly varying parameters and look at the most likely results. It's good and worthwhile science but what makes the project particularly worth noting is the work they've done with education and the media. They have created a schools pack so that the system can be used in school teaching (that's pre-college school in USA terms). As a result they have several schools running models on their IT labs. The project has also been fortunate to attract the interest of the BBC, who included it in a documentary on climate ch...

One of the events on our SC06 stand was a vendors' round-table discussion on Grid licensing. Representatives from Platform, NAG, Visual Numerics, Allinea, among others, contributed their thoughts on how we can produce more flexible licensing schemes for Grid use cases. There are two levels of concern: technical and business. At the technical level, some vendors expressed frustration that the licensing systems that are currently available are too restrictive. So even though they want to offer more flexible business models, the underlying technology prevents them from doing so. There was broad agreement that a usage-based monitoring technology would provide a more flexible substrate. There were several scenarios offerred at the business level. One of the simplest is to charge directly based on use. Ohio State University want to do this for local SME's who are just setting out down the road to Grid adoption; these SMEs don't want to commit to a large upfront cost when th...

Kim Cameron's Identity weblog is worth keeping an RSS eye on. In this article he reports an Australian politician who is seeking to limit the uses of medical identity cards by law. I wish our UK politicians showed this level of understanding.

I'm particularly taken by one of the projects we're demonstrating on the UK e-Science Stand at SC06. " e-Social Science in action: a prototype geo-simulation portal " is, as the name suggests, applying Grid computing to the social sciences. It is simulating the interactions of individual households across an entire city to produce estimates of various factors such as health, housing, and car ownership. The simulations are based on UK census data. Users can then model the effects of various policy changes on these factors. Under the hood, this is using standard Grid systems such as a compute cluster for running the simulations and SRB for managing the data. The system is easily scalable, so we might see more detailed simulations or see the system applied to larger cities such as London. There are some interesting research challenges left - I'm especially interested in the secure management of sensitive data - but the potential is clear. The UK is at the leading edg...

This week I'm at the SC06 Conference in Tampa, Florida, where I'm organising the content for the UK e-Science booth in the exhibition. This exhibition is large by the standards of most conferences, with booths from all the major vendors (IBM, Sun, HP, Microsoft, etc.) and research labs. It's a good opportunity to catch up with developments around the world. Our booth has 10 demonstrations running continously. Researchers will explain to visitors what their science is all about and how the computing infrastructure helps them to achieve it. The demos range from physics (astronomy & particle physics) through to the social sciences, taking in the life sciences and computer science too. We also have some interesting talks, including panel sessions on topics such as licensing and volunteer computing. We will also show a BBC horizon programme about climate change, which features one of our demos (climateprediction.net). For more info, see http://www.nesc.ac.uk/events/sc06...

ETSI recently announced they were entering the Grid standardisation space. This week I got the chance to learn more about this, thanks to Mike Fisher of BT, who leads the relevant ETSI Technical Committee. He explained that ETSI has a protocol and testing competence centre, along with the methods and experience to take specifications written in English and write conformance test suites that allow precise checks of interoperability. This is certainly a facility that the OGF does not have; indeed, the OGSA working group has carefully framed its work to avoid making any claim about conformance. So if the ETSI initiative can agree use cases and find sufficient consensus on standards, perhaps building on OGF work, it could add a valuable facility to the Grids standards community.

This week I attended a meeting at the ITU in Geneva on the topic of Next Generation Networks (NGNs) and Grids. NGNs are developed by the telco industry to provide a range of services over IP networks. The key to their design is the separation of service provision from transport mechanisms. Traffic may be routed over different types of network, each of which provides a common interface. The same traffic may in turn implement all sorts of services, including voice, television, videoconferencing, data transfer, multiplayer games, and whatever application designers come up with in the future. So NGNs have a virtualised infrastructure providing a range of user services. They are using Web Services technology and are tackling issues such as security (authentication, authorisation, audit), accounting & billing, service description and deployment. Which all sounds familiar from the Grid world. That's what this workshop was about. In particular, it was to investigate opportunit...

This week I chaired a web seminar on Grid Markets. We had two excellent speakers: Dennis Kehoe from the AIMES Centre in Liverpool and John Darlington from the Imperial College Internet Centre. Between them they covered what is possible for businesses now and what we might expect in the future. After the presentations we had a fruitful discussion and answered some questions from the viewers. This was the first time I've taken part in a web seminar and I know I made some mistakes - such as not introducing myself! A little more briefing by the hosting company would have helped but at least I'll know better for next time. Perhaps I'll even dare to watch myself on the stored video. In the meantime, if you would like to watch the seminar, click to http://brighttalk.com/event/gridcomputingnow/9b8619251a-130-intro-2-1 .

In the last two weeks, several people have suggested to me that the UK could take more advantage of data collected here. Public agencies such as the Met. Office, Ordnance Survey, Transport for London, the British Crime Survey, the Land Registry and others hold data on various aspects of our lives. Private companies such as Experian and major retailers collect much more. If these were available as web services (in the broadest sense), inventive developers could mix and match them in new ways, displaying the results in meaningful ways (such as in map form). Such activities are already popular in the USA, where more similar data sets are available. A widely-quoted example is Chicago Crime and there are many more. The question is, how can the UK catch up? A few example projects are already underway. The DEWS project is combining Met Office data with health information and (separately) the coastguard. The Ordnance Survey is holding a Mashups day. Paul Longley's team at UCL c...

This year's e-Science All-Hands Meeting has had a really good buzz going on. The conference is the same size as last week's OGF/GridWorld meeting in Washington, in a friendlier venue and with lots going on. This year's talks & demos seem more about applications than technology, which suggests that e-Science is maturing and becoming viable for more widespread use. I'm here mainly wearing my Grid Computing Now! hat. We have a booth in which we present the DTI-funded R&D projects alongside our industry case studies. We've had quite a good interest from the delegates. From my point of view it's been good to see what the R&D projects are doing now that they are up and running. We had excellent demos from BROADEN, DEWS and Healthcare@Home. Healthcare@Home is developing a system whereby diabetes patients can monitor key indicators (such as glucose level in the blood) at home, with readings transmitted to a server via mobile phone. This enables clin...

GGF18 included a series of sessions that were intended to capture requirements from the Pharmaceutical and Electronic Design Automation industries, with the intended output of guiding and prioritising the standards work in OGF. These started well. A session on EDA produced a small set of priotised requirements. Then the Pharma folks had their turn and produced a similar set, with some overlaps. Unfortunately things went wrong after that. These requirements were added to a veritable soup of other requirements, some of which were reasonable and some of which were high-level and vague. Then people made an attempt to order all these, which ran out of steam because the list was too long. The original focus from the industry speakers was lost. The goal of these sessions was highly desirable but this time it wasn't achieved. Afterwards I heard industry representatives bemoaning the outcome. They received the impression that the OGF is rather an academic organisation that is not ...

UK e-Science won an award in the first GridToday Readers' and Editors' Choice awards - see http://www.gridtoday.com/grid/888759.html .

I'm very pleased to say that the Grid Computing Now! KTN (see sidebar) will run our first web seminar on October 4. The title is The next-generation internet: Business opportunities and challenges for Grid markets . This webinar will describe new mechanisms for business enabled by the next generation internet and computing infrastructures. Prof. Dennis Kehoe, Director of AiMeS (Advanced Internet Methods and Emergent Systems) will talk about current opportunities and challenges for utility computing and software services. He will be followed by Prof. John Darlington, Imperial College London, with a look at a future for Grid markets. We expect this to be of interest to MDs of SMEs, CIOs, CTOs, business analysts and consultants. For more details, please contact tara.kelly@intellectuk.org .

What does the merger of the Global Grid Forum and the Enterprise Grid Alliance mean for the people actually developing Grid standards? This has been addressed in a couple of sessions here at GGF18. The first indications is that their efforts mesh quite well - or at least they avoid much conflict. The EGA have produced a reference model, a security model and use cases, concentrating on the management of data centres. Their focus on the provisioning of data servers should nicely complement the existing OGSA work, which addresses provisioning of compute servers rather than data and considers only higher-level data services. Similarly the EGA reference model describes the components of a data centre at a higher level than current CIM work, which is where OGSA is currently concentrating its efforts. And of course the Enterprise requirements have a major focus on SLAs, QoS, policy management, billing and chargeback - long recognised by OGSA but not something they've got around to addres...

This morning's sessions at GGF18 in Washington showed the interesting contrast between the academic and commercial ideas of what "Grid computing" means. The academic view came from Ian Foster's classic paper, The Anatomy of the Grid , which was referenced by Dan Atkins of the USA National Science Foundation in his keynote speech. This view stresses the importance of a Virtual Organisation - a group of people from different organisations working together on a project or task. The term "Virtual Organisation" comes from business economics, so is not unknown to the commercial world - classic examples are the supply chain of a manufacturing process or the various organisations working together on a civil engineering project - but in the Grid world, VOs are more commonly seen in science Grids. The commercial view was presented during the session on Enterprise Grid Requirements . Paul Strong of Ebay put it clearly: the focus is on removing silos between sub-organi...

One of my many hats is as co-chair of the OGSA Data working group of the Open Grid Forum . We work on the data-oriented aspects of the Open Grid Services Architecture (see also here ), covering the description, movement, access, replication, federation and storage of data in a Grid environment. Our aim is to provide a framework that links existing standards with those in development, and to show gaps that are not currently covered by standards work. We had a fruitful face-to-face meeting here in Edinburgh last week, in which we concentrated on the descriptions of the interfaces in the architecture. The architecture is really a toolbox of data services that can be composed in various ways to address a range of use cases. As I expected, this focus helped us to to describe in greater detail how the various components will work together. I believe we should have a draft of the architecture ready for public comment soon. I'm looking forward to actually completing this work. It...