Distributed Thinking

Development Services have a new blog.  We are using this as a place for members of the section to share information, experience and thoughts relating to our work.  Anyone in the section can contribute posts.  We expect most of the posts to cover technical material, such as useful coding techniques, experience from particular projects, and so forth. See http://www.appsdev.is.ed.ac.uk/blog/ .

Following from my previous post , this is my response to a second thought-provoking article that criticises a codification of agile software development into rigid project management frameworks.  In that article, Mike Hadlow asks why agile has come to mean just management practices (stand-ups, retrospectives, two-week iterations and planning poker), divorced from any base in technical practices.  He bemoans projects in which non-technical people are given the role of Scrum Master, enforcing agile rituals without understanding what the team are actually doing. I can see that Mike's scenario would be problematic.  I have seen examples in our organisation where some non-development staff may have thought they could control an agile team (e.g. as a business analyst) but the structure of the agile team has (correctly) worked against them.  In general, it is certainly easier to explain daily stand-up meetings and two-week iterations to non-technical people than it is to...

In the last week I've seen a couple of blog posts that bemoan a codification of agile software development into rigid project management frameworks.  Both are interesting and caused me to reflect on our own adoption of agile - because to a fair extent we are doing exactly what these posts criticise. The first such post is titled The Dark Side of Scrum .   Thomas Scranz asks why, if the first principle of the agile manifesto is that we value individuals and interactions over processes and tools, have people adopted Scrum as a dogmatic codified interpretation of agile?  I will note my thoughts in this post. (I hould clarify that we're not following Scrum per se; our process started nearer XP and has adopted some notions from Scrum and DSDM along the way). Thomas first argues that with continuous integration and automated testing tools, we can move beyond fixed two-week time boxes to a faster and more flexible delivery cycle .  Well, that may be true for an organisa...

The last month we seem to have spent a lot of time dealing with projects to upgrade various  systems.  We run a large number of such systems, covering everything from virtual learning environments and wikis to student record systems and timetabling, and also encompassing underlying middleware software.  I say "vendor systems" but some of these are open-source projects maintained by a global community.  Others are traditional vendors, ranging from large multi-national corporations to niche vendors of HE products. We have encountered problems with a number of systems, from various sources.    Some of these have been bugs in the third-party software.  I won't name the systems, nor even say what functions they perform, as I don't want to bring any particular vendor into disrepute.  Other issues have turned out to be caused by mistakes in our own systems.  In one failed upgrade, we kept the existing system running on one site while we upgr...

Friday last week was a bit different from the usual, as I went down to York to sit on an interview panel.  York are recruiting a full-time Enterprise Architect (similar to the post that Nikki Rogers holds at Bristol) with responsibility for shaping the whole contribution of IT to the university's business and my role was to provide an external point of view.  I was very interested to learn more about their plans and to compare the architecture situation at York with our own experience. Each of the candidates had to start their day with a ten-minute presentation about how they would persuade senior managers of the benefits of enterprise architecture.  These presentations were interesting in themselves and I may borrow one or two of the ideas put forward.  Obviously I can't say much about the interviews themselves, but I will say that this is hard role to fill. The ideal candidate would have a breadth of IT experience, ability to engage confidently with senior mana...

I'm beginning to wind down for the xmas break - just the Student Systems Board tomorrow, a meeting of the Applications Architecture Governance Group, a review of the first phase of development for the new university web site, then a xmas curry with the web site team on Friday, and that will be me off for a couple of weeks.  But before I finish, I want to record some thoughts about our IT Futures conference for 2013 , which was held last week. We started planning this in January, when we were thinking about ideas for a theme and potential speakers.  I had the idea of inviting a science-fiction writer to give an outsider's view of what higher education might look like in the future.  This was tossed around a bit and Jen Ross suggested Cory Doctorow.  Meanwhile, the theme of "disruption" rose to the surface.  We put the two together and a potential agenda began to take shape. The next step was to add a less technical and more people-oriented take on "disruption...

On Wednesday evening, I had an “Aha!” moment when listening to Steve Freeman talk about Test-Driven Development: That’s not what we meant at the local branch of the BCS.   Apparently there has been a bit of backlash against TDD recently, with people complaining that their test suites are so big and fragile that they take more effort to maintain than the actual code does. Steve was explaining his approach to writing tests and how to avoid these pitfalls. As part of his presentation, he advocated that tests should be written for interfaces not implementations, and in fact for protocols rather than interfaces.  He contrasted an example of good practice, using a combination of externally visible methods to test behaviour, with a poor example that used a backdoor to twiddle internal state.  He remarked that it was only when he started to write these sort of tests that he finally understood the oft-repeated mantra that logical models should be separated from the impleme...

I just passed a group of developers and project managers playing poker in the kitchen, surrounding by plates of cake.  This made me happy. The "poker" was planning poker - a technique for estimating user stories in agile projects.  The players simultaneously choose values for how much work they think a user story will take.  This approach ensures that everyone gets a say, without one person having to go first and possibly influence the estimate more than they should. The cake was real cake.  Some of our enterprising project managers are running a home-bake stall to raise money for Children in Need .  Everyone looks well-fed today. What I was actually going to write about today was the survey that we organised over the summer on the uptake of agile software development and agile techniques in Higher Education across the UK, America and Canada.  We posted the invitation on the UCISA PCMG mailing list and the EDUCAUSE Project mailing list in and ...

Bill Lee and Dawn Nicholls gave a very successful presentation about our experience with Agile project methods at this year's EDUCAUSE conference in California.  You can see their slides at the link below. Darwin... slides Bill tells me that he knew the session was going well when everyone in the audience closed their laptops and put away their mobile phones.  These days, thereare visible signs that your audience is paying attention!  Dawn and Bill had worked very hard preparing their joint presentation.  It took more effort working together than a one-person presentation would have done, but it produced a better show. This was the first time we have presented a session at the prestigious EDUCAUSE conference.  Members of IS have attended the conference for several years and we believe that the University of Edinburgh is "up there" with the world's best - ahead in some areas, lagging in others.  So this year we decided to submit some proposals for pres...

I have reorganised Development Services for the new academic year.  Instead of two large teams, one for software development and one for software configuration, we now have five software teams aligned with the main administrative units of the University.  We have also taken on new staff: six new people joined Development Services this month and we have three more on the way. This is not change for the sake of it, although I do believe in moving people around occasionally.  For one thing, the old structure was becoming too unwieldy, particularly in the student & academic area where we had one dedicated sub-team sharing a manager with the more general configuration team.  To grow the section, we needed more teams and more team leaders. Another reason for change was to create teams that work closely with particular partners.  The idea is to simplify the scheduling of projects throughout the year, with the team leader and programme leader working together to...

Last week we attended a workshop that brought together people from universities across the UK who are working on various forms of Service-Oriented Architecture (SOA) to underpin their business IT.  The workshop was organised by Nikki Rogers at the University of Bristol.  Bristol have the the foresight to employ Nikki as a full-time enterprise architect, which they should be congratulated on.  Nikki has written an excellent report of the workshop on her blog .  (Her blog is worth subscribing to, by the way). What struck me about the day was the variety of architectural patterns that people were pursuing under the general heading of SOA.  This echoed our own situation at Edinburgh, where we have got some traction for the basic ideas of service-orientation and now have to articulate our vision for the next steps.  Cardiff University, for example, are basically implementing a real-time data warehouse , with web services making the data available to downst...

As we attempt to provide round-the-clock IT services without employing out-of-hours staff, our worst nightmare for an "incident" is one that would affect the whole machine room, starting after everyone has left work on a Friday evening, with the whole weekend until normal work resumes on Monday morning.  It would be even worse if this were to occur just before the start of a new academic year. So guess what happened last weekend? One of the power supplies in our main machine room caught fire as a result of an electrical fault.  Although the fire was quickly contained, the emergency services shut off all power to the machine room as a precaution.  Several of our major services became unavailable and staff had to be called in over the weekend to fix everything. The good news was that those  services that are designed to automatically fail-over to the backup machine room did so. Also, the support team had all our top priority services back up by six o'clock on the...

I've been asked about the University's Research Data Initiative and what effect it will have on the work of Applications Division. I'm not actually involved with the initiative but I do know some of the background, so in this post I will attempt to explain briefly what research data is about in general and why it is important, then give a few links about the University's initiative. The root of all this is the computers and networking are introducing new ways for to academics to undertake and share their research.   Sensors, microscopes, telescopes, particle accelerators, satellites, social media, audio and video can all produce vast amounts of data that can be of interest to researchers. We can store, search and analyse petabytes of data, and make data available for others to share. For example, astronomers are collecting "sky surveys" - images of the entire sky, in various wavelengths (radio, Xray, visual, infra-red, ...), matching the objects between ...

I'm pleased to see that UCISA's Executive Secretary has written a blog post about the Unidesk shared service for service management.  This was set up by the universities of Edinburgh, St. Andrews and Abertay, who were joined last year by Sheffield Hallam.  Unidesk currently provides facilities for incident reporting and handling (i.e. service calls), enhancement request, and for managing change & release.  The next development of the service will add a configuration management database. I am impressed that Peter managed to write his blog post without once mentioning the ITIL framework for service management.  From our point of view, the entire aim of the Unidesk project was to provide a comprehensive service for the ITIL set of practices.  Peter is more interested in how the shared service was set up and managed. Unidesk has been a great success for us.  It hasn't always been plain sailing behind the scenes, as you would expect with any complex I...

On the rare occasions when we computer folk get to collaborate with artists, I'm always fascinated by the different ways we approach design.  I first noticed this back when I was working for the National e-Science Centre and an art project used our video conferencing system for a live performance project.   During the performance the video system started to glitch.  Instead of stopping and fixing the problem, the musicians and dancers started to incorporate the glitch into the work.  That was neat. More recently, some of my colleagues have worked with artists on a couple of web applications.  These collaborations have exposed wide differences in the ways people think.  For one project, we did our usual approach of producing wire-frame outlines of the user interface, with the aim of letting sample users step through a draft design.  To our surprise, the users couldn't handle this sort of abstraction.   They needed to see a completed gra...

I am delighted that Applications Division will be giving two presentations at this year's EDUCAUSE conference in California.  Traditionally we have found it hard to get proposals accepted, so it is good to have this opportunity to be recognised internationally for the quality of our work. The two talks will be: "Darwin, Finches and poker chips - an Agile journey", by Dawn Nicholls and Bill Lee "From Black Hole to Gold Mine - Adding Analytics to the University Portal", by Martin Morrey I've been closely involved with our adoption of Agile methods.  Dawn and Bill will be presenting what we have learned, especially regarding techniques for engaging users in the project prioritisation procedures.  They are aiming to use Agile techniques in the session itself, giving delegates the chance to vote on which topics to discuss most. Martin has done great work on making information available about which tools our students and staff actually use.  As thi...

I first came across the notion of " technical debt " in a web article about Agile development methods. The phrase instantly struck me as a pithy summary of all the problems that IT people experience with poor quality code that is rushed out to meet deadlines, code that will cause problems down the line.  We know the problems are there, waiting to hurt us, while our business partners tend to be oblivious to them. So I was attracted by a local BCS Event on the topic.  I went along this evening to hear Ali Law lead an interesting and wide-ranging discussion.  Ali gave me a lot to consider.  I've always thought of technical debt in terms of maintainability, i.e. of code becoming harder to adapt as it becomes twisted out of shape.  Tonight's discussion suggested that technical debt could include defects, redundant functionality, mismatch with a changing environment, and complexity (of requirements, of design, of code...), as well as maintainability and supportabil...

JISC have announced a programme to fund student projects over the summer.  They're calling it the Summer of Student Innovation and they are looking for projects that will use technology to improve student life  Students can pitch their ideas on the web site and the selected projects will get £5,000 to transfer the ideas into prototypes. This seems a neat idea and we have an example at Edinburgh of how this can work.  Last summer, two students from the School of Maths built a web application to help students choose the course modules most appropriate to them, taking into account their results in the previous year as well as their interests.  We are funding them for a year to develop this into a fully supported university system. It will be interesting to see what results are produced by the new JISC initiative. http://elevator.jisc.ac.uk/

A post by Jerry Fishenden has pointed me to the blog of the UK Government's Chief Technology Officer, Liam Maxwell , in which he describes the latest stage in the government's rebalancing or reform of its IT.  They have taken a significant shift away from incompatible vendor-dominated services, instead adopting cloud-based infrastructure, shared back-office services, common platforms, and open standards.  They intend to allow small, nimble vendors to fill particular niches, in a culture that is open to change.  This is a massive change and very welcome. In this latest iteration, they have released a set of guidance documents for CTO's .   This is worth reading - I particularly like the section on technology architecture .  Also this week, Chris Sexton of Sheffield Uni blogged about the Eduserv conference , which had an opening keynote about the same topic. It occurs to me that these documents are also useful for CTOs to share with their non-technical col...

Yesterday evening a group of us assembled on the Meadows and proceeded to play rounders for 90 minutes.  We were a mixture of people from Applications Division and from the University Web Site team, all involved to some extent in our investigation of the Drupal CMS .  As we work in different buildings, some 15 minutes walk apart, we rarely all congregate and our project manager decided that a social event was in order.  As we all turned down his offer of golf, rounders was the preferred option. I've been to many team social events in my life but I don't think I've played rounders since I was at school.  It was fun.  After 90 minutes, several of us were beginning to ache and we gladly retired to a nearby restaurant.  My legs still ache today. What does this achieve, from a work point of view?  Well, at the most basic, it help to puts faces to names.  More than that, it helps us to work together, because we've played together.  We learnt a ...