Skip to main content

EDUCAUSE: Learning analytics and business intelligence

Learning analytics and business intelligence were major themes at this year’s EDUCAUSE conference. I attended a couple of sessions to see how other institutions are using these technologies and to learn more about these areas for myself. Our learning technologists are all engaged with these ideas but I personally didn’t know much about it so this was a good opportunity to learn something about it.

John Doove gave a very interesting overview of several small projects funded by SURF, the Dutch equivalent of JISC. These were deliberately experimental, aimed at bringing people together to see what they could do. The results were both varied and neat and will form the basis for further work. This gave me a good overview of the sort of issues that learning analytics can investigate:

  • Project 1 gained insight in the use of Learning objects (e.g. short instructional videos), showing that use of certain materials correlated to successful student outcomes.
  • Project 2 compared course evaluation by students with student performance data.
  • Project 3 researched how students used the LMS. They found that short student sessions corresponded to the successful students, which goes against the evidence from other experiments. So they investigated this and found problems in the UI were holding up some students and interfering with their learning.
  • Project 4 investigated when to roll out a component of the LMS tool in use that everyone already bought but doesn’t use.
  • Project 5 used text mining of student notes in digital textbooks. E.g repeated occurrence of the word “understand” could indicate students not understanding a certain area.
  • Project 6 prototyped a dashboard for students, showing online behaviour in Blackboard.
  • Project 7 investigated learning paths through the curriculum and how these affect student performance. Now developing into a recommender service for students.

The other learning analytics presentation was on an altogether different scale. Three institutions collaborated to build an aggregated data set, containing over 600,000 student-level records, that they could use to analyse and compare outcomes. The talk didn’t really go into details of what they learnt from the data, focussing more on the process of building the data set in the first place.

On the business intelligence side of things, Henry Childers of the University of Arizona gave a thorough and illuminating presentation about their major BI intiative. He showed the amount of effort that had gone into their system and the organisation they used. Their approach was to start with operational data on an area-by-area basis (e.g. student data, finance, research, estates) and then gradually introduce management data and then aim for strategic data. They have a common data governance group, which sounds similar to our own Applications Architecture Governance Group.

I was particularly interested by one of the lessons Henry listed, that a specification-based approach did not work well for designing reports and that they needed a much more iterative and responsive approach. I think this matches our experience too and we should probably adapt the way we develop reports.

There were many other presentations on these issues, particularly on learning analytics. Reports I heard from other delegates suggested that the quality of presentations was rather mixed, so I’m glad that I found at least one good talk on each topic.

Comments

Popular posts from this blog

Presentation: Putting IT all together

This is a presentation I gave to an audience of University staff: 

In this seminar, I invite you to consider what the University’s online services would be like, if we worked together to design them from the perspective of the student or member of staff who will use them, instead of designing them around the organisational units that provide them. I’ll start with how the services might appear to that student or member of staff, then work back from there to show what this implies for how we work, how we manage our data, and how we integrate our IT systems. It might even lead to changes in our organisational structure.

Our online services make a vital and valued contribution to the work of our students and staff. I argue that with better integration, more consistent user interfaces, and shared data, this contribution could be significantly enhanced.

This practice is called “Enterprise Architecture”. I’ll describe how it consults multiple organisational units and defines a framework …

Not so simple...

A common approach to explaining the benefits of Enterprise Architecture is to draw two diagrams: one that shows a complicated mess of interconnections, and one that shows a nicely layered set of blocks. Something like this one, which came from some consultants:


I've never felt entirely happy with this approach.  Yes, we do want to remove as much of the needless complexity and ad-hoc design that litters the existing architecture.  Yes, we do want to simplify the architecture and make it more consistent and intelligible.  But the simplicity of the block diagram shown here is unobtainable in the vast majority of real enterprises.  We have a mixture of in-house development and different third-party systems, some hosted in-house, some on cloud infrastructure and some accessed as software-as-a-service.  For all the talk of standards, vendors use different authentication systems, different integration systems, and different user interfaces.

So the simple block diagram is, basically, a l…

2016 has been a good year

So much has happened over the last year with our Enterprise Architecture practice that it's hard to write a succinct summary.  For my day-to-day experience as enterprise architect, the biggest change is that I now have a team to work with.  This time last year, I was in the middle of a 12-month secondment to create the EA practice, working mainly on my own.  Now my post has been made permanent and I have recruited two members of staff to help meet the University's architectural needs.

I have spent a lot of the year meeting people, listening to their concerns and explaining how architecture can help them.  This communication remains vital, the absolute core of what we do and we will continue to meet people in this way.  We also talk to people in other Universities in order to learn from what they are doing and to share our own experience back.  A highlight in this regard was my trip to the USA last January.

Our biggest deliverable for the past year was the design of the data wa…