Skip to main content

Combining information in a data warehouse

We are looking at how best to store data so that we can map trends over months and years, in what is known as a data warehouse.   We do have some digital records already, for example we can looks at trends in student numbers for over a decade.  With more and more processes online, the University administration want to look at trends over a wider range of data, and to ask questions of data that go beyond the individual administration units. 

This is not trivial, especially as we don't know which questions people will want to ask in the future.  We need a flexible data model that allows reports to link data that was taken from more than one operational system.  For example, a question about the cost of running any particular course would need to know about the cost of the rooms used, the staff, library materials, any special equipment, and so on.



We have looked at a number of ways of tackling this problem.  The approach we are now proposing  is to have two levels in our data warehouse.  The first will be a very flexible data model that can integrate information from all sorts of systems.  The second will present greatly simplified views of the data that address a particular set of questions - such as student numbers, or course costs, or research income, and so forth.

The diagram shows the end-to-end model, as it would apply to users in the Estates department.  They would see the operational systems that they use day-to-day (shown on the left), and the strategic reports and dashboards.relevant to their work (shown on the right).  Inbetween, the data warehouse will integrate this data with information from other areas of the University, and preserve it over time.

In a more technical forum, I will write some notes about the details of the data modelling approach.  It's the sort of thing that is very interesting to technical people and rather less so to the people who just want to use the system.


Comments

Popular posts from this blog

2016 has been a good year

So much has happened over the last year with our Enterprise Architecture practice that it's hard to write a succinct summary.  For my day-to-day experience as enterprise architect, the biggest change is that I now have a team to work with.  This time last year, I was in the middle of a 12-month secondment to create the EA practice, working mainly on my own.  Now my post has been made permanent and I have recruited two members of staff to help meet the University's architectural needs.

I have spent a lot of the year meeting people, listening to their concerns and explaining how architecture can help them.  This communication remains vital, the absolute core of what we do and we will continue to meet people in this way.  We also talk to people in other Universities in order to learn from what they are doing and to share our own experience back.  A highlight in this regard was my trip to the USA last January.

Our biggest deliverable for the past year was the design of the data wa…

A new EA Repository

One of my goals since starting this job two years ago has always been to create a repository for architecture documents.  The idea is to have a central store where people can find information about the University's applications, data sources, business processes, and other architectural information.  This store will make it easier for us to explain our plans, to show the current state of the University's information systems, and to explain what Enterprise Architecture is all about.

It's taken a long time to reach this goal, mainly because we're often had more pressing and immediate work to be done.  The creation of a repository is one of those tasks that is very important but never quite urgent.  So I'm now very happy to say that we are in the process of deploying a repository and modelling tool.


This is the culmination of a careful process to select the most appropriate tool for our needs.  We began by organising several workshops to gather requirements from a rang…

A brief summary of our major initiatives

I notice that in 2016 I wrote 34 posts on this blog.  This is only my fifth post in 2017 and we're already three-quarters of the way through the year.  Either I've suddenly got lazier, or else I've had less time to spend writing here.  As I'm not inclined to think of myself as especially lazy, I'm plumping for the latter explanation.

There really is a lot going on.  The University has several major initiatives under way, many of which need input from the Enterprise Architecture section.

The Service Excellence programme is overhauling (the buzzword is "transforming") our administrative processes for HR, Finance, and Student Administration.  Linked to this is a programme to procure an integrated ERP system to replace the adminstrative IT systems. 

Enabling Digital Transformation is a programme to put the middleware and architecture in place so that we can make our processes "digital first".  We're implementing an API framework, a notification…