Category Archives: analytics

Part 3: Data, analytics and learning intelligence

I’ve been using the learning cycle as a framework for a strategic approach to technology in schools. This is the third post of the series, the previous two having focused on access (mobile) and action (cloud). The next stage is that of reflection. The manifestation of this aspect in my proposed strategy is analytics.

In the basic learning cycle, reflection is the all-important point in the process when we widen our awareness, take a breath and open our senses to some objective evidence of the efficacy of our efforts. Reflections may be fluid and continuous (usually resulting in micro adjustments) or periodic (usually resulting in more macro or strategic reflections). We may self-reflect (internal validation) or we may seek out reflection in the observations of others or in data (external validation). In our journey to becoming more effective learners, an important part of the process is calibrating our self-reflections to more closely match external validation. This is a lifelong process in which external validation continues to be important but we learn to learn more effectively because our internal validations are proved to be getting more accurate.

The calibration of internal and external validation is essential to the teaching and learning process. Without it, it’s quite possible for individuals to entirely miscalculate their progress and consequently focus on the wrong things to generate improvement. I’m reminded of the contestants in singing contests on TV who are convinced they are superstars in the making but who can barely sing. This is an extreme example on the spectrum (perhaps delusional) however the underlying issue is a lack of calibration between internal and external validation of effective learning.

Of course, this is (in part) precisely the purpose of the teacher. The challenge is that, being human, we’re not only capable of a little self-delusion at times but we can also project our delusions. In other words, the teacher as an instrument of reflection for learners also needs to be calibrated. Teacher calibration might come through the formative assessment process, summative assessment, experience and professional development. The challenge is to effectively and objectively benchmark our internal assessments.

This is the point at which I introduce the concept of data, analytics and learning intelligence (equate with business intelligence). Before you start telling me about the shortcomings of data in the learning and teaching process, hear me out. I know that human relationships underpin learning. What I also know is that human nature is such that we are simply not objective in our evaluations nor are we calculating machines. It is possible for us to miss patterns, to be ‘mis-calibrated’ or simply to be overwhelmed by too much data. We’re fallible.

‘Big Data’ and analytics are 21st Century phenomena emerging from the already enormous, and still rapidly increasing, speed and scale that technology affords us in capturing, aggregating, storing and analysing data. There is more data available about human behaviour than ever before and a great deal of value is locked up in that data. The promise of analytics is that new insights can be gained from analysis of the data trails left by individuals in their interactions with each other and the world, most particularly when they’re using technology.

The rapid evolution of big data methodologies and tools has, to date, been driven by the business world which recognises in them the potential for unlocking value for their customers and shareholders. In this context the term ‘business intelligence’ is often used to describe the intersection of data and insight. When applied to education, analytics may be sub-divided into two categories: learning and academic. The following table describes that categorisation:

Academic analytics are the improvement of organisational processes, workflows, resource allocation and measurement through the use of learner, academic, and organisational data. Academic analytics, akin to business analytics, are concerned with improving organisational effectiveness.

We can define learning analytics as the measurement, collection, analysis and reporting of data about learners and their contexts for the purposes of understanding and optimising learning and the environments in which it occurs. In the same way that ‘business intelligence’ informs business decisions in order to drive success, so learning analytics is the basis of ‘learning intelligence’ that is focused on improving learner success.

Learning analytics are not the goal in themselves. Learning intelligence is the goal. Learning intelligence is the actionable information arising from learning analytics that has the potential to deliver improved learner success. The evidence from analytics in business is that there is deep value to be mined in the data. The objectivity and rigour that is represented by learning analytics provides an empirical basis for everything from learner-level interventions to national policy making.The Society for Learning Analytics Research (SoLAR) is an inter-disciplinary network of leading international researchers who are exploring the role and impact of analytics on teaching, learning, training and development. Their mission as an organisation is to:

  1. Pursue research opportunities in learning analytics and educational data mining,
  2. Increase the profile of learning analytics in educational contexts, and
  3. Serve as an advocate for learning analytics to policy makers

Significant potential exists for analytics to guide learners, educators, administrators, and funders in making learning-related decisions. Learning analytics represents the application of “big data” and analytics in education. SoLAR is an organisation that is focused on a building a planned and integrated approach to developing insightful and easy-to-use learning analytics tools. Three key beliefs underpin their proposal:

  1. Openness of process, algorithms, and technologies is important for innovation and meeting the varying contexts of implementation.
  2. Modularised integration: core analytic tools (or engines) include adaptation, learning, interventions and dashboards. The learning analytics platform is an open architecture, enabling researchers to develop their own tools and methods to be integrated with the platform.
  3. Reduction of inevitable fragmentation by providing an integrated, expandable, open technology that researchers and content producers can use in data mining, analytics, and adaptive content development.

From my experience talking to educators, it’s clear they usually know that there is data available and they know how to act on learning intelligence when they have it, but they’re much less sure about the analytics phase. Whilst working on a national procurement for a learning management system last year I realised we really knew very little about the utilisation of key technology assets in the schools we were trying to build systems for. As it turned out this data was sitting, untouched, in log files in servers within these schools. I approached three of the schools and asked their permission to copy this data for the purposes of analysis. They knew it existed and were happy for me to analyse the anonymised data.

I was able to analyse the utilisation of technology assets (software and hardware) across these schools over a period of months in order to understand exactly how technology was used. This enabled me to show where the investment in technology was being dramatically underused and how it could be re-shaped to maximise utilisation of the investment in order to improve the chances of learning gains. I didn’t have time to, but could have mapped this data against the timetable and assessment data to explore how technology mapped against attainment. This would have allowed me to correlate technology utilisation by different teachers, departments and schools against the performance of their pupils.

This example is the tip of the iceberg in terms of analytics and big data in education. In terms of my technology strategy, identifying and analysing key data in your school to produce learning intelligence will maximise the learning bang for your technology buck in an objective manner. It is a critical part of your strategy because without the analysis, you may well be making unnecessary or ineffective investments in technology. Don’t be driven by technology; be driven by learning outcomes.

2012 and beyond (part 3)

“With a yo ho ho and a bottle of rum I bring you…” No, wait… I mean “Yo ho ho. Merrrrrry Christmas!” Yep, it’s almost here and Santa (minus the rum, honest) has another sacklet of goodies for you. This is part three of my ‘education-speak’ version of Techmarketview‘s 2012 predictions. My comments in blue. Enjoy!

1.  More SaaS vendors will lose money – Lack of consistent profitability is a telling sign that the commercial aspect of SaaS [Software as a Service] has not been mastered by SaaS pure plays. Higher prices and better cost control are one approach to addressing the problem but few will get the opportunity because acquisition activity will ramp up in 2012..

To make sense of this prediction, I think it’d be handy to understand what exactly “pure play SaaS” is. Helpfully, the September 12th 2008 Gartner report, “Market Trends: Software as a Service, Worldwide, 2007-2012,” states: “Gartner defines SaaS as software that is owned, delivered and managed remotely by one or more providers. The provider delivers an application based on a single set of common code and data definitions, which are consumed in a one-to-many model by all contracted customers anytime on a pay-for-use basis, or as a subscription based on use metrics.” Vendors that strictly adhere to this definition, and whose software is only available by SaaS, are often referred to as ‘pure plays’.

For some organisations, SaaS is the right answer. For example, if the organisation does not have sufficient capital budget to make the initial investment for an on-premise solution then a predictable monthly cost and payment out of a revenue budget might be attractive. Also, if the organisation does not have in-house technical expertise, a SaaS deployment managed remotely by the vendor might be the best option. Sound familiar? If you’re in education you’ll recognise these constraints. But what about the downside? Well, it is not necessarily cheap and it’s difficult (or impossible) to integrate with other systems. The May 29th 2009 Gartner report, “Dataquest Insight: SaaS Adoption Trends in the U.S. and U.K.” pointed out these issues with the top two barriers to purchase being high cost of services (42% of respondents) and difficulty with integration (38% of respondents).

With these points in mind, I think the SaaS vendors, and indeed the markets, are still working out where it really delivers value and 2012 will inevitably bring a distillation of vendors. As we know, education and particularly K12, suffer from the twin challenges of limited on-premise technical support and tight capital budgets. For this reason, I think there is a significant and largely untapped market in that sector. As is often the case, education is a little behind the curve in technology adoption. I would predict that through 2012 and beyond, schools in particular will begin to recognise the value in SaaS and that Management Information Systems will be a particular target. 

2. Social platforms will challenge enterprise platforms – Relentless pressure from employees and customers will ensure enterprises get the social and collaboration bug despite the negative pull of rigid and hierarchical organisational structures and traditional software.

It’s hard to argue with the mind-boggling adoption metrics of Facebook and Twitter, let alone fly in the face of millions of years of evolution. Homo sapiens is a social species. Who’d have guessed, eh? OK, so it’s easy to be wise after the event, but I think social platforms are leading the charge towards the general socialisation of software rather than displacing enterprise platforms as such. That is to say, pure play social (yep, even I’m doing it now) platforms offer the full social experience (like going to a Christmas party) whereas I think we will see the evolution of social features in enterprise software (like facilitating corridor meetings). Social features such as rating, reward, reputation etc will be integrated into enterprise platforms and converge the social with productivity, leveraging the benefits that we already know and love in the physical work place. For education, the convergence will be ‘social’ and ‘learning’ and here I believe there’s massive as yet untapped potential. Young people are exceptionally good at ‘social’ and harnessing this to support learning is going to be transformational. Which makes it all the more amazing that social platforms are usually non grata in schools. 

3. MEAPs will prove more ‘mobile’ than incumbents – Software suppliers will not have the mobile opportunity to themselves. They will be challenged by nimble Mobile Enterprise Application Providers (MEAPs) looking to claim a portion of the revenue software suppliers are eyeing up to help maintain growth.

Just one word: Darwinism. In a previous blog entry (Open or closed) I explored the role of natural selection in the technology ecosystem so I will not labour the points here. Suffice to say, the barriers for entry to development on mobile platforms are relatively low and so this market is wide open to massive competition. Massive competition generates rapid evolution (innovation) and so will continue to put pressure on even the big players, in some cases rendering them irrelevant because they simply take too long to bring their products to market. Small is beautiful (and mobile).

4. Big Data will move from hype to reality – Large data volumes are a fact, unstructured (and structured) data is a fact. They have to be managed and interrogated and this real need will convert hype to reality in a shorter period of time than is usual.

I’ve been listening to (thanks again Audible) ‘In The Plex’ by Steven Levy. This is the story of Googol (doh, my spelling is dreadful). And my memory… Where was I? Oh yes, so Google basically knows everything and I no longer need a memory. Well not quite everything, but you know what I mean. They know enough to be scary. One of the elements of Google I hadn’t appreciated was the centrality of Artificial Intelligence (AI) to their vision. The reason the dynamic duo of Page and Brin were so excited by large datasets is that massive amounts of data were required to enable their machines to learn meaningful things. And it’s still the case. Their view is: “If it moves, measure it.” And then they work out how to use the data. Clearly it works for them. We’re definitely in the age of ‘Big Data’ but the key will be how to convert that data into information, and that information into knowledge. Turning knowledge into wisdom may be a step too far although the idea of ‘Google Wisdom’ as our new deity is not entirely implausible.

From an education perspective, I’d say this is a largely untapped well. Culturally many education organisations find that data capture and analytics are too difficult. If you’re a parent of a child in school, you’ll certainly be aware that your child’s report is fairly one dimensional and more or less unchanged from your childhood. This is a massive missed opportunity and given that technology in schools is pretty ubiquitous, there are really exciting opportunities for capturing enormous amounts of data about learning pathways and using this data to understand what learning looks like, bringing incremental improvements in efficiency and effectiveness. I will probably blog on this specific subject in 2012, but meantime I think analytics is a significant trend to watch out for in education. 

5. Security will take centre stage – End-point security and data loss prevention will be hotbeds of activity as more businesses ramp up their use of the cloud and mobile platforms. It is not just data that is at stake, but reputations that once lost are hard to win back.

Data security and eSafety are significant concerns for education organisations too. Why? Because they hold potentially sensitive information about individuals and, for minors, they have a duty of care, acting in loco parentis. For this reason, issues of identity and security will continue to grow in importance through 2012 and beyond. Of the two issues, in education I think eSafety will be the greater concern.

While it’s easy to condemn schools for trying to control the experience of their young people while using technology, schools in particular are very vulnerable to accusations of carelessness and even negligence. As a consequence they over-compensate and ban platforms that might expose their young people to bullying, manipulation, grooming and so on. It is the reason why education so often tries to create walled gardens and why the issue of identity is particularly important for schools. Arguably this may indeed be appropriate in primary schools, but by the time young people reach secondary school, and indeed beyond, we should be supporting them to understand the risks and manage their own eSafety. The alternative – banning significant portions of the Internet experience – is ineffective (because they will usually access this experience outside of the school) and counter-productive for learning (because higher order skills required in the digital world need to be taught). Once again, this is a topic all on its own and I may come back to it in 2012.